Did I measure “hard numbers” to demonstrate increased productivity with Scrum/XP? No.
…and that’s a good thing.
The tyranny of metrics.
Metrics have unintended consequences. Particularly when they justify incentives or affect people’s workplace. In the short term, performance improves regardless of what you measure. Over time, behavior distorts in ways you don’t want.
What to measure?
Lines of code? What a terrible measure of productivity! Is “thethethethethe” 5x’s more valuable than “the”? I know it takes longer to copy edit.
Function points? Who has access to a qualified function point counter? Is 6 fp’s more productive than 10 fp’s? What if I only need 4 fp’s to get my job done?
Velocity? Too many agilists fall into the trap of thinking of velocity as something objective.
Example: A team of three performs 40 story points in one sprint. A team of ten does 50. Which team is more productive?
If the teams estimate in isolation, work on different types of projects or if the sprints were months apart the appropriate answer is, “who knows.”
People will adjust their sense of how long things take over time and in response to change. That’s good. 1pt <> 1pt. 1hr <> 1hr.
Velocity is a feedback mechanism for the team – a way of informing their own intrinsic motivators and refining gut estimation. Burndown is an early warning mechanism for iterations or releases. Leave it at that.
What’s my baseline?
Rigorous tracking is a hard-won part of agile adoption itself. I have no “before” to compare to an “after”.
Small businesses are volatile. Our team grew and our work changed dramatically during the course of our agile adoption.
We have better things to do.
Measurement is an overhead. Tracking a backlog and velocity are enough.
There are better reasons to adopt agile.
We sought improved customer satisfaction, reduced risk, improved quality, incremental delivery, and innovation. We obtained other benefits including: great recruiting and retention, rapid professional development, high employee engagement. [I’ll go over these benefits in a future post.]
Be the change
I saw an XP experience report by a big BPM vendor. Even they didn’t have quantitative metrics to support their adoption. Why? Because the reasons above apply to big companies as well.
If nothing else, agile practice should shorten our patience for doing things we know don’t add value. Re-frame the conversation.
Pingback: Cost Savings with Scrum | Ken H. Judy
re: What to measure
Why not measure throughput? If your job is to deliver software then measuring the rate at which you deliver features seems like the right thing to do. If you are already using an agile technique the calculation is simple. Number of stories delivered to your customer per month.
See also
http://en.wikipedia.org/wiki/Throughput_accounting
Interesting concept.
Here are some concerns I have:
1) Stories vary greatly both in their size and their perceived value to the customer.
2) I’ve talked to Jeff Patton about whether it’s meaningful to try to monetize something as small as a story. He and I both are skeptical. So the idea that a story is a better actual measure of value delivered than points or simple burn is debatable.
3) Quality of story writing improves drastically as product teams become more adept at agile and so you have another pseudo quantitative metric that does not mean the same thing over time.
3) There is no baseline to demonstrate a before and after. Before agile there were no stories to compare against.
4) This may be a dangerous metric to put in place if you actually tie it to rewards and a team’s ability to continue doing agile. Stories gameable. The team might be incentivize to break a feature into as many stories as possible and customers to roll up features into as few stories as possible. Shared investment in stories is difficult enough to achieve.
5) The point of the post is using metrics to champion an adoption. For management to buy into stories as a meaningful metric and not distort what that means the same way people distort velocity would require a pretty sophisticated grasp of both what stories are and what agile/lean management entails.
The long and short is that I don’t think this addresses many of my concerns for using any metric for productivity as a case for adopting agile.
I would be willing to try it in a healthy agile/lean organization as input into portfolio management. I wouldn’t have tried it four years ago when I was talking people into letting my team introduce Scrum/XP.
I do not believe this
This is differnt way of thinking.
If you dont measure, you dont know how and where are you going. Measuring will not give you perfection. But it will guide you and will show your capabilities which sometime you dont know.
Measuring scale can be wrong; but if you measure everyone using same scale; you will know whether you are improving or getting worse!
Okay, I can tell people are missing the point here. My point is not to avoid measurement. Clearly agile practices entail constant monitoring of progress. This is a feedback loop to the team so they can self manage.
What I do not believe is that you should not be using your early velocity metrics as proof of productivity gains to justify your agile adoption.
You will introduce all kinds of disfunction into your team if you encourage management to think of points as some literal measure of value.
Again, in an entirely agile organization where management “gets it” this is not an issue. But this is not the environment where one has to justify agile adoption in the first place.