Kallokain wrote a rebuttal to my post questioning the value of productivity metrics to justify agile.

In a recent article Ken Judy takes the stand that agile software development should not be adopted on the grounds of higher productivity. The reason for that, Judy claims, is that there are better ways to justify adopting agile than with hard numbers.

I can sympatize, because I have worked in my share of software development projects where the measurements did more harm than good. Nevertheless, I believe Judy is wrong in this instance. Most organizations measure the wrong thing. That does not imply that measuring is bad in itself. (read the post)

Just to be clear, my objection is not that agile should not be justified by hard numbers but that I haven’t seen a metric for productivity gain specifically that both stood systematic scrutiny and was economically feasible for the average business to collect.

In my experience, measures that are tied to some non-revenue related abstract concept of “productivity” such as lines of code, story points, etc. are more problematic than helpful.

My point about velocity is that it is a measure that should be a feedback system back to the teams and becomes problematic if it is considered some sort of intrinsically meaningful measure that can be reported to senior executives (as in “We completed 20 story points last iteration, 25 this iteration. We’re five story points more efficient!”)

I agree that the ultimate measure of success in business is profit. I understand that any business decision should somehow influence revenue gains or hard dollar cost savings.

The problem with justifying an agile adoption based on revenue gains is there are so many other considerations that attempts to credit any single factor become dubious.

Cost savings need to be real not theoretical. Who did you lay off? How much budget did you give back based on agile adoption? Jeff Sutherland has a paper that does show significant cost savings using Scrum. The subject company was a CMMI-5 organization and there commitment and rigor to measurement was higher than most companies would support and is cost justified based on the government regulations they perform under.

If someone can propose a relevant metric that is economical for a small to medium size business to collect, that can be measured over time in small enough units to show increased performance due to specific process changes, that has some relation to reality that can be meaningfully communicated to senior managers, and doesn’t create more problems than it solves, I will be happy to consider it.