Don’t Justify Agile Based on Productivity

Iteration Burndown by kjudyDid I measure “hard numbers” to demonstrate increased productivity with Scrum/XP? No.

…and that’s a good thing.

The tyranny of metrics.

Metrics have unintended consequences. Particularly when they justify incentives or affect people’s workplace. In the short term, performance improves regardless of what you measure. Over time, behavior distorts in ways you don’t want.

What to measure?

Lines of code? What a terrible measure of productivity! Is “thethethethethe” 5x’s more valuable than “the”? I know it takes longer to copy edit.

Function points? Who has access to a qualified function point counter? Is 6 fp’s more productive than 10 fp’s? What if I only need 4 fp’s to get my job done?

Velocity? Too many agilists fall into the trap of thinking of velocity as something objective.

Example: A team of three performs 40 story points in one sprint. A team of ten does 50. Which team is more productive?

If the teams estimate in isolation, work on different types of projects or if the sprints were months apart the appropriate answer is, “who knows.”

People will adjust their sense of how long things take over time and in response to change. That’s good. 1pt <> 1pt. 1hr <> 1hr.

Velocity is a feedback mechanism for the team – a way of informing their own intrinsic motivators and refining gut estimation. Burndown is an early warning mechanism for iterations or releases. Leave it at that.

What’s my baseline?

Rigorous tracking is a hard-won part of agile adoption itself. I have no “before” to compare to an “after”.

Small businesses are volatile. Our team grew and our work changed dramatically during the course of our agile adoption.

We have better things to do.

Measurement is an overhead. Tracking a backlog and velocity are enough.

There are better reasons to adopt agile.

We sought improved customer satisfaction, reduced risk, improved quality, incremental delivery, and innovation. We obtained other benefits including: great recruiting and retention, rapid professional development, high employee engagement. [I’ll go over these benefits in a future post.]

Be the change

I saw an XP experience report by a big BPM vendor. Even they didn’t have quantitative metrics to support their adoption. Why? Because the reasons above apply to big companies as well.

If nothing else, agile practice should shorten our patience for doing things we know don’t add value. Re-frame the conversation.

Our History of Agile Adoption

Birth of an Unusual Planetary System Courtesy NASA/JPL-Caltech.My development team began adopting Extreme Programming (XP) in January 2004.

Before this, we were hit and miss. Success relied on individuals. We had few shared practices. Our goal in going “Agile” was to consistently perform across projects.

“Agile” declares a set of common values and supportive practices. It fosters collaboration with customers and shared ownership within project teams.

In time, the team became proficient in core XP practices:

Planning

  • User stories
  • Iterative development
  • Tracked velocity
  • Daily stand-up meetings
  • Regular retrospectives with continuous improvement

Designing

  • Simple system metaphor
  • Use of development spikes
  • Refactoring

Coding

  • Onsite customer
  • Pair programming with switching
  • Test driven development (TDD)
  • Continuous integration
  • Collective ownership
  • Sustainable pace

Testing

  • Extensive unit test coverage
  • Bugs are resolved within the iteration
  • Acceptance testing by the on-site customer

The team Ript page by kjudy

Within a year the team’s performance was more consistent and visible. We were measuring our velocity and predictably delivering on our 30 day iteration goals.

We discovered our project management practices had become a bottleneck. We were clearly hitting idle periods within and around projects because of a failure to efficiently describe and prioritize work.

We introduced Scrum as a management framework on top of XP. It provided practices for organizing and prioritizing work. It helped us define roles and responsibilities.

We clarified our expectations of internal clients and achieved more efficient interactions overall. We created mechanism for reporting progress and costs to senior management.

In Q1 06, the team’s practices were evaluated by an Agile Coach, Jason Lewis. Among his findings:

Oxygen Media’s Agile software development process overall rates above average and is better then the benchmark team. The benchmark did have considerably more Agile experience, but less time together as a team.

In the evaluation of practices, the team was overall: 1) well above average to outstanding in the adaptive learning practices, 2) Above average in Sprint practices and 3) Average in planning practices. High points for the team’s individual practices were the retrospective and use of the wall for iteration tracking. The one low point was the maturity of acceptance testing.

When comparing roles to the benchmark team, the benchmark team had a much better customer role but the team was stronger in the developer and facilitator roles. When comparing the team’s adoption of the practice’s versus the benchmark the team was generally more effective. Iteration tracking was one key area the benchmark team was better, however, the team was much stronger in the all the adaptive learning practices.

Scrum Release Burndown by kjudyAfter the audit, we pre-staged our iteration planning, reduced our iterations to 2 weeks and formally planned releases.

We added discipline to our acceptance testing. We described acceptance tests in a narrative script authored by and exercised by our product owner (proxies).

We never automated acceptance tests for rich windows applications or systems tied to large, volatile back end data stores. But by Q3 2007, the team was using automated acceptance tests on it’s web applications.

The most drastic improvement however was in the customer role. Scrum defines the responsibilities of the product owner. In our case, that role was divided into two individuals.

The product owner, is an empowered single authority for prioritizing business value at the feature level. They are usually are executive level and work in the business unit “funding” the work. They also have working knowledge of the system to be built. Product owners participate in planning and review, and are available for ad hoc questions within iterations.

The product owner proxy is a member of the development staff who acts as onsite proxy for the product owner. This person assists in authoring user stories and maintaining a product backlog, meets regularly with the product owner, and acts in their place to broker decisions within the development team during iterations.

By Q2 2007, the team had product owner proxies for both our IT and our consumer facing work. Product owners included the VP of Broadcast Operations, VP of Ad Sales Traffic, our CTO, and our CEO.

Sprint Burndown by kjudyThroughout 2006-2007 our team performed exceptionally well, balancing two simultaneous lines of work and maintenance in both .NET and Ruby on Rails with four to six developers. Our projects delivered on client satisfaction, originality and early monetary goals.

Team members raised their skills and began contributing to our field. They were writing, presenting and speaking at conferences on topics of scrum, XP and platform as well as contributing to open source projects and developer knowledge bases. We were drawing positive attention from our peer community and within our company.

Our consumer product, Ript™, was recognized for its elegance in design and implementation by members of Microsoft’s platform and developer evangelist team as well as by members of the WPF team. It also achieved high ratings in usability testing with end users (avg rating 8 of 10) and showed potential to deliver on its revenue targets.

At the end of 2007 our company was acquired by a much larger television company. Software we wrote for internal use is considered valuable enough by the acquirer that they are hoping to transition into their much larger operations.

Ethical Dilemmas and Agile Software Development

“Doc” List and I proposed an ethics open space for Agile 2008.

We all experience pressure to compromise our work and our reasonable care for others. As software becomes more beneficial, more pervasive, and inter-connected, our potential to harm grows.

Agile practices are making a contribution to ethical practice in our field, but we can and should be doing more to help each other navigate the ethical dilemmas we face.

This session will attempt to frame professional ethics in the context of agile values, make the community aware of the regulatory environment we may face from both state governments and standards bodies, and engage the participants in a conversation about how our day-to-day actions affect our employers, customers, peers, end users, and society.

Here’s the proposal http://submissions.agile2008.org/node/1573. Please rate and comment!

The Functional Manager in Agile

Home Farm by Hellsgeriatric, on flickrTeam managers should till the soil with their teams.

Anything else is waste and waste must be rooted out.

Still it is hard.

Luke Melia wrote about how he performed as functional manager and dedicated 75% of his time pairing.

There are two tremendous challenges with this.

The first is limiting distractions in order to remain a reliable contributor.

Luke has tremendous reserves of focus and enthusiasm. As his manager, I did everything I could with our scrum master, Salim Divakaran, to support him, remove distractions and share workload.

The second challenge is being both the boss and a peer.

Luke recruited most of the team, he held weekly one on ones with each person, he insisted on unvarnished feedback, and is worthy of respect as both a peer and a manager.

So, here is the pattern: An experienced coach with people skills and authority over development practices pairing in with the developers. An experienced scrum master. Functional management residing in one or the other or divided up in some sensible and easily described way among the two of them.

This enables direct participation in the work, management attention to the team, and strategic contribution to the rest of the company.

Winning Hearts and Minds to Agile

My colleague, Wendy, posted a quote from our former CEO describing the benefit she gained from collaborating in an Agile environment.

The way to win over an entrepreneur is to out-perform expectations set from painful, past experience.

Before Scrum/XP

…six months later, they deliver what the assignment was. And you look at it and say, “Oh no, that’s not what I wanted.”

With Scrum/XP

…you work on a two-week cycle… You agree on what the priorities are in the meeting. You review the priorities. You evaluate where you are, and you move to the next step…

Interview with Geraldine Laybourne, Condé Nast Portfolio