[agile] or Else

Cork Board by kjudyJeff Sutherland said he was finding more developers who will only work in agile software development teams.

He also said that to his estimation about 10% of shops that claim to be practicing Scrum pass the Nokia test and have self-organized teams, product backlogs prioritized by a product owner and estimated by developers.

And that doesn’t even speak to refactoring, test driven development, pairing, continuous integration, built in quality, acceptance testing, etc.

And that doesn’t speak to knowledge creation and sharing practices across the entire organization, clarity of vision, understanding competitors, collaborating with customers, continuous improvement, and embrace of change.

I’ve come to understand that agile values place demands on development, management and business practices.

Two questions arise from this:

  • Would you only work in an [agile] shop?
  • What do you mean by [agile]?

for [agile] feel free to substitute: Lean, XP, Scrum, XP/Scrum, Crystal, Adaptive, etc. etc.

Ten Agile Fortune Cookies

  • Small is better than big
  • Great teams are better than star individuals
  • Courage and heroism are different
  • Earning reputation is better than acquiring status
  • Agilists in a non-agile business are not agile enough
  • Your career is more important than your job
  • We crave human contact with the people who use our software
  • We can earn wages and be fair, honest, and worthy of trust
  • Robbing people of joy is evil
  • There are worse fates than being a dead sheepdog

HICSS-41

I just presented Ilio Krumins-Beens’ [and my paper] on unbounded collaboration between the product owner and development team at the 41st Hawaii International Conference on System Sciences. (I’ll link off to the paper when the transaction is published on the IEEE site.)

20070110 177HICSS is an interesting mix of academics and practitioners. On the list of presenters in the agile mini-track were Jeff Sutherland, Stephen Cohen from Microsoft, and Gabrielle Benefield from Yahoo as well as researchers Ann Fruhling from the University of Nebraska at Omaha, Kevin Kwiat from the Air Force Research Laboratory, and David F. Rico.

HICSS is an instance where the academy has invited us developers into their living room to discuss what we do, the way we actually do it.

There’s a huge disconnect between what I practice as a software developer and what many institutions of higher learning teach.

Theoretical exercises in waterfall practices are not helpful precursors to TDD, pairing, continuous integration, refactoring, interdisciplinary collaboration, self-organizing teams, etc. etc.

Arguably, they are not even helpful precursors to waterfall as it’s actually practiced. If you think XP requires experienced developers what the heck do you get when you make someone with little experience architect a market trading system in UML!

We need the academy to understand us. They not only train our workforce, their research informs policies, standards and business management practices that shape government and industry expectations.

We need business schools that train prospective CXO’s to build lean businesses that will in turn build out agile/lean IT and product development organizations.

Another big barrier to agile adoption is lack of empirical support for the benefits of specific Lean, Scrum and XP practices. We need original research that correlates to the obvious things: quality, risk mitigation, market performance, productivity and cost reduction.

I’d also really love to see original research on how agile, highly collaborative practices correlate to ethical behavior on the part of individuals and organizations, gender and ethnic diversity, and sustained innovation.

Bounded Collaboration

This is the third pattern of collaboration that entrenches status quo.

Contrived collegiality” and “balkanization” suggest a certain amount of bad faith. Bounded collaboration is a subtler dysfunction.

A pragmatic and collegial relationship between a product owner and team can honor roles and feel like collaboration while barely tapping or actually working against the potential of a project and its participants.

We may simply define our contribution too narrowly.

Bounded CollaborationA development team may communicate to a product owner only during formal inspection points. They limit co-work to the immediate needs of the project and not range to larger questions and concerns. Under the pretext of “single, wringable neck” they shield themselves from the struggle to shape a business outcome and stand at a distance from the product owner.

“Bounded collaboration rarely reaches deep down to the grounds, the principles or the ethics of practice. It can get stuck with the more comfortable business of advice giving, trick trading and material sharing of a more immediate, specific and technical nature. Such collaboration does not extend beyond particular units of work or subjects of study to the wider purpose and value of what is taught and how. It is collaboration, which focuses on the immediate, the short-term and the practical to the exclusion of longer term planning concern.” — A. Hargreaves and M. Fullan

Seeming collaboration limits business opportunity and works against sustained invention and true innovation. “Contrived collegiality” and “balkanization” are forced upon us but what boundaries do we ourselves create? To what degree do we champion agile practices while surrendering the values that inspire them.

Jeff Sutherland cites the exceptional Borland Quattro Pro development team as a significant inspiration for what became Scrum practice. He also points out that Quattro Pro didn’t win in the marketplace.

Superior technical execution and transparency to a single, empowered product owner is not, unfortunately, enough. We developers need to move beyond how and when to engage a broader set of questions over what, for whom and why.

We need to work jointly with our product owners to understand the opportunity, the end users and the value our software brings to them.

Collaboration and Competition: Balkanization vs. Bounded Cohabitation

Small collaborative groups often exist in isolation or in competition with other groups within an organization.

Unhealthy Competition: Balkanization1

This is the second pattern of collaboration that entrenches status quo (see Contrived Collegiality).

balkanize:

  1. : to break up (as a region or group) into smaller and often hostile units
  2. : divide, compartmentalize <now pop culture has been balkanized; it is full of niches, with different groups watching and playing their own things — Richard Corliss>

Balkanization

In a Balkanized environment, one team’s win is another team’s loss or, at least, one team’s loss is not every team’s loss.

A company that organizes itself by specialty and doesn’t matrix well to projects lends itself to balkanization but leadership can encourage politics under any structure if they distribute rewards based on unclear, unfair or arbitrary criteria.

Valuable learning in one group is not communicated or is disputed and not widely adopted. Managers drive to surface shows of success. Individuals are not encouraged to true joint work across organizational boundaries.

Agile is often introduced bottom up without executive sponsors in less than optimal cultures. In this context, development teams have dependencies on teams that do not buy into agile values. Developers are separated from decisions about opportunities, product portfolios, potential revenues, and product features. This is both a fragile place for agile teams and also diminishes opportunity for the company.

Healthy Competition: Bounded Cohabitation

Internal competition can be used to spur original thinking and organizational change.

Nonaka and Takeuchi describe a concept of “bounded cohabitation” where teams are set in productive competition with each team pursuing a different set of premises and value propositions all geared toward the same outcome.2

The example they use is detective work. One approach is to form autonomous teams around different premises: premeditated murder, crime of passion, accident, natural causes, etc. Let the teams self-organize assembling the appropriate numbers with relevant skills and experience for their specific premise.

The teams investigate independent of each other. Under their premise, each team may look past evidence others find relevant but also follow leads other teams wouldn’t think to pursue. Eventually, one team establishes the most plausible course of events. The shared outcome is met and the teams re-organize around the next investigation.

Japanese manufactures often form multiple engineering teams around the same design challenge; e.g., an engine meeting novel requirements of size, efficiency and performance. They adopt the best solution incorporating other good ideas into the current or future products.

In one case, Sony merged two teams pursuing different product strategies: (1) an evolution in video tape players and (2) a revolutionary digital non-linear editor.

Synthesizing those world views resulted in the digital video editor with engineer-friendly analog controls that broadcast centers could rack into their existing facilities. This new technology with a familiar form factor created a new market that Sony decks dominated.

An executive sponsoring agile adoption must strive for healthy internal competition. Carve out self-organizing teams. Encourage them to follow their own paths to a clear, common goal. Mutually agree upon performance measures. Retrospect across teams to determine what’s working and why. Allow for wrong paths, allow for variation and embrace the unexpected.

The concepts and examples in this post are drawn from:

1 Hargreaves A. and Fullan M., What’s Worth Fighting for in Your School?, Teacher’s College Press, New York, 1991.

2 Nonaka, I. and Takeuchi, H., Hitotsubashi on Knowledge Management, John Wiley and Sons, Asia, 2004.