I am ambivalent about the Scrum community’s focus on process and tools.

Yes, it is this effort that has driven adoption and created an economy for us practitioners. But adoption is yesterday’s challenge. We’re kind of winning that one.

We need to place less emphasis on getting new organizations to try Scrum to more on getting existing teams practice Scrum better.

How many of us many, many Scrum adopters strive towards the potential of the practice?

• Where reliable software delivers monetary return to sponsors because it is truly valuable to end users.
• Where individual contributors are allowed to bring their most creative effort to the workplace to the benefit of both employers and end users.
• Where workers are allowed to live rewarding lives outside the workplace to the betterment of their families and communities.

Not just exceptional productivity – ambitious enough as that is — but exceptional productivity to a genuinely productive end.

Life is full of compromise but if that is not the aspiration — to fill our careers with as much of these achievements as possible — then why bother?

Why spend money on training and tools to deliver more waste on short, iterative cycles?

Why extract more lines of code that no one will test or use but only spend money to maintain?

Why use the Scrum process to perpetuate the alienation of the knowledge worker from their work?

Mastery means taking responsibility for ourselves and our peers. Grasping our practice is the sum of our intentions and actions in the service of something.

So here’s my plea to shift the conversation back to it’s roots.

“Agile” is about the material and human good we create when we respect our co-workers tell truth to our employers, strive to improve, and care for the people affected by the software we help build.

We use a tool or process to the degree it furthers that end and no farther.

Posted: April 19th, 2009 under development, scrum.
Comments: 1

Here are agile software development mistakes that kick my ass whenever I let them:

• Know the assumptions in plans. Recognize when they change.
• Don’t abuse time boxing. It is a toe hold for over-committing. When the time box ends, the work ends.
• Doing Scrum means DOING SCRUM. Sloppy backlog. No Scrum. No Product Owner. No Scrum.
• No iteration boundaries and no commitment doesn’t make me “lean”.

Posted: March 22nd, 2009 under development, scrum.
Comments: none

In response to “Doc” and my proposal for an open space on dilemmas agilists face in the workplace, we’ve been asked what the ethical issues are around Agile and whether the topic is worth the three hours we’ve requested.

Agile is built upon a set of values. They are normative in that they declare certain behaviors and outcomes as better than others. They hint at a vision of who we ought to be and what we ought to be concerned about.

As we quoted before, Jim Highsmith believes agile is engaged with the “mushy stuff of values and culture”. At Agile 2008, Bob Martin declared agile to be no less than a rallying cry to free developers.

As we strive to embody agile values in the world we will encounter dilemmas — tough, ambiguous decisions fraught with fears, limitations, risk, possible sacrifice and consequence to others.

As illustrated by the Scrum MLB simulation many of us have participated in, an intellectual understanding of agile practices doesn’t protect us from saying what people want to hear in order to get work, avoid conflict, or out of an earnest if self-defeating desire to please. Even in circumstances which offer no material stakes.

It’s more than falling short of ideals though, it is about how we fail to seriously consider these values in the first place — the hard shell of justifications for doing what we are told. I’m personally alarmed by the lack of reflection I see practitioners who take holding a job to mean we can proceed without consideration for co-workers , end users, and society — stakeholders who don’t pay our salaries.

Agilists are uniquely suited to contribute to emerging software ethical norms now largely defined by academics and waterfall practitioners (I say this non-pejoratively). We aren’t trying to refine human frailty out of the equation. We embrace essential complexity. We have techniques for reflection and continuous improvement. We turn community into our greatest strength.

But there is a conversation to be had about whether the stated agile values are enough. They don’t really speak to our obligations to our peers and our society.

Should we hold each other to higher standards? Can we challenge experienced practitioners to contribute to the field, create good will, mentor, and do good works? Should we abide self-proclaimed agilists who show no talent for initiative, courage, honesty, collaboration and love of team? Doesn’t their work reflect on us? Particularly since the growing demand for agile which they exploit is built upon the hard work of our peers and mentors?

Don’t we owe some responsibility to the reputation of our practice, the long term viability of our careers and our industry — in the interests of making software which benefits people more and harms people less — to demand more of each other?

This isn’t idealism, this is the demand society will make of us as our numbers grow and our works affect people’s daily quality of life. The pervasiveness and interconnectivity of technology and the growing dependency of ordinary people on software services is creating much greater potential for the average coder to create benefit and to unintentionally harm. There are historical precedents in fields of engineering, science, letters, and the arts where codes of conduct become hardened into enforced standards and even governmental regulation in reaction to some controversy or catastrophic failure.

Finally, we need a way to confide in each other about the actual dilemmas we face in the workplace. There are few safe spaces for community and peer coaching through the intolerable conditions some of us face. Whether it’s abuse, dishonesty, or some other harmful behavior by peers or the people who have power over us. I worked in a place (briefly and I do not list this place in my resume) where a pregnant developer quite two weeks into the job for fear of the health of her unborn child. Isolated and afraid for our livelihoods, this is a terrible time to be compelled by circumstances to do things that compromise our integrity or to sit silently while others face harm.

These are not conversations that should “presented”. This would be sanctimonious and unhelpful. Rather we need to consider these topics with humility as peers.

An open space lends itself to this. I’ll let Doc describe open space technology in more detail but it is a self-organizing process. You gather, give yourself the luxury of time and space to think and to discuss a pressing concern on your own terms with the people who choose to join you. The sessions flow and change as new topics arise and old ones wind down. Then the whole comes together to pull the experience into a coherent conclusion. The process is self-documenting with proceedings of the event created as you go, then collated and provided back to the attendees. All of this takes time. Whether the time is worth it is a matter for the participants themselves to decide.

I hope this helps explain my motivations for co-proposing this session. More feedback is greatly appreciated.

Posted: March 7th, 2009 under development.
Comments: none

The hot topic in the scrum community is scale.

Size may offer more opportunity (it may not) but it demands overhead and compromise.

Necessity forces us to skip to advanced topics but there are so many fundamentals to master:

• recruiting and developing talented and diverse individuals,
• forming collaborative and highly productive teams,
• crafting ambitious work at sustainable pace,
• excellence, invention, and joy that benefits ourselves, our peers, our customers, and end users.

I’m not as interested in learning how to work with more people as I am in learning how to invent valuable software really, really well.

Posted: May 23rd, 2008 under personal.
Comments: none

Construx has a white paper revisiting Stephen McConnell’s Software Development’s Classic Mistakes.

In it, they list ten mistakes most likely to produce catastrophic or serious consequences.

Half of them speak more to executive and product management than development:

• #1 unrealistic expectations
• #2 weak personnel
• #4 wishful thinking
• #7 lack of sponsorship
• #10 lack of user involvement

Given my experience of organizations that means projects are marked for failure well before agile methods are even applied.

Under these circumstances, we can hope frequent delivery will either morph the project into something more valuable or cause it to die a quick and merciful death.

A better answer disperses transparency, collaboration and continuous improvement from the team room out to sponsors, stakeholders, support units, suppliers, customers and end users — from development and project management to economies.

Posted: May 16th, 2008 under development.
Comments: none

Fail fast is a technique for improving the quality of software:

“failing immediately and visibly” sounds like it would make your software more fragile, but it actually makes it more robust. Bugs are easier to find and fix, so fewer go into production. – Jim Shore

Scrum aspires to a fail fast approach to building software.

It describes practices that surface problems:

• a backlog prioritized by the product owner and estimated by the team (accountability)
• short iterations
• frequent retrospection
• a role dedicated to removing impediments

It champions values that motivate individuals to address problems:

• delivering business value
• collaborating with customers
• empowering teams
• building quality in
• continuous improvement
• courage and honesty (a refusal to hide risk)

Possessing these values and practices, an organization is less likely to overlook or tolerate dysfunction when it materially affects the setting and achieving of project goals.

1. risks are identified before they become problems
2. simple problems are detected and resolved quickly
3. thorny problems are mitigated
4. catastrophic problems are aired to all concerned parties (informed consent)

Cases #1-3 increase a project’s chance of creating value.

Case #4 compels an organization to cancel a doomed project.

All four cases represent a better outcome for the business. Assuming that business offers value to the world, that’s better for our end users, our reputation, and our society.

Immediate and visible failure. Much preferable to hidden, prolonged and inevitable failure.

Posted: May 15th, 2008 under scrum.
Comments: none

Just posted a reply on the Scrum Developers Yahoo Group. Keeping up with that list would be more effort than becoming a certified scrum master.

What I am interested in is to find out how graphical and interaction designers can be eased into Scrum development.

In my previous team, our UX director, Bob Calvano, mixed in with the team: proposing UI elements in mockups but also pairing with developers to collaborate on implementations. The team and UX director shared decisions but the UX director retained authority over them.

The team and product owner learned to defer to him on thorny questions of emotion, aesthetic and interaction particularly where the product owner had no clear sense of how the decision impacted tangible customer value.

The team had to learn how to deliver constructive feedback on UX. They had to learn how to express personal opinion in that context.

The UX director needed incredible patience taking in well and poorly delivered feedback. He had to understand his own process well enough to use day to day input to enable his own creativity rather than shut it down.

We evolved this relationship in a small team in an environment of high trust and we took months getting there. He came from a more traditional agency approach but he did have a personality suited to collaboration.

He eventually left our team to become an Interaction Design Director at one of the top agencies. He did so because the high profile of the work and pay were irresistible, so this experience didn’t hurt his career progression or his ability to work other ways. Though I know for a fact he misses that team and is returning to a smaller environment where he can recapture that collaborative experience.

thoughts from people who have read Jeff Patton’s book and what they think about how his ideas fit with Scrum.

Haven’t read the book yet. Talked to Jeff about his ideas at Agile 2007 (He was my adviser on my presentation on product ownership) and at the Fall Scrum Gathering.

High praise for his thinking on user experience as a precursor in product development (why) not simply as part of execution (what).

We tend to focus on story writing as the first tangible step agile plays in product conception. There are whole worlds of collaboration in terms of understanding who the software is for and how it solves problems for human beings that should come first.

Jeff Sutherland says the vast majority of teams run Scrums without real backlogs. How many of those few product owners that have backlogs derive systems and features from a user-centered perspective?

Hoping Jeff Patton will give us practices to tackle that problem.

Posted: April 26th, 2008 under development, scrum.
Comments: none

To help some peers advocate for agile adoption, I prepared an experience report to demonstrate how my old team benefited from XP and Scrum practices. This is an extension and refinement of an earlier post on the benefits of XP.

Team Cohesion

Before and during our agile adoption, I informally administered the Gallup Q12 employee engagement survey. It is composed of twelve simple questions. Agreement correlates to retention, customer loyalty, safety records, productivity, and profitability.

From the beginning to the mid-point of our adoption, staff went from a response rate of 70% agreement 30% disagreement to 80% agreement, 15% neutral and 4% disagreement.

The most improvement was in daily opportunities “to do my best” and daily feedback on performance and expectations.

I’m convinced if I had administered the Q12 late in our adoption, we would have had even better results. The key un-addressed concerns were about having a best friend at work and feeling connected to the mission of the company. By 2007 our team grew to include people brought in by personal recommendation of other members of the team and our portfolio included consumer facing work directly for our CEO.

Rather than re-take the Q12, we undertook a 360° performance review. That we did this on our own initiative shows just how much trust we had built with each other.

Test Coverage/Code Quality

XP practices enforced methodical unit test coverage, mutually arrived at coding conventions, and real-time code inspection by multiple members of the team. The team went from no unit test practice to comprehensive coverage over the business logic and controller layers. (Unit tests against data access and gui were less comprehensive. I don’t intend to get in the middle of that debate here.)

A user story, test-driven approach to development has been shown to reduce defects in final testing by 40%.

XP and Scrum force conversations between the development team and product owner that incentivize all to build quality into the software rather than allowing technical debt to accumulate and relying on downstream QA process to fix the application.

In 2006-2007 there were no business impacting failures of our internally authored software. We were able to function as a project team with no dedicated developer maintenance staff. Change requests were minimal enough that we were able to prioritize them into our project sprints as overhead.

Reduced Risk

While any team has experts, “Agile” practices reduced our reliance on “specialists”. The entire team was capable of working on and maintaining any aspect of the code base. We passed the “bus test”; despite our small size, no project was at risk if any given member of the team became unavailable.

Leadership

Our team raised our skills and began contributing to our field. We write, present and teach at conferences on topics of scrum, XP and platform as well as contributing to open source projects and developer knowledge bases.

Recruiting and Retention

After establishing “Agile” practices we recruited skilled candidates from higher paying positions who desired to work in our culture and with our practices. We received inquiries from as far away as South America and Europe. Despite the reputation of our team and market demand we retained staff.

An additional benefit is that pairing provided an efficient on-boarding process for new hires. Developers joining the team provided immediate contribution. A metrics-based way to demonstrate this is to show that sprint commitments weren’t affected new hires first weeks. I observed that but mainly base this on comments from the team lead and existing members of the team.

Workplace Diversity

A 2006 paper by McDowell, Werner, Bullock and Fernald found that pair programming practice, “may help increase female representation in the field.”

Agile values and practices support a collaborative, empowering and sustainable work place. Such environments support diversity and take advantage of the breadth of experience each worker represents.

Client Satisfaction

We asked for quotes from our clients, vendors and even competitors which we included in our budget presentations (I’ve pretty aggressively scrubbed them):

“Working with the agile Software Development team has been rewarding on many levels…it’s a team that celebrates creativity, organization, listening, feedback, openness, honesty…and is proof positive that a great process results in great product. I look forward to our very regular meetings (I even readjust my travel schedule as much as possible to not miss anything) and am never disappointed. They are an engaging and engaged group of individuals.” – CEO

“[____ saved] half a head in [another team] and a full head in my team.” – VP

“The _______ written by our development team are the guiding-light to our decisions. [third party solution] has a vast wealth of information but no good reporting and our in-house [solution] enables us to divine meaning from the mountain of data.” – VP Traffic Operations

“We also use [third party solution] for all of our broadcast networks but I have heard about your software technology for ____. We currently do that through manual operators but I’d like to understand how you do that more sometime and how it works…” – Senior Executive, Competitor

“Given the complexities of ____ that includes the combined limitations of automation, graphic and traffic systems I believe [the team] has created a solution that has proven to be much more capable than most systems than I’ve worked with.” – Vendor

Frequent Delivery, Adaptability

Throughout 2006-2007 our team of 3-8 developers balanced two simultaneous lines of work on diverse projects built in Microsoft Windows Forms, ASP.NET to SQL Data Analysis Services Data Warehouses, Vista compatible Windows Presentation Foundation and XAML, open source .NET MVC frameworks and Ruby on Rails including a rich windows application built on beta Microsoft Technology.

The team completed eight IT and three consumer projects while doubling head count from 5 to 10 (+2 contractors). We initiated our consumer product initiative and achieved our first release of a rich windows application with a six month allocation of effectively 1.5 – 2.5 developers.

Invention/Innovation

Agile practices evolved from Lean management and associated knowledge creation theory. In this, it shares ancestry with Six Sigma.

Agile is based on empirical not plan-driven process control. It is closer to lean product development than lean industrial manufacturing.

Lean product development models sustained innovation as a process of knowledge creation and conversion within an organization that acquires and shares learning in an cycles within and across teams and up and down from leadership.

Agile fosters true joint work which is the only form of workplace collegiality that advances organizational change and innovation.

Our consumer product was recognized for its design and implementation by Microsoft’s platform and developer evangelist team as well as by the WPF team. It achieved high ratings in usability testing with end users (avg rating 8 of 10) and showed potential to deliver on its revenue targets.

Posted: February 11th, 2008 under scrum.
Comments: none

Jeff Sutherland published a paper at HICSS-41 that documents cost savings a CMMI level 5 organization delivers switching to Scrum.

As I just wrote, I would not invest in productivity measurement to justify agile adoption.

But a CMMI-5 company invests in more measurement than I would ever do so let’s take advantage of what they learned. While taking it with a grain of salt.

The company believes work costs about half as much if they use Scrum instead of their best of breed CMMI level 5 waterfall practice. For those of you who hate process, lack of a disciplined process (CMMI level 1) costs even more.

Jeff says this company does a diverse mix of work both in terms of size and platforms and that the results hold. I would want to know more about the company’s engineering practices both before and after but since Jeff is involved I’ll assume this is one of the small percentage of teams who claim to be doing Scrum that are really doing it.

Posted: February 10th, 2008 under scrum.
Comments: none

Did I measure “hard numbers” to demonstrate increased productivity with Scrum/XP? No.

…and that’s a good thing.

The tyranny of metrics.

Metrics have unintended consequences. Particularly when they justify incentives or affect people’s workplace. In the short term, performance improves regardless of what you measure. Over time, behavior distorts in ways you don’t want.

What to measure?

Lines of code? What a terrible measure of productivity! Is “thethethethethe” 5x’s more valuable than “the”? I know it takes longer to copy edit.

Function points? Who has access to a qualified function point counter? Is 6 fp’s more productive than 10 fp’s? What if I only need 4 fp’s to get my job done?

Velocity? Too many agilists fall into the trap of thinking of velocity as something objective.

Example: A team of three performs 40 story points in one sprint. A team of ten does 50. Which team is more productive?

If the teams estimate in isolation, work on different types of projects or if the sprints were months apart the appropriate answer is, “who knows.”

People will adjust their sense of how long things take over time and in response to change. That’s good. 1pt <> 1pt. 1hr <> 1hr.

Velocity is a feedback mechanism for the team – a way of informing their own intrinsic motivators and refining gut estimation. Burndown is an early warning mechanism for iterations or releases. Leave it at that.

What’s my baseline?

Rigorous tracking is a hard-won part of agile adoption itself. I have no “before” to compare to an “after”.

Small businesses are volatile. Our team grew and our work changed dramatically during the course of our agile adoption.

We have better things to do.

Measurement is an overhead. Tracking a backlog and velocity are enough.

There are better reasons to adopt agile.

We sought improved customer satisfaction, reduced risk, improved quality, incremental delivery, and innovation. We obtained other benefits including: great recruiting and retention, rapid professional development, high employee engagement. [I'll go over these benefits in a future post.]

Be the change

I saw an XP experience report by a big BPM vendor. Even they didn’t have quantitative metrics to support their adoption. Why? Because the reasons above apply to big companies as well.

If nothing else, agile practice should shorten our patience for doing things we know don’t add value. Re-frame the conversation.

Posted: February 10th, 2008 under scrum.
Comments: 5