Scrum, XP, Management and the Ethics of Agile Software Development

scrum

Making an iteration/sprint burn up chart with Thoughtworks Studio Mingle

As I’ve said before, we use Thoughtworks Studio Mingle to track our backlogs.

One thing Mingle has not provided for us in the time we’ve worked with it is a daily burn up. Kind of shocking.

Last month, we migrated our instance to the EC2 cloud. I took advantage of that migration to un-cruft our Mingle instance, apply the XP template and simplify cart types and states.

Now time to stop dumping data out of mingle to report to report status. Yes. Kind of shocking.

The first step was to track the data points we need for a burn up.

That involved creating a series of date fields to track the day on which a story moves from in analysis to ready for dev, in dev, in qa, ready for customer (QA done) and customer accepted.

We also need a way to have those dates recorded automatically as part of moving cards from one state to another.

Mingle card transitionSo we set up Mingle card transitions.

This replaces the drag and drop behavior of a card from one swim lane to another with a button link on the card. This a bummer for our product team but it allows us to script transitions to both change the status and set the appropriate date field.

Now to setup a burn up chart using Mingle charts and mql.

{% dashboard-panel %}
{% panel-heading %}Current iteration burnup{% panel-heading %}
{% panel-content %}
{{
data-series-chart
conditions: (‘Type’ = ‘Story’ OR ‘Type’ = ‘Defect’ OR ‘Type’ = ‘Task’) AND ‘Iteration – Scheduled’ = (Current Iteration) AND ‘Status’ > ‘In Analysis’ AND ‘Status’ is not ‘Deleted’ AND ‘Status’ is not ‘Blocked’ AND ‘Estimate’ IS NOT NULL AND ‘Iteration – Analysis Completed’ IS NOT NULL
labels: SELECT DISTINCT ‘Date Estimated’
x-title: Date
x-labels-start: 2010-08-10
x-labels-end: 2010-08-21
y-title: Estimated Scope in Story Points
show-start-label: false
data-point-symbol: diamond
data-labels: true
chart-height: 500
chart-width: 800
plot-height: 375
plot-width: 500
trend-ignore: zeroes-at-end-and-last-value
cumulative: true
series:
– label: Total Scope
color: black
data: SELECT ‘Date Estimated’, SUM(‘Estimate’)
– label: Development Complete
color: yellow
line-width: 1
data: SELECT ‘Date Dev Complete’, SUM(‘Estimate’) WHERE ‘Status’ >= ‘Development Complete’
– label: QA Complete
color: orange
line-width: 1
data: SELECT ‘Date QA Complete’, SUM(‘Estimate’) WHERE ‘Status’ >= ‘QA Complete’
– label: Accepted
color: blue
data: SELECT ‘Date Accepted’, SUM(‘Estimate’) WHERE ‘Status’ >= ‘Accepted’
}}
{% panel-content %}
{% dashboard-panel %}
{% dashboard-panel %}

Observations:

  • I had to hard code date start and date end in x-labels-start and x-labels-end but otherwise, I was able to use the project variable ‘Iteration – Scheduled’ = (Current Iteration) that’s part of the XP Template.
  • Burn up is accomplished by setting cumulative:true. Unfortunately, I can’t get trend lines to work as a result.

Here’s what the result looks like:

Burn up using Mingle data series chart

This chart along with selected summary counts and tables allows us a real time dashboard of the health of our sprints.

As you can tell from the burn up, we have work to do improving flow in our iterations.

Anyway, reporting is a work in progress. As is everything.

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

Time to shift focus: from Scrum tools and process to practice

I am ambivalent about the Scrum community’s focus on process and tools.

Yes, it is this effort that has driven adoption and created an economy for us practitioners. But adoption is yesterday’s challenge. We’re kind of winning that one.

We need to place less emphasis on getting new organizations to try Scrum to more on getting existing teams practice Scrum better.

DSCN1768.jpgHow many of us many, many Scrum adopters strive towards the potential of the practice?

  • Where reliable software delivers monetary return to sponsors because it is truly valuable to end users.
  • Where individual contributors are allowed to bring their most creative effort to the workplace to the benefit of both employers and end users.
  • Where workers are allowed to live rewarding lives outside the workplace to the betterment of their families and communities.

Not just exceptional productivity – ambitious enough as that is — but exceptional productivity to a genuinely productive end.

Life is full of compromise but if that is not the aspiration — to fill our careers with as much of these achievements as possible — then why bother?

Why spend money on training and tools to deliver more waste on short, iterative cycles?

Why extract more lines of code that no one will test or use but only spend money to maintain?

Why use the Scrum process to perpetuate the alienation of the knowledge worker from their work?

Mastery means taking responsibility for ourselves and our peers. Grasping our practice is the sum of our intentions and actions in the service of something.

So here’s my plea to shift the conversation back to it’s roots.

“Agile” is about the material and human good we create when we respect our co-workers tell truth to our employers, strive to improve, and care for the people affected by the software we help build.

We use a tool or process to the degree it furthers that end and no farther.

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

Oops… learning lessons over and over

Here are agile software development mistakes that kick my ass whenever I let them:

  • Know the assumptions in plans. Recognize when they change.
  • Don’t abuse time boxing. It is a toe hold for over-committing. When the time box ends, the work ends.
  • Doing Scrum means DOING SCRUM. Sloppy backlog. No Scrum. No Product Owner. No Scrum.
  • No iteration boundaries and no commitment doesn’t make me “lean”.
  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

10 Take Aways From the Bush Years

Trying to gather what I can from Bob Woodward’s column in the NYTimes, 10 Take Aways From the Bush Years. Basic management advice extracted the hard way from the record of our first MBA president. Among the lessons:

  • insist that everyone speak out loud in front of the others, even — or especially — when there are vehement disagreements
  • foster a culture of skepticism and doubt
  • insist on strategic thinking
  • embrace transparency
  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

HICSS-42

DSCN0234.JPG

This week, I’m presenting a paper at the Hawaii International Conference on System Sciences. My goal is to engage academic ethicists in a conversation about agile software development.

Given the year in employment I’ve had in the last year and what’s going on at my current employer this week, it is a gift that I was able to attend and I’m grateful for it.

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

Owning uncertainty

At Agile 2008, I attended Jeff Patton’s talk on embracing uncertainty and Alan Cooper’s keynote on interaction design.

I am convinced it is the role of product owner or customer that needs the most work in our evolving agile practices.

Sponsors express their desires as feature requests. But, as Alan Cooper argues, there is no linear progression from what people need, what they perceive they need, and how they express that in language.

At the same time, supporting departments, customers and management want a commitment to a scope and schedule. And in response, the team wants methodical decomposition to estimatable stories.

And so product owners dive into story writing, decomposing software into smaller bits in order to grasp the whole from the details. But the resulting release backlog looks only slightly more nimble software requirements specification and only slightly better at describing what customer’s really want.

What if regardless of our initial input from customers, product owners took Jeff Patton’s advice and focused our initial backlogs on specific, desired and attainable end user goals — not on interactions but why they are valuable to users? What if themes were something other than a less granular stories?

Could we retain this focus through release planning by sizing these themes not by committing to a single path and simple decomposition but by a more complex matrix of possible implementations, classifying how effectively those implementations might meet the end user goal?

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

Stop calling it an estimate. Stop pretending it’s a commitment.

A product owner describes work. The team estimates it. The product owner sets a delivery target. The team commits to it.

Estimates

People are good at estimating their own ideal effort on well-defined work within their realm of experience.

People are poor at translating ideal effort into calendar days, estimating how long others will take to perform work, and estimating work that is either poorly understood.

Estimation is time consuming with diminishing returns so the effort should be managed to cost, i.e. time-boxed. That is why Agile practices invest more energy and place more value in estimating immediate work than on more speculative work farther out.

All estimates contain uncertainty. Industry research says an upfront estimate can be 25% to 400% of actual performance. The range of uncertainty is deeply dependent on context: how much work is involved, development lifecycle, experience with the particular work, shared experience within the development team and maturity of the management organization.

It is poor practice to “pad an estimate”. Padding doesn’t match the scatter that surrounds upfront estimation. For large scopes of work a developer should express an estimate as a range of uncertainty (i.e. “four to eight months skewing to between six and eight”).

Middle managers should not pad or trim a developer estimate. That is undermining the developer’s authority and making them un-accountable. The estimate is the estimate.

That doesn’t mean that the business doesn’t make planning decisions based on estimates. It means those decisions are separate from, though informed by, the estimate.

Targets

When a product owner or sponsor takes a developer estimate of 4-8 months and sets a release date six months out, they are moving beyond the estimate to set a business target. This is a judgment of what expense and time to market promise sufficient value to justify the work.

The product owner is using the developer’s estimate to inform themselves of the risk they are taking with their investment. An aggressive target within an estimate with high uncertainty is a larger risk than a conservative target on a more certain estimate.

Commitment

Setting an achievable target and owning that decision, communicating the rationale for your decision and having that rationale inform your priorities earns trust and rallies a team to deliver.

wall target by janerc on flickrIt’s the targets, stupid

Don’t set arbitrary targets. Don’t burden yourself with unnecessary risk, demotivate your developers and thoughtlessly constrain the value built into your software.

Do set meaningful targets. Take calculated risks, manage costs, partner with your developers and know what and when you need to deliver to your customers.

It’s not an estimate. The developer cannot assume your risk.

It’s not a commitment. You’ve got to earn that.

At the end of the day, the product owner is responsible for understanding the business climate, understanding the customer, describing and prioritizing the work, and managing the company’s investment to a successful outcome.

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

Local Optima

Not to get super preachy on you all, but sometimes I think we’re full bore on the wrong mission.” — ‘Agile Shop’ by Dave Laribee

As people, we embrace change we can ourselves effect. Our conversations about value turn to story writing. Our conversations about competitiveness turn to scale.

But we risk engaging the surface of things and not the things themselves. Means to what end?

As brother bee preaches, I stand before you penitent of the sin of local optimization.

In my last job, I led a development team. We were an agile team in a non-agile company. We were engaged in the effort of years, championing organizational change bottom up.

In spite of everything we’d built — an excellent agile team, a direct relationship with our CEO, visible release backlogs and delivery — the business remained opaque. It was unable to rally to us and unwilling to provide the transparency and focus we needed to effectively rally to it.

As a result, our timeline didn’t match the life-cycle of the business. When it was acquired, our efforts were shelved and we all moved on.

An agile team in a non-agile organization is not agile enough.

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

Fail fast

Panic Button by aperte on flickrFail fast is a technique for improving the quality of software:

“failing immediately and visibly” sounds like it would make your software more fragile, but it actually makes it more robust. Bugs are easier to find and fix, so fewer go into production. – Jim Shore

Scrum aspires to a fail fast approach to building software.

It describes practices that surface problems:

  • a backlog prioritized by the product owner and estimated by the team (accountability)
  • short iterations
  • frequent retrospection
  • a role dedicated to removing impediments

It champions values that motivate individuals to address problems:

  • delivering business value
  • collaborating with customers
  • empowering teams
  • building quality in
  • continuous improvement
  • courage and honesty (a refusal to hide risk)

Possessing these values and practices, an organization is less likely to overlook or tolerate dysfunction when it materially affects the setting and achieving of project goals.

  1. risks are identified before they become problems
  2. simple problems are detected and resolved quickly
  3. thorny problems are mitigated
  4. catastrophic problems are aired to all concerned parties (informed consent)

Cases #1-3 increase a project’s chance of creating value.

Case #4 compels an organization to cancel a doomed project.

All four cases represent a better outcome for the business. Assuming that business offers value to the world, that’s better for our end users, our reputation, and our society.

Immediate and visible failure. Much preferable to hidden, prolonged and inevitable failure.

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter

Re: Interaction designer in a Scrum team

Just posted a reply on the Scrum Developers Yahoo Group. Keeping up with that list would be more effort than becoming a certified scrum master.

What I am interested in is to find out how graphical and interaction designers can be eased into Scrum development.

In my previous team, our UX director, Bob Calvano, mixed in with the team: proposing UI elements in mockups but also pairing with developers to collaborate on implementations. The team and UX director shared decisions but the UX director retained authority over them.

Concept Drawing from BrainstormingThe team and product owner learned to defer to him on thorny questions of emotion, aesthetic and interaction particularly where the product owner had no clear sense of how the decision impacted tangible customer value.

The team had to learn how to deliver constructive feedback on UX. They had to learn how to express personal opinion in that context.

The UX director needed incredible patience taking in well and poorly delivered feedback. He had to understand his own process well enough to use day to day input to enable his own creativity rather than shut it down.

We evolved this relationship in a small team in an environment of high trust and we took months getting there. He came from a more traditional agency approach but he did have a personality suited to collaboration.

He eventually left our team to become an Interaction Design Director at one of the top agencies. He did so because the high profile of the work and pay were irresistible, so this experience didn’t hurt his career progression or his ability to work other ways. Though I know for a fact he misses that team and is returning to a smaller environment where he can recapture that collaborative experience.

thoughts from people who have read Jeff Patton’s book and what they think about how his ideas fit with Scrum.

Haven’t read the book yet. Talked to Jeff about his ideas at Agile 2007 (He was my adviser on my presentation on product ownership) and at the Fall Scrum Gathering.

High praise for his thinking on user experience as a precursor in product development (why) not simply as part of execution (what).

We tend to focus on story writing as the first tangible step agile plays in product conception. There are whole worlds of collaboration in terms of understanding who the software is for and how it solves problems for human beings that should come first.

Jeff Sutherland says the vast majority of teams run Scrums without real backlogs. How many of those few product owners that have backlogs derive systems and features from a user-centered perspective?

Hoping Jeff Patton will give us practices to tackle that problem.

  • email
  • Print
  • Digg
  • Reddit
  • StumbleUpon
  • Google Bookmarks
  • del.icio.us
  • LinkedIn
  • Facebook
  • Twitter
ken h. judyExecutive manager, software developer, father and husband trying to do more good than harm.
Agile is about the material and human good we create when we respect our co-workers, tell truth to our employers, strive to improve, and care for the people affected by the software we help build.
CSPIEEE CSDP

Papers

Presentations

 

Site menu:


Meta

Creative Commons License
This work is licensed under a Creative Commons Attribution - Noncommercial - Share Alike 3.0 United States License.
Copyright © 2006-2010
Ken H. Judy.
This is a personal weblog. Views expressed are my own and not my employer.