Tag Archives: estimation

Estimates and the desire to be lied to

Posted on by

A manager who does not accept estimates with range for uncertainty demonstrates a desire to be lied to.

A mundane corporate drama

We’re at a staff meeting of a company reliant upon but not about software.

Halfway through the hour long meeting, they start talking about a new software dependent product. The product has been a bullet point and a few sentences for months but now has some artwork and wireframes.

After a general conversation, the sponsor turns to the development team, “How much work is this?”

The team makes an educated guess – whether on the spot or after some closed room review. They express this as:

  • a rough sense of effort (easy, hard)
  • analogous to some other project or effort (at least as much work as)
  • a broad timeframe (month, quarter, week)
  • a range (it will take 2-4 days/months)
  • The team also describes some of the risks and uncertainties in general terms.

“Yes, but on what date will it be done? How much will it cost?”

***

What is an estimate?

A prediction made at a point in time with best available knowledge.

An estimate is fraught with more or less uncertainty. Early on an estimate has a very high degree of uncertainty — as high as an order of magnitude. As work proceeds depending on the risks involved, an estimate can become more accurate.

An estimate is subject to risk tied to the nature of the work, the organization, and/or external conditions.

Formal estimates (intended for consumption outside the team itself) should be expressed at an appropriate precision for the point in time they are made and with an appropriate range of error. Formal estimates should always include some analysis of the risks that might affect that estimate.

In our drama, the range of error is wildly large.

  • What is being built has never been built before — or — if it has, not by this particular team.
  • The build requires new learning and original problem solving.
  • The product launch is dependent upon third party tools, existing systems or data.
  • The ‘team’ doesn’t represent all the skills and access required to complete the software.

More fundamentally:

The sponsor doesn’t know what they want and to the degree they think they know what they want it will change during the course of the project.

And, even more fundamentally:

The sponsor isn’t the end user and doesn’t exactly know what those end users will find useful or desirable.

***

An estimate is a planning tool. Not a bid. Not a commitment.

A bid is a proposal to do a certain amount of work for a certain cost. It may or may not lock in a minimal required delivery. If so, it is managed as a revenue stream and losses against that stream are made up by other streams. A certain overhead for profit is planned in. A contract formalizes this as a business arrangement.

A commitment is a determination by a group of people to meet a certain goal. It is a real commitment if people make it themselves, there’s a clear and demonstrable definition of completion, if they believe it is achievable, if it is achievable, and if achieving it actually makes a difference for the individuals involved and for their company.

***

“Yes, but on what date will it be done? How much will it cost?”

“Give yourself a margin for safety – say 10% to 15%. I want a date I can trust.”

***

Why do estimates become a contract?

  • Companies lock in resources and budgets months before any work is under way.
  • Managers are under pressure to provide hard dates and costs to their bosses.
  • Managers may be used to working with outside vendors on a fixed cost basis.
  • Managers may be experienced in another kind of business with well-established, repeatable processes.

When people are under pressure they embrace certainty – even when they know none exists. Short of that certainty they want to know their staff cares, will work hard and is committed to delivering for them.

And they want the team to demonstrate their commitment by making a commitment.

But if the manager is capable of trust and new learning, has good intentions, and is focused on achieving their goals a team can turn this dysfunction around over time.

If not, seriously, look for another job because this one will cause you pain and stress. It will likely waste your hard work. It will rob you of joy and it may drive you from the software field altogether.

Rather than refuse to commit or make a bullshit commitment, commit to working toward your mutual success:

  • Communicate an understanding of what a sponsor is trying to acheive.
  • Demonstrate through your actions that you are working to help them achieve it.
  • Acknowledge their need to make schedule and budget commitments to the degree they really need to do this
  • Provide your estimates but be honest about the risks and uncertainties.
  • Attack the idea of “done”. The software may (should) be releasable before the full scope is built and it will require work after it is released.
  • Challenge and test assumptions about what features/implementations are needed.
  • Demand the sponsor’s participation in negotiating scope all along the process.
  • Make your progress visible both in tracking metrics against the goal and in demonstrating actual working code. Revise your projections as you go based on better information.

And don’t just build what your told. Build what people need. Waste as little as you can to get there.

More estimates in real life

Posted on by

Constraints

(July 2008) “There are some 146,000 U.S. soldiers in Iraq, down from a peak of 170,000 in 2007” — Reuters

“Although no decision has been made, by the time President Bush leaves office on Jan. 20, at least one and as many as 3 of the 15 combat brigades now in Iraq could be withdrawn or at least scheduled for withdrawal, the officials said. The most optimistic course of events would still leave 120,000 to 130,000 American troops in Iraq.” — NYT

(July 2007) “More than 180,000 civilians — including Americans, foreigners and Iraqis — are working in Iraq under U.S. contracts… The numbers include at least 21,000 Americans, 43,000 foreign contractors and about 118,000 Iraqis — all employed in Iraq by U.S. tax dollars.” — LA Times

Goal

“(O)n my first day in office, I would give the military a new mission: ending this war… ensure that our troops were redeployed safely, and our interests protected.” — Barack Obama

Estimate

“Military experts believe we can safely redeploy combat brigades from Iraq at a pace of 1 to 2 brigades a month” — Barack Obama

Target

“…that would remove them in 16 months. That would be the summer of 2010” — Barack Obama

A lot of attention has been placed on the target of sixteen months and whether Obama will stick to it. Obama has said, “I am going to do a thorough assessment when I’m there,” he said. “I’m sure I’ll have more information and continue to refine my policy.” This has been called a “flip flop” or “reversal”.

But this is a simplistic interpretation of both Obama’s position and the nature of a target. The target is informed by the estimate in an attempt to attain the goal. The target should change as new information provides better estimates and if the adjusted target better attains the goal.

comparitive us force levels by the congressional research serviceIt is not the target but the estimate and goal that need to be debated.

Who are the military experts? Does this estimate represent a consensus among these experts? What are the assumptions surrounding this estimate? Does a range of 1-2 brigades per month represent the full range of uncertainty? What are the set of risks that might scuttle this estimate?

What does safety mean in the context of a war? What does it mean to ensure our “interests” are “protected”? What kinds of events would threaten our interests and change the redeployment schedule?

As long as our public debate focuses on positional bargaining around targets we will continue to miss the point.

Stop calling it an estimate. Stop pretending it’s a commitment.

Posted on by

A product owner describes work. The team estimates it. The product owner sets a delivery target. The team commits to it.

Estimates

People are good at estimating their own ideal effort on well-defined work within their realm of experience.

People are poor at translating ideal effort into calendar days, estimating how long others will take to perform work, and estimating work that is either poorly understood.

Estimation is time consuming with diminishing returns so the effort should be managed to cost, i.e. time-boxed. That is why Agile practices invest more energy and place more value in estimating immediate work than on more speculative work farther out.

All estimates contain uncertainty. Industry research says an upfront estimate can be 25% to 400% of actual performance. The range of uncertainty is deeply dependent on context: how much work is involved, development lifecycle, experience with the particular work, shared experience within the development team and maturity of the management organization.

It is poor practice to “pad an estimate”. Padding doesn’t match the scatter that surrounds upfront estimation. For large scopes of work a developer should express an estimate as a range of uncertainty (i.e. “four to eight months skewing to between six and eight”).

Middle managers should not pad or trim a developer estimate. That is undermining the developer’s authority and making them un-accountable. The estimate is the estimate.

That doesn’t mean that the business doesn’t make planning decisions based on estimates. It means those decisions are separate from, though informed by, the estimate.

Targets

When a product owner or sponsor takes a developer estimate of 4-8 months and sets a release date six months out, they are moving beyond the estimate to set a business target. This is a judgment of what expense and time to market promise sufficient value to justify the work.

The product owner is using the developer’s estimate to inform themselves of the risk they are taking with their investment. An aggressive target within an estimate with high uncertainty is a larger risk than a conservative target on a more certain estimate.

Commitment

Setting an achievable target and owning that decision, communicating the rationale for your decision and having that rationale inform your priorities earns trust and rallies a team to deliver.

wall target by janerc on flickrIt’s the targets, stupid

Don’t set arbitrary targets. Don’t burden yourself with unnecessary risk, demotivate your developers and thoughtlessly constrain the value built into your software.

Do set meaningful targets. Take calculated risks, manage costs, partner with your developers and know what and when you need to deliver to your customers.

It’s not an estimate. The developer cannot assume your risk.

It’s not a commitment. You’ve got to earn that.

At the end of the day, the product owner is responsible for understanding the business climate, understanding the customer, describing and prioritizing the work, and managing the company’s investment to a successful outcome.