Our weekly rhythm (iterations)

clock

(Clock image by simpologist on Flickr)

Establishing a rhythm to our weeks and days has provided our team a sustainable pattern for predictably delivering work with quality. Our performance is an outcome of executing within this pattern repeatedly over time.

I learned the concept of a team rhythm from Jeff Sutherland and ours is a scrum and has the shape of a game: a short time frame (one week), an achievable goal that constitutes “winning”, alternating periods of focused team work with opportunities to regroup, strategize and rest.

Rolling Planning

Each day after stand up, as needed, the developers and product people have a 30-45 min. conversation to co-author, clarify, scope and estimate stories for next week. The narrative of the story is not as important as that we have discussed it as a group and that it has mutually understood acceptance criteria.

By Thursday, the product people have force ranked next week’s work reviewed the work and ranking with our sponsor, the Chief Digital Officer.

Do we need to estimate stories? That depends on whether the conversation over the estimate adds focus to the conversation over the scope and acceptance criteria of the story, surfaces risks, raises alternatives. It also depends on whether estimates affect when or whether a story is played or how it gets prioritized. It also depends on whether your team makes an iteration commitment.

We estimate.

Commitment

On Monday morning, the team is ready to decide how far down that list of work we are confident we will complete this week. We usually start and complete additional work during the week but this is the floor.

Is the commitment necessary? It is a definitional practice in Scrum but the answer depends on whether anyone considers achieving the commitment a meaningful measure of success. It also depends on whether the conversation over what is or is not in that commitment will affect what the team is asked to work on and what gets delivered at the end of the week.

For us, the fact of a commitment helps maintain trust with our sponsor, the product people like the ability to make commitments to others with assurance we have their back, it helps the team work opportunistically on the backlog — to self-organize around the work and pull stories in a way they believe will lead to the best outcome that week while still having a reminder of which stories need to get done. So, we have a commitment.

Showcase

As a department, we hold a a 30 min. showcase where the product team and two developers present the work we delivered the prior week. That bleeds into another 60 minutes of roadmap conversations which the developers are free to duck out of at their discretion depending on the relevance.

This showcase cycles in and out of being a meaningful exercise for the developers themselves but it continues to be a useful touchpoint with the rest of our department. Whether the developers find the following roadmap discussion useful is the degree to which they feel a need to get in sync with our department goals and how our work feeds into those goals. It also depends on how well that meeting addresses that need — which varies.

Retrospect

Having made our commitment for this week, we hold a 30 minute retrospective with the product people and developers. A developer usually facilitates.

This is one of the most important tasks in our week and yet it is also the one practice we control that needs most improvement. We need to do a better job and focusing in on root causes of both what is going well and what is not. We need to do a better job of identifying concrete things we should do in response. We need to do a better job of holding ourselves accountable for doing things and observing whether they actually help.

Deploy

We deploy on Mondays. We do more as needed. We have a theoretical one touch deploy but the reality of our environment is Zombie Unicorns that sometimes wont die and so we are not yet at a point where we deploy without hesitation. We are working on our environment with our hosting provider to get past this.

Though the reality is, we find a balance point between frequency of deploys and branch management. A single weekly deployment has the advantage of allowing us a 1-2 day lag between QA’d and signed off without requiring us to maintain a rolling production deploy branch. We usually can simply deploy master. More frequent deploys would require either a more just in time sign off or a disciplined practice that connects sign off to which features are merged into/turned on in the deploy branch.

Pairing

After Monday, each day follows a rhythm: morning standup with product (15 min.), the product planning time (30-45min.), morning pair session (2.5 hours), lunch and personal work time (1.5 hours), dev team check in (15min.), afternoon pair session (3.5 hours)

The afternoon check in is something the developers themselves started that functions as part standup, part opportunity to share something they learned that morning or a problem they ran into. It also is a second opportunity in the day to rotate pairs if it makes sense.

Kaizen

One of the downsides of a joint product/dev retro is that many of the issues the developers want to discuss aren’t relevant to the product team. So, the developers themselves started holding a Kaizen at the end of the week, while things are still fresh, to share and improve our development practices and code quality.

Because the Kaizen arose from the team itself and is more focused in scope, it tends to work better than our retros.

Conclusion

So, this is how our week falls out. It is what we as a group have arrived at over time and it is continually evolving. New obstacles or constraints may drive new practices. Practices that were a solution to a specific problem can remain like vestigial limbs long after the problem has ceased to exist and need to be discontinued.

Hyperproductive? I don’t know but I’ll take it.

I’ve written about how our agile practice, particularly continuous improvement, has re-organized our product and development group into one, high performing team. How this has resulted in less code, less waste, higher quality and more value.

Jeff Sutherland talks about mature, agile teams achieving what he calls hyperproductivity. At this point, the team output accelerates and they outpace the organization around them.

annotated-burnup

Performing beyond even our own expectations allows us slack to help our product team and our sponsors define the work we do. As a matter of fact, last week, the dev team accounted for 9 of 15 stories in our backlog including 5 consumer facing additions or changes. Our sponsors have so much trust in both our ability to deliver work and our judgement that we have top down support behind the kinds of necessary efforts companies find hard to justify like platform and framework upgrades.

It is a virtuous circle, performance allows slack, slack allows time for reaching outside the immediate task to understand and optimize work for the next week, to keep up with our training and industry, to get out of the room and talk to the business, to imagine new features, to identify and solve problems.

Team members are allowed an opportunity to express their individual talents and interests: one developer can bring in a prototype of an entirely new implementation of our website, another can research and improve our search engine rank. All while working as a single collocated team that pair programs with daily pair rotation through a weekly iteration cycle and a shared backlog.

It is a synthesis of contradictions: deliver features and create less code, remove interruptions and make time out of the team room, strive for collective ownership and individual autonomy.

It results after years of dedicated practice in the techniques of Extreme Programming: pair programming, test driving, collective ownership, refactoring, iterative delivery — and determined Scrum project management: impediment removing, iterative delivery, retrospection, adapting our organization, earning trust, and protecting the team so that they could get their work done. A team is an organism that needs to close inwards, build strength and discipline before it can open up and expand outwards.

It’s the second time I’ve been a part of a team that’s reached this level of performance and both times it’s taken years to get there. I’ll take it.

Continuous Process Improvement — Deliver Less Code at Better Quality to Achieve More

In my last post I claimed we rebuilt our website properties with with half to one third the developers and with “about half 65%” the resulting code. All to justify my claim that “You accomplish more by doing less.”

Now, I’ll back that claim with some metrics.

Lines of code

There are a number of tools for measuring ruby code size and complexity. The metric_fu gem does a good job a aggregating them.

But because our retired rails application is six years old, I resorted to the perl based cloc to count lines of code. I excluded standard libraries and counted code in the app and lib directories supporting websites common to both codebases.

Old Application Lines of code (LOC) New Application Lines of code (LOC)
Websites 39,981 Websites 27,071
Admin websites 15,011 Data ingest/processing app 6,176
Shared library 2,716
Total Old 54,992 Total New 35,963

Our current codebase is 65% the size of our original*.

Code size is a predictor of defect count and ongoing maintenance. That our current codebase is 35% more efficient at delivering roughly the same perceived value implies it is materially less expensive to maintain.

Take from this that lines of code is a terrible measure of productivity.

Less code is not our only indicator of the health

As measured by rake stat, we now have 3.6 lines of test code for every line of production code. Our old codebase measures 0.4. Our new applications have 9 lines of test for every one in our old.

Test lines of code is not a measure of test coverage or quality but remember I also work in this code. I know from direct experience that the new tests are more thoughtful — less fragile and opaque — better at testing real business logic with real data. We have invested in improving our test practice.

Better tests and better test coverage allows developers to take more initiative in changing and optimizing code. We can turn around features faster and we can improve our code with less fear of breakage. It’s a virtuous circle.

Again, we rebuilt our corporate websites with half to one third the development resource, 35% less code and better test coverage. We did this over a similar time frame in one week increments without schedule pressure or extraordinary efforts.

For the product and executive managers out there

Spiral Staircase

Managers want predictable delivery in short cycles. But that is an outcome. You achieve that outcome by reducing waste and building with quality.

If you don’t make quality your priority, your developers — out of loyalty and best intentions — will not build with quality. And that leads to larger, less maintainable code, longer and less predictable delivery cycles. The opposite of what you want.

Allow developers their due as craftspeople and professionals. Instead of banging the drum on delivery and delegating responsibility for quality, commit time and attention to building with quality and your team will deliver.

Next time, I’ll dive into how and why our practices have changed over time…

When I first measured this it was 50%. In the meantime, we’ve added features, services and a new source data feed. Still 5% of our current codebase is localized terms of use and privacy policies that didn’t exist in our old website. So, growth of the codebase is something to watch but I’m not sweating it yet.

Continuous Process Improvement – Doing Less

burn-up2

I’m wary of the cliche’, “doing more with less.”

When I started in my current role, my team consisted a development manager, a product manager, a business analyst, three developers, an outsource/offshore arrangement of 6-18 developers with its own full-time relationship manager and me, a non-individual contributor tech executive. We worked with a product team of three. We built our main US and six other variations over the course of 12-16 months – which was a great success in the eyes of our sponsors.

Five years later, my team consists six developers and me. I have all the same responsibilities but get to code half-time. We still work with a product team of three. When we want to flex, we co-locate two or four contract developers onsite. These contractors rotate through pairing with staff and perform as part of our team.

To characterize this as “doing more” is profoundly mistaken. If you dive into each effort, you’d see we built less and wasted less. Less production code to change and maintain, less beloved but little used features, less idiosyncratic and unnecessarily difficult implementations, less waste in the process of defining and delivering work. Our newer codebase consists of roughly half 65%* of the production code of our old site.

You accomplish more by doing less. I know this sounds very affirmational and non-specific. I’ll describe how we accomplished this in future posts…

ProgressVisualizer – visual reporting for your Trello boards

ProgressVisualizerI’m working on a pet project I’m calling, for now, ProgressVisualizer. It is intended to be a quick and easy way to create visual reporting for Trello boards.

Trello is the first tracking tool my team finds so easy to use that we’ve largely stopped using our physical cork board. It offers simplicity, openness and doesn’t impose unnecessary assumptions. It offers an API that lets you interact with your data and webhooks which allow you to do so in near real time.

I’ve tried to follow Trello’s lead and make ProgressVisualizer easy to use. It is a tool intended for leads and managers who want burn ups, reports and other charts, don’t want to spend alot of time crafting reports for themselves, and believe they should impose as little overhead on their teams as possible.

ProgressVisualizer sample charts and reports

ProgressVisualizer sample charts and reports

ProgressVisualizer is a nights and weekends project. I don’t know what it will become. That said, it’s more than a prototype. I respect your privacy, user profile information is encrypted, and I have provided the ability to export and destroy any data collected by the site.

Right now, I’m gathering feedback and suggestions, so if you use Trello, please try ProgressVisualizer and let me know what you think.