As I’ve said before, we use Thoughtworks Studio Mingle to track our backlogs.

One thing Mingle has not provided for us in the time we’ve worked with it is a daily burn up. Kind of shocking.

Last month, we migrated our instance to the EC2 cloud. I took advantage of that migration to un-cruft our Mingle instance, apply the XP template and simplify cart types and states.

Now time to stop dumping data out of mingle to report to report status. Yes. Kind of shocking.

The first step was to track the data points we need for a burn up.

That involved creating a series of date fields to track the day on which a story moves from in analysis to ready for dev, in dev, in qa, ready for customer (QA done) and customer accepted.

We also need a way to have those dates recorded automatically as part of moving cards from one state to another.

So we set up Mingle card transitions.

This replaces the drag and drop behavior of a card from one swim lane to another with a button link on the card. This a bummer for our product team but it allows us to script transitions to both change the status and set the appropriate date field.

Now to setup a burn up chart using Mingle charts and mql.

Observations:

• I had to hard code date start and date end in x-labels-start and x-labels-end but otherwise, I was able to use the project variable ‘Iteration – Scheduled’ = (Current Iteration) that’s part of the XP Template.
• Burn up is accomplished by setting cumulative:true. Unfortunately, I can’t get trend lines to work as a result.

Here’s what the result looks like:

This chart along with selected summary counts and tables allows us a real time dashboard of the health of our sprints.

As you can tell from the burn up, we have work to do improving flow in our iterations.

Anyway, reporting is a work in progress. As is everything.

Posted: August 22nd, 2010 under scrum, software development.
Comments: none

- or – building the wrong thing, really, really well.

The epitaph of many agile teams.

You can optimize your construction practices as much as you want but if there is no discernible need for what you’re building…

The goal is not to build crap well. It is to find a way each day to do less and less crap and more and more not crap.

If this were easy, there’d be a lot more software that people should really get there hands on.

Let’s all look in the mirror on that one…

Posted: July 8th, 2010 under software development.
Comments: none

I’ve written about agile software development as an ongoing, perhaps excruciatingly gradual, conversation on the definition of value.

We need a place at the conference table. But we also need a forum and language to discuss the human values behind the way we aspire to work.

“… I’ve got to have more experience with junior [children] than a lot of the people who are telling me what I should be doing with them… I think I could help bring a lot to it and nobody ever asks…They just go ahead and proclaim and we have to follow.” -– Anonymous Teacher

This quote is from a study on the failure of education reform movements, called What’s Worth Fighting for in Your School?, by Andy Hargreaves & Michael Fullan.

Reform often fails because reformers don’t engage individual contributors in the goals. Rather, they impose practices upon them. The attempt to improve the classroom actually makes the teacher less effective and sets them against reform.

Sound familiar?

The landscape of broken agile adoptions is composed of attempts to make software development “faster, cheaper, better” without embracing the experience and creativity of our core asset, the software developer.

We need to focus a little less on specific practices and a little more on building consensus among our peers over a shared set of values which despite human fallibility and economic necessity inspire us to do good work.

The Manifesto for Agile Software Development is an attempt by thought leaders to capture these principles. As with all documents, it is an artifact of a moment created to address that moment’s opportunities and constraints.

If we take the “Manifesto” as terms and conditions, something to read and sign before we advance to practice then we glide over the surface of things. It’s not what was written but why people felt the need to write it.

If we take the “Manifesto” as a complete expression, then as happens with codes of ethics, we will use its silence on certain concerns and nuance in the interpretation words themselves to excuse actions by others and even justify the harm we do ourselves.

The “Manifesto”, as with any other artifact in agile, is simply a reminder to have a conversation.

Martin Fowler discussing gender disparities in the industry:

How about a community where women are valued for their ability to program and not by the thickness of their skin? How about a community that edgily pushes new boundaries without reinforcing long running evils?

Bob Martin lobbying to add a fifth principle to the Manifesto itself:

“We value craftsmanship over execution” (or “craftsmanship over crap”)

But the discussion isn’t between thought leaders. It involves every practitioner.

We are here to create long-term value. To build businesses, careers and an industry. This is a labor of years not the current iteration.

If your concerns are entirely short term, then there are less disingenuous ways to extract wealth from people’s effort than to claim you are “agile”.

If you are agile, then you are accountable to the creativity and well-being of the teams with whom you collaborate. You are accountable for the security of end users and the benefit they derive from the software you create.

We are not tools. We are knowledge workers. What we create carries with it the way it was created. What we create is in some profound way us.

Posted: July 5th, 2010 under software development.
Comments: none

As advocates of agile software development we focus on practices.

The hype on those practices is they produce software, “faster, cheaper, better.” And we sell our efforts with the promise of, “delivering value.”

We speak of value as if the definition is shared, self-evident, contained within our backlogs and measured by our burn ups.

At the same time we minimize the hard and long the struggle to achieve mastery, identify and address a material need, and sustain creativity and quality.

So, we win the opportunity to labor with our teams to incrementally deliver potentially shippable units of code to stated business priorities.

When those priorities are pointless, so is the software.

When those priorities are tactical and subjective, the values behind agile practice — sustainable effort, maintainable code, self-directed teams, collaboration and trust — become irrelevant.

The truth is there are definitions of “value” that sell us out whether or not material success accrues to someone as a result of the software development effort.

And so, an agile adoption that is true to its participants is an ongoing, perhaps excruciatingly gradual, but substantive conversation with the larger organization on the definition of value.

A set of practices is only companion to the human values that give our work meaning.

Posted: July 3rd, 2010 under software development.
Comments: 1

I had to install an instance of Thoughtwork Studio’s agile project management tool, Mingle. My favorite tracking tool is Excel but such is life.

The instance does not need to be highly-available but I don’t want to risk data loss and I want to minimize the cost of hosting. So I decided to put the instance out on the Amazon cloud.

What follows are instructions targeted to relative newbies to PostgreSQL and Amazon Web Services (AWS) but some basic knowledge of Unix. I’ve tried to give credit wherever I based what I did on content from the web.

The whole process takes several hours with a clean set of notes but took me a couple days of trial and error to get through.

Create and deploy an Amazon Machine Image (AMI)

I used Elastic Server to build a clean Ubuntu Linux AMI with the recommended versions of Java JDK (1.5) and PostgreSQL (8.3). Through Elastic Server site I deployed the AMI to a new small instance in our Amazon EC2 Account.

Note: An implementation detail of the Elastic Server image is that the shell account user is different than the PostgreSQL user account so in the following instructions, most bash shell comands run as postgres or root using the su command.

Assign an Amazon Elastic IP to the instance and register a domain name to that IP

Since this is the cloud, I can’t rely on the permanence of the instance. Amazon supplies the Elastic IP service to ensure you have a consistent way to reference your sites regardless of the particular instance the site is hosted on.

Once the AMI was running on a new small instance within our EC2 account, I logged into our AWS console and created and assigned an Elastic IP address to the instance and registered a DNS entry against that IP address.

Move PostgreSQL data to an Elastic Block Store (EBS) volume

The small instance created by Elastic Server uses the instance-store for the boot volume. This is fine for the AMI itself and for the Mingle installation which is easy to install and rev but not for the database storage itself.

We decided the best approach for us was to locate the data store on an EBS volume with copies of custom config files and scripts. We also wanted to create nightly database dumps on that volume. Finally, we’d enable snapshots of the EBS volume giving us a stable store with convenient backup and restorals.

Create an EBS Volume (AWS Management Console)

1. Create volume 10GB in the same zone as the instance
2. Create snapshot of volume
3. Attach the volume to the instance (button in console UI volume tab – takes a while…)

Format and mount the volume from within the instance

Requirements: An attached EBS Volume

Assumptions:

• Your EBS volume is: /dev/sdf and you are creating one volume. This mount point is listed in the AWS console in the Attachment Information column of the EBS volumes console.
• You want to mount the volume at /data

1. SSH into your instance. AWS Console -> Instances -> select the specific instances -> Instance Management -> Connect -> terminal opens login as cftuser
2. type sudo fdisk /dev/sdf
3. At “Command” type n
4. At “Primary or Extended” type p
5. At Number of volumes type 1
6. Press “Return” two times to accept the default cylinder values
7. Press “w” to write the partition table and exit
8. Type sudo “mkfs.ext3 /dev/sdm” to format an ext3 file system
9. Type sudo “mkdir /data” this creates a directory called data, which we will use to mount the new volume.
10. Type sudo mount /dev/sdm /data to mount the drive
11. (optional) edit /etc/fstab to add the drive mount to automount at boot, this will not automaticly attach the volume to your instance at boot, you need to do that through the RightScale interface or maybe i’ll write another tutorial on that later.
• sudo vi /etc/fstab
• add a line: /dev/sdf /data ext3 defaults 0 0
• type :wq and type return

Instructions based on notes from https://learningspaces.njit.edu/content/how-format-and-mount-ebs-volume

Move PostgreSQL data directory in PostgreSQL config

1. Stop the PostgreSQL server sudo /etc/init.d/PostgreSQL-8.3 stop
2. Edit PostgreSQL.conf sudo /etc/PostgreSQL/8.3/main/PostgreSQL.conf
• Change from: data_directory = ‘/var/lib/PostgreSQL/8.3/main’
• To: data_directory = ‘/data/PostgreSQL/’
3. sudo cp -r /var/lib/PostgreSQL/8.3/main /data
4. sudo mv main PostgreSQL
5. sudo chown -R postgres:postgres PostgreSQL/
6. move main directory sudo mv /var/lib/PostgreSQL/8.3/main /var/lib/PostgreSQL/8.3/data_bak
7. restart server sudo /etc/init.d/PostgreSQL-8.3 start
8. copy this change to /data/config_copies

Basic PostgreSQL server setup

Change the PostgreSQL postgres user password in order to access the server. As the postgres Linux user execute the psql command:

1. sudo -u postgres psql template1
2. \password postgres
3. ctrl-D (to exit posgreSQL prompt)

Based on notes from https://help.ubuntu.com/community/PostgreSQL

Create a PostgreSQL database called “mingle”

• sudo -u postgres createdb mingle

Turn off the software firewall via the Elastic Server Management Console on your instance at port 2999

Mingle needs access to traffic across multiple ports. These are prohibited by rule by the ubuntu firewall. In a situation where you need to lock this server down, you’ll have to figure out what specific ports to open traffic to. I just turned off the Ubuntu firewall. Our instance of mingle ain’t that mission critical.

1. Open up access to tcp traffic to port 2999 in AWS.
• Enable port 2999 in the security group to which your instance is assigned in AWS. This is listed in the Security Group column of the instance list in the console. Go to Security Groups, select the group and add a rule to allow tcp traffic on port 2999.
2. Browse to Elastic server manager and turn off firewall http://:2999/firewall

Install Mingle http://www.thoughtworks-studios.com/mingle/3.1/help/installing_mingle.html

1. Download Mingle
2. secure copy the install package to your small intance:
   scp mingle_unix_.tar cftuser@:~
3. sudo mv mingle_unix_.tar /usr/local
4. cd /usr/local
5. sudo tar -xvf mingle_unix_.tar
6. sudo ln -s mingle_3_1/ mingle
7. sudo ln -s /usr/local/mingle/MingleServer /usr/local/bin/MingleServer
8. sudo mkdir /home/mingle
9. sudo mkdir /home/mingle/data
10. sudo chown cftuser /home/mingle
11. sudo chown cftuser /home/mingle/data
12. Edit the mingle config to set the swap directory and to run on port 80


-Dmingle.swapDir=/home/mingle/data/tmp
-Dmingle.appContext=/
-Dmingle.memcachedPort=11211
-Dmingle.port=80
-Dmingle.logDir=/usr/local/log
-Dmingle.memcachedHost=127.0.0.1


13. Start the instance (see below) and wait for the server to startup – takes minutes

Starting Mingle: sudo MingleServer –mingle.dataDir=/home/mingle/data start

Stopping Mingle: sudo MingleServer stop

Run the Mingle setup (refer to thoughtworks studio help)

1. Browse to the server
2. Run the install wizard
3. Create your first user
4. Enter your license key

Manually managing dumps of the mingle database

• Dumping a single database (from cftuser folder)
• sudo -u postgres /usr/bin/pg_dump –create –oids –format=c –verbose –file=”dump/mingle.backup” mingle
• Renaming the mingle database
• Stop the mingle server: sudo MingleServer stop
• sudo -u postgres psql template1
• ALTER DATABASE mingle RENAME TO mingleback
• Dropping the mingle database
• sudo -u postgres /usr/bin/dropdb -i -e mingleback
• Restoring single database
• sudo -u postgres createdb mingle
• sudo -u postgres /usr/bin/pg_restore –create –verbose –dbname=mingle dump/mingle.backup

Setting up a schedule cron job to dump the mingle database nightly at 7pm

1. Create a backup script and place it in /data named backupmingledb.sh


#!/bin/sh

#Based on PostgreSQL DUMP RECIPE by Frederik Dannemare
# Must run as postgres user!

# Script to automate database backups onto the EBS volume
# at /data/backupmingledb.sh with owner group postgres (
# dump dir = /data/database_backups/ again owner/group postgres
# entered as crontab for postgres user

export PGPASSWORD=
export PATH=$PATH:usr/bin
DATE=`date +%y%m%d-%H%M`

echo "Beginning to dump mingle database..."
DUMP_DIR=/data/database_backups/
DUMP_FILE=mingledump-$DATE.backup
pg_dump --username=postgres --create --oids --format=c --verbose --file="$DUMP_DIR$DUMP_FILE" mingle 2> /$DUMP_DIR/pg_dump.out
echo "Mingle database has been dumped to $DUMP_DIR$DUMP_FILE.gz"

exit 0


2. Change the owner group to postgres: sudo -u postgres chown postgres:postgres backupmingledb.sh
3. Make the script executable by postgres user: sudo -u postgres chmod 755 backupmingledb.sh
4. Create a cron job to invoke the script
• sudo -u postgres crontab -e
• To append a new line type: o
• Type: 0 19 * * * /data/backupmingledb.sh

Posted: July 2nd, 2010 under software development.
Comments: 1

I realize Microsoft AutoUpdate and any particular Microsoft Office update are two different executables but since the first launches the second, this dialog box strikes me as a plea for assisted suicide.

The fact that it always comes up makes me think Microsoft could have spent time on a more elegant solution.

That fact that the naming and iconography is so confusing that I’ve accidentally killed the update instead of the auto-updater requiring me to start all over again is infuriating.

Posted: June 14th, 2010 under software development.
Comments: none

Here is a draft handout from my presentation at “Agile NYC”, Instilling Agile Values for Creativity, Self-Improvement and Organizational Change – A Manager’s Perspective.

http://idisk.mac.com/kenjudy-Public/papers/agile-values-outline.pdf

Posted: May 16th, 2010 under software development.
Comments: none

I have a week to prepare for this month’s APLN-NY which now goes by the zazzy name “Agile New York City”.

Last month, I was the last minute replacement for the planned speaker until the planned venue fell through. So this month, I am the scheduled speaker. This lends an air of improvisation to the event.

Right now, I am a year and a half into my current job and locked in the kind of agile adoption I’ve only presented with hindsight. Right now, I am about the day to day: acknowledging incremental progress, working through setbacks and full of my own limitations. I don’t know the ending. What I feel is an urgency to do better and be better.

So, I worked out the topic over some e-mail exchanges with Jochen Krebs. In the absence of a story to tell, it’s time to do a little a personal retrospection.

Instilling Agile Values for Creativity, Self-Improvement and Organizational Change – A Manager’s Perspective

The scale and speed of an agile adoption are external measures that don’t speak to the founding values of the practice. Collective ownership, continuous improvement and high trust are hard won, take time and discipline but lead to craftsmanship and joy. They are enabling conditions for innovation and beneficial change.

I will retrospect on my contributions both positive and negative towards cultivating these values in two teams. The first was a practice that matured over four years, led to a new mission for the team and direct collaboration with the founder and CEO. The second, is a new team finding its way at the end of its first year.

What will I do more off? What will I do less of? What impediments got in the way?

Posted: May 5th, 2010 under software development.
Comments: none

A Cameron Blow Out Preventer (BOP) is installed at the base of the BP leased, Transocean owned Deepwater Horizon rig. An unmanageable surge of oil led to the fire that sunk the rig and ended 11 lives.

The BOP is not preventing up to 200,000 gallons/day of oil from gushing into Gulf of Mexico. At this point, no one appears to understand why.

The Blow Out Preventer is only a very visible part of an intricate human and technological system under enormous economic pressure to deliver crude oil.

If interactive complexity and tight coupling — system characteristics — inevitably will produce an accident, I believe we are justified in calling it a normal accident, or a system accident. The odd term normal accident is meant to signal that, given the system characteristics, multiple and unexpected interactions of failures are inevitable. This is an expression of an integral characteristic of the system. System accidents are uncommon, even rare; yet this is not all that reassuring if they produce catastrophes. — Normal Accidents: Living with High-Risk Technologies

Posted: April 30th, 2010 under software development.
Comments: none

Facebook patents news feeds in a social network context

The method covered in the patent includes “generating news items regarding activities associated with a user of a social network environment and attaching an informational link associated with at least one of the activities, to at least one of the news items, as well as limiting access to the news items to a predetermined set of viewers and assigning an order to the news items.” — PC World

Apple sues HTC for violations on 20 patents

Here’s the actual filing that includes the titles of each of the patents including:

• Object oriented graphic system – granted 1995
• List scrolling and document translation, scaling, and rotation on a touch-screen display – granted 2008
• Object oriented event notification system with listener registration of both interests and methods – granted 2002

What kinds of Inventions can be Protected?

An invention must, in general, fulfill the following conditions to be protected by a patent. It must be of practical use; it must show an element of novelty, that is, some new characteristic which is not known in the body of existing knowledge in its technical field. This body of existing knowledge is called ” prior art“. The invention must show an inventive step which could not be deduced by a person with average knowledge of the technical field. Finally, its subject matter must be accepted as “patentable” under law. In many countries, scientific theories, mathematical methods, plant or animal varieties, discoveries of natural substances, commercial methods, or methods for medical treatment (as opposed to medical products) are generally not patentable. — World Intellectual Property Organization

Posted: March 3rd, 2010 under software development.
Comments: none