Negative perceptions about software development. Do you have a solution?

Feedback on my proposed session at Agile 2012 on whether principled Agile practice is capable of creating workplaces and an industry more inviting of women software developers…

I could not agree more. There are many negative perceptions about software development these day in the US (off-shoring, hostile env, long hours, …). As a result, my friends at North Carolina State University tell me that overall CS enrollment is down. There was a similar event in Japan with the creation of the “Software Factory” in early 80s. I believe that they almost decimated their software industry. But how do we solve this? It has to start early as the career begins way before their first job. Leadership? If we want to maintain the industry, you bet.

Agile thought leaders came together in the first place to challenge the rest of us to empower individual contributors, elevate the role of craft and quality, cultivate collaborative ways of working, and create better, more valuable software products.

Following their lead, principled Agile practice is a determined process of honest observation and incremental improvement. It is dedicated, courageous advocacy for removing obstacles — an effort supported by analytics and a track record of improved performance.

The ambitions of this change don’t stop at a team or a set of engineering practices (though those are hard enough to accomplish). It is a change program within an organization.

We should see our mission is aligned with creating a software workplace and definition of the software developer inviting to articulate people, with diverse interests and points of views, who reflect our actual end users, and who want careers that have meaning and purpose.

I’ve never participated an agile adoption that didn’t ultimately set its sites on the larger company, the products that organization is building and why it is building them. I’ve never been part of a prolonged and dedicated agile adoption that didn’t bring developers closer to creative people outside the team, that didn’t make work a more rewarding place to show up each day.

Agile practitioners need to battle workplace cultures that discourage women and other talented people from entering and remaining in our field one dysfunction, one bully, one obstacle at a time, in one workplace at a time, because they are obstacles to collaboration and trust, disempower and burn out talented individual contributors, and distance us from our customers and end users.

I’m not saying this happens everywhere or that the changes are permanent. Widespread adoption brings with it mixed and often disappointing results. But enough of us need to drive for this change in enough of our shops, enough of the time that Agile remains a path to excellence for those of us capable of striving after it.

And by doing this we will create enough change to influence the rest of the industry. Agile adoption itself is an example of this kind of change.

Does this effort provide a clear path to success? Clearly not.

But is this approach capable of driving large and dramatic changes in companies and our industry? Yes.

Power, dissent, and bullying in software developer communities

Grassroots developer communities form around shared values in dissent against institutions and norms that dehumanize their work and diminish their efforts. They attack these orthodoxies with humor, heretical thinking, and hard work.

This benefits society when developers defy those with greater power. It harms society when developers bully people with less power.

At the ThoughtWorks sponsored Agile East, Martin Fowler spoke to his post, SmutOnRails.

Part of the community was offended by a presentation at the GoGaRuCo (Golden Gate Ruby Conference). Others fought back saying that no offense was meant, the presenter apologized, and that the tone was in the spirit of the Rails community.

(T)he view of the rails leadership seems to be this: that the objections to the presentation are yet another attempt to foist empty corporate values on the thriving Rails ecosystem… (more)

This debate is not unique to the Rails community. It reminds me of concerns my friend, Luke Melia, raised over jokes and behavior at the first Austin Alt.NET. Martin Fowler links off to a similar controversy in the Flash community.

It is also not unique to developer communities but developers in particular need to be concerned about the outcome.

Women, African Americans and Hispanics are under-represented in IT and even more so in software development. In 2001-2002 74.4% of software developers were men. 78% of those men were white.

In 1986 the percentage of women in CS programs peaked at 37%. The percentage of women in computer science programs has gone down since then.

In 2001-2, only 28 percent of all undergraduate degrees in computer science went to women. By 2004-5, the number had declined to only 22 percent. — What Has Driven Women Out of Computer Science?, NY Times

There were 15,000 women in CS progreams in 1986. Riding natural cycles this number was not matched again until 2003. This latter number contains a higher percentage of non-resident aliens who will not necessarily contribute to the US workforce.

This despite higher percentages and numbers of women acquiring college educations than men. In 2007, 33% of women 25-29 held a four year degree or higher versus 26% of men. 55% of graduates with four year degrees or higher aged 25-29 were women.

Women are even receiving the majority of degrees in science and technology. They have shown steady progress in biology, chemistry, physics, mathematics and engineering.

Metrics can be misinterpreted but these quantitative measures support a stunningly obvious anecdotal observation. US software developers are a white male enclave.

This is a power imbalance and we developers are part of the problem.

Isolation is a key factor for a higher attrition rate among women and minorities, said Teresa Dahlberg, director of the Diversity in Information Technology Institute at UNC Charlotte. People tend to associate with “like communities,” where people have similar backgrounds and interests, she explained. — Computer science lacks women, minorities, SD Times

So when we behave in a way that marginalizes and intimidates talented women and minorities, we abuse power. We become bullies. We are oppressors.

“There is a good amount of research that shows that women are judged more harshly than men, for hiring, evaluations and promotions,” she added. “Virginia Valian [author of “Why So Slow? The Advancement of Women”] shows this for women in science, technology, engineering and math faculty jobs.” Virginia Valian is a professor at Hunter College. — SD Times

Part of the problem may be a perception that software development doesn’t contribute enough to society. To the degree this perception is true it is damning. To the degree it is just a perception we have work to do as advocates.

Our actions need to be judged not by our intentions but by the outcome.

Requisite variety within our teams remains an essential enabling condition for sustained innovation.

Access to technology is growing across all tiers of class, race and gender both in the US and overseas. Diverse teams can better address our market and build software better adapted to our end users.

A more diverse workforce provides the kind of social change that will help us create a more humane workplace for developers.

Finally, anything that limits the number of able US software developers hurts our ability to compete.

When developer communities marginalize women and minorities, we conspire to isolate ourselves from the larger society. We defeat our own attempts to change the power structures around us and improve our lot and our output.

HICSS-41

I just presented Ilio Krumins-Beens’ [and my paper] on unbounded collaboration between the product owner and development team at the 41st Hawaii International Conference on System Sciences. (I’ll link off to the paper when the transaction is published on the IEEE site.)

20070110 177HICSS is an interesting mix of academics and practitioners. On the list of presenters in the agile mini-track were Jeff Sutherland, Stephen Cohen from Microsoft, and Gabrielle Benefield from Yahoo as well as researchers Ann Fruhling from the University of Nebraska at Omaha, Kevin Kwiat from the Air Force Research Laboratory, and David F. Rico.

HICSS is an instance where the academy has invited us developers into their living room to discuss what we do, the way we actually do it.

There’s a huge disconnect between what I practice as a software developer and what many institutions of higher learning teach.

Theoretical exercises in waterfall practices are not helpful precursors to TDD, pairing, continuous integration, refactoring, interdisciplinary collaboration, self-organizing teams, etc. etc.

Arguably, they are not even helpful precursors to waterfall as it’s actually practiced. If you think XP requires experienced developers what the heck do you get when you make someone with little experience architect a market trading system in UML!

We need the academy to understand us. They not only train our workforce, their research informs policies, standards and business management practices that shape government and industry expectations.

We need business schools that train prospective CXO’s to build lean businesses that will in turn build out agile/lean IT and product development organizations.

Another big barrier to agile adoption is lack of empirical support for the benefits of specific Lean, Scrum and XP practices. We need original research that correlates to the obvious things: quality, risk mitigation, market performance, productivity and cost reduction.

I’d also really love to see original research on how agile, highly collaborative practices correlate to ethical behavior on the part of individuals and organizations, gender and ethnic diversity, and sustained innovation.

Scrum Gathering – Agile and Ethics

I’ll be presenting two talks at the November Scrum Gathering.

One of them is dear to my heart. I’ll be using this blog over the next few months to work up my ideas and document conversations I have around this topic.

The Ethics of Scrum and Agile Software Development.

Here’s what I proposed:

Presentation Description

Are Scrum and XP inherently ethical?

In the face of contradictory beliefs over what we do and how we do it, we software developers, agile or not, experience pressure to compromise our work and our due care for others. Meanwhile, as our products become more beneficial, more pervasive and inter-connected our potential to harm grows.

Attempts by the ACM and IEEE to engage us in a dialog on norms of conduct has resulted in a controversial code of ethics that borrows heavily from established engineering disciplines – mandating specifications to ensure effective software.

We, agile software developers are making an under-appreciated contribution to ethical practice in our field.

Whether our work is a profession or craft, we need to engage the larger community in a conversation about how our day to day actions affect our employers, our peers, and our society. This presentation will attempt to frame professional ethics in the context of agile values and practices.

Why is this topic of interest to Scrum Gathering attendees?

The discussion over norms of ethical conduct happens outside the earshot of most working developers. The day to day experience of Scrum practitioners is at a distance from those who concern themselves with software ethics.

As a Scrum community, we have a responsibility to help shape the expectations placed upon us by others. We cannot delegate our integrity. Nor can we defer concerns over negligence, recklessness, or intent to harm the human beings who use the systems we create. We openly discuss our projects, our working conditions, and our advancement but to protect those very interests we often deal with issues of conscience privately.

Yet the passion behind Scrum is, in part, an idealistic one – a hope that by dealing openly and responsively with our stakeholders we will build something of real value. We need to harness this idealism to encourage each other make better decisions in the interests of stakeholders who do not pay us and do not always have a seat at the project table.

Given the downstream effect ethical lapses large and small have on society, we need to engage in this discussion or have the wrong solutions imposed upon us by employers, institutions, and regulatory agencies.

Presentation Objectives

  1. Is it important for us to establish a shared commitment to ethical conduct?
  2. What obligations a software developer should feel beyond fulfilling the requirements of their employer?
  3. How the Agile Manifesto and Scrum/XP practices suggest a partial set of norms of ethical conduct.
  4. How agile organizations have started to provide their own statements of principles to extend agile values and encompass conduct towards our peers and society.

The buck stops… where?

A fundamental value of computer ethics, agile and Scrum is truth telling.

As developers we have an obligation to provide honest feedback around decisions that affect either the quality or value of software our employer asks us to create.

I am now the senior person dedicated to software in my company. If that means anything it is that my responsibility has expanded to protect the best interests of my employer on any project we undertake that has software development dependencies.

The profound challenge in this is that while the company has expanded my scope of Responsibility it has not necessarily expanded my scope of Authority. As a mid-size, growing company we give department heads great discretion. My company is also one of many with a structural distinction between online and IT related software development. Therefore my team has no direct role in some of the most visible outputs of our company.

As a result, I cannot have true accountability for some outcomes since I cannot change them. That does not absolve me of my ethical responsibility nor allow me to narrowly define success in my current role. That is why to take my job seriously is to take on a certain amount of anguish.

The obvious answer is that I never should have taken on the role unless I was given commensurate authority. Perhaps this is correct.

I accepted the promotion because I believe it raises the profile of my team and makes more visible the quality and value return they are producing. I also believe that while responsibility without authority is flawed, it is a common state of play for those of us who introduce agile practices into a company. Let’s be honest, introducing agile is an attempt to lead change. Since I am not an entrepreneur it is a given that I will have to earn trust at each step of the way in making that change.

So given these circumstances, what is my responsibility to my employer around truth telling? Who am I obligated to tell truth to? Let’s focus on questions of value and quality and leave aside safety or legal concerns because as dramatic as whistle blowing is, that is not my reality.

Basic risk management tells me that the person who’s interests are most damaged should a problem arise owns the risk. In a functioning Scrum organization this is the product owner — or as Yahoo calls them, the single wringable neck. This one person is held accountable to the performance of a product in the market place. It is to them that any concerns over the value or quality of a software product need to be raised.

As I’ve said, I have a Scrum team but I do not work in a Scrum organization. There are times when projects originating outside my team do not have a clear “single wringable neck”. This presents a huge challenge. Who will be most affected by a problem if no single person is assigned real accountability for the outcome of a software project?

I have decided after hard experience that in circumstances where there is no responsible and empowered product owner the person most affected by problems is the person most identified with our brand and our products, my CEO. Therefore my responsibility for truth telling is to her.

This is, to me, a freeing realization for in it lies great opportunity for my company. My CEO who has proven herself to be an exceptional product owner. If anyone on the business side understands agile principles and my ethical responsibility as a software developer it is her. If anyone is capable of creating positive change in my company it is her.

Therefore, in the spirit of my CEO’s own vision, I will expect the best of people while maintaining my integrity and independent judgment to serve the best interests of my company, our customers, my peers, and our society.