Organizing Teams With Collective Fictions

There is often an observable difference between what is considered the right thing to do, and what actually is being done.

Terry Pratchett said it best when he made Death declare mercy and justice nonexistent: “TAKE THE UNIVERSE AND GRIND IT DOWN TO THE FINEST POWDER AND SIEVE IT THROUGH THE FINEST SIEVE AND THEN SHOW ME ONE ATOM OF JUSTICE, ONE MOLECULE OF MERCY.” (Note that Death is not shouting, he simply speaks upper case.)

We can’t measure justice and mercy. These are collective fictions — things we agree to believe to enable us to get along — and finding consensus on the immeasurable extends to political systems, religion, and most of economics. In a recent article [zwischenzugs] makes the point that methodologies in software development fall into the same category. Like collective societal fictions, methodologies tend to elicit strong emotional responses among those dealing with them.

A software development methodology is a playbook for getting from nothing to something. It’s a control system for how people working on the project spend their time. And there are a lot of these prescribed methods, from Agile to Waterfall, and any combination of letters is likely to turn an abbreviation for a methodology. An interesting game when hanging out in groups of software engineers is to start the “Have you ever tried the…” conversation. Just don’t expect to move to another topic anytime soon.

One disheartening aspect of methodologies is their resistance to scientific scrutiny. Two samples of development teams will differ wildly in so many characteristics that a meaningful comparison of the way they organize their work is not possible. Which will leaves us with anecdotes and opinions when discussing these things.

Current opinions regarding the impact of methodologies on the success of a project range from ‘marginal’ to ‘essential’. The latter position is mainly propagated by consultants selling agile certifications, so you may want to take it with a grain of salt. Whether a team adheres strongly to the methodology or adopts it in name only, it’s obvious they serve a purpose — but that purpose may not match the face value of the method.

Misused and Disliked

Methodologies are often introduced from outside of the team when things don’t go as expected. It is very likely that at this point the project has already failed due to circumstances outside the team’s control. In his classic book ‘The Mythical Man-Month‘, [Frank Brooks] recounts his struggles to ship software on schedule in a time when the volume of documentation for operating systems was measured in shelf-meters required to store the manuals.

The introduction of some methodology or another then is the attempt to regain control. Ironically the ways people resist such attempts seems to be better researched than the effectiveness of methodologies themselves. In a paper from 2003 Alistair Cockburn states:

“I was shocked to discover that people were not using even the simplest form of [Responsibility Driven Design], although it is a simple technique, they had just gone through a week-long course on the subject, and I was available on the project to answer questions. This awoke in me the awful possibility that the smallest effective methodology one could design might already be too much for practitioners to practice on a live project.”

Among developers, scenarios like these have resulted in the feeling that methodologies just don’t work. Lamenting their failings is the equivalent of gathering around the fire and bonding over war stories.

Why bother?

First, there is no way of escaping them if you are a software developer, so understanding what they are and how they work may help you. A lot of these processes are built from observations of how people get things done, so even if they fail to deliver as a whole on their promises, they often contain useful hints on how to improve your work.

[A typical programmer] argues that most methodologies are an attempt to magically inject ‘people skills’ to development teams. His takeaway is that developers should stop caring about processes and start talking to their peers. While good advice in itself, this ignores another feature of established processes. When performing well, they buy developers time to do their jobs by streamlining communication with other parts of their company. It is not uncommon for teams to adopt some methodology or another in name only to get management off their backs or to satisfy nervous inquiries about their process.

In the paper cited above, Cockburn proceeds to introduce his own concept that is typical for the current generation of methodologies. Instead of presenting us with a fixed process, he offers a set of tools that can be adjusted to the needs of each team and project. This mirrors my earlier point: you can’t directly compare one team to another and so the same application of one methodology will also not perform as well (or as poorly) from one team to the next.

So I suggest that you read up on what’s out there  — the comments to [Typical Programmers] article are a great starting point. Combine the pieces you like and give it a fancy name to sell it to your boss. That is exactly what the pros do.

Filed under: Featured, Interest, Original Art, Software Development