Dreamfeed

Lots of IT projects are said to fail - many people are complaining, calling shame on the IT profession. An example is the recent article Okay to fail from the CIO Weblog, which talks about poorly run projects and underperformance.

Most IT projects are not in a predictable context, but in the complex domain - as I’ve argued before. In the complex domain, failures are a necessary part of the learning process, part of the way you make sense of what’s happening. IT projects are not in the domain of ‘best practices’, you don’t have complete (mental) models of how projects work, you can’t predict them.

A more sensible strategy is to probe, sense, respond (or apply, inspect, adapt in agile terms) - i.e. try something, look what happens, and base your actions on this information. What does trying something mean? Raymond Salzwedel describes it in 9 principles of safe-fail probes:

In the complex world (…), we rather need to create ways that allow unpredictable things to happen, and then to have a plan to deal with them, and learn from them. Moreover, we can even create situations that will test the stability (or lack thereof) of a complex system, and then observe the results. If it goes wrong, that is OK - it is allowed to. We can call these types of interventions or events, Safe-Fail.

So, failure is good, failure is a necessity. It’s a means to a higher end. Please stop complaining and whining about failing IT projects; stop calling IT such a bad boy; stop burning the guilty at the stakes - that doesn’t change a thing, it only creates fear and prevents any learning from happening. Leave out judgment and take what’s currently happening as information. Use failures as probes in the complex space of IT and learn from them.

Not only accept failures, also plan for failures as part of portfolio management: take into account that a part of your portfolio will not deliver what you want. That’s ok, it’s part of the organizational learning process.

Some tips on what to do when a project goes awry:

• Don’t panic! What’s happened, has happened; postpone judgment, regard the failure as information, run a retrospective to learn from it.
• Allow peope to fail in your organization. In particular, check your reward system - it probably rewards success and punishes failure. Your organization will learn most from its failures however (this is not an easy change…)
• Make sure the people involved get a soft landing and an opportunity to recover - failure is necessary for learning, but being part of a failing project can be ugly and stressful, provoking strong emotions.
• Work in an agile way to help you fail early and often. Working in an agile way helps address risks and uncertainties early on. You can detect wrong assumptions and other problems in a matter of weeks instead of months or years; this saves a lot of financial and emotional investment. Furthermore, it enables you to run a lot of small projects as experiments.
• Set up projects as safe-fail probes: small and incremental; make sure that if they fail, you can afford it and you will learn something.
• If you want to label a project as a failure, investigate why. When is something a failure? Donald Gause and Gerald Weinberg define a problem as a difference between what you perceive and what you desire. A failure then means that you tried to make the difference smaller, but it is still there (or it’s even bigger). Investigate the project from the viewpoint of your desires and perceptions, how these changed over time, and how the project has affected both.

References

• Donald C. Gause & Gerald M. Weinberg, Are Your Lights On?: How to Figure Out What the Problem Really Is
• Carmine Coyote, How to make more - and better - mistakes
• Norm Kerth, Project Retrospectives: A Handbook for Team Reviews
• Dave Snowden, Safe-fail or Fail-safe

Do you want to get more out of your failures? Would you like to learn how to use safe-fail probes to get more out of your IT? Feel free to contact us.

I’d like to invite you to propose session(s) for the Agile2008 conference (4-8 augustus 2008 in Toronto). It is the biggest international conference on agile development.

Would you like to tell about your agile adventures in an experience report? Do you have words of wisdom for a tutorial? Would you like to explore questions and issues in a workshop? Propose a session!

This year’s review process is iterative and transparent: until the end of February, you can update your session proposals based on the reviews you receive. You can also give feedback on other people’s session proposals. If you have a rough idea for a session, but not a perfect proposal yet, that’s also ok: just propose it and refine your ideas based on the feedback you receive.

I’m producer of the Agile and Organizational Culture stage. Linda Rising and Diana Larsen are doing a wonderful job in helping me organize it. So I’d like to ask in particular for sessions about topics like:

• interaction between agile and culture
• culture change
• agile itself as a(sub)culture
• agile in different cultures across the world
• working with different (sub)cultures in one organization
• …and anything else that relates to agile and culture!

See you next year in Toronto!

Here’s a summary of the six cultural patterns of software organizations we’ve been writing about the last months.

The Oblivious culture has taken away fear of computers and brought IT within reach of ordinary people (i.e. non-IT people). There is no separation of user and developer, which makes this culture very agile, adaptive, and customer oriented. The risk of an Oblivious culture is that the systems being developed grow complex without anyone noticing. If people finally notice, the impact of errors can be huge.

The Variable culture values craftsmanship and fosters innovation. There is close collaboration between user and developers. This culture is results oriented, management is hands off. Performance and quality are fully dependent on specific individuals. Sometimes it’s a culture of heroism. The risk of a Variable culture is the lack of knowledge sharing between individuals, and, as a result, no development of the organization as a whole.

The Routine culture brings order to chaos. It applies feedforward control, assuming a well known and predictable environment. The culture is process oriented. People assume there is one best way of developing software (methodologies) and look for silver bullets. Management manages by controlling, often resulting in micro management. In this culture you often encounter blaming and lack of trust. The risk of a Routine culture is going awry when getting into unforeseen circumstances.

The Steering culture makes extraordinary things ordinary. It applies feedback control, is results oriented and based on trust - act early, act small. Testing is an essential part of many feedback loops. The risk of a Steering culture is getting stuck in a local optimum: by focusing on stability you can miss process changes that lead to a much cheaper, better, faster way of working.

The Anticipating culture makes everything more efficient, moving out of local optima. Change is managed consciously and introduced deliberately. Everyone is involved with change. This culture is process oriented, continuously reflecting and improving - if it ain’t broke, fix it! Typical practices are full blown retrospectives, risk management, and scenario planning. The risk of an Anticipating culture is getting so involved with processes and meta processes that you miss out small, subtle possibilities for improvement.

The Congruent culture makes sure continuous change, reflection, and improvement are rooted in the culture: these become part of the organization’s DNA. A Congruent culture is highly customer oriented.

A nice article about moving towards a more congruent organization, is Beyond Blaming: Congruence in Large Systems Development Projects by Jean McLendon and Gerald M. Weinberg.

Together with Willem, I’m writing a paper about the cultural patterns model. We will first publish a Dutch version, an English version will follow soon. If you’re interested, let me know and I’ll send you a copy. We also welcome volunteers for reviewing - in return, you’ll receive an honourable mention in the paper

We’ve toured around with our People vs. Process session. At XP Days Benelux we did the session in 60 minutes, which proved to be a little on the short side for the exercises and discussion the session generates. At XP Day in London we had 90 minutes, which is about the right amount of time.

According to the feedback we received, most people liked (or even loved) the session; it made them think and gave them new insights, like why agile does not always work. The slides are available for download (PDF, 1 MB). I enjoyed both conferences a lot. Lots of people have written about their experiences, see e.g. the reports about XP Days Benelux.

If you missed us at the conferences and you would like to know what this fresh perspective on organizations and teams is, let Willem or me know. We’d be happy to run the session at your place and let you see things in a way you’ve never seen them before…