Dreamfeed

This is the next installment in a series of blog posts on Jerry Weinberg’s cultural patterns of software development organizations.

A Steering culture focuses on improving what you develop, through steering the software development system based on feedback about its outputs. Steering applies the concept of feedback control to software development management.

Oblivious, Variable and Routine cultures are based on lack of trust and power. Steering, Anticipating, and Congruent cultures on the contrary are trust-based: managers and customers trust developers to deliver a good job and create quality software, developers trust managers to remove impediments and to provide guidance instead of micromanagement, everyone is part of one team. I find a trust-based environment much more enjoyable to work in. It is also more efficient because it reduces the amount of communication and time spent on contract negotation.

Steering software development means observing the outputs (which includes software, bugs, happy team members, burned out programmers, etc.), comparing the observations to a plan, and taking action if a deviation from the plan is found. The actions are supposed to reduce the difference between what is observed and what is desired. In the field of organizational learning this steering process is called single loop learning. See also the illustration below (based on the diagrams in Quality Software Management vol. 1 by Jerry Weinberg).

The controller can be management, but this is not necessary. It includes anyone who influences the system, like a project manager, a Scrum master, an agile coach, or a technical lead.

A Steering culture introduces different ways of feedback and uses it appropriately. Testing is an important form of feedback. In Routine organizations, testing is a phase at the end, which in practice results in uncontrollable chaos and quality problems. In a Steering organization, testing is an measurement process that provides information for decision making and interventions. Testing is something that you can add to different processes: unit testing for rapid feedback on the impact of the code you’ve just changed, continuous integration for feedback on integration problems, automated acceptance tests for feedback on what’s done.

An example from an agile project:

After the review of the current iteration, you determine the current velocity, the amount of work remaining, and the number of open defects (observations).

You notice that you won’t be able to finish the work planned for the release next month unless you remove 5 stories from the plan (comparing to the plan).

You discuss this with the customer, who juggles priorities and cuts scope for this release (action).

Other examples of actions are: adjust the plan, add or drop requirements, add, remove, or train people, give a pep talk to the team, celebrate team success, and do nothing.

When everything is going well, Routine and Steering organizations look the same. When problems arise, a Steering organization has a repertoire of routines and is able to adapt instead of panicking. In a Steering culture people learn to pay attention to subtle signals and seemingly insignificant data, because big, runaway problems often start as small, manageable deviations - act early, act small. Daily standup meetings are an effective tool for surfacing issues quickly. Also pay attention to the emotions in your team, these are an indicator of other troubles. A Steering manager can use weekly informal one-on-one meetings with everyone as a tool to gauge emotions.

In a Routine culture, processes are defined but often not understood. This is why people drop them as soon as things run awry. In a Steering culture, processes are not always completely defined, but they are understood. You know why you do the things you do. As a result, you will be more inclined to stick to the processes in hard times. A Routine culture regards processes as explicitly definable and executable by interchangeable programming resources developers.

A Steering culture acknowledges that parts of the processes can be explicitly defined (explicit knowledge) but that there is also tacit knowledge involved - the know-how in the heads of the people, which cannot be written down easily.

Links

• Chris Argyris & Donald A. Schön, Organizational Learning II: Theory, Method, and Practice, about single and double loop learning
• Ikujiro Nonaka & Hirotaka Takeuchi, The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation, about explicit knowledge vs. tacit knowledge
• George Dinwiddie, It’s all about feedback

Do you wish to transition to a trust-based, steering culture? Would you like to learn how to steer your projects effectively? We can help you - feel free to contact us.

Routine - we follow our routines (except when we panic)

A Routine culture assumes that there is a optimal way of doing things and that we can determine this best way. This assumption can hold if the organizational context, the problem domain, and the technology are well-known and predictable enough.

A Routine culture might work if you mainly do small, simple, similar projects. I think a Variable culture fits this context better however, being more efficient and less risky - a group of skilled craftsmen who can oversee the work and who will do whatever they think is best to get results.

If your organization specializes e.g. in building accounting software and you have built over a dozen accounting systems with the same technology, the same DBMS, the same programming environment, a Routine culture could work. You should ask yourself however why you are doing the same thing over and over again - you’re better off making a product or service out of it and selling this product or service. Alternatively, you could make a framework or automate it (e.g. an accounting systems generator or a 4GL for accounting systems). This will give a much better return on investment.

So some IT projects are in a known, predictable context, but most of them aren’t. Usually the problem domain is not as predictable as you wish. The business context changes continuously. You have to discover requirements along the way. By deploying prototypes and early versions of software, you influence the users’ perception and understanding of problem and solution, making the whole system unpredictable.

Even if the problem domain and technology are well-known and predictable, you still have to cope with the complexity of people systems when you roll out a system. If you have one or two users, you might be able to oversee it…if you have to roll out your system to ten different offices and hundreds of users, you’re definitely in a complex space…

IT projects that deliver a strategic advantage for the business are by definition not in a known, predictable context - if it was known and predictable, there wouldn’t be any strategic advantage, only something you need to do to keep up with the competition.

And even if everything is currently known and predictable, it’s not going to stay that way:

• You tread into a new and unknown problem domain, thinking something like: nah, client relationship management is probably just like accounting.
• Someone else creates an accounting service that costs ¼ of what you charge, undermining your company’s reason for existence.
• Your programming language or 4GL isn’t supported any more and you have to migrate to .NET, a technology that is completely new to you.

This makes Routine processes break down. People start to panic. In practice, the software development system becomes more and more like a Variable culture, where most people just do whatever they think should be done in order to get over with it. This is not something that is discussed or admitted (don’t you dare mention it!). Officially, you’re still following your routines. In practice, you’re doing something completely different.

An example: in a regular waterfall project, a test phase is planned at the end of the project, followed by a few days of rework, followed by deployment to production. What happens in practice, is: the test phase uncovers a lot of defects all at once, developers work 70+ hour weeks to fix everything; they start cutting corners in code quality and unit testing as a response to the management pressure to finish before the release date; ultimately, even a few untested fixes slip into production.

How can you recognize a Routine culture going awry?

• Lack of trust; focus on role definitions, contracts, service level agreements instead of collaboration.
• People say “that is his/her responsibility, not mine!”
• Lots of blaming (and placating) - managers look for scapegoats and reprimand developers for not following the process - Who’s fault is this? Heads must roll!; developers blame managers, but not openly.
• Developers complain about all the routine meetings, but still attend them, silently nodding and yawning.
• People try to stick more and more to the routines, adding extra procedures and rules; but this makes things even worse.

So what can you do when this happens? You could let the routines fade away over time and proceed in the chaotic way things are already happening, i.e. transitioning to a Variable culture.

Alternatively, you can try transitioning to a Steering culture, but this is not for the faint of heart: you have to build trust and make the undiscussable difference between what is should happen and what actually happens discussable. One of the next installments will be about transitioning from a Variable or Routine culture to a Steering culture.

In the mean time, don’t forget to check out Nynke’s blog entry on scenario planning as an Anticipating culture practice.

Links

• Most IT Jobs Are Like Deodorant - about strategic, value adding IT work vs. essential but not strategic IT work.
• The new dynamics of strategy: Sense-making in a complex and complicated world - in depth article about the Cynefin model, about known, the knowable, complex, and chaotic spaces.
• Gerald M. Weinberg, Quality Software Management vol. I: Systems Thinking

Last week, I attended the Agile Business Conference in London. This conference was organized by the DSDM consortium, in cooperation with the Agile Alliance. The program seemed to be aiming at the early majority (people who recently discovered agile) and not so much at seasoned agile practitioners.

I enjoyed the numerous good conversations, sessions like Creating Shared Understanding by Angela Martin, Duncan Pierce, and Olivier Lafontan, and Chris Avery’s keynote about responsibility. Chris presented a model on how people respond to problems in different (not always effective) ways. It looks like a useful model about people’s behaviour in stressful situations, like Virginia Satir’s coping stances model (pdf).

Willem and I ran a short interactive session on Exploring the Agile Space. Originally we wanted to do a World Café, but because there was only a small group of people present (the session was at the end of the conference, people were already leaving), we had a lively plenary discussion about what participants had done to successfully apply agile across the business.

Routine - we follow our routines (except when we panic)

Willem wrote that a routine culture is not focused on results. Instead, it’s focused on methods, processes, and procedures. The underlying assumptions are that you are in a known, more or less predictable context and that you know that certain procedures and methods predictably lead to desired results. So to get to your results, focusing on steps, procedures, methods should be enough. It’s basically feed forward control of your software development system.

Working from these assumptions, it could make sense to blame the method or blame the person following the method if you don’t get the desired results: by following the method in the right way, you should get the right results, hence wrong results mean either the method doesn’t work or the people didn’t follow the method.

A Routine culture can work, as long as its underlying assumptions hold. A pitfall here is that when the methods are working well, they get reinforced, making them more and more efficient. The software development system gets more and more adapted to its context, but also less and less resilient to changes in that context. The assumption that the correct method produces a correct result becomes a ‘truth’. The system will get into trouble when either the context remains the same for a long time and then changes radically or when the context changes subtly over time and these changes are detected when they’ve grown big.

If a method doesn’t work any more, most people still try to make the best of the situation. There grows a tension between the espoused theory - the official method - and the theory in use - what people actually do to get results. This tension can be useful to stimulate process innovation (for more information about espoused theory vs. theory in use, see Organizational Learning II by Argyris and Schön).

A dysfunctional reaction is to cling more and more to the methods and blame the people for the results becoming worse. Now the theory in use and the espoused theory start to run widely apart. The link between them gets lost, some people focus on the method, trying to carve it in stone (without a defined process, how will we know who to blame?), others just try to get things done in spite of the Method. The painful tension between what actually happens and what ought to happen is covered up with a blanket of undiscussability - if you try to discuss the gap, you’ll be punished. Even the undiscussability is undiscussable. This is basically a Train-Wreck Management culture, with lots of blaming from management towards development. Now your organization is in deep trouble, a useful intervention could be to get a new job. There are other ways to get your organization out of this mess, but that’s a topic for a future blog entry.

A congruent response to the tension between espoused theory and theory in use is to acknowledge the end of the illusion that your methods are working, embrace the chaos, and move through practice and integration into a new status quo (in terms of the Satir model of change).

So within a known, more or less predictable context, a Routine culture can work. You should however frequently verify the underlying assumptions and improve your methods if they aren’t working any more (see also the Standardized Work principle of The Toyota Way). If you’re doing this, you’re in fact moving towards a Steering culture, by using feedback from your results to manage your software development system.

The next installment will be about why a Routine culture is often not appropriate for IT projects.

As promised, here is the first installment of a series of blog entries about cultural patterns of software organizations. Jerry Weinberg distinguishes the following patterns:

• Oblivious - we’re not aware that we’re developing software; there is no separation between user and development, e.g. a manager who writes Excel macros to support his decision making process.
• Variable - we do whatever we feel like at the moment; performance depends mainly on individuals (superprogrammers, “heroes”), e.g. a small startup with a few very skilled programmers developing the first product version quickly and efficiently.
• Routine - we follow our routines (except when we panic); e.g. an IT department that establishes a waterfall project management methodology as the right way to run all IT projects; this kind of works for projects that aren’t too complicated, but when applied to a brand new, important, complex project, everyone drops the standard process and works 60+ hour weeks towards the end of the project, in a desperate attempt to get the project finished.
• Steering - we choose among our routines by the result they produce; e.g. an agile project with short iterations provides insight in the results at the end of each iteration, through product demos and iteration retrospectives. This gives the product owner the opportunity to steer the product and the team the opportunity to adapt their way of working.
• Anticipating - we establish routines based on our past experience with them; e.g. explicit risk management activities that help to anticipate or prevent troubles instead of reacting to the troubles when they arise.
• Congruent - everyone is involved in improving everything all the time; it’s a culture of continuous reflection and improvement of processes, continuously improving the baselines; as a non-IT example, the idea of the Toyota Production System is to establish and sustain such a culture of continuous improvement.

If these patterns look familiar to you, that’s not a coincidence: they have the same origin as the CMMI maturity levels (they’re both based on the ideas from Quality Is Free by Philip B. Crosby), except Oblivious which was added by Jerry Weinberg. The patterns are not maturity levels however. Steering or Anticipating is not necessarily better or more mature than Variable or Routine. It’s all about whether a specific pattern fits your context or not.

In the context of strategically relevant IT projects, I believe it’s worthwhile to transition from a Variable or Routine culture to Steering and then to Anticipating. I’ll elaborate this in one of the next installments; for now, I’ll just mention that agile development principles and practices help you a great deal in making the shift to steering, but they don’t provide enough to help you establish an anticipating culture.

The next installment will be about the Routine pattern.

This week I got a notification that our proposal for an interactive presentation called People vs. Process: Cultural Patterns of Software Organizations has been accepted for both XP Days Benelux (15 and 16 November) and XP Day London (19 and 20 November). I’ll run this session together with Willem.

In this session, we are going to present different cultural patterns of software organizations, as described by Jerry Weinberg in his 4-volume Quality Software Management series. We’ve found this model of cultural patterns useful as a tool to understand IT organizations and to know where and how we can intervene effectively. The model gives a fresh perspective on software organizations, with a people oriented instead of process oriented view. It focuses on subculture and people’s behaviour.

During the next few weeks, I’ll write a number of blog entries about these cultural patterns. If you want to read ahead, here are a few links about the patterns:

• Welcome to Humane Architecture by Nynke Fokma and Becky Winant
• Our Management Process Can’t Tell us How to Get From “Repeatable” to “Defined” by Nynke Fokma and Erwin van der Bij
• Shooting And Aiming Stances on the systemsthinking.net wiki
• Blog entry by Willem van den Ende relating the Cynefin model and the cultural patterns