Over the years the technique has proven useful, not only in coaching and workshops, but in all kinds of situations:

• As a sensemaking technique – to make sense of what’s happening in your team or at your client, so that you can take more appropriate, effective action. I started doing systems thinking together with a number of other agile coaches around 2002. It helped us a lot to understand better what agile is all about. We realized among other things that it’s not about my Scrum being better or worse than your XP, but it’s about doing whatever works for your situation.
• To illustrate the dynamics of software development, e.g. technical debt (pdf) or the dynamics of test driven development and pair programming.
• As a means to do root cause analysis in retrospectives – diagrams of effects facilitate team problem solving processes (similar to what David Draper is doing with current reality trees, another systems thinking technique).
• To support sales conversations – we are using diagrams of effects more and more in sales conversations, e.g. clarifying how a website delivers value for a nonprofit organization or explaining the effects of portfolio and risk management.

picture  by Willem van den Ende

Like Rachel, we create diagrams of effects collaboratively, in small groups, focusing on the conversations and not so much on the diagrams. It is also possible to use the diagrams in a more mechanistic way, creating a precise model of the system and even using it to run simulations. This approach is suitable for knowable problems and situations, not for complex systems like teams and organizations.

Some of the effects we have seen in practice, are:

• Diagrams of effects work as a medium that enables deep conversations – the conversations that change organizations. The value lies in the process of creating, refactoring, and discussing a diagram collaboratively, less in the resulting diagram (although the diagram can be useful as an illustration, to retell the story).
• They help surface your assumptions. For each causal relation you draw, ask yourself: what assumptions is this relation based on? How can you test these assumptions?
• They shift focus towards the whole system and make clear how everyone/everything plays together, instead of blaming individuals. You’ll see the underlying patterns of behaviour through the incidents. It can help you build your own situated theory of the team/organization dynamics.

So, will diagrams of effects solve all your problems? No, but they are a easy to learn, simple, useful technique that will enrich your repertoire of skills and techniques. As a result you will have more options to choose from and you’re better able to do what fits the situation.

Would you like to know more? Jerry Weinberg’s Quality Software Management: Systems Thinking and Peter Senge’s The Fifth Discipline are a good start. Alternatively, take a look at systemsthinking.net or contact me.

Story Testing at the Scandinavian Agile Conference
15-16 October in Helsinki (Finland)

As a developer, I want to know story testing frameworks and practice one of them, so that I can test stories using the examples the product owner writes together with me.

As a product owner, I want to understand the process of testing stories and practice specifying acceptance criteria using example scenarios, so that I can help developers develop what I want.

This workshop focuses on behaviour driven development and writing automated tests for user stories based on examples, with story testing frameworks like Cucumber or JBehave. Learn about specification by example, tools for testing user stories; practice hands on writing and improving story tests using example scenarios.

Mapping Product Line Value Streams at the
Practical Product Lines conference
20-21 October in Amsterdam (NL)

The goal of this workshop is to apply and evaluate value stream mapping as a tool for understanding business value creation in a complete software development organisation from requirements to delivery. We wil use value stream mapping to analyse existing processes or product lines brought in by the participants.

We use simple tools like index cards to prevent discussion about drawing techniques and enable all attendees to participate in creating a value stream map.

Would you like to register for the Practical Product Lines conference at a 10% discount? Just let me know.

User story mapping offers an alternative for traditional agile planning approaches like the Scrum product backlog. Instead of a simple list, stories are laid out as a two dimensional map. The map provides both a high level overview of the system under development and of the value it adds to the users (the horizontal axis), and a way to organize detailed stories into releases according to importance, priority, etc. (the vertical axis). The map shows how every user story fits in the full scope.

Releases are defined by creating horizontal slices of user stories, each slice is a release. For the first release, it is recommended to build a minimal set of user stories covering all user goals, so that you build a minimal but complete system to validate functionality and architecture early.

An important contribution of user story mapping is its focus on the users, the user goals, and the (system independent) activities/processes that the system should support. This helps the team to focus on why they are building the software – business value instead of feature details.

Story mapping was a natural fit for us, after working with agile for a number of  years. It augments existing agile processes and solves a number of issues we have run into.

We love small user stories

We came to prefer small user stories that can be build in 1 or 2 days. Small stories have all kinds of advantages. They are easier to estimate, you deliver working software in small steps (working software daily), you get early feedback and a continuous feeling of accomplishment. We don’t need to do extensive task planning, because making a quick list of tasks just in time on an index card is sufficient.

Limitations of traditional agile planning approaches

We ran into a number of limitations of small stories. It becomes increasingly more difficult to understand the system as a whole through all the small stories. Some of our customers had trouble seeing the forest for the trees. We were also lacking a way to manage coherent sets of dependent stories. These start off as epics or themes, but once they are split up, it becomes hard to keep track of what belongs to what and how far each theme or epic has been done.

Where to Go by Jeff Hitchcock (found with PhotoSuggest)

We also ran into limitations of linear backlogs for managing and prioritizing user stories. With lots of small stories and a lack of a good overview, it becomes difficult to prioritize. You spend more and more time on rearranging and grooming the backlog grooming instead of delivering working, valuable software.

Release planning – determining what value to deliver when – is a complex problem with multiple dimensions. Reducing it to a one dimensional backlog can work for simple systems, but for most systems we work with, essential details are lost.

The traditional way of agile planning supports working incrementally well. It provides little support for iterative development (i.e. building a story by fleshing it out in a number of deliveries). The risk here is that customers expect a user story to be finished in one go, while it might need two or more iterations to be fleshed out well. Lack of support of iterative development also bears the risk of customers requesting complete, bloated features (because it should be done right the first time), where simpler features would be sufficient (but the customer cannot know that until he/she sees the working features).

Last but not least, we frequently encounter developers and customers discussing primarily in terms of software features. Teams get lost in the solution details without a thorough understanding of the actual problems the software will solve. It’s not only developers failing to speak the customer’s language! We have also encountered quite a number of customers and users talking databases, fields, screens, and other solution aspects, instead of focusing on what they actually want to achieve.

User story mapping to the rescue

User story mapping focuses on the different users of the system and their goals. To achieve their goals, users perform activities, consisting of tasks. These are all user-centric – focusing on what the user does, not on features of the software. Features should support the user in doing tasks and achieving goals.

Instead of a linear backlog, you create a map of user’s activities and tasks. This defines the system at a high level and gives a complete overview of the scope. You put activities and tasks along the horizontal axis. The map then tells the story of the system: the activities describe the bird’s eye view of the scope, the tasks tell how a user actually uses the system. Note that activities and tasks are closely related to business processes. Adapt the terms to your own context.

You define releases based on tasks. You iteratively build the system, fleshing out the different tasks by defining detailed user stories (which are software oriented). The vertical axis of your story map indicates importance: higher up means more important.

You define releases by drawing horizontal slices and moving stories up and down between slices. For the first release you define a walking skeleton – the bare bones of the system, minimal functionality that covers all activities. You defer splitting out the task level stories into concrete, detailed stories until the last responsible moment.

Detailed stories will become ‘done’; activities en tasks are never ‘done’ as a story; you can always put in more features, make it better.

There are no hard and fast rules for story mapping. There are a number of useful guidelines, but no strict boundaries of what activities, tasks, and stories are. The boundary between user oriented stories and software oriented stories is not strictly defined either.

Benefits

We see a number of benefits of using user story mapping.

• It helps to keep an overview of the whole system and makes visible what value the system adds, what goals and activities or processes it supports, and how all the small, detailed user stories fit in.
• It facilitates release planning and enables a better conversation about defining delivering value early and managing risks; building a walking skeleton can give early feedback on market and technical risks.
• By introducing the users perspective and the rationale behind the system, User story mapping shifts the dialogue to the actual business value of the software. We only build features when the story map shows a rationale for that feature. A story map helps focus on the essential parts of the system. We have also used story maps to make teams aware that you don’t need to build all those features. A subset is often sufficient to help users achieve their goals.
• By focusing on user goals, activities, and tasks first instead of software features, you keep more options open regarding ways of realizing / supporting those goals and activities. Manual workarounds can be a good alternative more often than you’d think (btw I prefer to call it sapient processes rather than manual workarounds)
• It augments rather than replaces practices from e.g. Scrum or eXtreme Programming and it plays well with other techniques like Dimensional Planning and agile business process analysis.

Would you like to know more? Jeff Patton has written very good material on story mapping. There’s however only so much theory, the beef is in doing it. We have developed a simulation called The New New NEW! Product Development Game to experience release planning with user story mapping. We can also guide you in applying user story mapping in your projects. Let me know if you’re interested.

This year we’re organizing it together with Agile Holland. Many thanks to VxCompany for helping us out by providing the venue. 

The conference theme is:

My method is bigger than yours…or is it?

A development method is nothing more than a named cloud – a set of practices, principles and values labeled under a common name. Whether you realize it or not, each of these named clouds have fuzzy boundaries and overlaps with others. Examples of named clouds you may be familiar with include Scrum, eXtreme Programming, Test Driven Development, DSDM, Lean, Refactoring and more recently Kanban, User Story Mapping, and Dimensional Planning.

Word cloud of agile, Scrum, Kanban & XP created with Wordle.

This year, we have space for 80 participants. See the announcements on the Agile Open website (English) and the Agile Holland website (Dutch) for details.

Hope to see you in Baarn in September!

I promise to get back to blogging and check pending comments more often…

He describes moving to agile development from the perspective of complex adaptive systems and game theory. Agility is about moving software projects from either an ordered context or a chaotic context into a complex context. Depending on where an organization comes from, different approaches work, e.g. Scrum with or without the XP practices. 

This fits with my experiences. I see more and more agilists focusing on agile according to The Book or saying it’s not agile if you don’t do the sacred practices. It is also fits with the stuff Willem and I have been writing and presenting about Jerry Weinberg’s Cultural Patterns and Dave Snowden’s Cynefin sensemaking framework applied to software projects.

Cynefin sensemaking framework

Cynefin describes four domains, as shown in the picture below (the green arrows represent the agile choreographies Jurgen describes): the simple and complicated domains are ordered, you can see cause and effect relations there, either simple or complicated but still knowable. This means you can devise methods and processes that will lead to desired outcomes in a bounded, predictable way. 

In the chaotic domain, no cause effect relationships are perceivable. The main decision model is to act quickly, trying to reduce turbulence, sense immediately what the effects of your action are, and respond accordingly. You can try to impose order on the system or reduce turbulence a bit to move towards the complex domain.

In the complex domain, the relationship between cause and effect can only be seen in retrospect. Looking back, causes and effects seem clear and we can explain what has happened and why – retrospective coherence. If you wouldn’t know better, it looks just like you’re in an ordered domain. But only in retrospect! You can’t reliably predict what will happen and you can’t reliably determine up front which actions will lead to desired results.

Retrospective coherence trap

I see a lot of agilists stepping into the retrospective coherence trap. They assume, consciously or unconsciously, an ordered context, where following a process like Scrum, XP, Scrum+XP, … will lead predictably to desired results (i.e. working software & a happy customer or product owner).

Most software projects that make a difference exist however in a complex context, because of a complex organizational context, ever changing customer requirements, complex problem domain, and/or unknown and rapidly evolving technology (I’ve written about this before in the context of Weinberg’s cultural patterns). 

If you’re not aware that your project is in the complex domain, you might be tempted to reuse the processes of a successful project for a next project, assuming it will be similar. It might work by accident, but more often than not the context changes, not radically but enough for the process not to work as expected. This can explain why many Scrum and agile transitions still fail.

The retrospective coherence trap is also a risk when doing retrospectives if you focus mainly on what worked well without realizing things only make perfect sense when you’re looking back. In the complex domain, success stories are only retrospectively coherent and not a recipe for success.

Summarizing

• Software projects that make a difference usually are in the complex domain.
• Managing in the complex domain requires an approach that is fundamentally different from managing in an ordered domain. In the latter, it makes sense to follow known methods leading to project success or steer a project based on your (mental) models of how the project works (e.g. doing agile by the book). In complex space, you need to do numerous small interventions, experiments, short iterations, do small probes, sense what happens, and respond accordingly – the agile principle of apply, inspect, adapt.
• We often assume that we’re in ordered context, even if we’re in a complex domain. 
• We step into the retrospective coherence trap: what happens makes sense only retrospectively, making it look like we’re in an ordered context. The context will however change in unknowable ways, so doing what worked yesterday will probably not give the expected results – although when you look back, it all makes perfect sense…

I do not really care whether you should do Scrum by the book, do all the 15.7 practices, get a high rating on an agile checklist, or even whether what I do is  officially ‘Agile’. I strive to find what works for myself, for my teams, and for the organizations I work with. It all depends (the main lesson I learned at university), or context is king as Jurgen states it, and anything goes (from Against Method, required reading for agilists, but that’s another blog entry) I feel a theme emerging for the next Agile Open Europe conference ;-)

A few recommended links

• Cognitive Edge website with more information about Cynefin and related work
• Systems thinking
• Jurgen Appelo’s blog about understanding development and management

Responsibility Driven Design with Mock Objects

Object oriented design is the art of assigning the right responsibilities to the right objects and arriving at a clean, loosely coupled, and highly cohesive design. Test Driven Development (TDD) will guide you in that direction, but not far enough. TDD helps to get loosely coupled objects, because coupling hinders the test-code-refactor rhythm.

Responsibility Driven Design is an approach that goes a step further. It shifts the focus from state to interactions and responsibilities. It helps to get a highly cohesive, loosely coupled object oriented design – an approach facilitated by test driven development with mock objects.

In this workshop, you will learn how to use the CRC card technique (Class-Responsibility-Collaborator) and mock objects to evolve a design in small steps, by looking at classes and their responsibilities, relations and interactions.

After a bit of presentation and a live programming demo, we’ll run a short coders dojo so you can experience it yourself. The dojo makes this session also interesting if you are already an RDD master.

Interested in this topic, but unable to participate? We offer courses on OO Design, Test Driven Development, Mock objects, and Refactoring in different forms, from short workshops to 2-day courses. Feel free to contact us.

• Finishing off a closed list with today’s work feels good. It helps me to get relevant stuff done in time.
• I liked setting my email and paper backlogs explicitly aside as official ‘backlogs’ and planning to process them (little and often). It took away the pressure I felt from the big stacks of papers on my desk and the long list of emails in my inbox.
• I often have emails and other stuff that I would like to do something with, but initially I don’t know exactly what. As a result, I postpone them, think frequently about them, and, after some time, force myself to do something. Then I often find that the actual action is quite easy to do – realizing I’ve wasted time and energy on worrying about it Focusing on doing stuff tomorrow helps me to find the smallest, simplest next action that could possibly work. The trick is to think of the first tiny step I could do to get started or make progress and make it a recurring tasks for the next day(s).
• I’m good at procrastinating on answering ‘difficult’ emails. I have found out that it becomes much easier if I split the work into first writing a rough draft version (where anything is good enough for a draft) and finishing it the next day. I feel much less resistance to taking a minute to ‘just jot down my ideas’ than to writing the ‘difficult’ email. The weird thing is that the ‘rough draft’ usually turns out to be a nearly finished email… 
• Sometimes, I prefer to go with the tasks I feel energy for. Last week for instance, I planned to spend two or three pomodori on end-of-year bookkeeping tasks, regarding it as a recurring task. After I started however, I continued doing it – I felt motivated to finish most of the (boring) work. This helped me to get a feeling of accomplishment and leaves me more time for enjoyable stuff in the rest of the month.
• I also prefer to have time for unplanned things each day, things that come up and/or things I feel energy for. A Will Do list can help to get both the important things done and leave time for unplanned stuff. 
• I’m not sure doing tasks tomorrow works for all my stuff: some tasks that I don’t do rightaway (i.e. today or tomorrow), end up being invalidated or changed after some time by new information. If I would have done the task right away, it would have been suboptimal, like buying at a price that’s too high or bugging people about issues that would otherwise get resolved automatically. Real options thinking would be useful here.  
• I’m keeping the Will-Do lists for planning and doing tasks at home. There it helps me to get all kinds of not-yet-urgent tasks done that I would normally postpone forever. By making a small, feasible list every day, I get things done, without stress.
• For my business, I’m also looking at the new Autofocus system Mark Forster is currently beta testing; I like the name and it looks interesting. Furthermore, I’m working on my “project management” system, to keep track of all my different projects and to make sure that imporant stuff happens. 

So there’s no easy solution for time management. Do It Tomorrow contains a lot of useful ideas, but in the end you’ll have to keep on reflecting what works and what doesn’t work for you, and grow your own system.

1. have a clear vision
2. one thing at a time
3. little and often
4. define your limits
5. closed lists
6. reducing random factors
7. commitment vs interest

Have a clear vision

Have a clear vision of your goals, of the things you want to do and the things you don’t want to do. A clear vision directs your priorities. Setting priorities is only meaningful between projects, not between tasks that have to be done anyway (’project’ is loosely defined here as an activity that leads to some desired result and that cannot be finished in one go).

Your vision is not something static: it will change over time. So frequently revisit your vision, to keep your priorities clear as well.

One thing at a time

Focus, focus, focus! Use for example timeboxing or working with a pair (like pairprogramming) to work in highly focused way. Don’t dilute your focus by having too many projects at the same time.

Little and often

Work on things frequently, in small bits, iteratively and increment, so that results grow over time. If you want e.g. to write a book or finish a Ph.D. thesis, work every day on it. Actually doing something and keep doing it is more important than the amount of time spent.

This works for writing, uncluttering your home or office, bookkeeping, and many other larger activities.

Define your limits

Creative thinking works better within clear boundaries. An example of limits is timeboxing your activities, e.g. using the pomodori technique.

Defining limits is also important for your projects: determine the boundaries (and frequently re-determine them) to get a clear focus of what you’re doing and what you aren’t doing, instead of being busy with a cloud of all kinds of vaguely interesting and possibly relevant stuff.

This week, I’ve started to make a map of all the projects that I currently have and that I want to take on this year. Being an independent consultant, I don’t have an organisational context that sets a lot of boundaries for me so I’ll have to set them myself in order to be effective.

Closed lists

A closed list is a list that has a line under it and that will not change. For every day, you make a Will Do list, a closed list with the stuff that came in the previous day and your recurring tasks. As the list is closed, it will only shrink when you’re finishing items from the list. This will give you a feeling of accomplishment at the end of each day, when all the Will Do items have been checked. 

Anything that comes in during the day and that is not a real urgency, will be put on tomorrow’s list or below the line of today’s list. You’ll first finish all the items above the line, before doing the newly added things.

This approach enables you to plan most of the work you do, so you can work much less reactively and much less governed by self-inflicted urgencies. Your day to day planning will become more predictable and you’ll get early feedback when you’re structurally overloaded.

The Will Do list is limited by your daily processing capacity (so you will need to find out what it is), so you prevent backlogs from building up. If you get more work each day than you can handle the next day, you’ll have to either cut down on your commitments, make your systems more efficient, and/or allocate more time for the stuff on your lists.

Willem asked, what do you do when the telephone rings? It depends: you can answer the call, make a note, and take action tomorrow (unless, of course, it’s about your house being on fire). You can also decide that you won’t answer the phone during certain activities, listen voicemail later on, and get back to the callers the next day. It depends on the nature of your work and your preferences.

Another advantage of closed lists is that you don’t have to prioritise between the items. They all need to be done and if the list is limited by your daily processing capacity, it will be finished. Prioritizing doesn’t make sense for stuff that needs to be done anyway.

Working this way gives peace of mind and reduces waste: you don’t have to spend your energy making difficult decisions about priorities. Prioritizing is waste: it’s work that adds no value, but just increases the pressure on you! You’ll have more time and energy left for actually doing useful stuff.

Forster’s recommendation is to start with the least urgent things first. If work has to be done anyway, why not do it right away?

A bright, grand idea like writing a book is not something you can finish the next day. This becomes a project, a task that recurs (a little attention every day) until the work is finished.

Reducing random factors

By preventing most ‘urgencies’, you will reduce a lot of (self-inflicted) variability in your day to day work. Closed lists system make the underlying systems problems visible. You can’t eliminate all variability and randomness, but you can reduce them substantially, giving you more freedom, making sure your important things get done, and enabling you to handle the remaining randomness better.

Commitment vs interest

You can be interested in a lot of things, but you can have only a limited amount of commitments. It is important to know your commitments, as these provide a framework for your decisions. It’s like the pigs and chickens metaphor used in Scrum (chickens are only involved, but pigs are committed). A pig only has limited ham and bacon it can provide… (the pigs and chickens metaphor has its limitations, but that’s another story)

I’ll give my interpretation of what Do It Tomorrow is about, based on what I’ve read and tried so far. Forster nicely summarizes it with the Mañana Principle:

The art of getting everything done by putting it off to tomorrow.

The motto is nothing is so urgent that it can’t be put off till tomorrow. It’s however not about letting things wait until they disappear by themselves! It is about actually doing the stuff that you need and want to do, earlier, with less stress and reduced cycle time (= the time between the arrival or conception of the task/email/thing and doing it).

You get rid of your backlogs (emails, tasks, papers, mail) and set up a system for yourself to keep backlogs away. Backlogs are ‘evil’: they are symptoms of problems in the underlying systems. With systems I mean the systems you use for managing your time and energy, for processing emails and other stuff, the way you manage all your other work and projects. Backlogs tend to hide problems. Stuff waits in your backlogs until it cannot be put off anymore, which makes it urgent without a good reason. In this way, you are driven by the urgencies of the minute, while other stuff in your backlog that does not make enough noise just starves.

Forster states that there are only a few real urgencies that really require an immediate response from you, like your house or office being on fire, your kid being ill at school, or your pc breaking down so that you can’t do your work. Most of our ‘urgencies’ are either misinterpreted as urgent or self-inflicted (caused by letting it wait until the last irresponsible moment). Most of the stuff that comes in can wait until tomorrow.

So the idea of Do It Tomorrow is to set up a system for yourself, in which you handle all of the stuff that comes in today, tomorrow. Of course, some days you get more stuff than you can handle on the next day, but that should even out over time. If not, you have a systems problem that cannot be solved in a sustainable way by just working a bit longer or putting in a bit more effort. The problem is caused by:

• working inefficiently
• having too many commitments
• having too little time for the work

According to Forster, these are the only three possible causes of time problems. Prioritizing and creating to do lists won’t help you in tackling these. 

I see a lot of parallels with lean: you establish flow of all the stuff you do, reduce variability in your work. It focuses on reducing muda (waste, inefficient way of working), mura (unevenness of work, caused by procrastination and waiting until nonurgent things become urgent), and muri (overburdening yourself with too many commitments).

In the next blog entry, I’ll give an overview of the 7 principles of Do It Tomorrow and say something about my experiences. I appreciate Willem for the evening conversations leading to these blog entries.