Agile Pain Relief Consulting

Scrum and Agile Training and Resources

Mark Levison

Agile Bonuses – The Damage They Do

by Leave a Comment

A design pattern is a description of a solution to a recurring problem. It outlines the elements that are necessary to solve the challenge without prompting the reader to address the issue in a specific way.
Unfortunately, we also regularly see recurring patterns of ineffective behaviour. These are called Anti-Patterns.
The following explores the common anti-pattern of a performance bonus. Micromanagement.

A few months ago a friend asked me about setting up a bonus scheme at his work. From working with Scrum and Kanban teams, I know that bonuses are counter-productive anywhere team members are doing knowledge work and their work is interdependent. The same might apply in other areas too, but I know this to be true in the context of software development in particular, because of my experience and study.

What kind of bonuses are we talking about here? Not Christmas bonuses, profit sharing, employee stock ownership plans or stock options (although, if not used carefully, even those have risks). We will focus on only performance bonuses for this article, which is what my friend was discussing.

“A performance-based bonus is an extra compensation granted to an employee as a reward for reaching pre-established goals and benchmarks.”[1]

In software development, individual performance bonuses are often geared to something like personal velocity if you’re a developer, or number of defects found if you’re a tester. For team bonus, maybe the target would be set as “greater than 75% of committed stories are completed in most Sprints”. We’ll use the developer example in our illustrations, but others would have similar effects.

After a short conversation with my friend, we managed to identify a number of ways that bonuses would be a bad idea and I offered to assemble research for him as ammunition to convince his boss. And what I found in the research was curious. It showed that researchers have a very narrow focus when studying bonuses. With a couple of exceptions, they assumed that the people designing the incentive had chosen a good target.

Of course, from our experience in the agile world, we know that most targets are either outright dangerous (e.g. velocity as a measure of throughput) or far enough from the truth that focusing on them may cause harm (e.g. defects as a proxy for quality). Researchers don’t tend to look very far forward when they assume that bonuses will have a positive effect and they end up measuring the wrong things. If they instead ignore any assumed target, and follow cause and effect to the unbiased end results, they would see that.

Other challenges with common research:

  • Many of the tasks being measured are too simple. For example, in online game play, the task is designed to measure collaboration and competition. Unfortunately, the limited complexity of game play doesn’t simulate product development work well.
  • Many studies are run over a short period of time – hours or days – so would not show the long-term effects of most bonus schemes.
  • Since the research is typically based on short time periods, it rarely examines the effects of bonuses on morale or intrinsic motivation. Adding more effective motivation tools to this article might double its length.

To see what goes wrong with bonuses, we’re going to draw on a few tools.

  • Goodhart’s Law – “When a measure becomes a target, it ceases to be a good measure”
  • Motivational Models – specifically ARC
  • Systems Thinking – many of the problems created by bonuses come from a lack of consideration to knock effects. I.E. If we do X, what other effects will that have downstream? What effects will it have in a few months?

And to visually illustrate cause and effect, we’ll use Causal Loop diagrams to show positive and negative links. Remember, “positive” and “negative” shouldn’t be read as good or bad, but simply as increasing or decreasing.

Individual Bonuses

Here’s a hint right out of the gate if you’re new to Agile: you never want to measure anything for one person.

Individual Bonus (-ve)-> Autonomy – Chasing a bonus lessens the opportunities we have to make decisions to better the team because meeting the bonus requirements becomes the higher priority. This reduces (i.e. negative link) a team member’s Autonomy (see ARC and SCARF models).

Autonomy (delayed ->) Engagement – When we have Autonomy —control over our own choices— we have a greater sense of Engagement. But the bonus has reduced Autonomy, causing an opposite effect. It turns out, Autonomy is a core driver for Engagement at work.

Individual Bonus -> Competition – An individual bonus also increases (i.e. positive link) Competition between team members. It automatically means you should put your own needs ahead of the team’s needs, to maximize your bonus payout. Since others are doing the same thing, your cooperative Scrum team starts to have competition between members.

Competition is a fun one because it sparks a lot of effects of its own…

Competition (-ve)-> Peer to Peer Help – You will still help others, but only after your own work is done and mostly on a quid-pro-quo basis. This isn’t healthy collaboration and reduces the amount of Peer to Peer Help that happens.

Competition (Delayed -ve)-> Quality – As Competition increases and communication declines, more errors creep in. There is reduced cooperation between individual developers and, much worse, reduced cooperation between developers and testers. Have you ever been in a bug triage meeting where one party claims they’ve found a bug and another person claims it works as designed? This happens enough even without incentives getting in the way, and bonuses just make this much worse. Reduced Quality is a delayed effect; it will take a lot longer to show up.

Competition (Delayed -ve)-> Relatedness – We have a need to care about, and feel cared for by, others. As the feeling of Competition increases even a little bit, that harms our ability to Relate to others.

Relatedness -> Peer to Peer Help – As we get to know our teammates better and learn to care about them, our willingness to provide support increases. Unfortunately, Competition harms Relatedness and, therefore, further undermines the Peer to Peer collaboration required for Agile to be effective.

Relatedness (Delayed)-> Engagement – When we feel we’re part of something bigger than ourselves, our Engagement goes up and then we become more productive. The competitive environment which harms Relatedness, also creates dis-engagement.[2]

Autonomy -> Peer to Peer Help – When we have freedom to make choices, it is easier to collaborate because you don’t feel as much pressure inside yourself to get your own work done first. Instead, you can help a peer and expect the help will be repaid.

Individual Bonus (-ve)-> Experimentation – Experimentation allows a team to learn and improve through learning what succeeds and what fails. As people focus on meeting their bonus, and avoiding failure, Experiments are reduced and so is team learning and improvement.

Individual Bonus -> # of Features – Yay, a net good; we got more Features! But at what cost? 😉

# of Features (-ve) -> Done – As the number of Features being worked on at any one time increases, there will be pressure to call them ‘Done’ sooner, even though they’re likely not done. The best part is, with a good bonus, this pressure doesn’t need to come from outside – the individual can create it for themselves.

# of Features (-ve) -> Quality – The more Features being worked on at the same time, puts increased pressure on the people doing the testing. This will have two effects: items will wait in a queue longer making their defects harder to fix, and more defects will slip through unnoticed. Reduced Quality will be the result.

# of Features (-ve) -> Peer to Peer Help – As the number of Features being worked on at any one time increases, there are fewer people who have capacity to help their peers.

Peer to Peer Help -> Throughput at the Bottleneck – We know from the Theory of Constraints that all systems have bottlenecks that limit its throughput. Bottlenecks are inevitable in a normal Agile environment, but when a bottleneck happens, we would expect another team member to help out until it is no longer the constraint for Throughput. If a bonus makes Peer to Peer Help less likely, Throughput at the Bottleneck will not improve.

Throughput at the Bottleneck -> Number of Features – If we improve the bottleneck, the number of features delivered will increase. Unfortunately, the use of bonuses means that there’s no incentive to help clear the constraint, so the bottleneck is likely to persist.

Peer to Peer Help -> Quality – Collaboration increases Quality through Pair Programming, Code Reviews, and even just having someone to bounce ideas off of. All this requires team members who feel that they have enough slack in their own work that they can help you. Bonuses ratchet up the pressure to meet a target and therefore reduce the amount of time anyone feels they have to provide help to others.

Done -> Quality – Scrum includes a Definition of Done as a quality leveler. Scrum challenges the team to ensure that level of quality is met whenever a Product Backlog Item is complete. As a team grows in skill, it normally strengthens its ‘Done’ criteria, therefore improving Quality. Unfortunately, in the land of bonuses and focusing on the # of Features, team members will feel pressure to skip parts of Done, as they feel it slows them down. Hint: over the long haul you never speed up by reducing quality standards.

Quality (Delayed) -> # of Features – When a team makes the effort to do quality work, through the Agile Engineering Practices, they increase the code quality. It is easier and faster to add new features in a code base where team members tidy up small messes and automate many of their acceptance tests. Unfortunately, in an environment where # of Features is consider a measure of success, corners get cut and quality degrades slowly.

Bonuses anti pattern CLD

Reviewing these results, we can see that bonuses might result in an increased number of features, but to stop analysis there is short-sighted and foolish. An increase in features doesn’t automatically lead to greater customer happiness. We should be focusing on Outcomes (features with good quality, that solved the customer problem) over Output (# of Features delivered). We may have got more of the target, but if the target was poorly chosen in the first place, the focus on it didn’t give us more of what we wanted.

Individual Bonuses can work in situations where we want competition. On a sports team, for example, a bonus for home runs might be helpful. Even here we have to be careful, because home runs alone don’t win baseball games. Bonuses may also help if there is a certain class of work that no one wants to do, for example update and maintain the build system. Again, this isn’t without consequences because, having provided a bonus for a mildly unpleasant task, all of sudden people will want bonuses for any task that is out of the ordinary.

In the context of Scrum and Agile teams we know competition is actively harmful and we want people to volunteer for the unpleasant tasks, so staying away from bonuses continues to be a good idea.

Team Bonuses

When we move bonuses to the Team level from the individual level, we reduce the competition within the team (Yay?), but now create competition between teams. The causal loop diagram won’t change very much. The effect of competition on things like cross-skilling will be reduced, but new elements will appear that affect cross-team dependencies and improvements.[3]

Thank You or Spot Bonuses

These are impromptu bonuses awarded to thank a team member for a particular success, such as delighting a challenging client or solving a really deep problem. These at least avoid the damage of encouraging changed behaviour to achieve the bonus, and they can have a positive impact on the team. But they still come with their own challenges.

  • How to decide the recipient(s)? There is a risk that the award will acknowledge an obvious team member but overlook the contribution of someone else who provided invaluable ideas and advice but not hands-on work.
  • Don’t reward overtime – this always backfires. Not everyone is able to do overtime (e.g. single parent or a team member going to school part time) so it creates unfair advantage. Overtime also leads to a variety of negative effects in the long run, both for the individual and the organization, so rewarding it sustains the harm.[4]

Fairness

Our causal loop diagram was already way too crowded, but fairness is another challenge. For many people, fairness is an important factor in motivation. (In the language of the SCARF Model[5] this is a threat and reward). If members of the team feel that a bonus is unfairly awarded to one person for doing something (perhaps maintaining the build system) and the contributions of others are missed, they will feel unfairly treated. This will become a demotivator. I can’t readily imagine a case where the use of bonuses was a fairness motivator.

Conclusion

Bonuses are a bit like telling someone, “We value you enough to pay you X, however we will only pay you the last 10% of X if you jump through the following hoops.” Or, to use a different analogy, like large quantities of sugar where they taste great when we eat it, but the long-term effects are harmful. Bonuses attempt to improve performance by supplementing a person’s internal motivation with the allure of a sweet reward.

Instead of fighting against people’s internal motivation, find a way to align a team member’s own motivations with the needs of the team. When our team is working in an Agile way, this happens organically as part of the way we structure work.

The worst thing about bonuses is they seem so compelling in the short term but, when we add up all the delayed effects, it becomes clear just how insanely expensive bonuses are. So don’t waste money on a bonus scheme. They don’t improve performance. They destroy intrinsic motivation. Pay your people well and then give them the tools to find their own sense of motivation, and the performance results will follow.

Not yet convinced? You will be after you read this research that wasn’t conducted with an assumed target.

Research That Doesn’t Suck

Why Motivating People Doesn’t Work and What Does – Susan Fowler
Intrinsic motivation—motivation that comes from inside us— is more sustainable than extrinsic or external motivation. The famous carrot or stick.

Self-Determination Theory in Work Organizations: The State of a Science – Deci, Olafsen, Ryan
When our work environment has bonuses, the bonuses reduce the effect of the intrinsic motivation.

The first important finding in the meta-analysis by Cerasoli et al. (2014) showed that intrinsic motivation had a moderate to strong relation to performance across all studies and all types of performance, whether or not incentives were also being used. Accordingly, this indicates that intrinsic motivation is extremely important for the workplace. Furthermore, in line with the undermining effect of rewards on intrinsic motivation, Cerasoli et al. found that intrinsic motivation had a weaker effect on performance when incentives were directly salient, and a stronger relation to performance when the incentives were not directly salient.

The same paper shows that rewards help when the work is simple, but when the work is more complex (e.g. knowledge work) they don’t help at all and just interfere with intrinsic motivation.

In more nuanced analyses, Cerasoli et al. (2014) found that intrinsic motivation was a stronger predictor of performance quality, whereas extrinsic incentives were a stronger predictor of performance quantity. In a similar vein, Weibel et al. (2010) in their meta-analysis found that extrinsic incentives led to better performance on simple tasks but to poorer performance on more complex tasks. In short, PFP appears to be effective for motivating performance as quantity of simple, algorithmic tasks, but not performance as quality of complex heuristic tasks. Furthermore, PFP seems to interfere with the relation of intrinsic motivation to high-quality performance.

Do incentive systems spur work motivations of inventors in high-tech firms  – Lazaric and Raybaut

Performance bonuses inspire competitiveness and resource guarding instead of teamwork and collaboration. The exact opposite of how we know Scrum — and productivity — thrives.

In a study of what happened to patent applications at Thales, a change in the bonus system:  ”… harms relatedness and autonomy, reduces the creative output and knowledge diversity in an organization. In addition, changes to an existing bonus system may harm the culture it created: “new incentive system can conflict with intrinsic work motivation, notably because of the corporate imprinting. Instead of spurring work motivation, it may produce crowding out effects with a decline in motivations of some groups.”

Want to Get the Most Bang for Your Buck? Try Impact Mapping

by 2 Comments

Work is humming along nicely when your Marketing VP stops by your desk and says, “We need to double monthly sales.”

That’s it. That their entire instruction.

After you recover from this bombshell,  you will need to decide what tool you will use and who you will grab to help you start to tackle this problem.

Do you use Story Maps? (A great tool when you have an idea of where you’re going.)
A Journey Map? (Also requires that you have an idea of where you’re going.)
A Traditional Roadmap?

I’m going to suggest that you create Impact Maps in situations like this, where you have a vague goal but haven’t yet done a good exploration of how to get there.

Impact Mapping is a strategic tool to help teams and businesses focus their work on a product feature that will give the most bang for their buck. I’ve used Impact Maps for teams seeking to figure out what/where to build next, and I’ve also used them outside of software development.

Many organizations struggle trying to use Traditional Roadmaps in an Agile environment. They don’t work because there is an implied linear flow and promised timeline for delivery. Linear plans of this nature aren’t suitable in our current world, where change can happen at a rapid pace and history often isn’t a predictable indicator (e.g. COVID, or when once-in-a-hundred-year floods occurs twice in three years). Impact Maps replace Roadmaps, making it clear that we have choices in deliverables, while still achieving the original goal.

As humans, we tend to jump from a goal to the actions that we think will help us achieve it. Impacting Mapping works by forcing us to consider who is involved in achieving our goal and how they can help us. Using this approach, we find a broader set of possible options –including some that we might not have considered– before narrowing down on which option is most likely to help achieve our goal.

Impact Mapping is a tool to help you figure out where you will likely get the most bang for your buck. You map out some options to achieve a goal. Once you’ve got a rough idea, you stop mapping and execute.

How to Use Impact Maps

Identify the Goal first, then Who (Actor), How (Impact), and What (Deliverable).[1]

Goal

Why are we doing this? What are we attempting to achieve?
The map will be focused on finding options or ways of accomplishing that objective.

Actor/Who

Who are the consumers or users of our product? Who can produce the desired effect toward our Goal? Who can obstruct it?
In describing the Who or the “Actor”, the order of preferred usage is: specific individual, persona, role, job title, group/department – from most specific to least specific. Consider users of the system first, stakeholders second. By focusing on specific people or personas, it makes it more likely the deliverable will meet a real person’s need.

Impact/How

What Impact can the Actor have? How can they help achieve our Goal? How can they obstruct progress towards the Goal?
These are the Impacts that we’re trying to create or prevent. Don’t list everything an Actor might do – just the key things. List business activities, not process features. Be specific and consider negative outcomes as well as positive. When used in an Agile environment, the Impact/How can be tied back to the value statement in a User Story. Check to ensure that both are aligned.

Deliverable/What

What can we do as a team to deliver the Goal?
These become the product features. These Deliverables are the least important end of the spectrum and, on the first pass, will be unrefined. Much of software development is done when someone powerful wants a feature because they think it’s a good idea, but they haven’t really proven the idea will deliver value. By placing Deliverables after the Impact, wasted resources are avoided. So evolution here is a good thing. We don’t need to find all the Deliverables, just enough to find a deliverable that will, for lower cost, deliver the highest Impact. If an early attempt at delivering value fails, we can always return to the map with new information and try again.

Example of an Impact Map:

Impact Mapping example Figure 2 – Creative Commons 4.0 as per: https://www.impactmapping.org/drawing.html. Credit: http://impactmapping.org/

Tips for Drawing an Impact Map

Impact Maps are intended to be drawn and not just written in words. Visualizing them helps make conversations more concrete. It makes it easier to understand the relationship between a deliverable and the Actor who it is for. If you’re not yet familiar with them, creating an Impact Map from scratch can be intimidating, but it doesn’t have to be. This simple process helps move you through it, and often works well when done over two separate sessions by a collaborative group that includes Product Owner, development team, and stakeholders:

Session One: define expected business Goals and measurements
Session Two: build the map

Session One: This is the preparation session.

1. Identify the core business objective. Do this by listing possible business Goals, and discussing which one is the current priority.
Goals often tie back to money (how to save money, make money, or protect money). However,  Goals may tie back to other tangible benefits such as number of readers for a book; improvement in the team’s morale, etc. All of these are measurable in a meaningful way and, if achieved, would make a positive difference in their context. To find real Goals, ask: Why is this feature important? Or: How would it be useful? Keep iterating until you find the underlying Goal. [2]

Using the fictional company of our Scrum by Example team, our Impact Map example Goal is: Increase Monthly Book Sales and Profitability for the World Smallest Online Bookstore

2. Define good measurements.[3]

    • what we’ll measure (aka the ‘scale’)
    • how we’ll measure it (aka the ‘meter’)
    • what the current situation is (aka the ‘benchmark’)
    • the minimum acceptable value (e.g. the break-even point)
    • for investment (aka the ‘constraint’)
    • and the desired value (aka the ‘target’)
Our example:
●    Measure – Monthly Sales, Monthly Profit
●    Current Situation – $100,000, Monthly Profit $20,000
●    Minimum Acceptable Value – $125,000, Monthly Profit $30,000
●    Investment: $200,000 (e.g. one month of expenses for one team)
●    Desired Value: $200,000, Monthly Profit $70,000

3. Plan the first milestone. Effectively, you’re picking a Goal/target combination.

Our example: One month after the team complete this work they have increased sales to $150,000/month and profit to $40,000/month. Since the investment is also one month of working time, we’re looking for payoff in two months’ time.

Session Two: This is the mapping session.

1. Draw the map skeleton. Start with one Actor (Who), Impact (How), and Deliverable (What). Use tools that promotes collaboration – whiteboard, sticky notes, etc.

    • Put the milestone (Goal/target combination) in the center.
    • Find the Actors who could have a positive or negative effect on the milestone.
    • If people already have ideas or features, put them around the outside. Then ask some questions: Does the feature contribute towards the Goal? Is the Impact valid for the Actor it is ascribed to? Gojko offers a user story-like technique for testing: “<someone> can help us achieve our <objective> by doing <something differently>”
Our example: For the “Actor” we decide to focus on current book-buying customers for our milestone:
impact mapping example

Then we ask ourselves: What “Impact” can the Actor have? Or: How can the Actor help us?

In our example, the answer might be “buy more books” or “refer friends”.

What Deliverable could we build to achieve this?

impact mapping example

2. Find alternatives. What else could these people do for us? Who else can help? Who can obstruct?
If you find that you’re struggling, try Hohmann Innovation Games[4] or Gray Gamestorming[5]. Also try asking which of the following factors to consider: New Business, Upsell, Retainment, Operational Efficiency[6]

impact mapping example

3. Identify key priorities. Look for obstructions. What can stop us before we start? Is there high-value, low-hanging fruit? What key assumptions should be tested?
These questions should all be focused on business activities, impacts, and outcomes, not software features.

In our example, questions we might ask: Are we currently getting bad reviews or negative comments through support? If so, is there a common theme? Do these issues harm our relationships with our current customer. (Retainment)

Assuming our customer relationships are in a good place, will we have happier customers if we sell them more books? (Upsell) Or will we have happier customers if we offer them a referral bonus for referring friends? (New Business)

Alternative: Prioritization. Select which Actor to satisfy first.
Consider modified dot voting (red dots for critical features, green dots for low hanging fruit). Given the limited number of options, you can use a Kano model[7] to categorize more effectively.  The Actor with the fewest red dots gets prioritized.

4. Earn or learn. Set a budget to learn whether or not a Deliverable will have the expected Impact. Use simple tools (often not software) to test the value or assumption. The sample questions will be very similar to those used in Lean Startup.

    • What is the simplest way to support this work?
    • What alternatives are there?
    • What is the simplest way to test this assumption?
    • Could we start earning money with a manual process?

If you’re struggling to find testable slices, try story mapping or Gojko’s Hamburger method.[8]

Our example: Can we test any of these ideas without implementing a feature in software? For example, can we offer hassle free returns simply by adding a note on the site or in every order saying if you need to do a return call support? This is an experiment – if no one uses the offering maybe it’s not valued.

“Offer similar books” requires a lot of work as we require some algorithm to guess which books are similar. Is there a simple version of this feature that we could try to start? Maybe we could look at our ten bestsellers, manually find books that are close to them, then when a top seller is purchased, offer the companion book. This experiment still requires software development work, however the cost has been reduced from several Sprints of work for one team, to a version that could be built in one Sprint.

Fit the Measures to the Map

Risks

The map becomes the target and we risk missing things that matter outside the picture.

Typical mistakes

  • Too many people involved in creating the map – too large a group will reduce the diversity of the opinions and ideas shared, as the quieter people are drowned out
  • Wrong people (e.g. no decision makers, no technical people)
  • Projector or one person doing the drawing. Impact Mapping is a collaborative technique. If you meet in person, consider using sticky notes or index cards and move items around on a table. In a virtual context, consider an electronic whiteboarding tool that invites contributions from everyone versus a tool that only one person can update.
  • Early criticism (we want more ideas while still in the divergent phase[9] , not an attempt to prune)
  • Leaders with outsized influence (leaders should always speak last)
  • Reverse engineering of the entire shopping list. Your existing list of preferred Deliverables may not map well to Actors/Impacts.
  • Drilling down to detail too early
  • Skipping levels in the Goal, Actor, Impact, Deliverable layout
  • Building a map without metrics to guide

Impact Mapping and User Stories

Many Agile Teams use User Stories as dumping ground for ideas, such as when someone has a technical need or a pet feature that they would like to see delivered. They express the idea in the format “As a <role>, I want <to do> so that <value>” and dump it the Product Backlog. Impact Maps can be used to keep these User Stories honest. If the User Story doesn’t map to a branch of the Impact that we’re prioritizing, then it can be dumped.

Summary

Impact Maps work well because they maintain a visual connection between the Goal, the Actor, and the Action (Impact + Deliverable). They can be used to generate a new list of features/requirements/ideas, or to prune an existing list by focusing only on the features that support the Goal.

Impact Mapping ensures that when we discuss a piece of work, it always remains in context of our overall Goal, and it can also help us see that there are many paths to it.

Never implement the whole map – instead, run experiments to see which path will get you to the Goal the soonest/with the least amount of effort.

Finally, you can use Impact Maps for more than just product development work; they also help when undertaking organizational change.

Impact Mapping – Glossary and Additional References

Want to learn more about Impact Mapping and other strategies to help your team identify the best way to get more bang for the buck? Our Certified Scrum Product Owner training uses real-life examples and hands-on practice to help you and your team understand how to create ideal customer and organization outcomes, not just produce widgets and complete tasks.

[1] The original book used “Who” for Actor; How for “Impact” and “What” for Deliverable – I find the newer versions far more clear
[3] Gojko offers this list as a simplified version of the advice from Competitive Engineering by Tom Gilb
[5] Gamestorming by Dave Gray – https://gamestorming.com/
[6] From the Business Value Game by Andrea Tomasini – https://www.agile42.com/en/business-value-game/
[8] “Splitting user stories — the hamburger method” by Gojko Adzic – https://gojko.net/2012/01/23/splitting-user-stories-the-hamburger-method/

Why are Group Decision-making Techniques Important?

by 2 Comments

decision-making techniques and tools matter

Whether we’re facilitating Retrospectives, Sprint Planning, or a Strategy session, there are always decisions needed. How you approach those decisions should vary depending on the degree of cost and whether the decision is easily reversible or not.

Given the speed at which Agile moves, we can’t afford a lot of time spent waiting for simple decisions to be made. So for low cost, simple, or reversible decisions, we need mechanisms to make decisions quickly and with a minimum of overhead. For more expensive, complex, or risky decisions, we need approaches that explore the problem in more depth and then create experiments that we can run to quickly see which path is a clearer choice before too much investment is made.

Another reason good techniques are needed is because of cognitive biases.[1] Hopefully we’re all aware of them and we’ve started to spot them throughout our lives. Decision making, like everything else, is made more difficult because of these biases, so we’ll also address how those should be factored into your choice of technique.

Establish Decision-making Policies

The first thing to decide is how your group is going to handle decisions. Once that’s determined, add it to your Working Agreements. A defined policy will help people feel that their ideas got the same reception as anyone else’s ideas. When a team doesn’t have an agreement on their decision-making rule, then a team member might feel blindsided if, for example, a majority vote is called for and they were not expecting that. There should be no surprises.

Techniques for Low Cost or Reversible Decisions

With low cost or reversible decisions the cost of minimizing cognitive biases may be more expensive than the decision itself. There are several options for techniques in these situations, but be warned that some of the most common methods have unexpected traps.

In evaluating different tools, I want to know: Does this help the group decide quickly? Does it help ensure quieter voices are heard? Does it avoid endless rounds of debate? If the proposer’s idea loses, will they feel that it got a fair hearing?

Here are some popular tools and how they work.

Majority vote

This can be used as a group decision-making technique and it’s probably the default of most teams, however, there is a significant risk. When we have quieter people on the team and they take the risk to vote against the group, they may easily see a majority vote against them as silencing their opinion. Also, when someone who is powerful votes early, other people may fall into line, either because they assume they know best or because they feel the safest following authority. I strongly recommend against using this in Agile teams; it tends to disenfranchise the voices we need to hear more from.

Consensus

This approach seeks to ensure the support of all members of the group for the decision. This doesn’t require complete agreement, only no disagreement. Usually when seeking consensus, the facilitator asks a question like “Can we all live with and support this decision?”[2]

Consensus decision-making is remarkably inclusive and does a good a job of inviting participation from all. With this approach, there are still a couple of risks:

  • Endless debate – a team can get stuck revisiting the same options over and over from different angles[3]
  • False consensus – a team member may disagree with a decision, but not voice it to avoid rocking the boat

After reaching consensus, the facilitator might ask the group to commit to helping to implement it.

Fist-to-Five rapid voting technique

This should be done blind – everyone makes their decision and votes are revealed simultaneously using one hand per person. This improves on majority voting because it invites more subtle opinions than just yes/no from a majority vote. It also invites further discussion where it is warranted. Where it would save time, it also signals an idea that has rough agreement.

A “no” vote, will block consensus No support, but won’t block. Will discuss and suggest changes Moderately comfortable, Discuss minor issues Not in total agreement but comfortable to let the decision pass Good idea, will work for it Great idea, complete agreement, support it fully
fist to five rapid voting

Decider Protocol

From the Core Protocols – a set of Protocols to help teams go about their work, minimizing the time wasted in meetings. The protocols are designed to help focus discussion and decision making.

One actionable idea is proposed.

Participants ask questions to gain clarity.

A vote is performed using hands again, but this time just a thumb, to indicate a choice.

  • Thumb up = yes: full support
  • Thumb sideways = yes: support the idea but may still have to think it through
  • Thumb down = no: do not support, provide a counter-proposal

The proposer counts the number of votes. If the number of yes votes (either thumbs up or sideways) is too low (usually below 70%) it is a weak proposal and should be withdrawn.

If there are a small number of “no” votes, use the Resolution Protocol to see what it would take to get them on board with a question like: “What will it take to get you in?”

(This is a brief summary – see all the Core Protocols[4] including the Decider section[5] and Resolution sections[6] for more depth.)

Dot Voting

This is a tool for selecting the top few options among many.

Before using dot voting, take a few minutes to group or eliminate similar items. Then each voter gets the same number of dots to vote with. The dots can be stickers or simply a dot written in pen.

  • A voter places all dots on one idea or spreads them out – it’s their choice.
  • The idea, or ideas, with the most dots wins.
  • How many votes per person? Five to seven has worked well for me in the past, but it’s up to you.
  • In remote teams, some of the tools build dot voting in. In others, I’ve seen attendees edit the titles of sticky notes to add a “#” per vote.

dot voting The above example was used by a group at an organizational retrospective to decide what items they should address.

Dot voting is the Agile community’s poster child for decision-making techniques but it, too, has some weakness. Since dot voting is usually done with people walking up to a poster or wall to place their votes, everyone can see which items you voted for. Seeing other people’s votes will influence you and, of course, seeing a leader’s vote will influence everyone. This weakness can be overcome if all the votes are placed blindly, i.e. voting decisions are made without seeing the votes of others.

The other challenge is that you can see that people voted for an item, but not why they voted against another, missing an opportunity to improve it.

It’s worth noting that dot voting is a popular tool for online interactions and distributed teams. The online tools typically cost money, but whiteboard tools such as https://www.mural.co/, https://miro.com and https://stormboard.com/ all allow the facilitator to create a dot voting session. They also have the advantage of being blind. Tools like https://www.groupmap.com/, https://consider.it/ and https://stormz.me/en have a richer set of tools to generate, organize, and vote on ideas. While these aren’t designed for retrospectives, they might be useful.

Techniques for Expensive or Irreversible Decisions

When the cost of the decision is higher or there is an element of risk or uncertainty, then we want to find decision-making tools that mitigate against those, as well as against cognitive bias, and take more time and ask better questions so we can have better information to make decisions with stronger odds of success.

When the cost and/or uncertainty is high, we should not look for an immediate decision. Rather, we should be looking for an experiment that we can run that will get us closer to a clear choice later. In designing our experiments we need to combat cognitive biases such as the following.

  • We often start our problem by thinking in too narrow of a frame of choices. This is often coupled with Blind Spot Bias, where we can see other people’s biases more easily than our own.
  • Confirmation Bias leads us to look for information that reinforces our existing thinking, and Motivated Reasoning causes us to see new information in a way that supports our existing beliefs.
  • Short-term emotions can paralyze us or shut down good choices via fear, sunk cost fallacy[7] and more
  • Overconfidence when we are certain about our guesses for the future is often coupled with Hindsight Bias where, once something happens, we see it as if it was the only thing that could have happened.

No single magic technique will overcome all of these biases, but we can take steps to minimize their effect.

  • Use Systems Thinking tools like change perspective (What happen if we take another person’s point of view?) or change time frame (What will the effects be in 5 days? 5 months? 5 years? This is also known as 10/10/10 Analysis – 10 days, months and years.) This ties back to the Future Perspective approach[8] of stepping mentally into the future and imagining afresh where we want to arrive.
  • Explore other sources of data or information. (From outside the team? From outside our organization? Who else has made similar decisions before?)
  • Use the Rule of Three.[9] If you don’t have at least three options, you don’t have enough.
  • Ask what assumptions are being made and test if those assumptions are valid.
  • Discuss the longer term unintended consequences of this decision. (How could someone game the system to exploit this decision?)
  • Ask whether there are any time-delayed effects that might show up down the road. This is an especially important question, as a lot of solutions that seem good on the surface cause deeper problems down the road.
  • Ask for someone to take a dissenting position and treat the dissenting view as important and not just an empty gesture.  Getting an opinion from someone who would make a different decision forces us to consider our own approach from another angle. If you can’t find a dissenting view within your team, ask outside your team.
  • Admit uncertainty.
  • Consider what would happen if you made the opposite decision. This broadens thinking and can uncover advantages or pitfalls that help make the choice easier.

After the Decision

Whatever the outcome, we need to be careful of “Resulting”[10], where we judge the quality of the decision based on the results. Luck plays a big role in work. Sometimes good decisions result in bad outcomes and sometimes bad decisions still get the result we wanted. The common example used comes from poker where, despite using logic and weighing odds, sometimes luck tips the scales. It’s important to acknowledge that good decisions can have bad outcomes despite a team’s best efforts, otherwise fear will make future decision-making even more difficult.

For additional practical tips and strategies on how you can help your team be more effective, we’d love to have you join one of our Advanced Certified ScrumMaster programs, where instead of dying from boredom sitting in a 2-3 day theory class, you will learn new material and be coached over an extended period how to build a sustainable improvement process.

[1] There are several book length treatments of this subject. Decisive: How to Make Better Choices in Life and Work by Chip and Dan Heath and Thinking in Bets by Annie Duke are a good start. We will touch only lightly on the challenges.
[3] Curiously, Planning Poker was originally invented as a tool to cut through endless debate in estimation meetings.
[7] https://thedecisionlab.com/biases/the-sunk-cost-fallacy – commonly known as “throwing good money after bad”
[9] Attributed to Gerald Weinberg: https://en.wikipedia.org/wiki/Gerald_Weinberg
[10] Courtesy of Annie Duke

Future Perspective for Change: Why Backcasting Helps Get You Where You Want to Be

by Leave a Comment

When a product team starts work on a product, it’s important that they understand the vision. This is, as they say, a bit of a no-brainer since otherwise they might build something that completely misses the mark. In a modern Agile world, we’re expecting a mix of Product Management, Developers, ScrumMaster, the Customer and perhaps key Stakeholders to spend time together in a collaborative effort to have, and maintain, a shared goal so many teams use exercises like the Product Box (originally from Innovation Games), Pixar Pitch, etc, to spark conversation between the people building the product and the people for whom the product is being built. Over the process of a few hours these activities get the Developers and Customers to a common understanding of what problem they’re attempting to solve.

Right. Great. So that, in an oversimplified nutshell, is how everyone can have a vision that they agree on to build a great product.

But what about building a great organization?

When I’m brought in to help people undertake Organizational Change, I always ask them, “Why are you undertaking this change?” Too often the answer is, “To be Agile”. The problem, of course, is Agility is not the goal, it’s the outcome of improving the system. Then I ask who was involved in creating this “why” and the answer is usually “Management”.

When change is started by Management alone, the Doers perceive the change to be imposed on them and not something they’re part of. Co-creating the Product Vision is critical for a Product to succeed, and the same applies to changing the system itself. We need the people who will be part of the change to participate in deciding the why of change.

Typically the why is because they want improvement of something, otherwise what’s the point? It can get trickier, though, when asked to identify what exactly that improvement should look like —what their vision for organizational change is. When we attempt to imagine the future, we tend to picture a slightly better version of the present. If I had asked you in the early to mid-2000s to imagine a future phone, it likely would have been some improved variant of a Blackberry with a physical keyboard. Few would have mentally pictured the iPhone features released in 2007, changing our expectations.

This happens because our brains are only able to extrapolate from the current state. So if we’re asked to look forward a few years into the future, we tend to project a version that is consistent with the current world but only about 20-30% better. If we want to create a new world, we must reverse that direction and, instead, imagine ourselves in the future state looking backward. This is referred to as “backcasting”, which is different from forecasting in that you start with the end in mind and “cast” backwards.

Variants of this technique have been used by Amazon,[1] Komatsu[2] and any number of groups imaging a world without the use of fossil fuels.[3] In this case, we will use this approach when starting or sustaining an Organizational Improvement Process. By slightly tweaking the initial question, this is can also be used for Product Vison.

backcasting - c.The Natural Step

 

Picture from: https://www.naturalstep.ca/backcasting-starting-with-the-end-in-mind. Used with permission.

Goal Discover what success looks like in a change process.
When to use At the start of a project or any time you’re beginning a new initiative.
Time 1-2 hours depending on facilitator experience and the size of the audience.
Who is involved The Organizational Improvement Team  – must include some doers, a few key decision-makers, and a few key stakeholders (the people sponsoring the change)

Characteristics

  • Long term scale, imagining a future at least five years out, sometimes longer
  • Participatory Process, not just the views or plans of one person
  • Identifies and helps prioritize preferred outcomes
  • Vision should motivate and lead to collective action

Facilitation Notes

(Adapted from Innovation Games[4] – Remember the Future)

The exercise starts off with the facilitator asking the attendees to imagine a moment several years into the future, when their world of work is much better now. The facilitator asks participants to “look back” and see what made the change a success.

Participants write down the changes on sticky notes or the virtual equivalent using as much detail as they can. Once everyone has had a chance to write, draw, and reflect (10 – 15 minutes), participants take some time to group common themes (another 10-15 minutes). Once the themes are found, they’re reviewed to see if they suggest a headline (e.g. “World’s most improved quality” or “Our clients are so happy with our work, they surprised us with a paid vacation for the whole team”). Some approaches have boxes or prompts that invite people to answer specific questions – e.g. a newspaper article has a headline, main story, a picture and related news:[5]

future perspective newspaper - blank

When the team have created enough examples, they select the best one(s) and that becomes their change vision. When choosing, look for one that gets people jazzed up about making change.

The change vision feeds into creating their change strategy – think Story Maps or Impact Maps, adapted from the Product world to the Change world.

Facilitation Notes

The key is, as the facilitator, to frame things as looking from the future state backward, to help reframe the thinking, and then asks open-ended questions. So instead of asking, “What will make our company more flexible/agile/better/happier/faster/have better quality?” the ask is more along the lines of, “It’s been five years and one month. Our team members are happier, our customers appreciate our quality, and our executives like our flexibility. What changes got us here?”

Other prompts that the facilitator might use:

  • “Our improvement team is walking up on stage to receive the _____ award; the presenter says at no time in the history of this award has the committee seen an organization more deserving. Fill in the award name and describe the accomplishment.”
  • “You overhear a group of your colleagues talking about the organization. They say they didn’t believe change was possible but, here we are five years later and change happened, this is now a better place to work. As you listen to them speak, what are they saying? Why is it a better place to work?”

If the group is larger than 7-8 participants, break into subgroups and share results back after a round of play. In this case invite the groups to self-organize into subgroups of 4 or more people. Ask that they work with people they haven’t worked with before. We’re trying to reinforce that all aspects of an Agile work world can be self-organizing.

Consider a second round of play where groups take ideas that inspire them from other groups, and see if they want to change their own “remembered future”.

Since most teams are used to working with a Product focus, you will need to remind them at the start, and perhaps also after the first round of work, that the focus isn’t about the product but making the process better.

Example

We know we want to improve our quality and how our customers see our product. We might start with an opening question: “It’s been 5 years and a few months, our customers are phoning support, but not with problems.” What are they saying and why? Or “It’s been 5 years and a few months, we’re at an industry awards ceremony, the presenter is on stage saying ‘never before in the history of this association has ….”.

The team work with the prompt and say:

future perspective example cards

From this they might create a number of  newspaper stories that shows the effect of their quality work from the customer perspective:

future perspective example newspaper headline

We’ve only shown one here, but a group will create many and then select the best idea(s) for their change vision.

Just by changing the opening question, this exercise can also work well for a Product Vision too.

Complementary tools

Liberating Structures 1-2-4-All might work well to help the group self-organize to synthesize their discoveries.

Blank newspaper headline template:

future perspective newspaper - blank

Also see our “Agile Change or Adoption Always Starts with Why” article.

Want to have a better understanding of the why and how of more things in Scrum, and not just the book-level what? Practising Scrum without proper understanding can bring poor results and, ultimately, frustration and resistance. In our Certified Scrum workshops, attendees learn in a fun and non-judgmental environment how to make Scrum work effectively in the real world, not just in principle. Join us to learn and to gain practical, hands-on experience.
[1] https://commoncog.com/blog/working-backwards/
[2] Ch 4 – The Knowledge Gap – The Art of Action by Stephen Bungay
[3] https://medium.com/age-of-awareness/envisioning-a-future-that-works-for-all-characteristics-of-the-visioning-process-ab02332a63ce
https://energyfutureslab.com/backcasting-starting-with-the-end-in-mind/
a href=”https://sustainablebrands.com/read/new-metrics/backcasting-a-roadmap-to-transformational-change” target=”_blank” rel=”noopener”>https://sustainablebrands.com/read/new-metrics/backcasting-a-roadmap-to-transformational-change
[4] Innovation Games: Creating Breakthrough Products Through Collaborative Play by Luke Hohmann
[5] For example: https://paraffin.ltd/visioning/

Two Key Things for Sprint Retrospective Facilitation

by Leave a Comment

You Got This sign by Prateek Katyal courtesy of Unsplash Retrospectives are a critical element in Scrum but they won’t work well if your team hates them. A Sprint Retrospective needs someone to facilitate and keep it on track to encourage effective discussion.  It’s more of an art than a science, but these key things will help.

1. Remain Neutral

From the Cambridge Dictionary, facilitation is “the act of helping other people to deal with a process or reach an agreement or solution without getting directly involved in the process, discussion, etc. yourself.”[1]

This is often the hardest thing to do in the role of facilitator. It’s natural that you have ideas and opinions on what the team could try to improve, but the moment you move to advocacy, the team loses trust in their facilitator. The team needs to know that your goal in the retrospective is simply that they have a successful event and make their own discoveries. If they feel their facilitator has a personal position or opinion involved, they’ll question whether the event is being run to help them, or to steer where the facilitator thinks they should go.

This doesn’t stop you from being a facilitator and a ScrumMaster or Developer, if that’s the case – it merely means that, during the retrospective, you must set aside your personal agenda. Step back from advocacy (which does have a small place outside of the retrospective) to neutrality. Neutrality requires that you don’t advocate for viewpoints, positions, or ideas during the retrospective and, instead, seek to ensure that viewpoints are understood and ideas are heard.

But what about those times when you have ideas/issues that you feel are important to be discussed in the retrospective? You have four viable options:

  • Don’t share the idea at all. If it is valuable for the team, someone else will come up with it.
  • Invite another team member or an outsider to be the facilitator in a retrospective when you intend to discuss your idea.
  • Write your idea down and give it to another team member prior to the retrospective. It becomes their job to advocate on behalf of your idea and, during the event itself, you don’t speak to it.
  • Have a hat with a “team member” sign that you wear when presenting and speaking to your idea. There are several caveats with this idea: it can only work when the team already trusts that you’re a good facilitator, it can only be used sparingly and, when used, it really should be for only a short fraction of the retrospective.

While it’s challenging when the facilitator is an active team member, having the team’s manager or executive as the facilitator is far worse. Managers have power and, because of that power, participants won’t feel safe being candid and honest when difficult subjects are discussed. Managers cannot facilitate retrospectives and, under most circumstances, they shouldn’t even be in the room.

2. Own the Retrospective Plan, Keep it Interesting

To be effective, you should have a plan but also change up your retrospectives so they don’t get boring and predictable. If they’re always the same old song and nothing improves as a result of the conversations, team members will become disengaged and miss out on the benefits of a well-run sprint retrospective.

As the facilitator, you own the design of the retrospective. You decide which activities fit the team’s circumstances and mood, so be mindful of your choices. For example, if your team had trust issues last Sprint, then don’t plan an activity like Story Oscars[2] that focuses on why a User Story went wrong, or there’s a good chance it will turn into a blame-and-shame session. In a situation like that, you’d be far better off to pick an exercise that is likely to promote a Systems Thinking mindset, so that the wider context becomes visible before you try to deal with trust issues.

Even with the best plans, you still must be prepared to adapt. I’ve facilitated a retrospective where I could just sense the energy of the room dying and the existing activity wasn’t working for them, so I flipped to a Sailboat Retrospective exercise, which felt a little more lighthearted and interactive. Likewise, if the conversation seems to be going in a different direction than you anticipated but it’s still productive, consider allowing it to unfold and adapt your plan based on the team’s need.

Facilitation Tips

  • Cover the meeting outline before beginning by giving the team a short outline of your agenda.
  • Don’t solve problems. Help your team communicate effectively so they solve it.
  • Allow silence. If the dialogue pauses naturally, don’t seek to fill the void. Silence often prompts someone new to speak up.
  • Ask open-ended, probing questions that seek to clarify, and test to see if your understanding of a statement is correct by rephrasing it back to the original person.
  • Direct questions to the quieter members of the group to help bring them into the discussion.
  • Monitor the energy of the group by observing body language, fidgeting, tone of voice, etc.
  • Pay attention to your own body language and reactions. What signals are you sending? Are you favouring anyone? Signalling irritation?
  • Don’t do all the writing. When writing summaries is necessary, consider inviting a team member to do it. You may find that their interpretation of events is different than yours would have been, which is valuable to learn from.
  • When you must be the scribe and you paraphrase, ask the audience if your rephrasing matches their original intention.
  • Encourage people to use “I” language if there are difficult situations. That is when the person describes the effect it had on them and their feelings, rather than makes statements that start with “You did … x.”
  • Refocus the group if they jump ahead segments. For example, if in the data-gathering phase, people start to solve the problem, gently guide them back to the task at hand.
  • Use a Parking Lot to set aside questions or issues that can’t be handled right now. It acknowledges that there is something that needs to be addressed but prevents the retrospective from being stalled as a result of it.
  • When you have team members who are more expressive than others, consider exercises where their opinions come later, after quieter team members.
  • Bring in an outside facilitator from time to time – someone from another team (not just the ScrumMaster), an Agile Coach etc. – so they can observe and offer objective feedback and suggestions for improvement.

This is just a quick post about one of the most essential parts of Scrum. Learn much more in our free Guide to Effective Agile Retrospectives.

 

Image credit: Prateek Katyal

Be Better with Better Data

by Leave a Comment

Data and metrics are not synonymous. Let’s tackle that misunderstanding right out of the gate.

Data is a number or set of numbers. “Three days.” There. That’s data. Is it useful to you? Of course not, because you have no context to go with it.

Metrics is the context and measurement of where, and under what conditions from which, that data is the result. Metrics tells you whether three days is how long it took one person (data) to unload a cubic yard (data) of rocks in a rainstorm, or whether it’s how long it took a team of five developers (data) to plan, create, test, and release a product that does X.

We talk about using better data but, really, we should say be better with well-chosen metrics. The problem is that just doesn’t roll off the tongue the same way.

In Scrum, most people instantly think of velocity as the key metric – how much work a team does in a Sprint, usually in the form of number of story points completed. But that’s a weak and easily gamed measure.

“Tell me how you measure me and I will tell you how I behave.” – Eliyahu Goldratt[1]

How could a team game (i.e. manipulate) this number? The ways are many, but here are just a few:

  • They could inflate their numbers during estimation, even without intending to mislead. Someone might say, “Let’s make it 13 or 20, just to be on safe side.” They didn’t purposely inflate the estimate, but that was the outcome.
  • Feeling pressure to deliver more Story Points by the end of the Sprint, they might skimp on testing thinking, “Oh, that’s good enough and, in the worst case, we can always fix the bug later.”
  • They might argue in Sprint Review that an item is partially complete so they deserve partial credit.
  • …

All of these and more are outcomes of Goldratt’s quote. We told the team they were measured with Story Points and they behaved accordingly.

Forewarned is forearmed, and metrics have a good side as well. so they still have a place in Scrum.

Metrics are a tool to help a team know where to focus its energy for improvement next.

Notice that we’ve shifted the focus from measuring productivity to where to focus for improvement. Measurement shouldn’t be individual vs. individual or team vs. team. Rather, it should be team vs. itself in regard to “Where can the team improve next” or “Where has the team improved?”

But when we encourage you to improve, we can hear you cry back, “How do we measure that? Much of it can’t be measured and, when it can, the effects will only appear months after the actions are taken.” Fair point. Good metrics help you spot trends and find problems. Unfortunately, the data might not tell you if the specific change you made had the effect you wanted.

The key problem is that all measures we can find are trailing, and some measure their effect long after the action was taken. An example might help. Suppose our heroic team decides to better their engineering practice with “trunk based development”, their hypothesis is that doing all work on the main branch with daily checkins will reduce the frequency of merge problems and make refactoring easier. In turn, this will reduce code complexity and decrease the number of defects. So our mythical team adopts this improvement, and finds a good measure for code complexity and defects. Yay, team. The problem is that, with a sizeable existing codebase, it will take months for the benefits to prove themselves in a measurable way. In that time our team will likely have tried several other things to make improvements (a wise idea). How do we tell which improvement lead to the change in the measure?

Or if you prefer a non-programming example, think of data like a blood pressure test. High numbers suggest that improvements could be made. So you immediately cut down on salt and coffee. Maybe you take some medication. A few weeks later you book some vacation time and go hiking. Months down the road, you feel much better. What was the “fix”? Diet? Medication? Reduced stress? Exercise? All of the above? None of the above? A combination of some but not others?

In the end, does it really matter? You’re healthier and happier. The measurement data was important because it showed where to look for trouble brewing, but don’t get hung up on using it as the thing that tells you if you were successful.

So the measures we offer here are tools to help your team to understand where it is in pain, but they won’t tell you whether or not you got better.

Start with Throughput

In The Guide to Effective Agile Retrospectives eBook, we mention many sources of data that you could collect for the team. We suggest replacing velocity with a simpler measure like throughput – the number of Product Backlog Items completed in a Sprint. (Note: “completed” means Product Owner accepted and the item met the Definition of Done.) Throughput is elegant because most attempts to game it improve the product we’re building. For example, in an attempt to increase perceived throughput without doing more work, the team split their features or user stories into smaller chunks. When features are split into smaller chunks, there are typically fewer defects since they’re better understood and the product owner gains greater flexibility.

To improve your use of metrics, find a few simple sources of data you could bring to your next retrospective. From the eBook, some highlights:

  • The Team calendar – helps to remember who was present and who was away
  • Did the team have any Production Support issues to deal with? When did they happen? Were there interruptions from other sources outside the team (e.g. VP or CEO)? When? Capture and display this information on a Kanban wall.
  • Items still In Progress at Sprint End
  • Number of User Stories that were thought complete but failed to meet the Definition of “Done”

Consider Morale

We all want to create a workplace that we enjoy. We spend the vast majority of our awake adult lives there, so it sucks if it’s something we dread.  No matter how good, or bad, it’s normal to have many ideas about how we’d change things if we were the boss for a day. The question is then, how do we measure that?

There’s rising popularity for using the Happiness Index – whether you’re happy at work – but there are many problems with this as a measure. Morale, on the other hand, is robust, less susceptible to mood, and vastly better studied by both Occupational and Military Psychologists:

‘(Team) Morale is the enthusiasm and persistence with which a member of a team engages in the prescribed activities of that group’ (Manning, 1991).

In the Retrospectives book, we shared some summary notes about Christiaan Verwijs’s advice on how to measure morale for a team in a way that more useful than the Happiness Metric. He also a created a free survey tool to help measure morale anonymously across a team: https://teammetrics.theliberators.com

Example Data – Morale

Example of data collected to measure team morale presenting results using red-yellow-green colour coding

In a retrospective, we could ask questions like:

–       Why is our morale at this level?

–       How does it compare to our historical morale?

–       Can we see if changes in morale correlate with any other measures?

–       Has there been a noticeable downward trend? If so, what has prompted that change?

–       Is there one experiment that might help in improving morale?

Address Impediments and Interruptions

We already know that the ScrumMaster doesn’t solve most impediments on their own and that it’s a team effort. That leaves the bigger issue of creating awareness of the impediments and their sources. There aren’t any miracle cures in this area, but it really helps to gather both quantitative and qualitative data.

Qualitative first – listen in Daily Scrum and make note of problems that your team members encounter. Both the things they describe as impediments, and even the ones that aren’t explicitly named (for example, if a team member says, “I spent half hour waiting for a build to complete or another person to do something for me”). Watch for interruptions from outside the team. Also note incidents of lack of knowledge, which can slow one or more team member as they search for an answer. All of these point to different sources of slowdowns or blockages for your team. A great ScrumMaster will record these and bring the data to a retrospective.

Quantitative – count the number of these impediments, interruptions, or blockages that occur each day. This is also information to bring to the retrospective.

The qualitative data can be used immediately; the quantitative will take a few Sprints until we have enough data that we can show people the correlations. There’s an example in the Retrospective book that shows where, in the Sprints where there are more outside interruptions, the team deliver less value. Take the time to assemble this correlative data, because it can be very powerful at helping to start bigger organizational changes.

Graph measuring number of defects found in a product over time, measured in weeks.

Remember, metrics are a tool to help a team know where to focus its energy for improvement. There is no magic, but the closest thing to it can usually be found with teams that make an ongoing effort to consistently make small, positive changes. Something as manageable as a 3% improvement rate will add up to noticeable benefit quicker than you might think. Data, both quantitative and qualitative, can be a source of ideas for those improvements. Or, if the desired improvements are outside of the team’s control, the same data can be used to spark conversation around deeper changes.

Want to have a better understanding of the why and how of more things in Scrum, and not just the book-level what? Practising Scrum without proper understanding can bring poor results and, ultimately, frustration and resistance. In our Certified ScrumMaster workshops, attendees learn in a fun and non-judgmental environment how to make Scrum work effectively in the real world, not just in principle. Join us to learn and to gain practical, hands-on experience.

[1]  https://en.wikiquote.org/wiki/Eliyahu_M._Goldratt – from The Haystack Syndrome – caveat this book is generally considered hard reading even for experts in the field.

Image attributions: https://medium.com/the-liberators/agile-teams-dont-use-happiness-metrics-measure-team-morale-3050b339d8af and Agile Pain Relief Consulting

Scrum by Example – Same Old Song in Sprint Retrospective

by Leave a Comment

The World’s Smallest Online Bookstore team has just come out of their Sprint Review. For those who’re keeping score, the team committed to a Sprint Goal: Have basic reader review system in place so book buyers can see a variety of opinions about a book. Their plan included completing seven Product Backlog Items and fixing two bugs as part of that Goal. By Sprint Review time they’d completed five of the seven. They walked into the Sprint Review with heads held high and, with the pummelling they took over not mitigating for fraud reviews, they left with their heads down. ScrumMaster Steve suggests they all go to lunch and reconvene for the Sprint Retrospective in an hour and a half.

Steve knows they need a great Retrospective to turn this experience around. He has heard from others that you should mix up your retrospective style every few Sprints to avoid boredom and complacency. However, this time around he just wants to stick with a retrospective that he knows well. His agenda:

  • Welcome
  • Read the retrospective minutes from the last retrospective
  • What went well?
  • What didn’t go well?
  • What can we improve next time?

Before the Retrospective starts, Don – the Director of Development and Steve’s boss – asks to join the Retrospective. Don explains that it was a bit of a bad Sprint and the team need to understand just how important it is to turn things around.

Steve has to think fast. He knows that if Don shows up at the Retrospective, even if he stays silent, his presence will cause the team to stop talking openly. Steve decides to take a chance. “Don, the team really appreciate the support you give them,” Steve begins, “but I’m concerned that if you attend the Retrospective, the team will say a great deal less. The less they say, the longer our problems will continue. So if you want us to improve more rapidly, please don’t attend the Retrospective.”

After a bit more discussion, Don gives in and says maybe he will address the team later in the week. Whew. Steve just dodged a tough problem.

The team return from lunch and the Retrospective starts. Steve welcomes the group and makes a few lame jokes to lighten the mood after the Sprint Review. He starts off by reading the minutes from the last Sprint Retrospective. He then asks the team, “What went well in the last Sprint? I will give people a few minutes to write at least three items on Post-it® notes.”

The team dutifully pull out their pens and start listing what they felt went well. It takes five minutes.

Steve asks the team to group together notes that are similar. Once that is completed, he asks what patterns they see. Tonia says, “There are a lot of notes highlighting how well we handled Quality this time round.”

“Several of us felt that we did a good job Limiting Work in Progress,” Brad remarks.

Jasmine noticed, “Looks like team members were happier with the code itself this Sprint. Most of the new code was easily readable.”

Doug and Ian are silent.

Steve asks the team to repeat the process to answer: What didn’t go well?  This time, the five minutes results in over forty sticky notes. Like the first round, they group them and Steve asks his killer question, “Looking over our notes, what is the biggest area for improvement in the next Sprint?”

Two members of the scrum team argue during the sprint retrospective while the rest of the team gets anxious or tune out

At this point Kirby and Martin start talking animatedly. Kirby is really interested in tidying some of their technical debt. He thinks simplifying the worst of their code will make it easier to add the new features that Paula is asking for. Martin, who keeps talking over top of Kirby, says the database is a bit of a mess and he wants help from his teammates. Jasmine and Brad struggle to get a word in, but they do at least try.

Doug and Ian are silent.

In the final portion of the Retrospective – What can we improve next time? – Ian speaks up for the first time and complains that the responses given on the sticky notes are almost exactly the same problems the team faced last time, and the time before that as well. He quips, “Nothing ever gets better around here.” You can feel the overall morale in the room shrivel, as many silently agree.

Steve digs around in his notes and finds a few things that the team has improved in the past few months, and offers them as reminders that help keep things in perspective.

Once they refocus, the team agree to tackle two problems from the last Sprint:

  • Features rejected by stakeholders in the Sprint Review
  • Unit Tests that were few and far between during the Sprint

They turn these problems into two improvements:

  • Paula will consult more with the Stakeholders prior to Sprint Planning.
  • The Team will write more Unit Tests.

Steve adjourns the Retrospective, confident that these improvements will help avoid another bad Sprint.

Analysis

Credit where credit is due. For a ScrumMaster early in his journey, Steve is well-prepared. He has a plan for the Retrospective. He knows that if their manager is present, psychological safety will go out the window. He uses a silent listing activity (the Post-it® notes) to ensure that the quieter team members have voice. Hint: quieter people have just as much to contribute as the rest of the team. Their quietness is usually a sign that they don’t yet feel safe speaking up in front of the group and taking risks (this is at the core of psychological safety).

Challenges:

–        Like many first-time ScrumMasters, Steve is using the same agenda for his Retrospective each and every time: Went Well? Went Poorly? Improve Next Time? This leads to a couple of problems: 1) asking the same questions over and over means that we tend to get the same answers, 2) worse, using the same format constantly puts people to sleep. Before they even enter the room, they know what questions will be asked.

–        Even worse, each activity has the same basic format: list ideas on Post-Itâ Notes and then have group discussion. I bet 3M is the only one enjoying this repetitive format.

–        Steve took minutes from the last retro and read them out at the start, which is something he saw recommended by many sources on running effective meetings. In the right meeting context, this can save time rehashing past decisions and help a group see that they’re moving forward. But in a Retrospective specifically, this will violate the team’s sense of safety. They will wonder what will get recorded and, as result, they’ll speak less and keep their comments are mostly positive.

–        Improvements like “consult more with Stakeholders” and “write more Unit Tests” aren’t action items, they’re vague goals. What is “more”? When should “consult” happen? What is the outcome we expect of these actions?

–        When you’re talking with your friends over coffee, a discussion that wanders all over the map is fun. In a Retrospective it can become frustrating and lead to team members feeling like the whole event is a waste of time.

–        The team don’t have real data from which to work. Lacking data, all the challenges observed, and the improvements claimed, are nothing more than opinion and conjecture.

–        The rounds of open discussion are off-putting for quieter team members, especially in a team new to Scrum. These team members haven’t yet found their place of safety. Steve did a good job using a silent listing activity to start garnering ideas, however, that benefit was lost in the open discussion that followed when two team members who are quite vocal dominated the discussion.

What could they do differently next time?

Add variety with the agenda and activities. Steve really needs to start mixing up his approach to Sprint Retrospectives. Ideally he would never use the same agenda twice. He would be better off if he could facilitate five to six different Retrospectives, and never used the same approach twice in a row. Our free eBook The Guide to Effective Agile Retrospectives has a good start, and the Retromat has more (>100 activities). Doing this will avoid Retrospective boredom. Done well, Steve will also select activities that promote more focused discussion, saving the pain of wandering all over the map.

Note action items, not conversations. Instead of taking minutes, Steve should really only record the Retrospective action items. A good ScrumMaster would take those items and make them front and centre in the next Sprint Planning. A great ScrumMaster would ensure that the same items are checked for progress in the next Retrospective.

When adding action items in the next Sprint Planning, include Improvements in the Sprint Backlog. Otherwise they’re too easily forgotten during Sprint, and you end up with a “Why bother with retrospectives when nothing ever gets better around here” attitude from team members.

Avoid vagueness. Speaking of action items, we need to turn vague statements into concrete, actionable steps. The simplest way is to call the vague action a goal, and then ask: what is the first step towards that goal?  For example:

  • “Paula will consult more with the Stakeholders prior to Sprint Planning” becomes “Paula will schedule a one-hour StoryMapping exercise with key stakeholders (Director of Support, Eric, and Marketing Director, Jan) before each Product Backlog Refinement session with the team.”
  • “The Team will write more Unit Tests” becomes “When writing a new Object/Class/Function, Team members will attempt to create at least three new Unit Tests.”

These actions can be raised in Sprint Planning, then tracked during the Sprint and reviewed for progress in the next Retrospective. If you want more tools to turn vague statements into actions, see The Guide to Effective Agile Retrospectives linked above.

Use data. The questions “What went well?” etc. lead to vague discussions that wander all over the map. More structured activities like Mad, Sad, Glad[1] or Sailboat[2] will help focus discussion but are still just opinions. Even better, have data that was tracked during the Sprint that can be shared in the Retrospective. Data can move the initial discussion from opinion to fact-based.

The simplest metric to track might be how long each individual story took to complete (fancy name: cycle time). Discussing why some Product Backlog Items took longer than others can lead to the team discovering things such as: some items were sliced small and were completed quickly (a good thing); some had dependency problems with an outside team; some items were poorly understood and took longer to complete. Discussion around why an item took longer might also shine light on an area of codebase that is harder to work in and needs improvement.

From one small set of data, many discoveries can be made. With a few more observations, deeper and more focused conversations can happen. Consider “Team Morale, Defects that Escape a Sprint and the use of Cumulative Flow Diagram”.[3]

Make it safe for all. Not only are open-ended discussions less productive, they’re also challenging for quieter team members to participate in. Remember, quieter people need to feel safe to share ideas. A good ScrumMaster will want to select activities that gives everyone a voice. For example, “Mad, Sad, Glad” invites the team members to share their thoughts on Post-it® notes. Since everyone gets the same number of Post-its®, they all have equal voice.

Other strategies might include: giving the team members a specific order to speak in for an activity. Where a quiet person needs more help communicating, some teams find them a partner/advocate to help explain the quiet person’s position. The goal in all of these cases should be to improve the psychological safety situation inside the team so that no one is afraid to give voice to ideas.

If your Retrospectives aren’t yet fun and effective…

Retrospectives are more engaging when they’re not repetitive, and they result in meaningful improvements. Improvements are more likely when we have data to act on, and everyone has an equal say. The role of ScrumMaster is to create a retrospective where this all comes together. If you want more depth on Retrospectives, grab a copy of our eBook – The Guide to Effective Agile Retrospectives. We’d also love to have you join us in our Certified ScrumMaster training.


[1] Original source – Agile Retrospectives Book – Diana Larsen and Esther Derby. Online description: https://retromat.org/en/?id=7
[2] Original source – Innovation Games – Luke Hohmann. Online description: http://www.retrospectivewiki.org/index.php?title=Sailboat
[3] All of these are covered in much greater depth in our free eBook – The Guide to Effective Agile Retrospectives

Image attribution: Agile Pain Relief Consulting

Scrum by Example – Don’t Let Sprint Review be a Missed Opportunity

by Leave a Comment

When last we left our World’s Smallest Online Bookstore team, their Sprint was rocky. To have an effective Sprint Review story, let’s assume that they magically wound back the clock (shazam!) and implemented the changes. As a result, they had the more effective Sprint Planning that we discussed in the last blog post.

So let’s quickly recap that last episode. The team set “Have basic reader review system in place so book buyers can see a variety of opinions about a book” as their Sprint Goal. (They had a weak Goal initially but, after they magically went back in time and did things different, that was the improved result.)

They then selected the following Product Backlog Items:

  • Reader can add star ratings to a book
  • Reader can write a review of a book (plain text only)
  • Reader can add pictures to their reviews
  • When a review is written by someone who purchased the book through our store, the system highlights their review as a “verified buyer”
  • Registered site users can click a button to vote that a review is helpful
  • Reader can format the text of their review
  • Reviews feature the date they were written and the location of the author
  • They also planned to fix two bugs that affect how books are displayed to buyers.

The Sprint went surprisingly well. They finished the top five of seven items and fixed one bug. (By the standards of many teams this is remarkably good.) They are now ready to hold a Sprint Review.

Steve, being a well-organized ScrumMaster, has an agenda in place:

  • Remind the audience that the Sprint Review has a focus on the Product
  • Restate the Sprint Goal and review if it has been met
  • For each feature:
    • Team members demonstrate the features they worked on
    • Team checks features against the Definition of ‘Done’
    • PO accepts or rejects an item[1]
  • PO, Stakeholders, and Team members discuss the Product Backlog and the Product Map (often a Story Map or Impact Map)

Steve has invited all stakeholders to attend, which include the Team’s Director, and the Directors of Support and Marketing.

Illustration of Scrum team meeting with shareholders, with the Product Owner moderating discussion.

At the start of the Review, Steve reminds everyone of their Sprint Goal: “Have basic reader review system ….”.

Paula reports, “The team achieved the core of that goal, with a reader review system ready on staging server. All I have to do is press a button and the new version is live on the site.”

She then hands over the stage to team member Ian to start walking through the features that the team have built.

Director of Support Eric’s phone starts buzzing and he disappears into it, apologizing to the room. “Sorry, we have a problem with some users not being able to see their account order history.”

The first few features that Ian demonstrates garner almost no feedback except people saying vague things like “Good job” and “I like that”. There are no substantive questions or suggestions offered.

When Ian demonstrates the Product Backlog Item “Reviews from a verified buyer are highlighted and displayed above other reviews”, Paula asks a few questions about the behaviour of the feature. She also checks the Definition of Done and learns from the team it is Done except for testing the text-only book review in Safari on MacOS and iPad.

Placeholder example of an online review with a 1-5 star rating

Jan, the Director of Marketing, asks, “How are we handling fraudulent reviews? Our reviews have always been from staff. That has been a differentiator. If we take this feature live, we lose our ability to differentiate on that.”

Paula replies, “But we’ve verified that the buyer bought the book, so their review is legitimate.”

“There is a black market for reviews, and the book seller might pay the reviewer $10 for a good review and reimburse them the cost of the book,” explains Jan. “Although we verified that they bought the book, we haven’t verified that they read the book or that this is their honest opinion.”

After a lot more discussion it’s agreed that it’s okay to leave the verified buyer tag on the reviews since most are legitimate. However, they won’t be featured or placed higher on the page. They will just appear in the normal sort order.

Ian is almost afraid to bring up the next feature, “Registered site users can click a button to vote that a review is helpful”. He wonders what unexpected problem will be found next.

As he shows the feature, Jan interrupts. “We can’t have that at all. It’s another source of fraud. Sellers will often pay for many users to click the helpful review button on any review that has 4+ star rating. Their goal is to bury critical reviews.”

A placeholder example of an online review from a verified buyer of a product

At this point Paula can’t contain her frustration. “Why didn’t you tell us this before? Why wait until we wasted the team’s effort on implementing these features?”

Jan snaps back, “I tried to warn you about fraud weeks ago, but you didn’t listen.”

Realizing that the event has spiralled out beyond his ability to facilitate, ScrumMaster Steve asks for a ten-minute break. When everyone returns to the room, he suggests that they wind down the Sprint Review early, since they now have the list of features that were accepted and the ones that need more work.

As they leave, Ian mutters under his breath, “That was a ton of fun. Next time I’m asked to do the demo, the answer is NO!”

Analysis

What happened? Steve was exceptionally well-prepared. He had a thoughtful agenda and yet most features got no feedback. Of the people present, some (like Eric) were engaged only with their phone and they never participated in discussing upcoming product priorities.

Done well, a Sprint Review is a point of pride for the team. This one was a downer for their morale, even though some of the work that was completed made the product better.

Challenges:

  • The Sprint Review was a Show and Tell. The goal is to increase engagement with stakeholders and end-users, not decrease it. If you ask the team members who worked on each feature to demo it, it’s likely to result in an unnecessarily long, tedious monologue, or demonstration of every nook and cranny of the feature, putting attendees to sleep. Having Paula demo alone would be just as bad. She might script a better story and make it seem more fun, but “Telling” — aka talking at other people — decreases engagement.
  • The Definition of Done wasn’t checked before the Sprint Review. This is critical. It is the way the team and the Product Owner know that they’re on the same page with respect to quality. However, much of the “Done” is just going to put the audience to sleep, so going through the process during the Sprint Review isn’t practical.
  • The Director of Support was more engaged with his phone than in participating in the Review. We missed the opportunity for his feedback and we don’t know if his team’s needs are being well represented.
  • The surprise around the two features and how fraud mitigation wasn’t included indicates that the Marketing Director wasn’t involved enough in the Roadmap (hint: StoryMap) or Ongoing Backlog Refinement.
  • No end users were involved in the Sprint Review. Without their input, all of the feedback that was delivered was theoretical and from people who are involved day-to-day with the Product, so hardly objective.
  • Underdeveloped facilitation skills showed through. As Steve grows in the role of ScrumMaster over time, he might prepare in advance for conflict. Knowing that it will or might happen, he could have a short reset exercise ready to go that will cool tempers and improve relations so the Review can continue and the conflict can get resolved, rather than postponed. In the early stages of a ScrumMaster’s journey, it is a little unfair to expect them to know how to facilitate through conflict.

What could they do differently next time?

It starts long before the Sprint Review meeting. When the strategic work is done in Product Backlog Refinement, ScrumMaster Steve and Product Owner Paula need to make sure that key stakeholders, like Jan (Marketing) and Eric (Support), are involved. ScrumMaster and Product Owner should also commit to spending a couple of hours a month reviewing the strategy and looking for things that might turn into surprises. When there are items that represent more risk or challenge, Paula can invite the Stakeholders to the Backlog Refinement session so they can provide input and clarity. These two actions will go a long way toward reducing surprises in the Sprint Review.

Seeing a feature get reviewed by the PO and checked against the Definition of Done has never been a recipe to increase engagement. As an alternative, Paula could review features as fully completed, having already checked them against “Done” prior to the Sprint Review. This approach fits well in a world of Continuous Delivery, something many modern teams are already employ. As a good final check just before Sprint Review, Paula could recheck all features to make sure they work well together.

Get End Users in the room. Where humanly possible involve actual users of the product to get feedback, to learn more about their needs, and generate enthusiasm. There are clear limits as most users won’t have the time or energy to participate every Sprint.

Replace Show and Tell with Show and Play. Have someone who knows the product well show the high-level features. Have each developer pair with a stakeholder/end-user and invite them to play with the product. As much as possible they should focus only on the work completed in this Sprint. Show and Play helps the users and other stakeholders discover how the product actually works, it generates a much deeper level of feedback, and it helps team members learn more about the needs of real users.

A good Sprint Review gets features accepted and deployed. A great Sprint Review engages the attendees and creates productive discussion around product direction and business priorities.

Learn more about how to hold great Sprint Reviews

Consider attending one of our Certified ScrumMaster workshops, where we discuss the how and why of Scrum, not just the what.  You’ll get hands-on practical experience, and learn about some of the challenges – and solutions – along with tips on how to help your Team learn and grow to realize their potential.

[1] Hint: this shouldn’t be the first time the PO sees a feature

Image attributes: Agile Pain Relief Consulting