Improvement Experiments
Agile teams attempt to use Retrospectives as a tool for Team Improvement. Yet our improvements often lead nowhere. Instead of stating vague improvements like: “We should get better at…” consider creating experiments to test your improvements.
Each improvement is a small experiment designed to test a specific idea. The team predicts that a specific change will lead to the desired outcome. Success or failure, we still learn something about how we work. Like everything else in Scrum, experiments are iterative and adaptive. (Much like Evolution.) Each Sprint, we try something new, observe how it works and make adjustments in the next Sprint.
Well-Structured Experiments
A few key elements make experiments clearer and so more likely to succeed.
- Measurable Outcome Look for a measurable outcome, instead of saying “We should get better at X.” Define what “better” means. Perhaps the team wants to improve automated acceptance tests. The experiment might be stated as: by the end of the next Sprint, a number of user stories will have automated acceptance tests. (That assumes the team has a clear understanding of what a good acceptance test looks like.)
- Part of the Sprint The other key element is to make the experiment part of the Sprint. I remind the team during Sprint Planning, and we add them to the Sprint Backlog. During the Daily Scrum, I want to hear the team talking about their experiments. Finally, in the Sprint Retrospective, we review how the experiments went and use the results to inform the next set of experiments.
Experiment Templates
- Improvement Stories: These follow a format like “In order to [achieve a goal], as a [role], I want to [perform an action]. We will know we have improved when [measurable outcome].” This structure ensures the value, action, and measurement are clearly defined.
- Experiment Templates (e.g., A3): These provide a more formal structure for defining the problem, proposed solution, expected outcomes, and how success will be measured.
Examples of Experiments
- Limiting Work in Progress (WIP): The team might experiment with limiting the “Development in Progress” column to a specific number of items, with the goal of reducing average cycle time.
- Reducing Testing Time: Experiments involving cross-skilling team members to run test cases independently or implementing regression test automation, with the aim of decreasing the time items spend in the testing phase.
Warning, many good experiments require more than one Sprint to demonstrate their value. For example both the WIP and Reducing Testing Time experiments will require 4-6 weeks before we start to see noticeable improvements. Setting expectations up front is important.
Experiments are Key to Effective Scrum
The improvement experiment is at the core of our eBook: The Guide to Effective Agile Retrospectives - hint: plan your improvements into the Sprint, track them on the Sprint Backlog, and review them in the next Retrospective.
Agile Pain Relief Blog Entries
- Scrum by Example – Same Old Song in Sprint Retrospective
- Be Better with Better Data
- Bringing your improvement experiments into your Sprint: The Humble Sprint Backlog
- Two Key Things for Retrospective Facilitation
- Improvement experiments can be applied at an organizational level: Agile Change or Adoption: Define Small Organizational Changes
Resource Links
- Plan of Action - the simplest approach here
- Improvement Stories - using an approach like User Stories to create actionable change
- A3 Experiment Template - a full blown experimental model
