Agile Metrics

July 2, 2010 in .NET, Agile, Agile FAQ, Java by Mark Levison

Business Graph I’m frequently getting requests for good Agile Metrics and I’m never quite sure how to respond. Courtesy of some time waiting at LGA, I’ve been giving this some more thought. For many organizations metrics are irrelevant, don’t bother collecting them as they will just waste your time (and money).

If you must collect metrics, here is what I would consider.

Running Tested Purchased Features – Ron Jeffries is famous the metric Running Tested Features (RTF). I suggest that you consider taking it one step further until you’ve sold the feature to the customer you don’t know if they value it or not. For most product organizations this is a bit of stretch to measure in which case stick with Ron’s advice.

Questions to ask:

  • What would you like to change?
  • If you had the information the metric provided what action would you take? Can you take that action now without the proof of the metric?
  • Your key measure (i.e. RTF), should measure your widest span of control – Sold Features > Deployed > Automated Acceptance Tests > …
  • Measure Cycle Time – i.e. how long it takes to get something done and not people.
  • Other measures can be used: i.e. test coverage from Unit Tests, but be careful they might measure what you think they mean.
  • Measures can be gamed/fooled (intentionally or otherwise): For example test coverage measures whether or not a line of code has been visited. It doesn’t measure if there its meaningfully tested. If you must use a measure like this pay attention to the trend and not the absolute number. In this case a large jump might indicate someone having written a not very useful test of the outside api of your application.
  • Metrics have best before dates. Eventually you will stop getting real value from them. At that stage throw them away.
  • Ask can I get this information by walking around, observing and asking questions.

Alright you made it this far you deserve some options:

Martin Fowler says: CannotMeasureProductivity. Dave Nicolette presented on this at Agile 2009 (this article links to heaps of others). I wrote this for InfoQ last year:  What is a Good Agile Metric?. InfoQ also has: Metrics in an Agile World.

The following tools will help you measure, but please remember they often have many bad measures (comments?) turned with the good ones. Think carefully when choosing your rulesets:

  • Sonar – has a bunch of interesting measures: Cyclomatic Complexity, Duplicated code, … . While there are other plugins, its of most use in the Java world.
  • JDepend – helps you spot good vs. bad dependencies.
  • PMD, FindBugs, JLint – see a comparison of all three (pdf). Some of these tools check some pointless things: method name too short or too long? missing Javadoc comments? Please configure these with the help of a grown adult. But they can also be configured to spot methods (> 30-40 lines) and classes (>300-400 lines) that are too long.
  • NDepend like JDepend and heaps more measures. Again please be careful configure only with an adults help :-) Caveat Emptor I’ve been given a free copy of NDepend (that I’ve never had a chance to use).
  • Sonar for C# – yes according to StackOverflow.

When paying attention to measures of the code, what matters is the trend and not the absolute numbers. Finally just because a tool can measure it doesn’t mean its worth measuring, conversely some of the best measures don’t have tools to measure them. In this case note that none of the above tools measure cycle time.

Updated to make clear the point that you shouldn’t measure people and the limitations of tools.

Agile Retrospectives

May 27, 2010 in Agile, Agile FAQ by Mark Levison

teamwork 2 Continuous Improvement and Short Feedback loops (think: Test Driven Development; Sprint Demo/Review; …) are at the core of any Agile process. Without a structured improvement process it can be difficult for teams to improve and without improvement we stagnate. For methods like Scrum, XP and et al Retrospectives are that tool.

What is a Retrospective? It is a moment for the team to stop, breathe and take a break from the day to day grind. Its a chance to step back and reflect on the past iteration. To find things that worked well, things that need improvement and what the team has the energy to improve.

How do Retrospectives differ from Post Mortems (see CIO Update and PragmaticSW)?

  • Post Mortems occur after the project is done (or even dead), when it’s too late to improve that project.
  • Post Mortems are long feedback loops, once per project might mean every 6-18 months
  • Post Mortems often generate nice reports that are placed on a shelf and ignored (also called write only documentation).
  • Post Mortems sometimes turn into blame and shame events.

Well run retrospectives provide an opportunity for small improvements. The keys to a well run Retrospective:

  • Retrospective Prime Directive (Norman Kerth): “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.” The key here is to remind participants at the start of every retrospective this is not a blame and shame. Its about understanding what happened in the course of the last iteration. The focus is on events and not the people.
  • A clear agenda – a simple one:
    • What happened in the last iteration (including what SMART goals did we achieve?)
    • What would like to celebrate/remember/…
    • What areas need improvement?
    • What do we have the energy to improve in the next two weeks?
  • Clear Ground Rules see: Meeting Ground Rules Updated
  • An Open Mind from all team members with the focus on solutions and not the problems.
  • Appreciations – just take a few minutes to share something that you really appreciate that someone else on the team did. Interesting twist I just noticed that Ellen Gottesdiener puts appreciations at the beginning of her Retrospective. That’s very interesting, that will help put people in a positive frame of mind and make it easier to tackle the problems later. Elegant.
  • Once you decide what you have the energy to tackle, set SMART Goals (Specific, Measurable, Attainable, Realistic/Relevant, Timely). See: SMART Goal Setting. In the context of an Agile/Scrum team I would always make timely less than two weeks, so that you check back in the next retrospective.
  • A great facilitator who is able to stay out of the conversation and maintain the flow. Bring an outsider in occasionally, just to see a different approach.
  • Mix-up your retrospective activities to maintain the energy and interest

Follow-up:

  • Post your SMART goals on the team’s Information Radiator and check up on them in the Daily Scrum.
  • If you don’t make the improvements that people choose then Retrospectives will quickly lose their value as people say: “Nothing ever happens from these”.

When: At the end of every iteration or sprint. Allow one hour for every week of iteration. So a 1 week sprint would have 1 hr, 2 weeks a 2 hour retrospective. 3 weeks a 3 hour retrospective, 4 weeks – no one does those anymore right?

Who: The whole team – I like to see (or hear) the Product Owner and the Scrum Master. Some people will tell you that the PO isn’t necessary. Fine, but if they’re not there they can’t help make things better.

Pathologies of the Daily Scrum or Standup

April 19, 2010 in Agile, Agile FAQ by Mark Levison

Glenn Waters gave a great session at Agile Ottawa a few days ago, it was a combination of conversation, workshop and talk. As part of the session we identified many ways standups breakdown.

So what did we find going wrong:

  • Not held daily
  • Boring – no commitment or engagement
  • No perceived value
  • Too much talking and chit chat (socializing, wandering off topic)
  • Not adhering to the 15 minute rule
  • Being late and not showing up
  • Directed, i.e. one person taking over and telling team members what to do.
  • Reporting Directly to a manager vs. sharing with the team.
  • Not reporting to the team
  • Mumble, mumble disease, i.e. team member not saying anything that the rest of the team can understand.
  • Trivia report – reporting in so much detail that no one understands or cares.
  • Repetition – someone saying that they worked on one story several days running.
  • Not raising impediments
  • Not prepared
  • Problem solving

Misuse of Velocity in Agile Projects

February 18, 2010 in Agile, Agile FAQ by Mark Levison

measure

Like clockwork every month, someone on one of the Agile Mailing lists asks a question about “comparing velocity between teams” or “benchmarking data on velocity.” Before we examine the problem, let’s recheck what velocity is—the amount of work completed by the team/time taken to complete it. The amount of work is usually measured in Story Points (a relative measure of size). So velocity is just the number of points the team completes in the average iteration/sprint.

The underlying point is that Agile/Scrum teams use relative estimation (i.e., is this story/feature bigger or smaller than our standard story?) vs. the traditional approach of absolute estimation. The problem with comparisons, benchmarking, or any other attempts to compare velocity is that my story points ≠ your story points, because different projects use different standard stories. They work in different problem domains, and they have different people.

Retrospective Agility

September 21, 2006 in Agile, Agile FAQ, Books, Software Development by Mark Levison

Four years after I originally wrote this it is still getting hits, since that time I’ve written a much better post: Agile Retrospectives

Retrospective Agility” by Tim MacKinnon (part of a long pdf – too bad you can’t easily link inside these beasts) is a great starting point for running a retrospective. I was trying to find a short description of an Agile Retrospective to share with my team when I stumbled across this article. Tim has some excellent ideas especially for those of us just starting down the path of holding retrospectives.

Key Ideas:

  • Create Safety – how to create a safe environment for developers to say whats on their mind.
  • Types of Retrospective: Iteration, Incident, Release, Project
  • Lists of Common Things that went Well and Common Things to Improve – these could be very handy as a starting point for teams that don’t have much experience do retrospectives.

I just wish that more of the ideas mentioned in the article (like Gold Cards: intended to allow programmers to work on tasks of their own choice leading to increased innovation and a higher sustainable pace on the project) had been defined in the paper itself. It made the paper difficult to read on the bus ride home.

Over all Tim’s article made for a great introduction to Agile Retrospectives while I’m waiting for my copy of ” Agile Retrospectives: Making Good Teams Great” (Esther Derby and Diana Larsen) to arrive.

As for the description In the end I wrote my own (feel free to reuse it):

What are Retrospectives?

A meeting where a team looks back on a past period of work so that they can learn from their experience and apply this learning to future projects. Traditional waterfall driven projects normally hold these meetings after a release or some other significant event. Following the original description (Norman Kerth ” Project Retrospectives: A Handbook for Team Reviews“) these were 3 day offsite events.

Agile Retrospectives/Reflection?

Agile retrospectives differ from Kerth’s approach by being held at the end of every iteration and by being short thirty minutes to two hours (they tend to get shorter over time). They involve the team, management and the product owner. The scope includes: Reflect on the product: did the Product Owner get what they wanted, etc.; Reflect on the Team’s Progress: What did we accomplish? Was it what we set out to do; Reflect on the Team’s Process: What worked well and what worked poorly?

What Retrospectives aren’t

Blame fests/Witch-hunts – Retrospectives aren’t about who did what wrong – they’re about what the team can do to improve

Many of ideas for this description were taken from Alistar Cockburn’s “Agile Software Development: The Cooperative Game (2nd Edition)

If you enjoyed this post, subscribe to my RSS Feed