Estimation Risk
This technique for communicating the risk of estimates being wrong on a software project was supplied by ThoughtWorker Robin Gibson. Robin is a process improvement guru currently working in Sydney, and keeps this is his kit bag to help bridge the communication gap between business and IT groups.
The data collection is quite simple. Robin collects all of the stories in a agile project into a spreadsheet, where he groups them by functional area. In this case, there are 350 stories in 18 functional areas (the functional are names have been changed for confidentiality reasons). Then every story is characterised as follows:
- It is prioritised as either critical, essential or non-essential
- Its completeness (do we know all of the story details?) is set as complete, incomplete or unknown.
- Its volatility (is it likely to change?) is set as low, medium or high
- Its complexity (how hard is it to build?) is set as simple, standard or complex.
The priority levels can be averaged across a functional area to find the priority of the area overall. The completeness, volatility and complexity together represent the risk that the estimate for that story will be wrong.
This data can now all be charted like this:
Each bubble represents one functional area. The size of the bubble changes depending on how many stories are in that functional area. The highest priority functional areas are on the right-hand side of the graph, and the lowest are on the left. The highest-risk functional areas are at the top of the graph, with the lowest-risk ones at the bottom.
Its now easy to see all of this information at a glance. Any functional areas in the bottom right corner of the graph (high-priority and low-risk) are no-brainers to do. Functional areas in the top left corner of the graph (low-priority and high-risk) might never get done.
But wait, there's more... The whole project can now have its estimates summarised in one simple pie chart for everyone to see. Our example yields the following result:
It can plainly be seen from this chart that only a very small part of the system is unlikely to have its estimates change. Even more startling, a full 19% of the system is highly likely to have the estimates change.
This information can be used in a bunch of different ways, from initiating requirements workshops of architectural spikes (both to help reduce estimation risk) to including appropriate contingency funds in the project budget. Whatever the outcome, sharing this information with your team will mean everyone is better informed.
Comments
Interesting! Now we only need this kind of functionality integrated into Eclipse and other IDE's... ;-)
Mats
Posted by: Mats Henricson | March 15, 2004 06:49 PM
I'm wondering how to aggregate the 3 parts of estimation risk, each of which has a 3 point scale, into a single 3 point scale (low, medium, high) overall estimation risk.
Would you do something like what follows? Or something else?
convert to integers like this:
completeness (do we know all of the story details?): (complete = 1, incomplete = 2, unknown = 3).
volatility (is it likely to change?): (low = 1, medium = 2, high = 3).
complexity (how hard is it to build?): (simple = 1, standard = 2, complex = 3).
and then add up the 3 integers for a story, then divide by 3, then map to the closest integer (1 = low, 2 = medium, 3 = high)?
Posted by: Murray Bishop | March 16, 2004 07:34 AM
Love 'ya work Murray. Did you expect it to be anything other than the simplest thing?
Posted by: Marty Andrews | March 16, 2004 08:27 AM
I should have known it would be simple, but I wasn't sure and really wondered. Don't mind me, just thinking out loud.
Posted by: Murray Bishop | March 16, 2004 05:27 PM
Hummmm... looks very close to what we have been doing for quite some time!
The problem with what you describe is to make sure that you ask yourself the right questions to identify the risks. In short, if you have little experience of projects, you will not see where risks may occur. Only people who had learned the hard way will potentially be good at this exercice!
To do so, we have a list of 200 questions to force you to think about anything that could go wrong in a project (put yourself in a paranoid mode for a few hours) and try to foresee if this could happen to you. These questions are classified by topics (Scope, architecture, processs, customer, COTS, Resources, etc...) and you assign them with a risk level (if any: H, M, L) and a potential impact on the project (H, M, L) + the reasons why you think there is a risk (just to make sure you're not answering the questions blindly!). To this, you may start adding "actions" you would eventually take if the risk occurs (or to prenvent it from happening).
From the output of the these 200 or so categorized questions, the excel spreadsheet produces interesting charts (including a "Risk impact on the project" matrix, ).
The good news is, we recently reviewed a risk analysis from an old project and happily found out that quite few "high" risks we identified at the start did occur but that we failed revisiting the list regularly to prevent them... a good "lessons learned"!
Posted by: Hervé Bouvet | May 10, 2004 07:59 PM
Do we really need this sort of documentation?
Posted by: Mark Kamoski | April 1, 2005 11:54 PM
Ghost town?
Posted by: bonder | March 19, 2006 05:39 AM
Great reading, keep up the great posts.
Peace, JiggaDigga
Posted by: JiggaDigga | April 8, 2006 03:12 AM