Geeks With Blogs

News

Dylan Smith ALM / Architecture / TFS
Today I was involved in some discussion of how we can improve our estimating procedures. Being a believer in agile techniques, my approach to estimating is quite a bit different than the traditional approach to estimation.

Traditional estimation involves breaking down the work into tasks, then assigning an estimate of effort in hours/days/etc to each task.  This is then combined into a gantt chart or something similar to create a schedule of the work to be done.

Agile estimation doesn't do estimates at the task level, but rather focuses on the throughput of the team over a fixed period of time.  If the team uses an iterative approach to development the iteration length is a natural time-box to use to measure throughput/velocity, but even if a non-iterative approach is used an arbitrary time-box works just as well (eg. 1 month).  The work to be done also needs to be able to be broken down into "units" of work.  You will want to use something that is easy to break it down into with a minimum of effort/analysis.  Some common units of work are features, bugs, use cases, user stories, etc.  Something such as classes, lines of code, # of methods are typically poor choices as they require a non-trivial amount of effort/analysis to come up with up-front, and even then are typically wildly inaccurate.

Once you've decided on the time period to use, and the unit of work metric to use the only estimating you need to do is to estimate the "velocity" of your team in units of work / time period.  For example, if the team is responsible for fixing defects you would estimate in terms of how many defects per month you can resolve.  Once you have been doing this for a number of months you can simply use the historical velocities from previous months to figure out the average velocity and std deviation to allow you to make an estimate along the lines of "there is a 90% chance we can resolve 25+ defects in the next month".  Or you can make an estimate something like "there are currently 100 defects in the backlog, at our estimated velocity there is a 90% chance we can finish all 100 in 4 months time".  Another common scenario is if you have a target deadline, say a major release in 2 months, you are able to make a statement to the effect "in the 2 months left before the release we estimate - with 90% confidence - that we can resolve 50 of the 100 defects in the backlog, we'll need the stakeholders to prioritize the defects and identify the 50 they wish to make it into the next release".

A common reaction is to think that some defects/features take more time than others, how can such an broad estimation possibly be accurate enough?  But in practice if you are calculating velocity on a batch of work units (eg. The amount of defects resolved by a 3 person team over a month) then the variability tends to average out (the law of large numbers), with the especially long work units being offset by the shorter work units, and if you were to measure estimated velocity (from historical data) against actual velocity you'd find that it is much more accurate than if you were to measure estimated time per task against actual time per task (I believe that studies like this have been done, if anybody has a link to something like that please leave a comment).

There are lots of other benefits to this approach, one is that it provides a concrete source of feedback/input to assess schedule impact - being the velocity measurement at the end of every time period.  For instance, if you have 100 defects in the backlog and have planned on resolving them all over 4 months (based on historical/estimated velocity), at the end of the first month you will have a new velocity metric that you can use to adjust your plan if necessary.  If the measured velocity at the end of the first month is only 10 defects/month then as a PM you will have to take some action to either adjust the plan out to 10 months instead of 4, or do some investigation as to why the velocity has dropped off so drastically and remove some of the impediments to the team to allow them to regain their historical 25 defects/month velocity (and then adjust the schedule as necessary).  This turns out to be a much more reliable process for updating the schedule/plan than relying on developers to provide feedback when they realize an estimate won't be met.  Developers tend to be overly optimistic, and even if they realize that they are 4 days into a 6 day task and not even halfway done yet, rather than having to tell the PM to adjust the estimate, they tend to try to work harder/longer/smarter to squeeze the rest of the work into the remaining 2 days.  This typically results in them waiting until they are almost at the 6 day mark before finally "admitting defeat" and informing the project manager that the estimate won't be met.  More importantly is the fact that the schedule estimates can only really be updated by the developers once they actually start work on the tasks, so after the first month of work you only have estimate updates on 25% (or less) of the tasks, and usually the other 75% of the estimates are left in tact since they haven't been started yet (a strong PM may notice a trend and extrapolate that trend out to adjust the estimates on the other 75% of the work, but in my experience that doesn't happen too often).  Whereas with the velocity approach, it's very intuitive that if you notice your velocity is only 10 vs the 25 estimated this month it makes sense to expect that it will remain at 10 for the upcoming months (unless something is done to affect it, eg. Add more resources, remove distractions, etc).  Then the replanning effort is a simple division of the # of work units divided by the expected velocity, and voila you have your expected project length.

In my experience that is really the information that is needed to perform planning activities, and estimating at the individual task level is usually inaccurate, time-consuming, frustrating, and usually totally unnecessary.

Comments on this post: Agile Estimating