Why your boss should not be upset by high-uncertainty estimates in software

When we do detailed software planning, I almost always give a time range for each item. (I prefer time estimates rather than story points.) After all, managing cone of uncertainty is an integral aspect of managing your backlog.

And too often I hear back from the management/business development/marketing: we need a precise number. If you tell us that all user stories will take 1–5 days. And we have 500 user stories there. Assuming 200 business days in a year, it means that for the team of five the estimate for the whole project is 100–500 days, i.e. somewhere between 0.5 to 2.5 years. We cannot tell the client that!

Hence give us more precise estimates on each user story.

Which is a wrong conclusion. The right way, of course, is to stage the project: the intermediate deliverables will provide the value for the client early reduce the cone of uncertainty, a kind of agile 101.

However, your boss/marketing/etc still may haunt you with their intuition that 1–5 days multiplied by 100 turns into 100–500 days. If your superiors know mathematics, you have a chance to get them off your back:

The estimates are probabilistic by nature. When you give a range of 1–5 days, you essentially are saying: the item is unlikely to take less than a day or longer than 5 days.

The actual time the work is going to take is a random variable. Assume without a proof it is normally distributed just to illustrate the principle. Assume that we have 100 user stories and each user story is estimated to take 1–5 days. How long will it take one engineer to complete all of them?

The sum of N normal distributions is normally distributed with its mean equal to the sum of means of those distributions and variance equal to sum of variances. It means that the standard deviation of the resulting normal distribution will be a square root of sum of variances. (See e.g. https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables)

If all your random variables have the same standard deviation std, then the standard deviation of their sum will be square root of N multiplied by std.

The average in our example is 100*(5+1)/2=300, i.e. 1.5 years. But how uncertain is this estimate?

For simplicity (it is easy to generalise), assume you gave each user story a range (confidence interval) of one standard deviation, i.e. for the 1–5 day standard deviation it is 2.

Thus, we have N=100 and std=2. Square root of N is 10 and thus the standard deviation for our whole project is 2*10=20 days. Therefore, even though the time estimate for each user story is pretty uncertain, it results in the estimate for the whole project of 280–320 days, which is not so scary at all, just plus-minus a month.

Nevertheless, why not just follow your boss’s requirements and give her/him narrower estimates? When one gives a range, it forces bracketing the work by thinking: “this work is unlikely to take less than X days or longer than Y days”. A single-number estimate does not impose such a mental discipline. Therefore, it is less clear where in the range your estimate sits and hence it is likely to end up less accurate.

Last but not least, this note is about the estimate range. However, there always is bias in the time estimates, i.e. in our example, the average of 3 days per user story may be skewed. Due to the business pressure and human nature, that bias is usually (often way too) optimistic.

Estimating and managing the bias is the whole different story. There are many techniques for it. To mention a few:

• continuously measure project velocity (burndown charts, individual estimate coefficients, etc)
• (almost) never do detailed planning and estimation beyond 1–2 sprints
• never have items in detailed planning with estimates greater than 3–4 days; I prefer 3 days (otherwise, they either need to be decomposed, or, if decomposition is hard, it means that early uncertainty study needs to become a separate item)
• stage the project so that CoU (cone of uncertainty) for some areas in the project collapses to zero either continuously or at some checkpoints (e.g. milestones)
• if CoU for the whole project does not contract sprint to sprint, something is wrong