A Note on Agile Estimation
When it comes to software development a common question that always arises is “how do we estimate our work?” especially in an agile environment.
Estimating is one of the most widely debated areas of technology and being “agile” – everyone has their own opinion and approach, so there is never a “one size fits all” solution. When teams embark on their agile journey, often starting with Scrum, there is a natural tendency to “point” your work.
The big question is, what is a “point”, and why do we do this?
First, for a bit of a history lesson before the days of agile, every project or product was managed with a waterfall approach to delivery. Process aside, when in waterfall, you often had to estimate project duration and cost, among other things, in order to put together a project plan. These estimates were never easy to derive, particularly because the approach here was to plan an entire project before you did any work, so they were often wrong or required continuous changes.
Fast forward many years to when agile and Scrum became a thing, and people weren’t ready to let go of some of these old habits. There was a desire to hold onto estimation as a way to answer the question “when will it be done” however what many proponents of estimation failed to realize was that humans are not capable of making absolute estimates, let alone relative estimates.
In certain circumstances, absolute estimates make sense. For instance, if you’re building a house or painting a room, you generally know how long it will take to gather your materials, do your prep, and complete the task. For software and data engineering, there are a number of variables that impact how long something takes – whether it’s skillset, experience, dependencies, risks, technology and so much more.
This was the reasoning behind the “relative” estimation approach commonly adopted by those that practice Scrum.
When we talk about relative estimation in agile software development, there are two primary approaches
- Pointing: Using a numerical value to denote the estimate
- Sizing: Using a descriptor to denote the estimate (or size)
Starting with the latter, sizing is a quick approach to estimating the overall size of a story, bug, or task as a means to see how large it is. Most commonly, teams that practice this form of estimation or sizing will use T-Shirt sizes (e.g. Small, Medium, Large, X-Large) and have a discussion about the size. Many teams will say that if an item is >Large, it should be broken down into smaller pieces.
On the flip side, pointing as a way of estimating entails applying a point value to a unit of work to denote the estimate. Unfortunately, it is easy to fall into the trap that points are just another way to represent time, and so these estimates are often incorrect. We already know that humans are not good at absolute estimates such as hours, days, or weeks, so why would disguising it as a relative estimate change that fact?
Story Points are actually intended to be an estimate of the effort thought to be needed for a particular work item across three dimensions.
- The amount of work to be done
- The complexity of the work
- The risk or uncertainty in doing the work
It’s a common thought that these dimensions will all correlate to time anyways, such that if an item is more complex and has more “code” to be changed, then it will take longer but, research has shown that this isn’t always the case. In many instances, it has been proven that there is no correlation, and 1 point stories, for instance, will often take longer to complete than their 5 point counterparts.
With that said, there is immense value in the discussion that takes place when a team points work as a team. It often leads to further discussion about what the work is, what the goal is, what it entails from a technical or functional perspective, and much more. In a planning or refinement session, a team would typically perform some sort of planning poker for everyone to cast a vote, and then the team would talk through the logic for their vote, particularly the individuals that are not in line with the majority. This very conversation is what can make estimation valuable, as it can uncover things the other team members might not have thought of.
Estimation loses its value when points are viewed as time, or when the team is working in silos. The idea is that we plan and estimate as a team so that other members of the team can work on and contribute to similar goals or units of work.
As a team evolves, it is believed that they will get better at estimating the work. What really happens is that they get better at having the conversation that takes place, and their leadership gets better at anticipating the team’s needs.
In all cases, this estimation aids in team predictability across the board, and after many sprints of thoughtful estimation, a team can use this as a means to forecast when work may be done. If there ever comes a time where the team is asked to increase their velocity though, it is easy to game the system, inflate the estimates, and then you’re right back where you started.
So, now that you have a better understanding of what estimation and story pointing is all about, what might estimation look like in practice?
Scenario A:
A team has 5 stories on virtual sticky notes. The team identifies the lowest effort story, and the highest effort story, and puts them on a whiteboard. The lowest effort story becomes a 1, the highest becomes an 8. The team then reviews the remaining stories, and places them on the board in the spots 1, 2, 3, 5, 8.
Scenario B:
A team has 50 stories on their backlog, and they need to plan their next sprint. The product owner orders the stories, but the team has yet to estimate the work. Starting with the highest priority, the team reviews it, and then they do simple planning poker. Each person casts their vote for the first story; 5 people on the team say it’s a 3 pointer, 1 person says it’s a 1 point, and the remaining 2 people say it’s 5 points.
A discussion takes place. The person who said it’s only 1 point is very experienced, and they are the one doing the work, it’s very low effort for them. The people who said it is 5 points have never done it, they are unfamiliar with the work, and would need to do some research to learn how to do it. The team quickly realizes that they need to work as a team, and help each other learn a bit more about the product they are doing. While the most senior developer still plans on taking on the work, they decide that they want to teach one of the more junior engineers to do it. They agree that the complexity and amount of work will increase as a result, so the agreed upon point value is 3.
Scenario C:
A team has 50 stories on their backlog, all which have been prioritized by the product manager. Rather than estimating the work, the team looks back on their historical delivery to learn that they can usually deliver 11 stories every two weeks. They take the top 11 stories, review them, and determine if they are sized appropriately using T-Shirt sizing. They determine that two of them can be made smaller, but still deliver something of value, so they make them smaller and commit to those stories for the sprint. The team swarms as much as possible, while focusing on as few stories in progress as they can. They grow their skills as a team, reducing potential bottlenecks, and building camaraderie getting more done in the long term.