Should the team reduce future estimates after becoming competent at a new skill, because estimates were increased while learning?

Question

I have been pushing unit testing lately. This is a new skill for my team. I have had 10+ years experience writing unit tests, but I am basically the only person on the team with any experience with this at all. I have been struggling lately with how to budget for learning these skills. Forcing people (me included) to learn all new skills outside work hours doesn't work. We have families. Work at work. Home at home. We are all allotted training hours each quarter, which is great. However blog posts, YouTube videos and PluralSight tutorials only get you so far.

I got this hair brained idea to increase story points for stories where unit tests are required. This effectively reduces the amount of functionality we can deliver per story point. At the time it felt fine, since we are increasing the total effort. In my mind this increase was justified by the "unknowns" of writing unit tests. I also expect story point estimates to come back down after our team members have become competent at unit testing.

---

I originally got this hair brained idea from another hair brained idea to increase story point estimates for stories that required writing automated end-to-end tests with Selenium. This resulted in features that used to be 1 story exploding into 6+ stories. Story #1 included development and writing a single automated test. This usually turned out to be a 13 point story. As a general rule the team feels comfortable delivering an 8 point story in a 3 week sprint. Anything higher and our confidence goes down exponentially. A 13 point story is worrisome. A 20 point story in one sprint? Yeah, and while we're at it I'd like a pony too.

So that first story would be 13 points, then we would have 4-5 stories estimated at 3 to 5 points each. The smaller stories were literally the effort required to write the automated test, including the addition of any test infrastructure code, like Selenium page models. These tests all verified distinct, testable end user behavior.

Team velocity initially suffered, but eventually went up. Story point estimates never came back down. We continued our story breakdown of a single 13 point story and then a bunch of 3 to 5 point stories to write automated tests.

---

Now we fast forward to my current situation of learning unit testing. The team estimated a story at 13+ story points again, and there is no way to break this story down into anything smaller. For our team, a "story" is basically something an end user can interact with. Pretty general, but if an end user cannot see or interact with it, then it is not a user story.

I requested we do unit tests that require mocking a single method on an interface used to send an e-mail. We create and sent the e-mail using the Postal NuGet package, which makes sending an e-mail no more complicated than rendering a web page with a view model and razor template (our team has extensive experience with ASP.NET MVC).

The unit tests would cover a "service" class invoked when removing people from a business customer account. Anyone who is removed should get an e-mail notification. The new unit tests should cover the fact that e-mails get sent to each person who is removed. They do not need to assert the contents of the e-mail, just that the e-mail gets sent. This involves mocking the IEmailService.Send(Email) method.

This 13 point story makes me nervous. We are half way through our 3 week sprint and I am still getting basic questions about unit testing fundamentals. I'm afraid we are going to miss our goal this sprint, which is why the story got a 13 point estimate. Each time I tried introducing unit tests, even in smaller, simpler stories, the team always gave me a 13+ point estimate. I'm afraid no story is small enough for a single sprint anymore once you factor in development, automated tests and unit tests. This is simply too much for the speed and skill level of this team — a trend I have noticed the entire 4 years I've lead this project. I'm just simply hitting a brick wall.

We do not adjust story points based on who gets assigned the story. To be honest, no single person works on a story anyhow. I've read Where does learning new skills fit into Agile?, but at some point you must utilize the new skill, and this is my conundrum. Since I am the team lead, scrum master, business analyst, graphic designer, BDD practitioner and architect of this project I frequently do not have time to pair program with every person on the team. This large number of responsibilities is not changing any time soon, either.

It seems we must deal with a reduced velocity, or increase the estimates. I've chosen the latter of the two.

After increasing story point estimates in order to learn unit testing, should the team reduce future story point estimates for similar work based on the assumption that the "unknowns" of learning to write unit tests are no longer unknown?

Answer 1

I see several potential problems here.

The whole point of using story points and velocity is to hand-wave away hourly estimates, but ultimately story points must eventually correlate in some way to how long it takes your team to get things done. If your team can complete 30 story points in each three week sprint (without working any overtime), that means each story point takes roughly 4 hours to complete.

In my opinion, the story points and velocity should inform your estimation process, not the other way around. Simply increasing the estimates is not going to work; your team has to figure out how to get things done in a more timely fashion so that the story points and velocity eventually normalize.

If the team estimates 30 story points for a task, but completes it in Week 1 of the sprint and has time to complete 10 more story points in other priorities before the end of the sprint, that's a good problem to have. That isn't, however, the problem you appear to have.

So here are my thoughts, in no particular order.

1. Unit testing with mocks is difficult and expensive. In my experience, it's better to engineer your API so that it doesn't require mocks to test, and you get a better design in the bargain. Consider writing your tests first, so that they inform your API's design and serve as a partial "definition of done."

2. Find a way to increase the granularity of your tasks. Smaller tasks that are easier to complete are also easier to estimate. Twenty story points for a task is too large for a team that's only capable of 30 points per sprint, unless your team is especially disciplined.

3. Let the velocity and story points of the team speak for themselves. If the team consistently estimates more story points per task than it takes, gradually dial the estimates back and make sure there's plenty of work in the backlog to fill the void. If it's taking longer than it should to complete things, stretch the estimates and work on the root causes of the work slowdowns.

4. Pragmatism rules. If the team was consistently producing reliable software before unit testing was introduced, it might be time to re-evaluate your approach. Check your staffing levels; you may need more developers to accommodate the increased workload.

Your velocity and story points are telling you there's a problem. Don't try to re-engineer those metrics; work on the root causes.

---

_{True story: A former boss once told me that the story point system and software development process had become so institutionalized and corrupt at one of his jobs that simple changes like adding a dropdown to a form took three months to complete, because the development team had hijacked the estimation process. Don't let this happen to you.}

Answer 2

Its 2020 you would have thought all developers were onboard the unit test train by now.

In terms of points and estimates, I would say you are getting hung up on details. You know that unit tests will speed up development in the long term and have accepted it as a requirement.

Have a "definition of done" that includes units tests and let the devs estimate the tasks. Don't challenge or worry about their estimates, just keep track of velocity and use that to predict end dates. I'd wager that your stress over estimates pushes them up and eats up time in meetings

I'd also say the points seem a bit big if 8pts = 3weeks. I would recommend 1 week sprints and estimate in days. Let the team set their own targets.

The definition of story might also be part of the problem. "Make the button green on mouse over" can be a story

Answer 3

I'm afraid no story is small enough for a single sprint anymore once you factor in development, automated tests and unit tests.

I think that's the root of the problem - your stories are simply too large. I find it hard to believe that you can't break a 13 point story -- which represents roughly 5 weeks from your entire team -- into three or four smaller stories.

My recommendation would be to challenge the team to write better, smaller stories. With smaller stories will come more accurate estimates. Based on the numbers you gave in your post, I would suggest requiring that no story be larger than 4 points, including time for all testing for that story. If it's bigger than that, break it into two stories.

After increasing story point estimates in order to learn unit testing, should the team reduce future story point estimates for similar work based on the assumption that the "unknowns" of learning to write unit tests are no longer unknown?

The team shouldn't artificially reduce story points. If you were artificially adding extra story points, however, you should stop doing that.

The story points should naturally fall as the team develops skills. Story points should reflect the honest opinion of the team for fully finishing a story, including all testing, documentation, etc. As they become more proficient in testing, the time will naturally decrease.

Answer 4

You write in a comment: I'm legitimately afraid that requiring the devs to write unit tests for a method will take longer than 3 weeks, but might take a work day without unit tests.

Sounds like the team does not want to do unit tests, and thus arbitrarily increases the estimated effort. Use the training budget to do unit test workshops. Convince them to want unit tests before forcing them.

Creating unit tests will increase the complexity of a story. So, the number of points will increase, and less features will be done per sprint. In the beginning the effect will be bigger (although not as big as you experience it).

Because of the unit tests future refactorings will become easier. Release efforts may drop or disappear. Less bugs may show in the release, reducing bug fixing efforts. These safed times can be spend on realizing features, thus increasing velocity.