Dealing with failed sprints and deadlines

Question

Many Scrum books and articles say that a failed sprint (when the team fails to complete some features from the Sprint Backlog) is not something that bad, it happens from time to time, and it can actually be useful if the team learns from their mistakes and improves something in the following sprints. And the team should not be punished for not completing the work they committed to.

This looks great from the developer's point of view, however, let's say we have a software company "Scrum-Addicts LLC" developing something for serious clients ("Money-Bags Corporation"):

1. Scrum-Addicts managers suggest making a piece of software for Money-Bags
2. They agree on a list of features, and Money-Bags asks to provide a shipping date
3. Scrum-Addicts managers consult their scrum team, and the team says it will take 3 week-long sprints to complete all of the features
4. Scrum-Addicts manager adds 1 week to be safe, promises to ship the software in 1 month and signs a contract with Money-Bags
5. After 4 sprints (shipping deadline) Scrum team can only deliver 80% of features (because of inexperience with the new system, the need to fix critical bugs in previous features in production environment, etc...)
6. As Scrum suggests, at this point, the product is potentially shippable, but Money-Bags needs 100% of features, as mentioned in the contract. So they break the contract and pay nothing.
7. Scrum-Addicts is on the brink of bankruptcy because they got no money from Money-Bags, and the investors were disappointed with the results and are unwilling to help the company any more.

Obviously, no software company wants to be in Scrum-Addicts' shoes. What I fail to understand about Agile and Scrum is how they suggest teams should deal with planning and deadlines to avoid the situation described above. So, to summarize, I have 2 questions:

Who is to Blame?

1. Managers, because it's their job to do the proper planning
2. The team, because they committed to doing more work than they could
3. Someone else

What Is to Be Done?

1. The managers should move the deadline 2x (or 3x) times later than the original team's estimate.
2. Team members should be encouraged to do all the work they committed to no matter what (by issuing penalties for failed sprints)
3. The team should drop Scrum because it doesn't fit the company deadline policy
4. We should all drop software development and join a monastery
5. ???

Answer 1

I see several fundamental management issues in your example:

• if a Scrum-Addicts manager signs a "hard-deadline" contract, but adds only a safety margin of 33% in a situation where "a new system is involved", that is pretty reckless.

• the availability of delivering at least x% of the features after one month could have been used to negotiate a contract where the customers pays the money at least partially when he gets only 80% of the features at the deadline. An all-or-nothing contract is something neither the software vendor nor the customer will benefit from - this means not just 0 money for the vendor, but also 0 features for the customer. And an all-or-nothing development methodology like "Waterfall" will only let you write such contracts, an agile approach offers additional possibilities.

• looking at the results of the first one or two sprints should have made obvious to the manager that the team cannot meet the deadline. So he should have taken earlier actions, and re-prioritize the remaining tasks and features, or try to re-negotiate with the customer earlier. For example, the manager could have tried to downsize the scope of some of the remaining features, so the team could have delivered all features mentioned in the contract, but each of them in a reduced scope.

If a task turns out to take longer than you thought, no development methodology will save you from that. But an agile approach like Scrum gives management more opportunities to control what happens in that situation. If they don't make use of those opportunities, it is clearly their fault, not the team's, not the fault of "Scrum", and not the customer's fault because "he does not accept agility".

Answer 2

One of the value statements of the "Manifesto for Agile Software Development" is :

Customer collaboration over contract negotiation

The fact that Scrum-Addicts LLC negotiated a contract instead of establishing a collaboration with a customer makes me question their agility.

One thing is clear : Agility needs to be accepted by EVERYONE. Agility is not only for developers. Managers and customers also need to accept the values of Agile Manifesto. If customers don't accept agility and still require rigid contracts and minimal collaboration, then either don't use agile or find better customers.

It is customer's fault they are locked in their contract bubble with deadline-driven development.

Answer 3

Who is to blame?

Managers, legal dept, accountants - take your pick...

I know the example is somewhat contrived but the fact that the company could walk away without paying a dime if they weren't 100% satisfied should have rung immediate alarm bells as should mixing waterfall and agile thinking.

Customers want to have their cake and eat it - they're happy to accept waterfall, mini-waterfall, agile, la-la-land as long as they get product X for $Y by date Z.

Agile development absolutely requires the development team and customer to be on the same page from a methodology point of view. Thinking differences in culture will just come out in the wash is wishful thinking.

Answer 4

IT projects deal with unknowns; some of these unknowns are even unknown unknowns. What does that mean?

Take for example a toy bridge for your model railroad. There are all parameters known to you:

• You know how big the valley is

• You know the material of the mountains, their height, stability etc.

• You know how much material you need

• You know from earlier "projects" how long it took you to build similar things

There are many variables involved which influence your calculation of investing spare time and money. But you could say without a thought, whether it is finished next weekend.

Take the example a step further:

• Say you do not build the bridge for your own model railroad, instead you build it for a complete stranger: your job is to build a railroad bridge between two mountains

• Say you have near to no information in advance of the model landscape

• The information about the landscape is, that is has two mountains, which seem not too big

• The consistency of the mountain is between rock and jelly

• Total cost has a max of 10$

• The workplace is complete dark and there is no chance of light: you only have a box of 8 matches

• The deadline is 3 hours

That would be the analogy to an IT project. You have experience in building bridges and it is easy to walk on known terrain. What it makes hard is the darkness. There are many things you can hardly predict: The measures of the mountains are only known after poking some time in the dark. So is the consistency of the mountains. From that you could make estimates on how long it will take you and how much it will cost. Here the unknowns are things, which you do not know on the beginning of the project like the concrete terrain etc. But there are things you can not foresee, even with the most experience and most conservative estimations. These things are the unknown unknowns which bear a bit of chaos.

Every IT company should know that. They have to deal with project risk.

1) There are several ways to minimize the (financial) risk: the deal could include tha the customer pays for every working increment. So after increment 1 is delivered, a partial rate has to be paid. As long as Scrum-Addicts LLC delivers, there is minimal financial risk. The more finegrained sprint goals are planned, the lower the total risk of every sprint. That means, if Money-Bags Corporation received 80% of the contract they should at least paid 80% of the contract value. If they refused to pay after a failed sprint the risk isn't as high as the refuse to pay 100%.

2) Scrum-Addicts LLC has a problem with their developers

Scrum-Addicts managers consult their scrum team, and the team says it will take 3 week-long sprints to complete all of the features Scrum-Addicts manager adds 1 week to be safe, promises to ship the software in 1 month and signs a contract with Money-Bags

That suggests, that a) the developers aren't experienced with scrum or b) they are doing scrum wrong The longer teams work with scrum, the better are their estimates. If the teams makes estimations and the manager adds a "buffer" as "safety", the manager seems to know better than the team, which is a bad sign. If you have an experienced team, there is no need for a "managerbuffer", the team included that already in the estimation. The idea is, the more sprints the team has worked together the more the team knows its strengths and weaknesses and has some metrics to make realistic estimates. Of course there are - as already mentioned - the unknown unknowns which tend to make estimates hard; or at least imprecise. But in the long run, the estimates should get better and better.

Who is to Blame?

1) Management

As said above: there is clearly a failure in risk management. If the company is on the brink of bankruptcy, the company deserves it. If you work at such a company: leave!

2) The Team

Even if management is utterly stupid, the team should have spoken up against such a project. A good manager should know the risks; but a good developer should point out risks. And most of all: The team should report early if something fails.

What Is to Be Done?

Now: Take Money-Bags to court

For the future: Do not make such contracts

Scrum is not to blame for management failure. Scrum was developed based on the experience of many IT projects failing. It can not prevent failure, nor can it cure incompetence of teams or management. The basic idea is:

• to structure communication ways (who talks to whom when about what)

• to encourage developer reporting failure early

• to divide problems in tasks and subtasks

• to structure time and capacities (who works when on what)

• to distribute the tasks over the time slots

• make the unpredictable a bit more predictable (planning poker)

or overall: to minimize risk.

Scrum is a tool as a hammer is. Whether it is a good tool depends on your knowledge how to use it. But sometimes a screwdriver fits better. It's up to you.

Answer 5

First off, "Who is to blame?" is the wrong question to ask. Assigning blame is fun and all, and will probably make everyone except the blamed person(s) feel relieved (in a "hey, it's not my fault, the boss said so!" sense), but it's not a productive use of your time, and can actually be counterproductive and cause a drop in employee morale.

A better way to look at it is "What caused the delay?". Was it lack of experience in the technology? Critical bugs which were not detected in testing/QA? Lack of testing/QA? Too optimistic in the estimation? Not taking into account the team's not-so-optimistic estimations? Somebody got hit by a bus? Whatever the cause, the next question is "How do we make sure this doesn't happen again?". In some (hopefully rare) cases, the answer for that might be "get rid of so-and-so", but if you start from "I need to punish whoever's responsible", you're unlikely to see the majority of the cases where it is not the right solution.

Within the project, you're already sunk. Deadline's come and gone, you warned the customer as soon as was apparent it was going to slip (because you did do that, right? If not, that's part of the problem), and now it has to be handled however it was spelled out in the contract (it is actually spelled out in the contract, right?). Generally speaking, that should involve negotiating with the customer how you are going to deliver what is missing. Lots of people like to think of a contract as something that cannot be changed, but faced with either a) dropping the contract and not having what you bought, b) suing the company for breach of contract and spending a lot of money in court, and c) negotiating how to get their product with the least amount of trouble possible, most companies choose c.

Looking forward, before quoting a price/deadline to a customer you should analize the risks involved in a slipped deadline or cost overrun (what are possible causes for such a thing? Which causes can you mitigate somehow, and which you can't and just have to plan for), and use that information to help decide on what you are going to promise. If it is a case where it's 100% or nothing, you'll obviously quote higher prices and longer deadlines, because the risk is higher.

You'll notice I didn't talk about Agile in this whole answer. That's because (I'm going to forget about the customer's involvement in Scrum for a second, although it is very, very important) at this point it doesn't really matter. You'll be faced with this problem in Agile, Waterfall, or whatever development process you use. Yes, Agile is supposed to help you manage risks better, by letting you see if they've become actual problems earlier, and involving the customer in the process itself so they are always informed, but it's not a panacea.

Answer 6

Firstly, this is a problem with any development methodology. At least with an iterative development system you have something to show the customer at the end of the deadline, which may be enough to get an extension to complete the product (even if the customer doesn't pay any more!).

There are cases where a deadline is a deadline however, imagine you're writing a game and it absolutely has to be released in time for the Christmas holidays. Getting that wrong has bankrupted many a company!

For agile methods that have to complete a certain amount of features by a certain date, scrum is probably not the best method to use (as I've always found that scrum makes dev go slower and does not allow enough agility to alter the process when needed.

What you need, regardless of methodology, is to set up a backlog of required features to give visibility of progress. Progress on a per-sprint basis is not good enough, you will not know if you're meeting the ultimate target. So a kanban-style methodology would be better: set all the features on the left, and work them through the system to to show progress to completion.

That focusses people's minds on what still needs to be done in a way that Scrum doesn't handle, and allows people other than the dev team to see what remains and whether you're likely to meet the target (and thus manage customer's expectations early, or arrange those overtime bonuses before they're needed).

Scrum is a system that potters on forever, continually defining and refining something. Its simply not suitable for this kind of development. Others can manage this sytle and still keep the iterative development concept, Kanban is one like that, Crystal another. But what is essential to understand is that if you're following Scrum religiously, you're not being agile. Any true Agile system should be able to morph to cope with these particular issues, that's why it was called agile in the first place, its about getting done what needs to be done, and if a fixed deadline is part of that, then you should be factoring that into the way you work.

Answer 7

The development paradigm is out of sync with the contract paradigm. Ideally, the way contracts are written would change, but that's unlikely to actually happen. However, even if you were to drop scrum, you would still have surprises and missed deadlines (only you'd likely be much later because you did all that design up front and it was all wrong!!).

With or without a change to how contracts are written, you ship what you've got working. Then fulfill the contract by eating a cycle of development time in order to finish the features you didn't get done.

Is it good that you failed to deliver everything you promised on the day you promised? No, but your customer will be much happier if you can deliver something that works on time, then deliver the rest of it quickly after than if you're simply running late and don't have anything at all to give them.

Answer 8

Many Scrum books and articles say that a failed sprint (when the team fails to complete some features from the Sprint Backlog) is not something that bad, it happens from time to time, and it can actually be useful if the team learns from their mistakes and improves something in the following sprints. And the team should not be punished for not completing the work they committed to.

The way you "punish" this kind of behavior is by limiting the amount of work the ones who didn't finish can take on next sprint. Chances to work on cool stuff are vanishing. The reward for doing good work is more work.

This looks great from the developer's point of view, however, let's say we have a software company "Scrum-Addicts LLC" developing something for serious clients ("Money-Bags Corporation"):

Scrum-Addicts managers suggest making a piece of software for Money-Bags They agree on a list of features, and Money-Bags asks to provide a shipping date Scrum-Addicts managers consult their scrum team, and the team says it will take 3 week-long sprints to complete all of the features Scrum-Addicts manager adds 1 week to be safe, promises to ship the software in 1 month and signs a contract with Money-Bags After 4 sprints (shipping deadline) Scrum team can only deliver 80% of features (because of inexperience with the new system, the need to fix critical bugs in previous features in production environment, etc...) As Scrum suggests, at this point, the product is potentially shippable, but Money-Bags needs 100% of features, as mentioned in the contract. So they break the contract and pay nothing.

Scrum-Addicts is on the brink of bankruptcy because they got no money from Money-Bags, and the investors were disappointed with the results and are unwilling to help the company any more.

If on Monday I bet you $100 that it will rain on Thursday and it doesn't rain until Friday you'd be right to take my money. If, rather than a chance to gamble, what you want is a weather forecast then we need a contract that lets me give you an updated forecast on Tuesday.

Obviously, no software company wants to be in Scrum-Addicts' shoes. What I fail to understand about Agile and Scrum is how they suggest teams should deal with planning and deadlines to avoid the situation described above.

Think about WHY MB wants to take their ball and go home. MB didn't demand the work be done in a month at the outset. SA promised 100% of critical features in one month and didn't deliver. SA set the deadline not MB. SA even arbitrarily added a week to the deadline. So why is this a deadline?

Occasionally when competing for work software companies give in to the temptation to show off and promise the moon. Professionals carefully establish whether a moon is even required. Which is the more critical need for MoneyBags? 100% of features or a functioning product in a month's time? Do they even know what is truly critical? Is there some upcoming event setting a hard deadline?

If I was Scrum-Addicts negotiating this contract I'd want to know a lot more about Money-Bags business needs and structure the contract to grant as much flexibility as Money-Bags is comfortable with. I'd teach them how the agile process works so they know what to expect from us.

This way rather than expecting everything to be suddenly working perfectly in a month they'd be expecting to be evaluating the first deliverable in 1 to 2 weeks.

So, to summarize, I have 2 questions:

Who is to Blame? Managers, because it's their job to do the proper planning
The team, because they committed to doing more work than they could
Someone else

Anyone could have stopped this travesty before we got a month down the road.

I could go as far as blaming Money-Bags Corp for hiring a team that obviously fraudulently represented a waterfall process as being agile. The contract itself makes it clear this is not agile. Planning to be done in a month doesn't make it agile.

If you insist that it's agile, it's agile with only one sprint that is a month long. Which, yeah, I wouldn't recommend because that's, again, the same thing as waterfall.

What Is to Be Done?

How about agile? Deliver something every sprint? Get feedback before the deadline? Week long sprints? How about renegotiating the draconian contract the very moment you suspect the deadline is in danger rather than hiding and praying? At the very least you can stop wasting time on a doomed project and find a more reasonable customer.

The managers should move the deadline 2x (or 3x) times later than the original team's estimate.

Deadline multipliers are about as useful as setting your watch 15 minutes early so you'll never be late. You can only fool yourself so long before you realize what you're up to.

Early estimates are wrong. Try to capture how wrong. 5 weeks, give or take a few weeks is a simple expression that lets you express how uncertain the completion date really is. Rather than trying to guess accurately, you guess how wild your guess is. Do some real work and get some real data. Then you can start making estimates with a narrower range. One to two weeks is plenty of time to do this.

Team members should be encouraged to do all the work they committed to no matter what (by issuing penalties for failed sprints)

Team members should be encouraged. Failed, committed, or otherwise. Rather than constructing any artificial consequence such as punishments or even bonuses (carrot and stick) studies have shown that people doing creative work such as programming respond best if provided three things: Autonomy, Mastery, and Purpose.

Daniel Pink has a TED talk about this. The talk is about motivation not agile but I easily saw how to map these points to agile:

Autonomy - I want to direct my own life - Let me pick work from the backlog.
Mastery - I want to get better at something that matters - Customer feedback.
Purpose - I want to be part of something larger than myself - A collaborative team.

The team should drop Scrum because it doesn't fit the company deadline policy Scrum can hit a deadline more accurately than waterfall. Given a deadline scrum can meet it. It might meet it with only 1 of 47 features depending on time, feature, and skill but it can meet it.

An agile project can be styled so extremely that every night when the team goes home it's ready to ship. This seems silly unless you think of shipping as asking the customer to test and provide feedback. The sooner that happens the sooner you can make adjustments. This hits every possible deadline. Just not every feature. But it steers you to the features that matter.

We should all drop software development and join a monastery

Right, like locking me up in a room away from real life is gonna make me write LESS code.

I've edited this answer down to size. If you're curious read the edit history.

Answer 9

Everyone has to be agile. Whatever you decide that will be, look like, who does what, how, when, where and why by all parties. Agile customers, management and developers.

You cannot give a shipping date too far into the future. You give an estimate.

Someone needed to manage the client's expectations. The reason you don't worry too much about having a couple of sprints fall behind is because you adjust to prevent the entire project from falling behind. If you come to the conclusion after a sprint or two that you're not going to finish meet the "shipping date", that's when you tell the client.

Now what do you want to do? Get rid of features you don't need or move the date. If you could deliver on time, you would. Don't hesitate to bring bad news.

Who knows, on some projects, you may ship sooner.

You can't be agile if the client doesn't want to.

Answer 10

Goal

I believe the following two "metrics" should be the basis for any business decision:

• is the work profitable (for the client)
• are we working as efficient as possible

These are quite universal. Of course it get's more complicated very fast, for example, work being profitable is about the product doing the right thing, the user being able to use the product, the product being marketed correctly etc. - for a lot of these "Scrum-Addicts LLC" is not bearing responsibility.

Issue

The contract is not focusing on the goals outlined above. There's an "all or nothing" clause - get everything done and be paid, or get nothing done and don't be paid. However, this does not directly relate to value being created. Another downside which follows: now we need to spend time & money assure and verify that the contract is being followed. Why on earth would we want to spend this money? How does making sure a contract is fulfilled help when the requirements have changed in the meantime and we've found out that the ordered piece of software is not creating any value? There's just more money going down the drain! Now, of course, there's a reason for this behavior:

• Culturally we are used to shop for things like that. We expect shop for software as we would for a car: pick a configuration, be given a price and deadline, and be very unhappy if those two aren't met.
• we want to offload risk and accountability
• we want stability, it helps with planning and makes us feel better (and also our client, which is an important aspect!)

At the end of the day, we'll have to choose a compromise which will allow us to satisfy our goals as good as possible.

This is how it should work

• have a contract for services & work instead of for a product
  • needs to be terminable on relatively short notice
• work closely together to ensure optimal efficiency
• involve all the necessary parties, both from "Scrum-Addits LLC" and "Money-Bags Corporation" - a "single point of contact" tunneling all the information is not going to work here

well i basically just said "be agile". Now here's why:

• process & contract are optimized to spend as much money on the Goal as possible
• you'll have to trust the contractor to do his job, and not need to invest a lot of money verifying that he's up to the job.
• the ability of suing your contractor if your expectations / contract isn't met usually doesn't help, because doing so will cost more than just dropping it. Some of the main concerns here are time-to-market. You most likely will lose way more money/business by going to court than you'll get.
  • at the end of the day you'll just have to bear some risks yourself.