Does Scrum create additional overhead for projects where requirements don't change?

Question

I'm reading the Scrum - A Pocket Guide by Gunther Verheyen and it says:

The Chaos report of 2011 by the Standish Group marks a turning point. Extensive research was done in comparing traditional projects with projects that used Agile methods. The report shows that an Agile approach to software development results in a much higher yield, even against the old expectations that software must be delivered on time, on budget and with all the promised scope. The report shows that the Agile projects were three times as successful, and there were three times fewer failed Agile projects compared with traditional projects.

So I have an argument with one of my colleagues who says that for some projects (like medicine/military where the requirements don't change), Agile (and, particularly, Scrum) is overhead with all of the meetings etc and it's more logical to use waterfall, for example.

My point of view is that Scrum should be adopted in such projects because it will make the process more transparent and increase the productivity of a team. I also think that Scrum events won't take much time if it's not needed because we don't need to sit the whole 8 hours in Sprint Planning for 1 month sprint. We can spare 5 minutes just to be sure that we are all on the same page and start working.

So, will Scrum create additional overhead for a project where requirements don't change?

Answer 1

I believe that it's a faulty assumption to say that there are projects where the requirements don't change. Having worked in both the defense industry and the pharmaceutical industry making software, I can tell you that once software ends up in the hands of subject matter experts (either internal or external), there is feedback. Sometimes, this feedback is on the way the requirement was satisfied and in other cases it's actually on the requirements themselves being wrong or incomplete.

Agility is about reducing that feedback cycle and getting working software into someone's hands faster, getting that feedback, and deciding what the next step should be to make sure that what is delivered adds value when the customer decides to accept the software. Even in realms like embedded systems with custom hardware (like you may find in domains like aerospace, automotive, or medical devices), delivering thin slices of functionality quickly to integrate and prototype with can help make sure that the software and hardware system is going to work as intended and in a way that will help the end user.

The reduction in the length of the feedback cycle is a huge factor in risk reduction. From the project management perspective, if you fund a project for 2-4 weeks and get regular visibility into progress, that assures you that you are on track. By being able to deliver thin slices of functionality, you incrementally work toward the target state and can begin to forecast when you will get there. If time becomes a constraint, you can descope the lower value functions since the work done first should either be a high value function or an enabler for a high value function. At any point, you can decide if it's worth continuing to fund the effort or go in a different direction and stop a project before it's too late.

Answer 2

The very short answer is that yes, Scrum is by design a more expensive approach, but if you're calling it a project, it almost certainly doesn't matter and in the end will almost certainly always result in a better ROI.

The more complete answer is this:

Generally speaking there are three forms of process control: Defined Process Control, Statistical Process Control, and Emperical Process Control. Defined Process Control is by far the cheapest. This is possible with frequently-repeatable work has been refined over time to find the "best" way to do the work. CI/CD in software development falls into this category. You don't want variation in your build process so you standardize the process, adjust until you're happy with it, then automate it. That automated process is obviously far less expensive to run through than manually fighting through a deploy.

Statistical Process Control is the next least expensive, but it accounts for variations in a known process. Medical procedures that go according to plan fall into this category. I don't want to reinvent a bypass surgery each time. I follow the basic process and adjust for variation. This has a relatively low cognitive load and a fairly high rate of success.

Next is Empirical Process Control, which is by far the most expensive because you have to discover the process as you go. Learning is incredibly high, but at the price of productivity and efficiency. However, nearly all project work requires this because few project have been done before. There are, of course, exceptions. Setting up a large active directory environment is more Statistical because you work from some tried-and-true instructions that you deviate from slightly as circumstances require. But unless you're project is to do the exact work that has been done before, it almost certainly requires Emperical Process Control.

To bring it back to Scrum, Scrum is designed to solve problems with Emperical Process control. Therefor, yes, it has more overhead than other approaches. However, since most projects require this approach, it's a moot argument.

To the counterpoint about medicine and military projects, it sounds like flawed logic. If you are fulfilling an order for 500 airplanes, then yes, you are recreating something exactly and Scrum is probably not beneficial. If you are building a new plane and your requirements never change, I wouldn't fly that plane.

Answer 3

Sure, if you have a project where you have crystal clear requirements up front, then you could waterfallishly dump them in front of developers and come back two years later to meet the software of your dreams.

But the vast majority of software projects is not like this.

Usually, the customer doesn't know what they need. They are unable to provide complete and specific requirements. Iterative approaches help here: build a small thing, then ask the customer for feedback. Yes, this "wastes" time on demos and planning the next iteration. But building the wrong thing for one sprint and then quickly correcting the requirements is a lot better than building the wrong thing for the entirety of the project. I.e. while requirements up front may allow for more efficient development, iterative approaches will be more effective.

Developers must understand the requirements correctly if they are to build useful software. What is a good way to discover misunderstandings before it is too late? Again, iterative approaches can help. But it is also important that developers themselves collaborate with the customer instead of only getting filtered information through a requirements document author.

Finally, the world doesn't stand still during the project. External systems change, priorities change, people change. Pretending that the requirements of a software project won't change is a bad idea, except for short projects.

All of these process-level benefits miss the big day to day advantage of agile approaches: if done correctly, agile makes everyone happier. The biggest one of these is that agile techniques focus on providing real value over short time frames. That brings visibility into the development process, gives stakeholders a reasonable level of control over the project, and is much more motivating than working towards a distant goal. Related to this is the idea that agile teams will be largely self-organizing. Feeling in control over their day to day work makes people feel valued, and therefore more likely to give their best.

Your colleague is not wrong that waterfall style projects can have their place. And you are not wrong that some agile-ish practices can be time-wasting rituals. But it is completely foolish to ignore the benefits of agile and iterative approaches, especially better risk management and respect for individuals. These are things you want in every project. If necessary a team can try to implement some of this internally, but processes work better when everyone is on board.

Answer 4

I think this may well be paraphrasing what @Cort Ammon is saying, but here is my take:

The external requirements (describing the "deliverables") aren't the only requirements in a project. Even if the external requirements don't change, the "internal" requirements will change, or need to be allowed to change, as you work. Developers will discover obstacles or problems with an approach, and this will affect the work of the other people in the team. A daily stand-up will keep everyone up to date with these internal changes.

Answer 5

Consider that:

• Even with fixed functional requirements you need to traslate them into technical requirements. And this may be better done by iterations. You may discover better ways to solve the problem at the middle of the project.

• Some requierements may be too generic or ambiguous: "be easy to use", "be secure". It is hard to analyze he security or usability of a system that it is not finished. Some may have hidden implications or may not be well understood.

• Some requirements may be improved. Responding in 200ms may be good but 100 may be better. You may target the best possible result but sacrifice it if needed during the project.

• You may discover some hidden requierement that won't be written on the contract but may change the project from failure to success. Even if you deliver the project the client may not be happy. May be they even need to change the contract to add (and charge) for new features that you may design in the project cheaper in the early stages.

• You may discover that you can't fullfill your requirements in the given time. Is not as if software projects never got late. So delivering the best value will allow you to renegociate what features to drop.

• Delivering something sooner will help integration and will show that this project can deliver results.

Answer 6

One can make the argument that if all requirements are laid out perfectly, then there exists a top down approach which achieves those requirements as fast as possible. However, if these are good requirements, they tell you what to make, not how to make it. If they tell you how to make it, I'd choose to call it "work instructions" instead of "requirements" and we'd be discussing a different sort of problem.

Accordingly, there is always a process of developing the "how" which is internal to the company or team implementing the requirements. Empirically speaking, we rely strongly on a hierarchical approach where a team of designers design the high level system to meet those requirements, and then use the specifics of that high level system to provide "requirements" to smaller teams which flesh our the details.

In the waterfall process, this can be seen in the one-way arrow between design and implementation. However, these requirements are not set in stone, like the customer provided ones were. These are internally defined and have room for the iterative process. In practice, we find designers either put considerable margin in the process to account for this lack of iteration or seek an iterative process.

SCRUM, and many other related agile methods, simply provide a rigorous framework within which to do this iterative process. A trademark of the agile approaches is that they consider optimizing this iterative pattern to be the core of the process, rather than focusing on the outer layer of hard requirements. As others have mentioned, actual fixed requirements are rare, but even in their presence, SCRUM uses the iterative approach as a methodology to control the contractual approach it fits inside of.

Whether it succeeds at doing this is a matter of open debate. Others have provided many metrics to this end. I will merely note that it is up to the strength of the leadership to make sure the iterations which occur below them dovetail correctly into the contractual system above. This is true with any approach to development, but it is more visible in agile approaches because we have been raised to assume the more top down approach is "normal" and trained leaders as such.