Is it a fallacy to say that system migrations don't suit an agile methodology as the requirements are known up front?

Question

I saw a project manager recently stand up at an Bank's internal staff meeting and say:

We're running this system migration not using an agile methodology. We know the requirements up front so agile isn't required.

I've worked on a few migration projects. The challenge is that you discover requirements along the way that weren't known up front. It's a normal IT project.

I went to a Martin Fowler talk we he was in town and he gave a provocative talk, "Why I will never work on a non-agile project." His points revolved around not being people focused, software projects being genuinely unpredictable and needing to have a delivery focus on working software.

My question is : Is it a fallacy to say that system migrations don't suit an agile methodology as the requirements are known up front?

Answer 1

Yes, it's a fallacy.

What your Manager call: "requirements are known up front" falls into the category of Predictive Planning. "We know what to do, so it just takes to assign tasks to people and stick to the planning".

Plan-driven engineering expects us to come up with a predictive plan that precedes development. The plan lays out the people, resources and timelines for the overall project. Software design is also done up-front, with implementation expected to conform with this design. Success is measured according to how well development follows this plan.

Martin Fowler -Co-author of the Agile Manifesto-

But it doesn't mean that changes are not going to happen.

Agile and Fluency by Martin Fowler - Barcelona 2017

There could be changes in the requirements or just a changes of priorities. There could be so many things that can go wrong or change during the development process. Unless your Manager drives a DeLorean with a flux capacitor, he can not assure you that there won't be changes.

One purpose of Agile is to be able to adapt ourselves to the changes. Whether we expect them or not. Not being able to adapt ourselves to the changes, these may result in opportunity costs to the customer.

Agile plans are a baseline that we use to help us control change. Agile teams plan just as carefully as traditional teams, but the plans are constantly changing to reflect the things we learn during a project. Success is based on value delivered by the software.

Martin Fowler -Co-author of the Agile Manifesto-

There could be other reasons for not practising Agile, but the one given here is not a valid one.

"The only constant in the Universe is change".

Heraclitus

---

Later edit

The industry of the software has tried unsuccessfully to mimic typical production processes from other industries with a longer history.

These industries have refined their production plan over the decades and some of them over the centuries. These came to a point where it's possible for them to reduce the uncertainity of the changes almost to the insignificance. At a cost. They removed from the plan any human influence on the process.

That's is not possible for us to do at the moment. Today's software is still made by people for people.

Whether the process have been refined up to the industrialization or not, nothing escapes from the changes. Overall when the human factor comes into play.

Answer 2

Your question holds a bolder statement than the quoted manager's.

The quoted manager states it is not needed, as if it would require a bigger investment to do an agile project compared to a non-agile project. It sounds like he would not mind going agile, he just believes the extra costs are not worth it. Which raises questions. Perhaps they never did agile before and they are expecting some adjustment costs.

But agile is not just about responding to new or changing requirements so the statement as quoted makes no sense.

Answer 3

Yes. The set of all requirements is never known at the start of a project. By proudly saying they are not using an agile methodology, it's as if they are saying "We reject the chance of having to adjust our plan and will try our hardest to resist changing our plan."

The potential damage this line of thinking will cause depends on the size of the migration.

Answer 4

Depends. There are many types of migration.

In many cases, you will be migrating from one fully functional application to another fully functional application. Since the new application is a fully functional application, the functional requirements are completely known-- after all, somebody built the thing.

The migration effort is primarily around converting the database structures to suit the new application. The goal is a big bang event where the old application is retired and all users are directed to the new application.

In this case, it would be pretty hard to stuff it into an agile or iterative approach. There is only one release. The data have to be converted all at one time, during the go-live window.

So while you may have several dry runs, each of which will produce fallout reports (data that could not be converted), in the end there is no functioning software until the very end. This is counter to the agile idea that you are making small changes in a manner where each release is a functional product with a subset of functionality.

In other cases, you might have two applications running in parallel, and may convert modules one at a time. For example, if you are converting from Clearquest and Subversion to TFS, you might migrate your code in one go, then migrate your defects, then migrate your tasks. You might first work out a federated identity system to allow a user to seamlessly switch between applications, then configure the menu system to point them at different software depending on what has been converted at the current point in time. Arguably this could be approached in an Agile manner.

In yet other cases, you are not really migrating but re-implementing. For example, if you are migrating from a customized application to another application that you just purchased, and a gap analysis has shown you will need to customize the new application too. In this case the project could be Agile, because you are re-implementing features as you go.

I wouldn't say that the statement was fallacious without knowing more about what the project is. At a bank, if they are migrating their core host, it would most certainly be a big bang event, because of the data integrity requirements-- a bank account can't have two master ledgers or else you will run into serious accounting problems. But if they are keeping their host but switching out a couple customer-facing web sites, it could conceivably be done in stages. Then again, it is quite likely that any one of those stages would be much too large to fit into what is typically thought of as an iteration.

Answer 5

If your manager supposes that the migration will be done in steps defined by sets of changes in old and new versions, while both of them will remain working, testable and comparable after every step, it IS agile, even if he does not know it and does not use SCRUM.

If he wants to make all changes in one huge step and after that start to test it, it looks more like a suicide than a fallacy.