Should I use scrum for big projects? [closed]

Question

Closed. This question is opinion-based. It is not currently accepting answers.

---

Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.

Closed 4 years ago.

Improve this question

I've been working as a programmer on a project designed for generic software for gas stations (to be redistributed for many customers) for 18 months. The project is big. Today we have around 150 tables. We haven't used a specifc approuch, it was not well managed.

The person table has today around 70 columns, but 15 months ago it had around 30 columns. These new fields emerged in order to integrate with other modules such as sales, financial and accounting. Also many fields were created then deleted.

As a result, we had many refactoring and rework. The project never gets ready because there are always new requirements emerging.

Here's my doubt: if we had used a usual approach of specification, we would had interviews, a requirements document, activity, sequence and class diagrams, so we would know since the begin that the "person" table would need 70 fields, then we had avoided a lot of refactoring.

Could scrum help in this project? I have a feeling that in this case scrum would end up in a lot of refactoring as well.

I'm only a programmer, not a project manager. I'm wondering how it should have been done: with scrum or with big design up front.

Edit

Just to complement the end of this story. Eight months later I asked this question, after put the project in production in some "test costumers" the project officialy failed. The product owner decided to abandon the project. It got hard to fix problems and a lot of perfomance problems occured.

Answer 1

It looks like you have churned your way in an uncontrolled development process to create a never-ending development system. This occurs in agile systems too.

The root problem is a lack of requirements, and while your solution might seem to be to use an agile methodology to fix this (as agile is designed around changing requirements) it would not solve the problem.

Even agile methods require a fairly firm starting point. They respond well to changing requirements, but they are just as useless as any other methodology if you start with no requirements. You still have to have a plan that you're heading towards before you start. Agile helps when that goal drifts, it does not help you define that goal.

Now its true that a fixed up-front design is too rigid if you do not know exactly what you are building.

So, I think you're going to have to move from your current 'chaos' methodology to something more organised and will have to implement a fair amount of design and planning. You can try to do this in one go with a heavy methodology, or you can be more flexible with an agile one. What you cannot do is expect any methodology to fix your lack of initial planning.

Incidentally, Kanban sounds more appropriate for your needs. Scrum works best with small teams and projects. Kanban can work with larger projects but it also works with a throughput of work approach, so design changes are continual and not chunked into sprints. I think you'd have more success with that.

Answer 2

18 months, 150 tables and still not in production? Sounds much like a death march for me. The only way you can fix this, if there is any chance to save this now, is to narrow the scope of your project dramatically - at least for your first production release. What you need is proper release planning, small, reachable goals and getting the system to the end user as soon as possible.

And when you have your first release in production, with only a tenth of the requirements implemented, you will have to extend it step-by-step by the next block of requirements, which will cause "refactoring". You will also get feedback, which means bugfixing and changed requirements, which will cause refactoring, too.

Now to you question - will Scrum help? Maybe, maybe not. Scrum is a tool to support iterative development and changing requirements, and focussing on the important things first. Other "Agile" methods do this too, and a not-so-formalized process might handle that also. But as long as you try to bring a monster like this in one "big bang" into production, it does not matter if you use "agile" or "upfront" development, both will fail.

So before thinking about Scrum, first rethink your goals and your release strategy, and then check if Scrum is the right tool for that, not vice versa.

Answer 3

If you can't manage requirements and don't have people capable of implementing requirements properly, SCRUM isn't going to help you (much), and that seems to be the real problem you're facing.
SCRUM can help you better deal with changing requirements than more static project management systems, but it's not the holy grail that will magically make everything work. In fact, unless your people are on board, willing and capable of working with SCRUM, and so is the rest of the organisation, it may end up making things worse.

If you've a table that's grown so much to fit in stuff to link with other systems, I postulate your database design is seriously flawed for example. No amount of SCRUM is going to improve your database design without you including people who are good at database design in your team and not being afraid of those design changes and the changes they will cause to the rest of your system.

Answer 4

Please note that when I wrote this answer, I didn't realise that the system was not in production yet.

They way you describe your product, I don't think that your immediate problem is that of managing requirements, nor your development process. It is that of your system architecture.

You have managed to create a monolith - and a rather large one at that. 150 tables is a lot for one system*. In particular, you mention that you have 40 new fields during the last 15 months just for integrating to external systems. I would seriously consider splitting your system into several autonomous services, probably starting with services for integrating to external systems - but later identifying separate business areas implemented in you monolith, and then perhaps split those concerns into separate services.

If you manage to split this monolith into separate maintainable code bases, you can also split your developers into smaller teams with well-defined responsibilities in specific areas of your business, and you can have a lot of smaller agile teams maintaining their own code base, instead of all hacking in the same code base.

As to why you have arrived at such an architecture, the root cause for your problem, there can be many answers. Maybe it's rooted in you development process, maybe all your developers are only experienced with transactional consistent software, or maybe it's a consequence of how your organization is structured (you are the victim of Conway's Law). I think there's a good chance it is a combination of the latter two.

I don't think that implementing scrum, or being better at managing requirements will help solve your immediate problem, nor the root cause. Adjusting the architecture for the complexity of you system will, and addressing the root cause for why you have build such a system will.

* Some will probably argue that they can maintain a system with 150 tables - or they have maintained much larger systems, but I do believe that most developers will consider this a large number of tables for one system.