Can agile be effectively used for initial development on a substantial project?

Question

I think the short answer to this question is probably "yes", but I thought I'd ask anyway!

We are in the process of taking a number of existing similar-but-different applications and, as part of what is for us possibly the largest development project ever, attempting to replace them and any subsequent similar-but-different applications with one over-arching super configurable system to rule them all. I fully expect Difficult Second System Syndrome to give us plenty of problems delivering the behemoth in the first place, but I don't want our choice of methodology to make it all worse. We've dabbled in agile, from something mostly waterfall, but this would be our first substantial project using some new improved methodology.

We have Big Plans for the new system, and while it doesn't all have to be delivered at once we will need to implement large chunks of it from the word go to allow anything to be given to the customer that they would even understand. My concern is that, since we're dramatically changing so much at once, there is little scope for small chunks of work that lead to useful/demoable deliverables. I'm worried that, even if we did manage to find a way to do it in small chunks, we'd need those chunks to fit well into a bigger picture, and so would find it hard to develop them in isolation for that reason too. We're aiming for quite a decoupled SOA approach, which is something.

Timescales are tight, and reputations have been staked on delivering on time.

So I'd like to know if, in this case:

• it is advisable to do loads of up-front design, put together the overall shape of the app and get the crucial modules working in one big push (which might be sprintable in terms of keeping track of work, but not so much in terms of feedback and improving design and so on), before maybe going more agile from then on, or...
• try to be as agile as possible from the start - agree things like coding standards, maybe some API basics, general module splits, and then trust to independent coding while bearing flexibility in mind to let the other stuff gradually flesh out; maybe some stuff has to be reworked, but anything that can be better steered during development rather than up-front will benefit, or...
• maybe something else, and...
• any general tips?