Suitable architecture for rewrite with integration + migration scenario

Question

Recently we started a project at the company I work at to restructure and rewrite our systems landscape and rescue the future of our children.

We have 3-4 legacy systems which absolutely can not be adapted to new use cases because of the horrific code, but still handle quite a big amount of orders per day via different interfaces and formats like email, xmlrpc, webinterface.

So we thought about writing a new system from scratch based on a completely redesigned domain model. Because of we cant simply switch the old systems off and are a really small team, we came to the conclusion that we need an architecture and approach which allows us the gradually develop the new system and put the parts of it live and easily (read; fast) integrate new interfaces, partners and with the legacy applications and interfaces.

The idea was to completely redesign the whole domain model from scratch, create an Order-service and to use Apache Camel with an OSGi container to mimic the old interfaces routing the orders to the legacy and the new systems and decouple the format processing and transport itself from the new system. Because of the gradual development we would like to chose are more "service centric" architecture which will allow us gradual improvement, reusability and scalability. It all sounds nice on paper, and I read quite a bit about "SOA" but until waited for the hype to die off and the "good parts" to stay, but most of the talk is still on a very abstract and unprecise "technical sales" level it seems.

I find the "Basic/Shared Data Service" approach quite problematic if I have entities with lots of relations like order, with quite a large graph. If I would create an individual service for CRUD operations on Orders and the other entities to base the more abstract ones on, it would be really problematic or impossible to handle ACID, relational integrity or you would have to sacrifice the autonomy of those services and interconnect them, which would make the services quite useless (and possibly quite slow). Or did I understand something wrong?

So my idea was to simply create a "traditional" DAL using nice JPA POJOs, of course with interfaces, and deploy it as a simple, versioned OSGi bundle for the business and process services to use, have a more abstract mapping. Those services would then simply use it and expose their interface to the bus. To serve the rare case of needing to access individual data, like for content enrichment in camel or bulk data import or reporting, an ugly "all entities included service" could be created which would solve the ACID and integrity problem.

So far so good, BUT: how should the WebUI (which is mostly CRUD as I mentioned and thus not really an abstract process) access the data? Directly using the JPA POJOs would make it coupled quite tighly, but creating a mapping and introduce another, almost identical model and use the aforementioned "monster DAL service" doesnt sound too good too.

What do you think? Where is a good balance between sense, elegance and practicability?

I'm sorry that those are several questions and the text is quite long, but I felt it was important to depict the situation we're facing here in a little more detail.

Thank you for your time :)

Answer 1

This isn't so much of an answer as a note wishing you good fortune.

We have a situation that is the same/different form yours, and it's proving to be a very hard proble to deal with effectively.

I think your approach is OK, sounds like you're striking an OK balance.

You can take enterprise integration patterns like those published by Martin Fowler, but whether that will get you to where you want / need to be remains to be seen.

Other options: there's the "sausage" machine: where you push the old system into a system which generates "new" code (in Java or .Net) which becomes your new platform. the good thing is that you now have a codebase in a language you can work with, the bad news is that it will be the biggest most awful pile of spagetti you ever imagined.

Even then it's a huge undertaking. A government agency here spent 2 or 3 years and $5Million getting this done. It wasn't pretty or painless but seems to have worked. If you ask around enough (i.e: not on StackOverflow) you should find people who have dealt with migrating off the platforms your stuck with, get into a conversation with them.