Managing the migration of breaking database changes to a database shared by old version of the same application

Question

One of my goals is to be able to deploy a new version of a web application that runs side by side the old version. The catch is that everything shares a database. A database that in the new version tends to include significant refactoring to database tables. I would like to be rollout the new version of the application to users over time and to be able to switch them back to the old version if I need to.

Oren had a good post setting up the issue, but it ended with:

"We are still in somewhat muddy water with regards to deploying to production with regards to changes that affects the entire system, to wit, breaking database changes. I am going to discuss that in the next installment, this one got just a tad out of hand, I am afraid."

The follow-on post never came ;-). How would you go about managing the migration of breaking database changes to a database shared by old version of the same application. How would you keep the data synced up?

Answer 1

If the old version has to be maintained, the changes simply can't be breaking. That also helps when deploying a new version of a web app - if you need to roll back, it really helps if you can leave the database as it is.

Obviously this comes with significant architectural handicaps, and you will almost certainly end up with a database which shows its lineage, so to speak - but the deployment benefits are usually worth the headaches, in my experience.

It helps if you have a solid collection of integration tests for each old version involved . You should be able to run them against your migrated test database for every version which is still deemed to be "possibly live" - which may well be "every version ever" in some cases. If you're able to control deployment reasonably strictly you may get away with only having compatibility for three or four versions - in which case you can plan phasing out obsolete tables/columns etc if there's a real need. Just bear in mind the complexity of such planning against the benefits accrued.

Answer 2

Read Scott Ambler's book "Refactoring Databases"; take with a pinch of salt, but there are quite a lot of good ideas in there.

The details of the solutions available depend on the DBMS you use. However, you can do things like:

• create a new table (or several new tables) for the new design
• create a view with the old table name that collects data from the new table(s)
• create 'instead of' triggers on the view to update the new tables instead of the view

In some circumstances, you don't need a new table - you may just need triggers.

Answer 3

Assuming only 2 versions of your client, I'd only keep one copy of the data in the new tables.

You can maintain the contract between the old and new apps behind views on top of the new tables. Use before/instead of triggers to handle writes into the "old" views that actually write into the new tables.

You are maintaining 2 versions of code and must still develop your old app but it is unavoidable.

This way, there are no synchronisation issues, effectively you'd have to deal with replication conflicts between "old" and "new" schemas.

More than 2 versions becomes complicated as mentioned...

Answer 4

First, I would like to say that this problem is very hard and you might not find a complete answer.

Lately I've been involved in maintaining a legacy line of business application, which might soon evolve to a new version. Maintenance includes solving bugs, optimization of old code and new features, that sometimes cannot fit easily in the current application architecture. The main problem with our application is that it was poorly documented, there is no trace of changes and we are basically the 5th rotation team working on this project (we are fairly new to it).

Leaving the outer details on the side (code, layers, etc), I will try to explain a little how we are currently managing the database changes.

We have at this moment two rules that we are trying to follow:

1. First, is that old code (sql, stored procs, function, etc) works as is and should be kept as is, without modifying too much unless there is the case (bug or feature change), and of course, try to document it as much as possible (especially the problems like: "WTF!, why did he do that instead of that?").

2. Second is that every new feature that comes in should use the best practices known at this moment, and modify the old database structure as little as it can. This would introduce some database refactoring options like using editable views on top of the old structure, introducing new extension tables for already existing ones, normalizing the structure and providing the older structure through views, etc.

Also, we are trying to write as many unit tests as we can provided the business analysts are working side by side and documenting the business rules.

Database refactoring is a very complex field to be answered in a short answer. There are a lot of books that answer all your problems, one http://databaserefactoring.com/ being pointed in one of the answers.

Later Edit: Hopefully the second rule will also answer the handling of breaking changes.