How can I keep order of refactors using database refactoring software?

Question

I have been trying to use either Liquibase or DBDeploy. I'm more drawn toward Liquibase because of its non-SQL interface (I.E. I can just use JSON or Yaml changesets). However, there is a problem I have with both of these software.

Liquibase Workflow

1. I create a master changelog. All it does is to includeAll a folder which contains small files with small changesets inside.
2. I then create changesets, and prefix them with a number (E.G. a timestamp, or a simple integer like 1, 2, etc).

DBDeploy Workflow

1. I just start making delta sql files, prefixed by the same strategy as number 2 above.

The problem

Well, the problem is so trivial I'm feeling stupid for asking it, but here it goes. Consider this scenario:

1. I make a branch to work on my feature, say, adding orders to the system.
2. My colleague, Bob, makes his own branch to add products to the system.
3. When it comes time to merge, there is no telling whose changesets or delta sqls will run first. This may break the database.

Doesn't this happen to anyone? If so, what's the way to solve it in the PHP land?

Thanks.

Answer 1

I've used a couple of different migration frameworks - Liquibase, Rails, and something from the C# world called Tarantino. Each used a similar strategy, where changes were recorded in separate files. Each was on a small team (<= 5 developers).

Rails is the most dogmatic about how files are named, and is where I have had the most conflicts due to branches. Most of those conflicts were name based rather than logical conflicts in the database.

In projects using Liquibase, we used a master/include pattern, and developers did their work on branches. Because the file names had a 3 digit sequence numbers plus a brief description (i.e. 009-add-customer0index.xml) we did not have name conflicts. We avoided database level conflict mainly by just talking to each other - daily standups, etc.

We had similar experiences using Tarantino, although it just uses a directory full of files for its migrations. As with Liquibase, we adopted a naming convention that kept things in order. Even when two changesets had the same number, they would have different names, and 99.9% of the time, the order of those two changesets was not dependent on each other.

Just for a datapoint, the project that used Tarantino leveled off with about 400 changesets.

Answer 2

(I only know liquibase - so my answer is valid only for liquibase)

As the includeAll doc states the files will be run in alphabetical order. So I'd expect that your numbering should be sufficient for the files to be in the right order. However you will have to synchronize those numbers between you and Bob anyways to find out what has to run first.

We don't use includeAll though. We include files into the master changelog manually. So whoever wants to change the database has to take care of including this into the master changelog. If there are two changes the developer that comes last has to make sure to include/merge his changes at the right place in the master change log.

EDITED - to explain the include mechanism we use

Within development we just flunk the whole database and let liquibase create the database from scratch whenever we change anything (on our development database). We always checkin the changeset file to our code repository. So with this we track everything that was done in development since we can checkout every version of the changeset and let liquibase create the database with it.

Only when development of the version we are working on (or the sprint) is done. Then we really let the changesets run und let them change the productive database.

Then this cycle is repeated again and again.

This way only the final db changes are tracked with liquibase and not all the tryouts that might take place during development.

During development you could end up changing a table multiple times because you try out different things. You add a column, then realize the whole thing will not work so you remove the column again and add another one. Why would you want to have all these changes in the databasechangelog ? Then it will fill up with unnecessary stuff - just like you were afraid of.

Hope this clears things up. But in the end you might have a total different development approach. So feel free to use liquibase just the way you need it.

There is no reason to fill the databasechangelog table with all the ideas during development. Only when we are done with development and release a version