Automated database versioning and Mercurial (and probably git)

Question

We've been using subversion for years, but are now moving to mercurial.

We have a nifty automated database versioning system using teamcity and a script. (It's .net and msbuild and powershell, but that's irrelevant). ]

We have a database directory as such:

/database/
/database/functions
/database/migration
/database/storedprocedures
/database/views

the migration folder contains things DDL and data modifications.

The source / build system works like so:

1. Developer makes a change to a stored procedure
2. Developer commits change to svn
3. Teamcity checks out code and builds
4. Database Migration Generation Script runs that gets the svn revision that kicked off the build
5. Script does a diff in the database directory between the last known revision and this revision. For all files that are in the diff, it concatenates them into a file with filename %buildnumber%_%revision%.sql
6. Script puts a final 'update databaseinfo setversion = '%buildnumber%' into fiel
7. Script then commits this new file back into subversion
8. build is packaged

Then at release time, or dev time, it's matter of running the SQL migration files in order and voila, database is upgraded and migrated.

It's good system, because it means you do your work and don't have to worry about writing migrations, or noting your changes, no "sorry about the test environment, I forgot to write the migration for that stored procedure, wait 30 minutes and I'll give you a new build"

However, with the move to mercurial, step 7. Script then commits this new file back into subversion doesn't work well.

With subversion it's not a big deal because you're just committing back to a database directory that never gets modified outside the build system.

With mercurial (and I'm a bit of a newbie here), we need to commit and push the change which entails doing a pull and update. I can get this to work no problem, but it comes at the cost of putting the source in an invalid state and it just feels as though it's the wrong thing to do.

I'm thinking of utilising the commit / pre commit hooks inside mercurial, but then we lose the the build number versioning and buy ourselves a problem in finding a naming convention to version the database on that works and then works again when we have a branch.

I'm wondering what other people do around this?

Answer 1

For what it's worth. I ended up doing this with a mercurial commit hook.

Since we're using windows & powershell.

I put this in the hgrc

[hooks]
commit.MigrateDatabase=powershell -File ".\BuildAndDeploy\DatabaseMigration_HgHook.ps1"

And the in the powershell file, to cut along story short

$hg_node = (gci -Path env:hg_node).Value $hgCommandLine = "hg status --change $hg_node > .\Migration.Working\ModifiedFiles.txt" Invoke-Expression $hgCommandLIne

Then parsed the ModifiedFiles.txt for any added or modified files, concatenated them into another file and versioned this based on a combination of the the version and the date it was comitted.

Answer 2

If your automated change is this:

1. Pull and update
2. Create a new file in a directory
3. Commit that new file into the repository
4. Push

Then what you risk doing is timing-wise that you get an extra head. In the time from you pulled, someone else might have pushed a new head (on top of the old one), that your local repository (on the build server) doesn't know about.

This is all that is going to be wrong. If you're not modifying existing files, then there is not going to be a merge conflict here, and since you're just committing on top of an existing changeset the build server is not going to merge at all.

ie. timing-wise, this is what will happen. This is the current main repository timeline:

1---2---3---4---5

You pull, and get this to your build server. Then you create a new file with the changes, and commit, forming this timeline:

1---2---3---4---5---6

In the meantime, another developer has also pushed a new changeset to the main repository, so when you push, you get this:

1---2---3---4---5---6
                 \
                  \-7

(6 or 7 is your new changeset, the other is from the other developer).

This is the worst that can happen. You can try to automerge on the build server. Again, this should not produce a merge conflict.