Database versioning – How to go back to a previous version and keep data?

Question

The situation

I’m developing a web application that use SQL server to store thousands of records. We are currently using a source control software to save each version of the application. There are 2 main versions of the application:

• Test version (the one that we are actually developing)
• Live version (the one that is currently on the website)

We publish a new “major” Live version every month or so, but between that time we might find many little bugs on the current live version. To fix these bugs, we go back to the version the Live is currently running on our test machine (we use a shared database for testing…). Reproduce the bug, find the cause, fix it then I apply a patch to the buggy version that we then push live right away and we merge the bug fix in the test version. During the month between each “major” version, we can publish a lot of little bug fix patches.

The problem

The current problem is that only the source code is under version control; the database is not under version control. The reason why this is a problem is that while developing the next “major” version, many things might change in the database that are not compatible with the previous versions. So, even if we can go back to the code of a previous version, the database is not able to do the same and hence we can’t test the previous versions unless we had a database backup from that time.

The obvious solution seems to put the database under version control. While putting the database schema and the static data under version control is not really a problem (I think I will use Visual Studio Database Project), I struggle to see what to do with the user entered data.

The question

What do I do with user entered data since I need to have such data entered in my tables in order to test the application?

1. How do I go to a previous version of the database with the data that was in the database at this time?
2. Should I put user entered data under version control? For example using a complete backup of the database for each version we publish…
3. Is auditing a good solution for keeping a history of the changes to the user entered data?
4. Is it worth it to even bother with the user entered data since it’s just a test version? We could always recreate manually all the data we need to test the application but that might take a lot of time just to test a little bug.

Answer 1

Based on my experience, I would suggest some combination of database backups and database upgrade scripts. You should keep backups of the production or production-like data (you might be contractually obligated to purge or alter data with names, addresses, bank account numbers of your customers, etc.) for major releases. Starting from that you should be able to get to any intermediate version of your database because you are writing database upgrade scripts and keep them in your source control system (you are currently doing it, right?)

For the practical reasons, you should have at least two separate QA environments: one with the database schema and the application matching your production environment, and another one - matching the version under development.

While database backups are large, you would need to keep only a few latest ones, unless you anticipate a need to do some post-mortem bug analysis on a version that was defunct for several years.

Answer 2

If you're trying to avoid keeping a backup around for each historical version you might need to restore, why not instead try to downgrade a restored copy of your current production backup to the required version.

If you're using VS database projects, you'll have a version history you can go back to. You can use VS Schema Compare to compare the historical version of your database project to the restored production database. Provided there haven't been data motion changes (eg, table/column splits/merges) then it should successfully downgrade your data as well as your schema (otherwise you might need to correct the auto-generated script manually). Some people will maintain downgrade scripts alongside their upgrade scripts to simplify this process, but this takes discipline.

When you're done, and you want to get back to the latest version again, you can either run your existing upgrade process on your database to get it back to the latest version. Or maybe it's just simpler to restore a backup of your test database.

This is also the approach I'd recommend if you choose to use SQL Source Control and SQL Compare.

Answer 3

In SQL Server you can detach database files from the server and put them under version control, if you like. They can be re-attached to the same or another server at a later date.

The procedure is:

1. Right-click on database in SSMS and select properties
2. Click on "Files"
3. Scroll to the right and note the full path of where the files are located
4. Close database properties
5. Right-click database name and click Tasks/Detach
6. Check on "Drop Connections" and "Update Statistics"
7. Click OK
8. Copy the .MDF and .LDF files from the location above to wherever you want to save them
9. Right-click on "Databases" and click "Attach..."
10. Click "Add"
11. Locate the .MDF file from the location noted above and select it
12. Click "OK"

That's it. You've got a copy of the database that can be re-attached at a later date.