How do you version your database schema?

Question

How do you prepare your SQL deltas? do you manually save each schema-changing SQL to a delta folder, or do you have some kind of an automated diffing process?

I am interested in conventions for versioning database schema along with the source code. Perhaps a pre-commit hook that diffs the schema?

Also, what options for diffing deltas exist aside from DbDeploy?

EDIT: seeing the answers I would like to clarify that I am familiar with the standard scheme for running a database migration using deltas. My question is about creating the deltas themselves, preferably automatically.

Also, the versioning is for PHP and MySQL if it makes a difference. (No Ruby solutions please).

Answer 1

See

Is there a version control system for database structure changes?

How do I version my MS SQL database in SVN?

and Jeff's article

Get Your Database Under Version Control

I feel your pain, and I wish there were a better answer. This might be closer to what you were looking for.

Mechanisms for tracking DB schema changes

Generally, I feel there is no adequate, accepted solution to this, and I roll my own in this area.

Answer 2

You might take a look at another, similar thread: How do I version my MS SQL database in SVN?.

Answer 3

If you are still looking for options : have a look at neXtep designer. It is a free GPL database development environment based on the concepts of version control. In the environment you always work with versioned entities and can focus on the data model development. Once a release is done, the SQL generation engine plugged on the version control system can generate any delta you need between 2 versions, and will offer you some delivery mechanism if you need.

Among other things, you can synchronize and reverse synchronize your database during developments, create data model diagrams, query your database using integrated SQL clients, etc.

Have a look at the wiki for more information : http://www.nextep-softwares.com/wiki

It currently supports Oracle, MySql and PostgreSql and is in java so the product runs on windows, linux and mac.

Answer 4

I make sure that schema changes are always additive. So I don't drop columns and tables, because that would zap the data and cannot be rolled back later. This way the code that uses the database can be rolled back without losing data or functionality.

I have a migration script that contains statements that creates tables and columns if they don't exist yet and fills them with data.

The migration script runs whenever the production code is updated and after new installs.

When I would like to drop something, I do it by removing them from the database install script and the migration script so these obsolete schema elements will be gradually phased out in new installs. With the disadvantage that new installs cannot downgrade to an older version before the install.

And of course I execute DDLs via these scripts and never directly on the database to keep things in sync.

Answer 5

I don't manage deltas. I make changes to a master database and have a tool that creates an XML based build script based on the master database.

When it comes time to upgrade an existing database I have a program that uses the XML based build script to create a new database and the bare tables. I then copy the data over from the old database using INSERT INTO x SELECT FROM y and then apply all indexes, constraints and triggers.

New tables, new columns, deleted columns all get handled automatically and with a few little tricks to adjust the copy routine I can handle column renames, column type changes and other basic refactorings.

I wouldn't recommend this solution on a database with a huge amount of data but I regularly update a database that is over 1GB with 400 tables.

Answer 6

You didn't mention which RDBMS you're using, but if it's MS SQL Server, Red-Gate's SQL Compare has been indispensable to us in creating deltas between object creation scripts.

Answer 7

I'm not one to toot my own horn, but I've developed an internal web app to track changes to database schemas and create versioned update scripts.

This tool is called Brazil and is now open source under a MIT license. Brazil is ruby / ruby on rails based and supports change deployment to any database that Ruby DBI supports (MySQL, ODBC, Oracle, Postgres, SQLite).

Support for putting the update scripts in version control is planned.

Answer 8

http://bitbucket.org/idler/mmp - schema versioning tool for mysql, writed in PHP

Answer 9

We're exporting the data to a portable format (using our toolchain), then importing it to a new schema. no need for delta SQL. Highly recommended.

Answer 10

I use Firebird database for most development and I use FlameRobin administration tool for it. It has a nice option to log all changes. It can log everything to a one big file, or one file per database change. I use this second option, and then I store each script in version control software - earlier I used Subversion, now I use Git.

I assume you can find some MySQL tool that has the same logging feature like FlameRobin does for Firebird.

In one of database tables, I store the version number of the database structure, so I can upgrade any database easily. I also wrote a simple PHP script that executes those SQL scripts one by one on any target database (database path and username/password are supplied on the command line).

There's also an option to log all DML (insert, update delete) statements, and I activate this while modifying some 'default' data that each database contains.

I wrote a nice white paper on how I do all this in detail. You can download the paper in .pdf format along with demo PHP scripts from here.

Answer 11

I also developed a set of PHP scripts where developers can submit their deltasql scripts to a central repository.

In one of the database tables (called TBSYNCHRONIZE), I store the version number of the latest executed script, so I can upgrade any database easily by using the web interface or a client developed on purpose for Eclipse.

The web interface allows to manage several projects. It supports also database "branches".

You can test the application at http://www.gpu-grid.net/deltasql (if you login as admin with password testdbsync). The application is open source and can be downloaded here: http://sourceforge.net/projects/deltasql

deltasql is used productively in Switzerland and India, and is popular in Japan.

Answer 12

Some months ago I searched tool for versioning MySQL schema. I found many useful tools, like Doctrine migration, RoR migration, some tools writen in Java and Python.

But no one of them was satisfied my requirements.

My requirements:

1. No requirements , exclude PHP and MySQL
2. No schema configuration files, like schema.yml in Doctrine
3. Able to read current schema from connection and create new migration script, than represent identical schema in other installations of application.

I started to write my migration tool, and today I have beta version.

Please, try it, if you have an interest in this topic. Please send me future requests and bugreports.

Source code: bitbucket.org/idler/mmp/src Overview in English: bitbucket.org/idler/mmp/wiki/Home Overview in Russian: antonoff.info/development/mysql-migration-with-php-project

Answer 13

I use http://code.google.com/p/oracle-ddl2svn/

Answer 14

I am interested in this topic too.

There are some discussions on this topic in the Django wiki.

Interestingly, it looks like CakePHP has schema versioning built-in using just cake schema generate command.

Answer 15

For MySQL

When I land on a new DB:

Firstly, I check structure:

mysqldump --no-data --skip-comments --skip-extended-insert -h __DB_HOSTNAME__ -u __DB_USERNAME__ -p __DB1_NAME__ | sed 's/ AUTO_INCREMENT=[0-9]*//g' > FILENAME_1.sql
mysqldump --no-data --skip-comments --skip-extended-insert -h __DB_HOSTNAME__ -u __DB_USERNAME__ -p __DB2_NAME__ | sed 's/ AUTO_INCREMENT=[0-9]*//g' > FILENAME_2.sql
diff FILENAME_1.sql FILENAME_2.sql > DIFF_FILENAME.txt
cat DIFF_FILENAME.txt | less

Thanks to stackoverflow users I could write this quick script to find structure differences.

src : https://stackoverflow.com/a/8718572/4457531 & https://stackoverflow.com/a/26328331/4457531

In a second step, I check datas, table by table with mysqldiff. It's a bit archaic but a php loop based on information_schema datas make job surely

For versioning, I use the same way but I format a SQL update script (to upgrade or rollback) with diff results and I use version number convention (with several modifications the version number look like an ip address).

initial version : 1.0.0
                  ^ ^ ^
                  | | |
structure change: - | |
datas added: -------- |
datas updated: --------

Answer 16

I am using strict versioning of the database schema (tracked in a separate table). Scripts are stored in version control, but they all verify current schema version before making any change.

Here is the full implementation for SQL Server (the same solution could be developed for MySQL if needed): How to Maintain SQL Server Database Schema Version

Answer 17

After long investigation, I figured out that there are some 3rd party tools or Visual Studio project types that does not satisfy me, or just blogs about the theory but no implementation. So I implemented a working system, which is used almost a year, and explained here:

http://nalgorithm.com/2015/11/09/database-versioning-part-1/

depending on the interest, will continue writing more.