Best practices for change management with indexes

Question

Our IT shop is first starting to build a group of DBA's. All of us (myself included) have come over from the application development/architecture world, so the DBA world is still fairly new to us.

Along with building a DBA group, we are looking to build change manage procedures and processes (hopefully based on best practices) for when we need to move changes.

I've found the following post which is helpful for mostly trigger, stored procedure, and/or DDL changes. But it doesn't necessarily address indexes or vendor databases.

We have a mix of both our own and vendor databases. In our case some of the vendors (though not all) are working with our company to build the database(s) and applications. We are in the process of performance testing our applications now before they "go live". Thus we are analyzing indexes (or the lack thereof) pretty heavily.

As we come across indexes that we feel should be made, how do we best deal with change management with regard to these, both for our own databases as well as for any vendors?

What do you do in your shop? I'm worried less about tools then about the process.

EDIT: So far, I'm appreciating the feedback, comments, and answers for this question. I have noticed that some of the answers are a bit tool specific. I'm looking for more "agnostic" practices, if that can be had.

However if agnostic is not possible, then for tool sets, we use IBM DB2 LUW (and that actually on AIX) mostly. We have some DB2 on Windows and DB2 for i (IBM's i5/OS), but we are mostly AIX DB2. We do use source control, specifically Subversion.

Again, looking for general best practices, but above is what we use that would be vendor specific.

EDIT: Current Decision: We intend to track our reasoning as well as our changes. So we are going to open an issue in our issue-tracking software (which in our case is JIRA). Now we can add in documentation as to what priority the change has, data that backs up what the change should be, the change, and the results of the change from another environment where the change was tested.

We then also intend to keep track of our changes in scripts in SVN (much like was suggested below). This way we can track what version of what exists where. This can be recorded in our JIRA issue (and in any other auditing software we use, ie. pasted links). We can know with more certainty what change went to what environment and why. We can then also track if the index was something we added beyond the vendors implementation or ahead of their implementation, etc.)

Answer 1

I would strongly recommend that you treat your database basically the same way as you treat your application code. You can script your database out to it's component parts and check those into source control and then use the same labels & versions there that you use for your apps.

To get the objects into source control there are a number of tools you can use. Microsoft has a tool that is nicknamed Data Dude. It works with Visual Studio. They're also preparing to release a new tool called SQL Server Database Tools (SSDT), again, working with Visual Studio. My company, Red Gate Software, makes a tool that works with SSMS called SQL Source Control.

In terms of process, I wrote several chapters for the book Red Gate Guide to Team Development. It's available as a free download (or if you want to kill a tree you can purcahse one from Amazon). I go into a lot more details about working with databases in development teams there.

Answer 2

1. We maintain database scripts as part of our application codebase which is maintained under version control. However we use different "processes" for development and production code

2. Development we maintain the following scripts:

   • base.sql - creates the schema tables and sample data
   • stagingchanges.sql - makes changes to the base.sql for staging environment, mainly email addresses, paths and other assets which might change
   • prodchanges.sql - makes changes to the base.sql for a production deployment. For new projects we usually get to test these out in actual production environments

3. Maintenance

   • base.sql - a sanitized version of the production database
   • stagingchanges.sql and prodchanges.sql - as above
   • changerequest.sql (usually has the id of the change request) which applies any schema changes for the current change request we are working on
   • changerequest-rollback.sql - reverses the changes made for the change request and resets the database back to production
   • archive (folder) for previous change request scripts

So in maintenance mode all we need to do is apply the changerequest.sql script during deployment to production

Answer 3

Being in the database version control space for 5 years (as director of product management at DBmaestro) and having worked as a DBA for over two decades, I can tell you the simple fact that you cannot treat the database objects as you treat your Java, C# or other files.

There are many reasons and I'll name a few:

• Files are stored locally on the developer’s PC and the change s/he
  makes do not affect other developers. Likewise, the developer is not affected by changes made by her colleague. In database this is
  (usually) not the case and developers share the same database
  environment, so any change that were committed to the database affect others.
• Publishing code changes is done using the Check-In / Submit Changes / etc. (depending on which source control tool you use). At that point, the code from the local directory of the developer is inserted
  into the source control repository. Developer who wants to get the
  latest code need to request it from the source control tool. In
  database the change already exists and impacts other data even if it was not checked-in into the repository.
• During the file check-in, the source control tool performs a conflict check to see if the same file was modified and checked-in by another developer during the time you modified your local copy. Again there is no check for this in the database. If you alter a procedure from your local PC and at the same time I modify the same procedure with code form my local PC then we override each other’s changes.
• The build process of code is done by getting the label / latest version of the code to an empty directory and then perform a build – compile. The output are binaries in which we copy & replace the existing. We don't care what was before. In database we cannot recreate the database as we need to maintain the data! Also the deployment executes SQL scripts which were generated in the build process.
• When executing the SQL scripts (with the DDL, DCL, DML (for static content) commands) you assume the current structure of the environment match the structure when you create the scripts. If not, then your scripts can fail as you are trying to add new column which already exists.
• Treating SQL scripts as code and manually generating them will cause syntax errors, database dependencies errors, scripts that are not reusable which complicate the task of developing, maintaining, testing those scripts. In addition, those scripts may run on an environment which is different from the one you though it would run on.
• Sometimes the script in the version control repository does not match the structure of the object that was tested and then errors will happen in production!

There are many more, but I think you got the picture.

What I found that works is the following:

1. Use an enforced version control system that enforces check-out/check-in operations on the database objects. This will make sure the version control repository matches the code that was checked-in as it reads the metadata of the object in the check-in operation and not as a separated step done manually
2. Use an impact analysis that utilize baselines as part of the comparison to identify conflicts and identify if a change (when comparing the object's structure between the source control repository and the database) is a real change that origin from development or a change that was origin from a different path and then it should be skipped, such as different branch or an emergency fix.

An article I wrote on this was published here, you are welcome to read it.