Twist to the standard “SQL database change workflow best practices”

Question

Twist to the standard “SQL database change workflow best practices”

Background

ASP.NET/C# Web App MS SQL

Environments Production UAT Test Dev

We create patch scripts (XML and sql) that are source controlled in Mercurial. We have cmd line utility that installs patches to DB (utitlity.exe install –patch) from a Release folder the build packages. Patches have meta data that helps with when patch should run and we log patches installed in a table in the target DB. All these were covered in the 3 year old question:

SQL Server database change workflow best practices

Our Problem/Twist

I think this works well for tables, views, functions and stored procedures. We struggle with application configuration data. Here are some touch points on application configurations.

1. New client. BA performs system study and fit analysis. Out of this comes a configuration word document of what application configurations need to be setup. Note some of these may also come in phases over time. We need to get these new configurations into the system for the developer and client UAT.
2. Developer works on feature request or bug fix. A new configuration change comes out of that change. The configuration needs to make it into the system for testing and promotion to UAT and up.
3. QA finds that the developer missed an associated configuration change. That configuration needs to make it into the system for promotion to UAT and up.
4. Build goes to UAT. Client performs acceptance testing but find they really want to change another unassociated configuration and have it promoted with the changes. In other words they found they want to change a business process by a configuration. The configuration needs to make it into the system for promotion to PRD.
5. As the client operates in PRD they may tweak application settings. These configurations need to make it into the system for future development and testing.

The general issue is making sure we are accounting for all the configurations and accidently not miss any during promotions which causes grief.

Our Attempts At A Process

a. We have had member of the QA team to write patches (xml and sql) and check those in. This requires a build to make sure those get into the package. With this approach it really just took care of item 1 above and we fell apart on the other items. The nice thing is for the items that made it into the patches it was just an install with the utility. b. A developer threw together a Config page on the application. All the configurations could be uploaded and downloaded via XML document but it requires the app to be running. For item 1, member of QA team would manually setup configurations in the application and then would download the Config.xml file. This XML file would be used to upload configurations in other environments. We would use text diff tool to look at differences between config.xml files from different environments. This addressed item 1 and the others items but had problems. Problems were not all configurations made it into the XML document (just needs to be fixed by developer), some of the configurations didn’t have a UI in the application so you still had to manually go to the database on some, comparing the XML document with text diff was difficult at time (looked mostly due to sorting but I’m sure there are other issues), XML was not very human readable and finally the XML document did not allow for deleting existing incorrect or outdated configs. c. Recently we went with option B, but over time for a new client we just started manually tracking configs and promoting them manually by hand (UI and DB) through the promotions. Needless to say lots of human errors.

So we have been looking at solutions. Eventually it would be great to get as much automation in as possible. I’m looking at going with the scripting approach and just focusing on process, documentation and looking at using Redgate data compare in addition to what we had been doing with compare on config.xml. With Redgate we have to create views though and there is no way to create update scripts from that approach except to manually update the scripts. It does at least allow a comparison without the app running. I’m also looking at pulling out the configs from our normal patches and making it a system independent of the build (utility.exe –patch –config). When I say focus on process it will be things like if we compare and find a config change either reported by client or not, we still script it, just means we have to have a process in place to quickly revalidate config install before promoting to the next level. As for documentation looking at making the original QA document a living document instead of just an upfront document. The goal is to try and enhance clarity and reduce missing configurations during promotion. Unfortunately it doesn’t improve speed of delivery.

Does anyone have any recommendations or best practices to pass along. Thanks.

Answer 1

Can I ask exactly what you mean by application configuration. I'm interpreting that as both:

• Config files in the web application
• Static reference data inside the database

Full disclosure I work for Red Gate. You might be interested in taking a look at Deployment Manager, it's a deployment tool that deploys applications, databases and configuration. It's free for up to 5 projects and target servers.

The approach it uses is to package application code and the database state into packages. These packages can be deployed into dev, test, staging and production environments. The same package is deployed to each environment.

Any application configuration that needs to change between environments is handled in one of the ways below:

• Variable substitution in web.config. The tool allows you to specify override values for variables in these files, and set these per environment/server
• Substituting the web.config file per environment.
• Custom powershell scripts that are run pre/post deploy. You could use these to execute custom SQL based on the environment or server.
• Static data within the database, using SQL Source Control's static data feature. I've written a blog post about how to supply different sets of static data to different environments/customers.

This allows you to source control the application configurations and deploy them to different environments.