Designing classes to cope with frequent database schema changes

Question

I am working on a database that is maintained by another system, so I am unable to make any drastic changes to the tables. The tables in this database have quite a few fields (upwards of 30+) with no two tables having the same naming convention, types, or same number of fields. These tables also change fairly frequently (fields being added, removed or their types being changed) and to make matters worse new tables are created on a frequent basis as well. Now I do have one table that can be strictly typed which is used as a queue.

The problem:

I have to get data from a table given the table name and some columns in that table. The column names are supplied as strings. Since these tables change so often it becomes difficult to maintain strictly typed entities for each table.

How would I design my classes in a way that they are loosely coupled but allow me to work with these tables?

Thanks for the help and sorry if my explanation sucks.

Answer 1

One idea is to use SQL Management Objects (SMO) with Tables to dynamically create strong types.

Server srv = new Server(conn);
Database db = srv.Databases["AdventureWorks"];

foreach (Table table in db.Tables)
{
    Console.WriteLine(" " + table.Name);
    foreach (Column col in table.Columns)
    {
        Console.WriteLine("  " + col.Name + " " + col.DataType.Name);
    }
}

I've written unit test generators for Data Access Layers this way and you could make up Classes using the column DataTypes, eg (look online for a better implementation - in C#):

Public Function SQLParameterType(ByVal ParameterDataType As String) As String

    ParameterDataType = ParameterDataType.ToUpper

    If ParameterDataType.IndexOf("NVARCHAR") > 0 Then
        Return "string"
    ElseIf ParameterDataType.IndexOf("VARCHAR") > 0 Then
        Return "string"
    End If

    Select Case ParameterDataType

        Case Is = "BIGINT"
            Return "Int64"
        Case Is = "INT"
            Return "Int32"
        Case Is = "SMALLINT"
            Return "Int16"

        Case Is = "BIT"
            If gIsVBdotNet Then
                Return "boolean"
            Else
                Return "bool"
            End If

        Case Is = "DATETIME"
            Return "DateTime"

        Case Is = "DATETIME2"
            Return "DateTime"

        Case Is = "SMALLDATETIME"
            Return "DateTime"

        Case Is = "MONEY"
            Return "single" 'float
        Case Is = "FLOAT"
            Return "single" 'float
        Case Is = "REAL"
            Return "double"
            'Case Is = "INT"
            '    Return "int32"
            'Case Is = "INT"
            '    Return "int32"
        Case Else
            Return "666"
    End Select

End Function

With this simple ORM that connects to your database in a schema-neutral way you can loosely couple dynamically generated classes to the dB. The new Dynamic type in .Net 4 seems to be a good polymorphic datatype candidate for this application.

Answer 2

I'm not really sure what sort of business logic you can derive from unstable data. My assumption is that you are doing some work on the meta data associated with the records. Working with the aggregates of the data (counts, averages, etc) rather then actual data.

I would use a DataSet, use a generic SQL statement like:

SELECT * FROM {dynamic table name}

Answer 3

An quick fix (not a solution) would be to use a code generation utility like this one one which would update/create your class files based on your templates or the datatabse schema definitions themselves. With a little bit of work this could be fully automated in your project via custom MsBuild task which would run when you build your project & your files would be updated directly into you solution folder.

But most importantly it would break your code upon build when your tables definitions get updated - showing you where you have to fix things due to table definition changes. For this to work your custom MsBuild task must run before the main build task as explained here.

This would definitely benefit you and make things easy for you because I've seen such design benefit a similar project but in the end the 'crap in - crap out' principle would still prevail so the above 'fix' would only treat the symptom not the cause which is likely a poor design and lack abstraction and analysis of what your solution data needs are. Hence you get a program which needs to be re-programmed in order to be re-configured:) but for LOB programming, it's not unusual:)

** Note: If you decide to use the design described in the first paragraph, in order to have a complete solution you may need to have an additional MsBuild task which would automatically checkout the files which need to be updated from you source control when the schema changes. Comment bellow if you need more info on this.

To summarize: what will happen is when your press F5 to build your solution in Visual Studio all your classes will get magically updated with the latest definitions from the database. You can also constrain this to happen on Release builds only so that you don't get this done every time you build in Debug mode while you are developing.