How to detect any changes to a database (DDL and DML)

Question

There are a lot of databases on my client's SQL server. These databases are under development, so developers can design, refactor, do data modifications and so on. There are some databases that change rarely. My client has to keep all of them safe (backed up) and spend some time managing the environment. (There is no DB administrator position at the company.) After lengthy discussion, the client has decided to use a daily full backup strategy, due to the ease of restoring.

So here is the summary of the situation:

• Number of databases can vary every day.
• Databases that were changed (meaning data and/or structure have been changed) shall be backed up.
• Databases that were not changed shall NOT be backed up.
• Solution shall not impact database structure (it's not restricted requirement)
• This "backup engine" shall work automatically.

The main problem: how to detect that a database has been changed. The first part of the problem (DDL changes) can be resolved by using DDL triggers. But the data changes (DML changes) are a problem. It is impossible to apply DML triggers to all tables of all databases to track changes (performance, management of extended objects...). The backup engine has to track all changes to mark each database as ready to backup.

• Change Data Capture is a solution but it seems too heavy (it requires SQL Server Enterprise Edition as well).

• Another way is to track database file changes (size or last change time), but it does not work correctly: A database can change its size when it exceeds all reserved free space and sp_spaceused is not a solution.

• Tracing is a solution but it causes performance issues and requires additional management.

Are there any solutions to calculate the actual database usage size without impact on other database management objects (like statistics..)? Granted that a change to a table's data that doesn't change the table's size would not trigger (I think), but it's better than nothing. Really I am looking for a direct or indirect solution for SQL Server 2008.

Thank you for any comments, solutions, and thoughts.

ADDED:

Here is the solution (thanks to Marian):

Select
    NextLSN = MAX(fn.[Current LSN])
    ,Databasename = DB_NAME()
 from fn_dblog(NULL,    NULL) fn
     LEFT JOIN sys.allocation_units au
         ON fn.AllocUnitId = au.allocation_unit_id
     LEFT  JOIN sys.partitions p
         ON p.partition_id = au.container_id
     LEFT  JOIN sys.objects so
         ON so.object_id = p.object_id  
    WHERE 
    (
        (Operation IN 
       ('LOP_INSERT_ROWS','LOP_MODIFY_ROW',
            'LOP_DELETE_ROWS','LOP_BEGIN_XACT','LOP_COMMIT_XACT') 
            AND so.is_ms_shipped = 0)
        OR 
        ([Lock Information] like '%ACQUIRE_LOCK_SCH_M OBJECT%')
    )

Answer 1

One idea would be to make a snapshot every day and monitor the snapshot file size on the disk using a file monitor. The snapshot is increasing its size only when data is added there, so it would be a valid idea if you would find a tool to monitor the real size (reported size).

Now.. I didn't use this, so can't give you technical insights :-).

Another idea would be to verify the transaction log of each db (if you're using full recovery mode on them, of course) with some function I've seen on the forums (db_fnlog.. or something) that reads operations from the log, and see if you have any deletes/inserts/updates.

Those are no easy things to do.. but I hope you'll find them useful.

PS: found the article with the log read function (it's fndblog, by the way :-): Read the transaction log by Jens K. Suessmeyer.

Answer 2

• For DDL changes you may read the Default Trace.
• For DML modifications since you find CDC to be bit heavy, you may run your own lightweight server side trace which traces only the relevant events

Answer 3

For DDL changes you DDL Triggers, But DML Changes you can try using 3 different options

1) Change tracking 2) CDC(Change data Capture) 3) Audit Feature

For Change tracking.. you can see the below link http://www.mssqltips.com/sqlservertip/1819/using-change-tracking-in-sql-server-2008/

this change tracking will be only used wheather the table has changed or not...but it is very difficult to find what data has changed.. if you want to find what data has changed then you can go Chnage data Capture.

For Aduit in sqlserver ..you can check the below link http://blogs.msdn.com/b/manisblog/archive/2008/07/21/sql-server-2008-auditing.aspx

Answer 4

For DML changes you can utilize any of the followinh native SQL Server auditing features:

• SQL Server Change Tracking
• SQL Server Change Data Capture
• SQL Server Auditing

Each has its advantages and disadvantages, but the Auditing is the latest introduced by Microsoft, so it would be a good idea to build your current and future solutions wrapped with it.

Note that only the Auditing feature provides information about Who / When / How

Answer 5

You can detect any ddl changes by using trace file. below is script to get changes.

SELECT 
    te.name AS eventtype
    ,t.loginname
    ,t.spid
    ,t.starttime
    ,t.objectname
    ,t.databasename
    ,t.hostname
    ,t.ntusername
    ,t.ntdomainname
    ,t.clientprocessid
    ,t.applicationname  
FROM sys.fn_trace_gettable
(
    CONVERT
    (VARCHAR(150)
    ,(
        SELECT TOP 1 
            value
        FROM sys.fn_trace_getinfo(NULL)  
        WHERE property = 2
    )),DEFAULT
) T 
INNER JOIN sys.trace_events as te 
    ON t.eventclass = te.trace_event_id 
WHERE eventclass=164

You can detect any modification on table and stored procedure using this script:

SELECT 
    SO.Name
    ,SS.name 
    ,SO.type_desc 
    ,SO.create_date
    ,SO.modify_date 
 FROM sys.objects AS SO
INNER JOIN sys.schemas AS SS 
    ON SS.schema_id = SO.schema_id 
WHERE DATEDIFF(D,modify_date, GETDATE()) < 50
AND TYPE IN ('P','U')