Definitive list of steps for SQL Server baseline testing?

Question

Before running a performance test / baseline for an app that uses SQL Server, I want to be able to set the instance to a "clean" state, without restarting the instance. There are steps I tend to follow, but I want to build a definitive list that is in the correct sequence, and has no redundant steps.

Does this list of steps accomplish setting SQL Server to a "clean" state?

Is the sequence logical / correct?

Are there any redundant steps?

CHECKPOINT              -- Write all dirty pages

DBCC DROPCLEANBUFFERS   -- All should be clean after checkpoint?

DBCC FREEPROCCACHE      -- Clear the plan cache

DBCC FREESYSTEMCACHE    -- Is this necessary after FREEPROCCACHE?

DBCC FREESESSIONCACHE   -- May not be necessary if distributed queries aren't used, but want to catch all scenarios

EXEC SP_UPDATESTATS     -- Refresh stats

'BEGIN TESTING!'

Answer 1

First, I'd step back and ask what measurements you plan to collect during the test. If you're counting logical reads by query, for example, then you don't need to free the cache. I'm a big fan of using logical reads because it's independent of whether the data is cached or on disk - and in production, it's hard to guess whether a query's data will be cached or not (unless you cache the entire database in memory). If you tune to minimize logical reads, then the app will go faster whether the data's in cache or not.

Next, I'd question what's changing between runs. For example, by running EXEC SP_UPDATESTATS in each database as you've suggested, you're going to resample stats for tables that have been updated. However, unless you're updating statistics with fullscan, you're getting random rows from the table - that's not too repeatable, and I don't think you really want to do that. Instead, you might want to restore the databases between each run so that you're always testing exactly the same data. If your tests are doing inserts/updates/deletes, they might have different performance profiles on each run if you're not restoring the database (because they're adding/changing data, plus changing statistics on the data) - and even worse, you could have queries fail if they try to insert repeated data for the same values.