Integrity Audit

Question

So here is my problem.

I need to get a count of all distinct values in each table in each database.

Example:
[db 1]
[table 1] 
[column 1] count()
[column 1] distinct()
[column 1] count()
[column 1] distinct() 
[column 2] count()
[column 2] distinct() etc

[db 2]
[table 1] 
[column 1] count()
[column 1] distinct()
[column 2] count()
[column 2] distinct() etc

Right Now i have this so far:

DECLARE @TableName   VARCHAR (MAX) =     'sales'

SELECT DISTINCT
     'SELECT '
   + RIGHT (ColumnList, LEN (ColumnList) - 1)
   + ' FROM '
   + Table_Name +' group BY '+ (ColumnList)','
  FROM INFORMATION_SCHEMA.COLUMNS   COL1
   CROSS APPLY (SELECT ', COUNT ( ' + COLUMN_NAME + ')'+ ','+COLUMN_NAME
                  FROM INFORMATION_SCHEMA.COLUMNS COL2
                 WHERE COL1.TABLE_NAME = COL2.TABLE_NAME
                FOR XML PATH ( '' )) TableColumns (ColumnList)

WHERE 1 = 1 AND COL1.TABLE_NAME = @TableName

So I just need help with the group by.

Answer 1

Ohh noo @SaUce used dreadful CURSOR again.

Well this is only part of the solution and this is not the best part, but here how I would do it.

The following code will return COUNT() of all rows and COUNT(DISTINCT Column) from each table in current database you just have to figure out how to execute it for each DB

No matter how you do it you have to consider that COUNT(*) and COUNT(DISTINCT) are expensive operations, so it is best to save results will date you last executed it. Depending on the size of your databases this could take hours (maybe days) to complete.

IF OBJECT_ID('tempdb.dbo.#tempTable') IS NOT NULL
    DROP TABLE #tempTable;

CREATE TABLE #tempTable
    (
     TableName VARCHAR(250)
    ,ColumnName VARCHAR(250)
    ,TotalCount BIGINT
    ,DistinctCount BIGINT
    )

DECLARE @column_name VARCHAR(250)
   ,@table_name VARCHAR(150)
   ,@SQLStatement NVARCHAR(500)

DECLARE table_cursor CURSOR
FOR
    SELECT DISTINCT OBJECT_NAME(object_id)
        FROM sys.tables t
        ORDER BY 1

OPEN table_Cursor
FETCH NEXT FROM table_cursor INTO @table_name

WHILE @@FETCH_STATUS = 0
    BEGIN

        DECLARE column_cursor CURSOR
        FOR
            SELECT DISTINCT name
                FROM sys.columns c
                WHERE OBJECT_NAME(object_id) = @table_name

        OPEN column_cursor

        FETCH NEXT FROM column_cursor
        INTO @column_name

        WHILE @@FETCH_STATUS = 0
            BEGIN
                SET @SQLStatement = N'
                INSERT INTO #tempTable
                SELECT ''' + @table_name + '''
                ,''' + @column_name + '''
                ,COUNT(*)
                ,COUNT(Distinct ' + @column_name + ')
                FROM ' + @table_name + ' WITH(NOLOCK)'

                --PRINT @SQLStatement

                EXECUTE sp_executesql
                    @SQLStatement

                FETCH NEXT FROM column_cursor INTO @column_name 
            END
        CLOSE column_cursor;
        DEALLOCATE column_cursor;

        FETCH NEXT FROM table_cursor INTO @table_name
    END
CLOSE table_cursor
DEALLOCATE table_cursor

SELECT *
    FROM #tempTable