Index Fragmentation - log issues

Question

I have a few questions that I want to understand in order to take informed decision. I am not a DBA by profession, I have done some administrative tasks in the past and now I would have to create a maintenance plans to some of my new servers and understand the issues with my old servers.

1. My database log size is 50GB(converting MB to GB) and when I use the following query:

   SELECT name, 
      size/128.0 FileSizeInMB,
      size/128.0 - CAST(FILEPROPERTY(name, 'SpaceUsed') AS int)/128.0 AS EmptySpaceInMB
   FROM sys.database_files;
   

   it returns 25GB as the Empty space consistently. My database is in FULL recovery mode with hourly log back up. Doesn't this log file get trimmed at some point in time after hourly transactional backup and reduce the size of the log file?

2. We have a defragmentation plan set up at night. This plan runs for 15 mins and rebuilds all indexes above 40% fragmentation level. The way the DBA from one of our vendors explain is that during the rebuild the tables are locked and hence make the log file grow. I don't understand this clearly. Doesn't the log files grow in both rebuilding and re-organizing? Cant a long running re-organize fill up logs quickly?

3. How frequently should we update statistics after a daily index re-organize - immediately or weekly/monthly?

Answer 1

Doesn't this log file get trimmed at some point in time after hourly transactional backup and reduce the size of the log file?

No, it does not reduce the size of the log file on disk. The physical log file is made up of virtual log files (VLFs) and each VLF represents a portion of the transaction log. When a VLF is included in a transaction log backup (because there are no active transactions using that VLF) that VLF is marked inactive and can be overwritten. These inactive VLFs are considered free space within the log file because they can be overwritten at any time by an active transaction if required.

It is possible to shrink the physical file and reclaim the disk space, however, it is generally not recommended as the log file most likely requires that space for large operations, such as index maintenance, and regrowing the log file is a performance bottleneck.

Overview of Transaction Log in SQL Server

The way the DBA from one of our vendors explain is that during the rebuild the tables are locked and hence make the log file grow. I don't understand this clearly. Doesn't the log files grow in both rebuilding and re-organizing? Cant a long running re-organize fill up logs quickly?

Index maintenance operations (REBUILD, REORGANIZE etc) are logged in the transaction log, even ONLINE operations that don't lock the underlying table. So the vendor's DBA is correct that rebuilding indexes can cause log growth, but it is not due to table locking, it is because it is a logged operation.

Check out these links for additional information on space utilisation during index DDL:

Transaction Log Disk Space

Disk Space Requirements for Index DDL

How frequently should we update statistics after a daily index re-organize - immediately or weekly/monthly?

It depends on a number of factors, how many rows in the underlying table/index, rate of data modifications, are you using trace flag 2371 and auto-update statistics on your databases etc?

Also, how frequently are you rebuilding your indexes, because the statistics related to your indexes will be getting updated at the same time, and if you're tables have clustered indexes then your column statistics should also be getting updated.

If you're using Auto-Update Stats, then you can read about the threshold for updating stats here. There's also some useful information in the answers to a similar question here.