SQL - Partitioning effect on indexes and Transaction log

Question

Question: I'm wondering if I can apply an index on a table without filling the transaction log up.

Detail: I have a table that is 800GB. A few months ago I went to apply an index on it and the transaction log filled up (for obvious reasons - was dumb of me to even attempt it). Instead I had to create the table over again, apply the index I wanted and then copy the records over.

Now we are going to be setting up partitions on this table. I'm wondering if I remove the clustered index and apply the new clustered index on a partitioning layout if it will still fill up the transaction log (assume I have 10 million rows per partition)? Or will the tlog not fill up because the indexes would also be partitioned which would allow the SQL server to finish an index faster and be able to checkpoint?

Does anybody have any idea? Otherwise I can just re-create the table again and apply the partition stuff to it and re-fill it but obviously that is much more involved.

Thanks!

Answer 1

Applying the partition scheme will result in a huge log, yes.

First I would recommend you to see if you can, if possible, achieve a minimally logged operation. Offline CREATE INDEX is subject to minimally logging, provided that the database is in bulk-logged mode.

If not possible (eg. full recovery is required) I would look at doing the move to the partitioned table online. Online operations operate in small batches and commit frequently. As long as you take frequent log backups, it will not grow that large.