Table Insert Rate Slows AsTable Size Increases

Question

Parsing some data and inserting it into 3 tables from .NET. Using Table Valued Parameters to pass the data as some inserts are 600,000 rows. Passing objects (not DataTables) and they are passed by reference (nature of TVP). Got 100:1 gain over straight value insert as insert value is limited to 1000 rows at a time. In the stored procedure the insert to the actual table from the TVP is sorted by the clustered index. These tables have no index other than the clustered index. The SP takes a TABLOCK as these are write once tables and one data loader. Fill Factor 100. No increase in data or transaction log size - it is sized for the total data load. Finally to the question. In the last 4 hours have inserted 200 million rows. The insert response time has dropped in 1/2. If the fill factor is 100 and I am inserting sorted by clustered index then why the drop in response? What can I do fix this?

I did not get TVP until I used it - it is like a reverse DataReader.

I would like to thank you for your help and apologize for an incorrect problem statement. For each parse (in this case I am parsing 200,000) the insert is sorted by the clustered index. However for only 1 of the 3 tables is the next parse as a whole in clustered index order. After parsing 70,000 the good table has a scan density of 99% but the other two tables are severely fragmented with a scan density of 12%.

Set a fill factor of 50 on the two fragmented tables and re-indexed. Now I am getting about 1/3 the max speed. I will just need to stop the process and re-index every few hours.

What I ended up doing is changing the clustered index to match insert order. Created a unique index on what used to be the clustered. I disable the unique, index insert the data, and then rebuild the unique index. With that scenario I get a 300:1 performance on a 10 hour run. That is not an extra 0 - three hundred to one. And that is not fudging - compared to starting with the index ordered and a fill factor or 30. Even with the extra index my table size is smaller as I can have both fill factors at 100.

I use #temp on some queries so I can get the rows in an order only known to the query. I converted the #temp to a TVP and gained 1/2 second (about the time it take to create and delete a #temp).

Answer 1

Per OP converting comment to answer...

In addition to auto-stats as @SqlACID mentions, constraint checking could get more expensive as the table fills up. If I'm going to seriously load a table I usually plan to disable or drop indexes and constraints, and re-create them after, if speed is my ultimate goal. This may mean deleting rows after the fact if the constraint is violated, or doing better validation on the bulk data up front when possible.