How to set up SQL Server Enterprise for Partitioning

Question

We have a database that is currently 1.5TB in size and grows by a gigabyte worth of data every day (a text file) that is 5 million records - and it grows daily

It has many columns, but a notable one is START_TIME which has the date and time -

We run many queries against a date range -

We keep 90 days worth of records inside of our database, and we have a larger table which has ALL of the records -

Queries run against the 90 days worth of records are pretty fast, etc. but queries run against ALL of the data are slow -

I am looking for some very high level answers, best practices

We are THINKING about upgrading to SQL Server enterprise and using table partitioning, and splitting the partition based on month (12) or days (31)

Whats the best way to do this?

Virtual Physical, a SAN, how many disks, how many partitions, etc. -

Sas

Answer 1

You don't want to split by day, because you will touch all partitions every month. Partitioning allows you not to touch certain data.

Why do you want to partition? Can you clearly articulate why? If not (which I assume) you shouldn't do it. Partitioning does not improve performance per-se. It improves performance in some scenarios and it takes performance in others.

You need to understand what you gain and what you loose. Here is what you gain:

• Fast deletion of whole partitions
• Read-Only partitions can run on a different backup-schedule

Here is what you loose:

• Productivity
• Standard Edition
• Lower performance for non-aligned queries (in general)

Here is what stays the same:

• Performance for partition-aligned queries and indexes

If you want to partition, you will probably want to do it on date or month, but in a continuous way. So don't make your key month(date). Make it (year(date) + '-' + month(date)). Never touch old partitions again.

If your old partitions are truly read-only, put each of them in a read-only file-group and exclude it from backup. That will give you really fast backup and smaller backups.

Because you only keep 90 days of data you probably want to have one partition per day. Every day at midnight you kill the last partition and alter the partition function to make room for a new day.

There is not enough information here to answer anything about hardware.