Creating an 'archive table'

Question

I have a table that will contain, let's say, tasks. Some tasks are new or in work-in-progress stage, but other tasks will be in archive stage which means they have been dealt with and the chance of going back to them is low. I was thinking that placing the archived tasks in a separate table with the same schema would be wise so that queries to the 'current' tasks are faster. Is it right?

If I need to bring up current tasks with archived tasks (like in a search result), I will simply union the two tables.

Is this right? Will I gain any benefit? I think it is called horizontal fragmentation. I am using MySQL InnoDB. Do I need to do something extra to the table definitions in order to really gain performance benefits?

Thanks!!

Answer 1

a separate table with the same schema

I'm not sure this is ever a good idea, remember, DRY (don't repeat yourself). If you have to change the schema for one, you will have to change the schema for the other, this can lead to bugs.

Also,

premature optimization is the root of all evil

Are your database queries currently running too slow? I am guessing they are not.

Answer 2

Partitioning is a technique that addresses specific requirements like yours. Basically you could do a logical separation of data based on a measure in data such as date.

However, if you are not willing to partition your table, it is wise to separate the tasks into two tables with the goal of increasing the overall throughput assuming you will need frequent table scans over the current tasks or if, for any reason, you are reluctant to create indexes on the table (for example, due to difficulty of predicting the queries type). In such cases, this separation will help to keep the table's size for the current tasks to minimal. So, the queries that normally require full table scan could be improved (especially for ad hoc queries).

However, please note that over the time, the size of the archived tasks table will also grow in size and this growth will affect the performance of the queries which are run against this table and it might become necessary to create some indexes on the archives table.