Can I have several 'similar' database tables to reduce retrieval time

Question

It is best to explain my question in terms of a concrete example.

Consider an order management application that restaurants use to receive orders from their customers. I have a table called orders which stores all of them.

Now every day the tables keep growing in size but the amount of data accessed is constant. Generally the restaurants are only interested in orders received in the last day or so. After 100 days, for example, 'interesting' data is only about 1/100 of the table size; after 1 year it's 1/365 and so on.

Of course, I want to keep all the old orders, but performance for applications that are only interested in current orders keeps reducing. So what is the best way to not have old data interfere with the data that is 'interesting'?

From my limited database knowledge, one solution that occurred to me was to have two identical tables - order_present and order_past - within the same database. New orders would come into 'order_present' and a cron job would transfer all processed orders older than two days to 'order_old', keeping the size of 'order_present' constant.

Is this considered an acceptable solution to deal with this problem. What other solutions exist?

Answer 1

Database servers are pretty good at handling volume. But the performance could be limited by physical hardware. If it is the IO latency that is bothering you, there are several solutions available. You really need to evaluate what fits best for your usecase.

For example:

1. you can Partition the table to distribute it onto multiple physical disks.
2. you can do Sharding to put data on to different physical servers
3. you can evaluate using another Storage Engine which best fits your data and application. MyISAM delivers better read performance compared to InnoDB at the cost of being less ACID compliant
4. you can use Read Replicas to deligate all (most) "select" queries to replicas (slaves) of the main database servers (master)

Finally, MySQL Performance Blog is a great resource on this topic.