Design recommendations for a history/audit table

Question

I need to keep track of many items and their states throughout time.

Example

ItemId Location  DateTime           State
   1   Mall A    2010-02-03 07:00   on_sale
   1   Mall A    2010-02-20 08:22   sold
   2   Warehouse 2010-02-02 09:00   on_sale
   2   Transit   2010-03-02 16:20   transit
   2   Mall B    2010-03-03 10:10   on_sale
   2   Mall B    2010-03-12 12:11   sold

Right now, this is a huge table and I use MySQL's pseudo rank function to perform the queries.

However, it is very slow. The queries are unable to make use of the indexes since for any particular item it will have to find the item with rank = 1 before it can perform any filtering.

SELECT 
    item_sorted.*, IF(@prev <> item_sorted.item_id, @rownum := 1, @rownum := @rownum+1) AS rank,
    @prev := item_sorted.item_id
FROM ...

What are your experiences in implementing such feature? What is the database design that you would recommend? Would a database like Oracle or T-SQL capable DBs be a lot better due to their support for partitioned queries?

Thanks in advance for any ideas!

Answer 1

Let me start with a question :

What is the main goal of this table ? To insert data in it or to retrieve data from it?

Two possibilities :

• For inserting : I would recommend you to normalize this, you will have a lot of redundancy.
• For retrieving : It is good the way it is. I would recommend a surrogate key for each set.

If you want to compare time consumtion, try to install PostgreSQL and see the result. You should not need to use Oracle or MS-SQL, PostgreSQL should be good enough.

I know that my recommendation has nothing to do with data-partitioning, but having a similar table at work this is how I do it and it works fine.

Hope I could help,