How to partition MySQL table with two indexes

Question

I have a table game_log with fields id, game_id and several varchar fields.

id is primary key and game_id is non-unique key.

There're two frequent queries:

SELECT * FROM game_log ORDER BY id DESC LIMIT 20
SELECT * FROM game_log WHERE game_id = <value> ORDER BY id DESC

The table is huge (6.1GB and 32M rows). InnoDB. Rows in it are being added randomly (one per query). Also, some games are being deleted.

I need to reduce disk IO and imrpove responsiveness.

Should I use key or range partitioning? If range, then by id or by game_id? Is there any theory?

Answer 1

Use partitioning by range.

If you partition by key, both of your example queries have to touch every partition.

The theory is that partitioning by KEY is like partitioning by hash, in that consecutive values of the primary key are bound to be stored in separate partitions. By querying a range of id values, you spoil the partition pruning.

Demo:

CREATE TABLE `game_log` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `game_id` int(11) NOT NULL DEFAULT '0',
  `xyz` varchar(15) DEFAULT NULL,
  PRIMARY KEY (`id`,`game_id`)
) 
PARTITION BY KEY ()
PARTITIONS 13;

INSERT INTO game_log (game_id) VALUES (1), (2), (3), (4), (5), (6);

EXPLAIN PARTITIONS SELECT * FROM game_log ORDER BY id DESC LIMIT 3\G
           id: 1
  select_type: SIMPLE
        table: game_log
   partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12

EXPLAIN PARTITIONS SELECT * FROM game_log WHERE game_id = 4 ORDER BY id DESC LIMIT 3\G
           id: 1
  select_type: SIMPLE
        table: game_log
   partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12

Whereas if you partition by range on game_id, you can get partition pruning to help you at least when you query for a specific game_id. But your query for any game_id order by id desc is still bound to touch every partition.

CREATE TABLE `game_log` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `game_id` int(11) NOT NULL DEFAULT '0',
  `xyz` varchar(15) DEFAULT NULL,
  PRIMARY KEY (`id`,`game_id`)
)
PARTITION BY RANGE (game_id)
(PARTITION p1 VALUES LESS THAN (3),
 PARTITION p2 VALUES LESS THAN (6),
 PARTITION p3 VALUES LESS THAN MAXVALUE);

INSERT INTO game_log (game_id) VALUES (1), (2), (3), (4), (5), (6);

EXPLAIN PARTITIONS SELECT * FROM game_log ORDER BY id DESC LIMIT 3\G
           id: 1
  select_type: SIMPLE
        table: game_log
   partitions: p1,p2,p3

EXPLAIN PARTITIONS SELECT * FROM game_log WHERE game_id = 4 ORDER BY id DESC LIMIT 3\G
           id: 1
  select_type: SIMPLE
        table: game_log
   partitions: p2