Common-practice in dealing with high-load tables in MySQL

Question

I have a table in MySQL 5 (InnoDB) that is used as a daemon Processing Queue, thus it is being accessed very often. It is typical to have around 250 000 records inserted per day. When I select records to be processed, they are read using a FOR UPDATE query to eliminate race conditions (everything is Transaction Based).

Now I am developing a "queue archive" and I have stumbled into a serious dead-lock problem. I need to delete "executed" records from the table as they are being processed (live), yet the table dead-locks every once in a while if I do so (two-three times per day at).

I though of moving towards delayed deletion (once per day at low load times) but this will not eliminate the problem only make it less obvious.

Is there a common-practice in dealing with high-load tables in MySQL?

Answer 1

InnoDB locks all rows it examines, not only those requested.

See this question for more details.

You need to create an index that would exactly match your search condition to get rid of unnecessary locks, and make sure it is used.

Unfortunately, DML queries in MySQL do not accept hints.