Large number of INSERT statements locking inserts on other tables in MySQL 5.5

Question

I'm trying to understand an issue I am having with a MySQL 5.5 server.

This server hosts a number of databases. Each day at a certain time a process runs a series of inserts into TWO tables within this database. This process lasts from 5 to 15 minutes depending on the amount of rows being inserted.

This process runs perfectly. But it has a very unexpected side effect. All other inserts and update's running on tables unrelated to the two being inserted to just sit and wait until the process has stopped. Reads and writes outside of this database work just fine and SELECT statements too are fine.

So how is it possible for a single table to block the rest of a database but not the entire server (due to loading)?

A bit of background:-

• Tables being inserted to are MyISAM with 10 - 20 million rows.

• MySQL is Percona V5.5 and is serving one slave both running on Debian.

• No explicit locking is called for by the process inserting the records.

• None of the Insert statements do not select data from any other table. They are also INSERT IGNORE statements.

ADDITIONAL INFO:

While this is happening there are no LOCK table entries in PROCESS LIST and the processor inserting the records causing this problem does NOT issue any table locks.

I've already investigated the usual causes of table locking and I think I've rules them out. This behaviour is either something to do with how MySQL works, a quirk of having large database files or possibly even something to do with the OS/File System.

Answer 1

After a few weeks of trying things I eventually found this: Yoshinori Matsunobu Blog - MyISAM and Disk IO Scheduler

Yoshinori demonstrates that changing the scheduler queue to 100000 (from the default 128) dramatically improves the throughput of MyISAM on most schedulers.

After making this change to my system there were no longer any dramatic instances of database hang on MyISAM tables while this process was running. There was slight slowdown as to be expected with the volume of data but the system remained stable.

Anyone experiencing performance issues with MyISAM should read Yoshinori's blog entry and consider this fix.