Partitioning increases CPU usage in mariadb

Question

I'm using mariadb 10.2, and I had a table with a structure similar to the one below, currently with 15gb, where I insert a few thousand records per minute all the time, each batch for a specific account (AccountId1).

AccountId1 bigint PK,
AccountId2 bigint PK,
ActionType int,
ActionDate timestamp

Looking at the process list, multiple inserts were waiting in line for this table, and they were always one insert running at a time, so I decided to partition this table by hash on the AccountId1, and created 32 partitions.

The inserts now jumped from 1 or 2 a second to 8 during peak times, and that worked great. The issue is that the CPU usage jumped from 15% to 40% after the change, and even stopping the insert process completely for a few hours, the CPU won't go down.

There are a few places where we select from this table that could span all partitions, but these are processes that run every few hours and don't take that long. Other frequent queries always use the AccountId1 field and are indeed querying a single partition by looking at the query plan.

Is there any overhead associated with partitioning, even if no sql is touching that table? What could be the reason for this increase in CPU?

Note: there are around 2000 concurrent connections at any time, but the # of connections before and after the partitioning were the same.

Answer 1

OVERHEAD

What needs to be understood is a partitioned table's physical implementation.

For example, in my old post (Aug 31, 2014 : How does subpartitioning actually work and what physical files are created?), I displayed what a partitioned table looks like:

C:\>cd \MySQL_5.6.15\data\test

C:\MySQL_5.6.15\data\test>dir nums_comp*
 Volume in drive C is TI10665200H
 Volume Serial Number is A273-2EFF

 Directory of C:\MySQL_5.6.15\data\test

08/31/2014  04:09 PM            98,304 nums_composite#p#p0#sp#p0sp0.ibd
08/31/2014  04:09 PM            98,304 nums_composite#p#p0#sp#p0sp1.ibd
08/31/2014  04:09 PM            98,304 nums_composite#p#p1#sp#p1sp0.ibd
08/31/2014  04:09 PM            98,304 nums_composite#p#p1#sp#p1sp1.ibd
08/31/2014  04:09 PM            98,304 nums_composite#p#p2#sp#p2sp0.ibd
08/31/2014  04:09 PM            98,304 nums_composite#p#p2#sp#p2sp1.ibd
08/31/2014  04:09 PM             8,594 nums_composite.frm
08/31/2014  04:09 PM                96 nums_composite.par
               8 File(s)        598,514 bytes
               0 Dir(s)  686,691,409,920 bytes free

C:\MySQL_5.6.15\data\test>

I have another example in Feb 16, 2015 : MySQL Create Table with Partition Slow on server machine

Each partition is treated as a separate InnoDB table with a distinct tablespace.

What overhead exists ?

• Memory Usage for the Information Schema (Adding new tables -- memory usage increases)
• Index Stats Separated By Partition (Do I need to index a key that I partition by?)
• Open File Handles (In your case, 33 [1 File Handle for Each .ibd file, 1 for .frm])

While these things help explain the overhead, you must still be wondering where the CPU activity is coming from ???

CPU

There will still be some CPU activity based on the open file handles. How ?

The option innodb_open_files places a limit on the number of open file handles against all InnoDB tables through the MySQL Instance.

If table info needs to be examined by monitoring software, more queries make more CPU.

If there are any queries that do not access disk (such as SHOW PROCESSLIST; or SHOW GLOBAL STATUS;) or reads from INFORMATION_SCHEMA, more CPU. (See my old ServerFault post from 2011-08-10 1 billion mysql queries in 24 days? Can something be wrong?)

Opening and closing files handles to access certain tables and read 16K pages into the InnoDB Buffer or marking pages in the InnoDB Buffer Pool as old.