In memory database error: table is full

Question

I got hit by error "Tale is full" on my in memory database.

My database is 4.5GB in size and has 300 partitions.

I run MySql 5.6.17 version_compile_machine=x86_64 on 64 bit Windows 10, 16GB ram. Disk are NTFS and have 100GB spare place.

I've setup tmp_table_size and max_heap_table_size to 12GB.
Those values ware 30 MiB untill "table is full" error, so it seems that they ware and are ignored (changing to 12GB did not help.)

If it has to do something with this setting innodb buffer pool size is 12GiB also.

System runs on 50% of RAM memory.

What else can do to increase allowed size of in memory DB?

Answer 1

I was changing tmp_table_size and max_heap_table_size both via PhpMyAdmin and via SET GLOBAL ... in my scripts. And it didn't work.

Changing those varaibles via my.ini, restarting server and then loading DB into memory allowed for bigger DB.

Answer 2

"I've setup tmp_table_size and max_heap_table_size to 12GB." -- TERRIBLE!

One query that has an ORDER BY and needs a tmp table could allocate up to that much space. 10 connections... Well, it is mind boggling.

I recommend no more than 1% of RAM (160M in your case) for both of those VARIABLES.

"System runs on 50% of RAM memory." -- I'd say you have a "ticking bomb".

"300 partitions" -- as in "CREATE TABLE ... PARTITION BY ..."? A single table with 300 partitions is inefficient; the practical limit is about 50.

"memory DB" -- You are trying to "CREATE TABLE ... ENGINE=MEMORY"? And fill it with lots of data? Why? It is usually quite adequate to use ENGINE=InnoDB and let the caching of that engine give you about as much performance as MEMORY.