different settings for max_heap_table_size and tmp_table_size (MariaDB 10.1.x)

Question

I run pt-variable-advisor and got a note about different settings for max_heap_table_size and tmp_table_size.

With a web search I found only old articles (around 2007).

pt-variable-advisor h=localhost,u=root,p=Quule0juqu7aifohvo2Ahratit --socket /var/vcap/sys/run/mysql/mysqld.sock
(...)
# NOTE tmp_table_size: The effective minimum size of in-memory implicit temporary tables used internally during query execution is min(tmp_table_size, max_heap_table_size), so max_heap_table_size should be at least as large as tmp_table_size.
(...)

Our configuration

max_heap_table_size = 16777216
tmp_table_size = 33554432

We didn't modify the defaults from cf-mysql-release. I saw that MariaDB KB recommends other default values.

  cf_mysql.mysql.tmp_table_size:
    description: 'The maximum size (in bytes) of internal in-memory temporary tables'
    default: 33554432

  cf_mysql.mysql.max_heap_table_size:
    description: 'The maximum size (in rows) to which user-created MEMORY tables are permitted to grow'
    default: 16777216

I found also Optimize MySQL tmp_table_size and checked our values and the configuration:

MariaDB [(none)]> show global status like 'created_tmp_disk_tables';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Created_tmp_disk_tables | 12727901 |
+-------------------------+----------+
1 row in set (0.01 sec)

MariaDB [(none)]> show global status like 'created_tmp_tables';
+--------------------+-----------+
| Variable_name      | Value     |
+--------------------+-----------+
| Created_tmp_tables | 115714303 |
+--------------------+-----------+
1 row in set (0.01 sec)

MariaDB [(none)]> select (12727901*100)/(115714303 + 12727901) as "Created disk tmp tables ratio" from dual;
+-------------------------------+
| Created disk tmp tables ratio |
+-------------------------------+
|                        9.9094 |
+-------------------------------+
1 row in set (0.00 sec)

Is something wrong with our (default) configuration? We don't know our workload. We run about 500 small databases for small web apps with different usage pattern.

Answer 1

Short Answer: None of those is "wrong".

How are those settings used?

There are (if I am not mistaken) exactly 2 uses for those two settings. They are, as alluded to in some of what you quoted:

• When you do CREATE TABLE ... ENGINE=MEMORY, it is given a limit of the current value of max_heap_table_size. You can, and "should", change that setting just before doing that CREATE. Too large a value could waste precious RAM.

• When complex SELECT needs to create a temporary table, such as in preparation for ORDER BY, it first tries to use a MEMORY table; if this fails (for any of several reasons), it resorts to using MyISAM. The limit on the MEMORY table size is min(max_heap_table_size, tmp_table_size).

I do not believe tmp_table_size is every used by itself.

If you never explicitly create a MEMORY table, the having those two settings the same avoids confusion and discussions like this.

So, the admonition to make them equal is weaker than the wording you quote. It is "sorta, kinda, oughta", and not "you MUST!".

A different perspective: Let me approach your question from another point of view.

Intra-select tmp tables can happen often -- in every connection, and even multiple times in a single select. So, keeping tmp_table_size "low" is important to avoid blowing out RAM. I recommend no more than 1% of RAM, but this is rather arbitrary.

If, meanwhile, you are playing around with MEMORY tables, you need control over their sizes.

OK, I have hit a dead end. Why do they have the "min(...)"? Why not just leave them separate?

I don't know.

Future: The above discussion applies to at least MySQL versions 4.0 to 5.7, plus all versions (so far) of MariaDB. MySQL 8.0 uses a "temp table engine", which may lead to some of the above discussion being moot.