Question

We are running a site (Moodle) that the users currently find slow. I think I have tracked down the problem to MySQL creating temporary tables on disk. I watch the variable created_tmp_disk_tables in Mysql Workbench server administration and the number increases with roughly 50 tables/s. After a days usage, created_tmp_disk_tablesis >100k. Also, the memory does not seem to be released. The usage keeps increasing until the system becomes pretty much unusable and we have to re-start MySQL. I need to re-start it almost every day and it begins with using about 30-35% of available memory and finishing the day with 80%.

I have no blobs in the database and no control over the queries either so I can't attempt to optimise them. I have also used the Percona Confirguration Wizard to generate a configuration file but that my.ini didn't solve my problem either.

Questions

  1. What should I change to stop MySQL from creating temporary tables on disk? Are there settings I need to change? Should I throw more memory at it?

  2. How can I stop MySQL from eating up my memory?

Edit

I enabled slow_queries log and discovered that the query SELECT GET_LOCK() was logged as slow. A quick search revealed that I had allowed persistent connections in the PHP configuration (mysqli.allow_persistent = ON). I turned this off. This reduced the rate at which MySQL consumes memory.It is still creating temporary tables though.

I also checked that the key_buffer size is large enough. I looked at the variable key_writes. This should be zero. If not, increase the key_buffer_size.I have zero key_reads and zero key_writes so I assume that the key_buffer_size is large enough.

I increased the tmp_table_size and max-heap-table-size to 1024M as an increase in created_tmp_disk_tables may indicate that the tables can't fit in memory. This didn't solve it.

Ref: http://www.mysqlperformanceblog.com/2007/08/16/how-much-overhead-is-caused-by-on-disk-temporary-tables/

Edit 2

If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the sort_buffer_size value. I had 2 sort_merge_passes in an hour so I consider the sort_buffer_size to be large enough.

Ref: Mysql Manual on sort_buffer_size

Edit 3

I have modified the sort and join buffers as suggested by @RolandoMySQLDBA. The result is displayed in the table below but I think the created_tmp_tables_on_disk is still high. I restarted the mysql server after I changed the value and checked the created_tmp_tables_on_disk after a day (8h) and calculated the average. Any other suggestions? It seems to me that there is something that doesn't fit inside some kind of container but I can't work out what it is.

+---------------------+-------------+-------------+--------------------+
| Tmp_table_size,     | Sort_buffer | Join_buffer | No of created      |
| max_heap_table_size |             |             | tmp_tables on disk |
+---------------------+-------------+-------------+--------------------+
| 125M                | 256K        | 256K        |  100k/h            |
+---------------------+-------------+-------------+--------------------+
| 125M                | 512K        | 512K        |  100k/h            |
+---------------------+-------------+-------------+--------------------+
| 125M                | 1M          | 1M          |  100k/h            |
+---------------------+-------------+-------------+--------------------+
| 125M                | 4M          | 4M          |  100k/h            |
+---------------------+-------------+-------------+--------------------+   



This is my configuration:

+-----------------------+-----------------------+
|DATABASE SERVER        |WEB SERVER             |
+-----------------------+-----------------------+
|Windows Server 2008 R2 |Windows Server 2008 R2 |
+-----------------------+-----------------------+
|MySQL 5.1.48           |IIS 7.5                |
+-----------------------+-----------------------+
|4 Core CPU             |4 Core CPU             |
+-----------------------+-----------------------+
|4GB RAM                |8GB RAM                |
+-----------------------+-----------------------+

Additional information

+--------------------+---------+
|PARAM               |VALUE    |
+--------------------+---------+
|Num of tables in Db |361      |
+--------------------+---------+
|Size of database    |2.5G     |
+--------------------+---------+
|Database engine     |InnoDB   |
+--------------------+---------+
|Read/write ratio    |3.5      |
|(Innodb_data_read/  |         |
|innodb_data_written)|         |
+--------------------+---------+
|Avg table size      |15k rows |
+--------------------+---------+
|Max table size      |744k rows|
+--------------------+---------+

This setup was given to me so I have limited control over it. The web server is using very little CPU and RAM so I have excluded that machine as a bottleneck. A majority of the MySQL settings originates from a config auto-generation tool.

I have monitored the system using PerfMon over a few representative days. From that, I conclude that it is not the OS that is swapping to disk.

My.ini

[client]
port=3306
[mysql]
default-character-set=utf8

[mysqld]
port=3306
basedir="C:/Program Files/MySQL/MySQL Server 5.1/"
datadir="D:/DBs/Data/"
default-character-set=utf8
default-storage-engine=INNODB
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
max_connections=125
query_cache_size=350M
table_cache=1520
tmp_table_size=125M
table-definition-cache= 1024
max-heap-table-size= 32M
thread_cache_size=38

MyISAM Specific options
myisam_max_sort_file_size=100G
myisam_sort_buffer_size=125M
key_buffer_size=55M
read_buffer_size=1024K
read_rnd_buffer_size=256K
sort_buffer_size=1024K
join_buffer_size=1024K


INNODB Specific options
innodb_data_home_dir="D:/DBs/"
innodb_additional_mem_pool_size=32M
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=16M
innodb_buffer_pool_size=2G
innodb_log_file_size=407M
innodb_thread_concurrency=8

No correct solution

Licensed under: CC-BY-SA with attribution
Not affiliated with dba.stackexchange
scroll top