For 768M RAM server is 617M allocation in my.cnf good or bad?

Question

I have a write-intensive web app, and I bought a VDS with 768M RAM

• Is this my.cnf good for my purposes?

I specified bulk_insert_buffer_size to 126M and some other parameters as below:

[mysqld]
bulk_insert_buffer_size=126M
set-variable = max_connections=500
safe-show-database
max_user_connections=200
key_buffer_size = 16M
query_cache_size = 50M
tmp_table_size = 50M
max_heap_table_size  = 200M
thread_cache_size = 4
table_cache = 80
thread_concurrency = 8
innodb_buffer_pool_size = 100M
innodb_log_file_size = 25M
query_cache_limit = 50M
innodb_flush_log_at_trx_commit = 2

Answer 1

This depends on the amount of MySQL Data and what storage engines your are using

When it comes to MyISAM and InnoDB, they cache differently. I wrote a post about this back on April 14, 2011.

Since your machine only has 768M of RAM you have to do a nice balancing act depending on which storage engine you are using the most.

All MyISAM

Please run this query

Query #1

SELECT CONCAT(ROUND(KBS/POWER(1024,
IF(PowerOf1024<0,0,IF(PowerOf1024>3,0,PowerOf1024)))+0.4999),
SUBSTR(' KMG',IF(PowerOf1024<0,0,
IF(PowerOf1024>3,0,PowerOf1024))+1,1))
recommended_key_buffer_size FROM
(SELECT LEAST(POWER(2,32),KBS1) KBS
FROM (SELECT SUM(index_length) KBS1
FROM information_schema.tables
WHERE engine='MyISAM' AND
table_schema NOT IN ('information_schema','mysql')) AA ) A,
(SELECT 2 PowerOf1024) B;

This query will tell you how big a MyISAM Key Buffer you data needs in MB. Of course, the amount installed RAM takes precedence, so use common sense. You should run this query:

Query #2

SELECT
    DataSize/POWER(1024,2) DataMB,
    IndexSize/POWER(1024,2) IndexMB,
    (DataSize/IndexSize) DataToIdexRatio
FROM
(
    SELECT SUM(data_length) DataSize,SUM(index_length) IndexSize
    FROM information_schema.tables
    WHERE engine='MyISAM' AND
    table_schema NOT IN ('information_schema','mysql')
) A;

If the DataToIndexRatio is less than 3, look over all the indexes. You may have indexes that are too big or just too many indexes. If you remove all unused indexes, this will raise the DataToIndexRatio and then you can run Query #1, and get a smaller recommendation. Again used common sense since you only have 768M. Personally, I would not exceed 256M. Therefore, you should choose either 256M or the result of Query #1, whichever is smaller.

If you have no InnoDB Data whatsoever, you added this to my.cnf (my.ini)

[mysqld]
skip-innodb

and restart mysql. That way, mysqld will startup faster because it not have to check for InnoDB crash recovery.

You mentioned your app is write intensive. A mix of INSERTs, UPDATEs, and DELETE will cause table and index fragmentation in a hurry.If this is the case, you need to convert the row format of the MYiSAM tables to Fixed. Why?

The book MySQL Database Design and Tuning recommends using ROW_FORMAT=FIXED on pages 72,73. This will internally convert all VARCHAR fields to CHAR. It will make the MyISAM table larger, but executed SELECTs against it will be much faster. I can personally attest to this. I once had a table that was 1.9GB. I changed the format with ALTER TABLE tblname ROW_FORMAT=FIXED. The table ended up 3.7GB. The speed of the SELECTs against it was 20-25% faster without improving or changing anything else.

All InnoDB

Please run this query

Query #3

SELECT CONCAT(ROUND(KBS/POWER(1024,
IF(PowerOf1024<0,0,IF(PowerOf1024>3,0,PowerOf1024)))+0.49999),
SUBSTR(' KMG',IF(PowerOf1024<0,0,
IF(PowerOf1024>3,0,PowerOf1024))+1,1)) recommended_innodb_buffer_pool_size
FROM (SELECT SUM(data_length+index_length) KBS FROM information_schema.tables
WHERE engine='InnoDB') A,
(SELECT 2 PowerOf1024) B;

This query will tell you how big an InnoDB Buffer Pool you data and indexes need in MB. Of course, the amount installed RAM takes precedence, so use common sense. You should run Query #2 and apply the same DataToIndexRatio rule even more strictly because secondary indexes in InnoDB tend to be more bloated than their MyISAM counterparts.

If you are running MySQL 5.5, please tune InnoDB for Multiple CPU engagement:

• Possible to make MySQL use more than one core?
• About single threaded versus multithreaded databases performance
• Multi cores and MySQL Performance

Because you only have 768MB of RAM, I again recommend 256MB for the innodb_buffer_pool_size or the result of Query #3, whichever is smaller.

You should resize the InnoDB Log Buffers. For example, if the innodb_buffer_pool_size you choose is 256M, you need to resize the log files to 25% of that number, 64M like this:

STEP 01) Shutdown mysql

• service mysql stop for Linux
• net stop mysql for Windows

STEP 02) Delete ib_logfile0 and ib_logfile1

STEP 03) Add this to my.cnf (my.ini)

[mysqld]
innodb_log_file_size = 64M

STEP 04) Startup mysql (log files are recreated to the new size)

• service mysql start for Linux
• net start mysql for Windows

InnoDB and MyISAM Mixed

I would apply the above rules and split up 128MB for key_buffer_size and 128MB for innodb_buffer_pool size.

SUMMARY

I choose 256M because regardless of the Storage Engine Mix you have, you have a write intensive app. You will need to do good caching for InnoDB as well as MyISAM indexes.

On the flip side, I only set aside 256MB of one or both storage engines because the OS will need as much RAM as possible for the following aspects:

• Operating System
• Query Cache
• DB Connections (Buffers Per Connections,Max Connection Count)
• File Handles (open_files_limit, innodb_open_files (if using innodb_file_per_table))

Have fun configure all these aspects !!!