Determine theoretical max connections for MariaDB

Question

I want to upgrade the ram our MariaDB server (10.1.25-MariaDB-1~xenial) to handle more connections.

I have been determining the max connections mostly based on this article which suggests to use this formula

(Available_RAM - Global_buffers) / Thread_buffers

Global_buffers it the sum of these variables.

key_buffer_size
innodb_buffer_pool_size
innodb_log_buffer_size
innodb_additional_mem_pool
net_buffer_length

Thread_buffers is the sum of these variables.

sort_buffer_size
myisam_sort_buffer_size
read_buffer_size
join_buffer_size
read_rnd_buffer_size

This appears to work and for me it gives approx. 15 connections as my theoretical max.

When I query SHOW STATUS LIKE 'max_used_connections', which to my understanding gives me the historical maximum connections requested, I get 46 connections.

For background when I run SHOW VARIABLES LIKE 'max_connections' I get 151 which I believe is the default setting value.

Is 15 connections actually my theoretical max and did the additional 31 connections failed but MySQL registered them?

Is my math wrong or more likely am I missing some variables?

Answer 1

Every formula I have found is wrong in some respect. Almost every computer I have encountered has "overcommitted" RAM; that is, the formula would say that too much RAM could be used.

Their list is missing two important settings: max_heap_table_size and tmp_table_size. The min of those two is what is used for internal temp tables in complex SELECTs. And there could be multiple uses in a single query! I recommend limiting each to 1% of RAM.

Overall...

• Leave most VARIABLES at the defaults.
• The 1% mentioned above
• use InnoDB, not MyISAM.
• Set key_buffer_size to just 20M.
• Set innodb_buffer_pool_size to about 70% of available RAM if you have more than 4GB of RAM. (A lower percentage for tiny machines.) This is the first thing to reach for if swapping occurs.
• When you find problems (swapping, high cpu, high I/O, etc), ask for help. It will usually be solved by better INDEXing and/or reformulating queries, not by tuning.

Ignore the formulas like what your question is posing.

As for "too many connections", there are two main causes and tuning MySQL/MariaDB is not the cure.

• The queries are running so slowly that they need help -- tuning queries is a solution here.
• The clients are flooding the database. Some web servers think that they can efficiently run hundreds or thousands of threads. But, in reality, they stumble over themselves and cause the database to also stumble over itself. Throttle them.

On this last bullet item, think of a grocery store where so many shoppers are in there that they can't even push their carts to the next aisle, much less ever get finished.

Put another way, if you let too many connections in, throughput stalls (hits some limit) and latency suffers badly.

Things to watch for:

• Swapping means things are tuned too high -- discussed above
• 100% CPU or I/O -- Fine and fix the queries. For help in finding them: http://mysql.rjweb.org/doc.php/mysql_analysis#slow_queries_and_slowlog
• Poor latency (queries taking "too long") -- See the other two items.

Answer 2

In your case the default MAX_CONNECTION is 151 connections & the MySQL instance has only reached 46 connections as MAX_USED_CONNECTIONS ( till now the instance has only 46 concurrent connection ).

For increasing RAM according to max_connection limit, you can get estimate by filling you instance parameter value on following site, this will get approx RAM value which is going to used when your instance hit MAX_CONNECTION limit :

https://www.mysqlcalculator.com/

Answer 3

Adding to what Rick James says in his answer, human-initiated database connections may be long-lived but most of the time they tend to remain idle, unnecessarily hogging database server resources. Consider using connection pooling in your application, and your perceived maximum of 15 connections could very well be enough to support the pool without causing waits on the application side.