Mariadb Delete with Join query stuck sending data
https://dba.stackexchange.com/questions/259585
-
24-02-2021 - |
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianDomanda
I am using Mariadb (10.4.10-MariaDB-log) with Galera Cluster (3 Nodes).
I am running a large data queries with joins , but
No idea why delete with join is not working.
DROP TEMPORARY TABLE IF EXISTS tmp_VendorRate_stage_1;
CREATE TEMPORARY TABLE tmp_VendorRate_stage_1 (
TimezonesID int,
VendorConnectionID int,
RowCodeID int,
RowOriginationCodeID int,
OriginationCodeID int,
CodeID int,
INDEX Index1 (OriginationCodeID, CodeID),
INDEX Index2 (TimezonesID, VendorConnectionID),
INDEX Index3 (RowOriginationCodeID,RowCodeID,TimezonesID)
);
DROP TEMPORARY TABLE IF EXISTS tmp_VendorRate_stage_1_orig;
CREATE TEMPORARY TABLE tmp_VendorRate_stage_1_orig (
RowOriginationCodeID INT,
RowCodeID INT,
TimezonesID int,
INDEX Index1 (RowCodeID,TimezonesID)
);
explain delete v from tmp_VendorRate_stage_1 v -- has 22 Lacks records
inner join tmp_VendorRate_stage_1_orig vd on -- has 50K records only
v.RowOriginationCodeID = 0
and v.RowCodeID = vd.RowCodeID
and v.TimezonesID = vd.TimezonesID
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----------+--------------+---------------------------------+--------------------+---------+--------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| 109798498 | cluster-user | web-01-ic.colo.local:37024 | RM | Execute | 148498 | Sending data | delete v from tmp_VendorRate_stage_1 v inner join tmp_VendorRate_stage_1_orig vd on v.RowOriginationCodeID = 0 -- vd.RowOriginationCode and v.RowCodeID = vd.RowCodeID and v.TimezonesID = vd.TimezonesID | 0.000 |
my.cnf is as follows
[mysqld]
# 62GB RAM, 80% RAM innodb_buffer_pool_size 52G = 55834574848
innodb_buffer_pool_size=55834574848
#1G= 1073741824 500M = 524288000 50M= 52428800
sort_buffer_size=52428800
join_buffer_size=52428800
read_rnd_buffer_size=52428800
max_sort_length=8388608
max_length_for_sort_data=1048
# 1.5G = 1610612736 500M = 524288000
innodb_lock_wait_timeout=90
query_cache_type=1
query_cache_size=52428800
query_cache_limit=52428800
thread_cache_size=90
max_connections=500
#9G = 9663676416
max_heap_table_size=9663676416
tmp_table_size=9663676416
max_allowed_packet=52428800
table_open_cache=300
#Support Large Txn -- 1 GB
innodb_log_buffer_size=1073741824
#Log
log_output=FILE
log_warnings=2
log_error=mariadb.err
slow_query_log
slow_query_log = 1
slow_query_log_file = mariadb-slow.log
long_query_time=600
log_queries_not_using_indexes=0
tmpdir=/vol/data/mysql/mysqltmp
character-set-server=utf8
collation-server=utf8_unicode_ci
skip-character-set-client-handshake
#character-set-connection=utf8
innodb_buffer_pool_instances=4
innodb_page_cleaners=4
group_concat_max_len=1048576
# 2GB for last txn
#wsrep_max_ws_size=2147483647
#binlog_row_image='MINIMAL'
auto_increment_increment = 1
My engine innodb status output is
=====================================
2020-02-13 07:22:51 0x7f239f9d8700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 0 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 1742407 srv_active, 0 srv_shutdown, 226747 srv_idle
srv_master_thread log flush and writes: 1969115
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 59432844
OS WAIT ARRAY INFO: signal count 158954620
RW-shared spins 0, rounds 130112974, OS waits 24302409
RW-excl spins 0, rounds 627640588, OS waits 13136007
RW-sx spins 9528206, rounds 140611307, OS waits 1141519
Spin rounds per wait: 130112974.00 RW-shared, 627640588.00 RW-excl, 14.76 RW-sx
------------------------
LATEST DETECTED DEADLOCK
------------------------
2020-02-11 14:45:18 0x7f239e88f700
*** (1) TRANSACTION:
TRANSACTION 212382101, ACTIVE 68 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
0: len 6; hex 343338323431; asc 438241;;
1: len 4; hex 80000d3c; asc <;;
*** (2) TRANSACTION:
TRANSACTION 212381915, ACTIVE 72 sec fetching rows
mysql tables in use 1, locked 1
1023 lock struct(s), heap size 139472, 125877 row lock(s), undo log entries 53625
0: len 6; hex 343338323431; asc 438241;;
1: len 4; hex 80000d3c; asc <;;
------------
TRANSACTIONS
------------
Trx id counter 224340465
Purge done for trx's n:o < 224340456 undo n:o < 0 state: running but idle
History list length 4
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421265576491064, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421265576478296, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 224340464, ACTIVE 0 sec
mysql tables in use 6, locked 1
0 lock struct(s), heap size 1136, 0 row lock(s), undo log entries 1
MySQL thread id 120410187, OS thread handle 139790965274368, query id 10436889809 web-01-ic..colo.local 172.27.16.172 cluster-user Sending data
INSERT INTO tmp_AccountSOA(AccountID,Amount,OutPaymentType)
SELECT
i.AccountID,
itr.TaxAmount as Amount,
'1' as OutPaymentType
FROM Billing.tblInvoiceTaxRate itr
INNER JOIN Billing.tblInvoiceDetail id ON itr.InvoiceDetailID = id.InvoiceDetailID
INNER JOIN Billing.tblInvoice i ON id.InvoiceID=i.InvoiceID
INNER JOIN Billing.tblProduct p ON id.ProductID=p.ProductID AND p.Code='outpayment' AND id.ProductType=1
INNER JOIN tblAccountBalanceLog bl ON bl.AccountID=i.AccountID
WHERE ( (i.InvoiceType = 2
Trx read view will not see trx with id >= 224340464, sees < 224340449
---TRANSACTION 421265576474040, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421265576457016, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421265576503832, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 224340449, ACTIVE 7 sec starting index read
mysql tables in use 9, locked 9
5064 lock struct(s), heap size 696528, 418698 row lock(s)
MySQL thread id 119790053, OS thread handle 139790967731968, query id 10436883673 web-01-ic..colo.local 172.27.16.172 cluster-user Sending data
UPDATE
tblRateTableRate rtr
INNER JOIN
(
SELECT
rtr.RateTableRateID, rtr2.EffectiveDate,
@RateTableRateID := rtr.RateTableRateID
FROM
tblRateTableRate rtr
INNER JOIN tblRateTableRate rtr2
ON rtr2.RateTableId = rtr.RateTableId
AND rtr2.RateID = rtr.RateID
AND rtr2.OriginationRateID = rtr.OriginationRateID
AND rtr2.TimezonesID = rtr.TimezonesID
WHERE
rtr.EffectiveDate <= NOW() AND
rtr2.EffectiveDate <= NOW() AND
rtr.EffectiveDate < rtr2.EffectiveDate AND
rtr.RateTableRateID > @Last_RateTableRateID
---TRANSACTION 421265576439992, ACTIVE 136417 sec fetching rows
mysql tables in use 2, locked 2
0 lock struct(s), heap size 1136, 0 row lock(s)
MySQL thread id 109798498, OS thread handle 139790977562368, query id 10152749737 web-01-ic..colo.local 172.27.16.172 cluster-user Sending data
delete v from tmp_VendorRate_stage_1 v
inner join tmp_VendorRate_stage_1_orig vd on
v.RowOriginationCodeID = 0 -- vd.RowOriginationCode
and v.RowCodeID = vd.RowCodeID
and v.TimezonesID = vd.TimezonesID
---TRANSACTION 421265576444248, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421265576448504, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
20595129 OS file reads, 246623491 OS file writes, 115845287 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 72000.00 writes/s, 42000.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 756107, seg size 756109, 2758171 merges
merged operations:
insert 69433878, delete mark 108367687, delete 18915129
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 13812451, node heap has 7365 buffer(s)
Hash table size 13812451, node heap has 5079 buffer(s)
Hash table size 13812451, node heap has 9187 buffer(s)
Hash table size 13812451, node heap has 3256 buffer(s)
Hash table size 13812451, node heap has 84483 buffer(s)
Hash table size 13812451, node heap has 56920 buffer(s)
Hash table size 13812451, node heap has 79064 buffer(s)
Hash table size 13812451, node heap has 2251 buffer(s)
188229000.00 hash searches/s, 1780000.00 non-hash searches/s
---
LOG
---
Log sequence number 829435448161
Log flushed up to 829435448151
Pages flushed up to 829430785856
Last checkpoint at 829430271613
0 pending log flushes, 0 pending chkp writes
90297766 log i/o's done, 30.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 55868129280
Dictionary memory allocated 1253168
Buffer pool size 3334240
Free buffers 15422
Database pages 3071171
Old database pages 1133612
Modified db pages 613
Percent of dirty pages(LRU & free pages): 0.020
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 44554490, not young 2541174114
42000.00 youngs/s, 0.00 non-youngs/s
Pages read 20587562, created 9411808, written 144652301
0.00 reads/s, 0.00 creates/s, 36000.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 3071171, unzip_LRU len: 0
I/O sum[14760]:cur[144], unzip sum[0]:cur[0]
----------------------
INDIVIDUAL BUFFER POOL INFO
----------------------
---BUFFER POOL 0
Buffer pool size 833560
Free buffers 3802
Database pages 767800
Old database pages 283406
Modified db pages 137
Percent of dirty pages(LRU & free pages): 0.018
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 11079344, not young 671122572
0.00 youngs/s, 0.00 non-youngs/s
Pages read 5290427, created 2356093, written 42288396
0.00 reads/s, 0.00 creates/s, 9000.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 767800, unzip_LRU len: 0
I/O sum[3690]:cur[36], unzip sum[0]:cur[0]
---BUFFER POOL 1
Buffer pool size 833560
Free buffers 3607
Database pages 768085
Old database pages 283511
Modified db pages 102
Percent of dirty pages(LRU & free pages): 0.013
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 11127071, not young 628315259
0.00 youngs/s, 0.00 non-youngs/s
Pages read 5097290, created 2348852, written 23470594
0.00 reads/s, 0.00 creates/s, 9000.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 768085, unzip_LRU len: 0
I/O sum[3690]:cur[36], unzip sum[0]:cur[0]
---BUFFER POOL 2
Buffer pool size 833560
Free buffers 3887
Database pages 767769
Old database pages 283394
Modified db pages 232
Percent of dirty pages(LRU & free pages): 0.030
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 11126210, not young 618766159
0.00 youngs/s, 0.00 non-youngs/s
Pages read 5072830, created 2340780, written 46913362
0.00 reads/s, 0.00 creates/s, 9000.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 767769, unzip_LRU len: 0
I/O sum[3690]:cur[36], unzip sum[0]:cur[0]
---BUFFER POOL 3
Buffer pool size 833560
Free buffers 4126
Database pages 767517
Old database pages 283301
Modified db pages 142
Percent of dirty pages(LRU & free pages): 0.018
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 11221865, not young 622970124
42000.00 youngs/s, 0.00 non-youngs/s
Pages read 5127015, created 2366083, written 31979949
0.00 reads/s, 0.00 creates/s, 9000.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 767517, unzip_LRU len: 0
I/O sum[3690]:cur[36], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Process ID=23611, Main thread ID=139733626447616, state: sleeping
Number of rows inserted 6548565481, updated 80996548, deleted 249968847, read 989518208378
343000.00 inserts/s, 6000.00 updates/s, 0.00 deletes/s, 308361000.00 reads/s
Number of system rows inserted 21, updated 0, deleted 21, read 47
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
My mysqltuner output is as follows
>> MySQLTuner 1.7.19 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[--] Skipped version check for MySQLTuner script
[!!] Successfully authenticated with no password - SECURITY RISK!
[OK] Currently running supported MySQL version 10.4.10-MariaDB-log
[OK] Operating on 64-bit architecture
-------- Log file Recommendations ------------------------------------------------------------------
[OK] Log file /var/log/mysqld.log exists
[--] Log file: /var/log/mysqld.log(0B)
[OK] Log file /var/log/mysqld.log is readable.
[!!] Log file /var/log/mysqld.log is empty
[OK] Log file /var/log/mysqld.log is smaller than 32 Mb
[OK] /var/log/mysqld.log doesn't contain any warning.
[OK] /var/log/mysqld.log doesn't contain any error.
[--] 0 start(s) detected in /var/log/mysqld.log
[--] 0 shutdown(s) detected in /var/log/mysqld.log
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +Aria +CONNECT +CSV +InnoDB +MEMORY +MRG_MyISAM +MyISAM +PERFORMANCE_SCHEMA +SEQUENCE
[--] Data in InnoDB tables: 118.3G (Tables: 449)
[--] Data in CONNECT tables: 0B (Tables: 27)
[!!] Total fragmented tables: 2
-------- Analysis Performance Metrics --------------------------------------------------------------
[--] innodb_stats_on_metadata: OFF
[OK] No stat updates during querying INFORMATION_SCHEMA.
-------- Security Recommendations ------------------------------------------------------------------
[!!] There is no basic password file list!
-------- CVE Security Recommendations --------------------------------------------------------------
[--] Skipped due to --cvefile option undefined
-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 22d 23h 50m 39s (510M q [257.069 qps], 120M conn, TX: 1671G, RX: 1991G)
[--] Reads / Writes: 47% / 53%
[--] Binary logging is disabled
[--] Physical Memory : 62.9G
[--] Max MySQL memory : 634.8G
[--] Other process memory: 0B
[--] Total buffers: 61.3G global + 1.1G per thread (500 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 128M
[!!] Maximum reached memory usage: 281.5G (447.44% of installed RAM)
[!!] Maximum possible memory usage: 634.8G (1008.87% of installed RAM)
[!!] Overall possible memory usage with other process exceeded memory
[OK] Slow queries: 0% (398K/510M)
[OK] Highest usage of available connections: 38% (192/500)
[OK] Aborted connections: 0.00% (152/120455146)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[!!] Query cache may be disabled by default due to mutex contention.
[OK] Query cache efficiency: 35.6% (255M cached / 718M selects)
[!!] Query cache prunes per day: 2832
[OK] Sorts requiring temporary tables: 0% (8K temp sorts / 113M sorts)
[!!] Joins performed without indexes: 139393129
[OK] Temporary tables created on disk: 23% (97M on disk / 419M total)
[OK] Thread cache hit rate: 99% (1K created / 120M connections)
[!!] Table cache hit rate: 0% (897 open / 54M opened)
[!!] table_definition_cache(400) is lower than number of tables(639)
[OK] Open file limit used: 1% (31/2K)
[OK] Table locks acquired immediately: 99% (17M immediate / 17M locks)
-------- Performance schema ------------------------------------------------------------------------
[--] Performance schema is disabled.
[--] Memory used by P_S: 0B
[--] Sys schema isn't installed.
-------- ThreadPool Metrics ------------------------------------------------------------------------
[--] ThreadPool stat is enabled.
[--] Thread Pool Size: 16 thread(s).
[--] Using default value is good enough for your version (10.4.10-MariaDB-log)
-------- MyISAM Metrics ----------------------------------------------------------------------------
[!!] Key buffer used: 18.2% (24M used / 134M cache)
[!!] Cannot calculate MyISAM index size - re-run script as root user
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[!!] InnoDB buffer pool / data size: 52.0G/118.3G
[!!] Ratio InnoDB log file size / InnoDB Buffer pool size (0.180288461538462 %): 48.0M * 2/52.0G should be equal to 25%
[!!] InnoDB buffer pool instances: 4
[--] Number of InnoDB Buffer Pool Chunk : 416 for 4 Buffer Pool Instance(s)
[OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances
[OK] InnoDB Read buffer efficiency: 100.00% (519447941557 hits/ 519456596570 total)
[!!] InnoDB Write Log efficiency: 83.44% (453045190 hits/ 542967312 total)
[OK] InnoDB log waits: 0.00% (0 waits / 89922122 writes)
-------- AriaDB Metrics ----------------------------------------------------------------------------
[--] AriaDB is enabled.
[OK] Aria pagecache size / total Aria indexes: 128.0M/344.0K
[!!] Aria pagecache hit rate: 37.2% (152M cached / 95M reads)
-------- TokuDB Metrics ----------------------------------------------------------------------------
[--] TokuDB is disabled.
-------- XtraDB Metrics ----------------------------------------------------------------------------
[--] XtraDB is disabled.
-------- Galera Metrics ----------------------------------------------------------------------------
[--] Galera is enabled.
[--] GCache is using 0B
[--] CPU core detected : 16
[--] wsrep_slave_threads: 1
[!!] wsrep_slave_threads is not equal to 2, 3 or 4 times number of CPU(s)
[!!] gcs.fc_limit should be equal to 5 * wsrep_slave_threads
[!!] gcs.fc_limit should be equal to 5 * wsrep_slave_threads
[!!] gcs.fc_factor should be equal to 0.8
[OK] Flow control fraction seems to be OK (wsrep_flow_control_paused<=0.02)
[OK] Binlog format is in ROW mode.
[!!] InnoDB flush log at each commit should be disabled.
[--] Read consistency mode :OFF
[OK] Galera WsREP is enabled.
[OK] Galera Cluster address is defined: gcomm://172.16.1.173,172.17.1.173,172.18.1.173
[--] There are 3 nodes in wsrep_cluster_address
[OK] There are 3 nodes in wsrep_cluster_size.
[OK] All cluster nodes detected.
[OK] Galera Cluster name is defined: web_cluster
[OK] Galera Node name is defined: web-db-01
[!!] Galera Notify command is not defined.
[!!] Galera SST method is not xtrabackup based.
[OK] TOI is default mode for upgrade.
[--] Max WsRep message : 2.0G
[OK] Node is connected
[OK] Node is ready
[--] Cluster status :Primary
[OK] Galera cluster is consistent and ready for operations
[OK] Node and whole cluster at the same level: 1422a148-0f6c-11ea-bdad-5b132c2a2ccc
[OK] Node is synced with whole cluster.
[OK] There is no certification failures detected.
-------- Replication Metrics -----------------------------------------------------------------------
[--] Galera Synchronous replication: YES
[--] No replication slave(s) for this server.
[--] Binlog format: ROW
[--] XA support enabled: ON
[--] Semi synchronous replication Master: OFF
[--] Semi synchronous replication Slave: OFF
[--] This is a standalone server
-------- Recommendations ---------------------------------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
OPTIMIZE TABLE `RM`.`tblTempRateTableRate`; -- can free 315 MB
OPTIMIZE TABLE `RM`.`tblRateTableRateArchive`; -- can free 2522 MB
Total freed space after theses OPTIMIZE TABLE : 2837 Mb
Reduce your overall MySQL memory footprint for system stability
Dedicate this server to your database for highest performance.
Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1
We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
(specially the conclusions at the bottom of the page).
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64: https://bit.ly/2Fulv7r
Read this before increasing for MariaDB https://mariadb.com/kb/en/library/optimizing-table_open_cache/
This is MyISAM only table_cache scalability problem, InnoDB not affected.
See more details here: https://bugs.mysql.com/bug.php?id=49177
This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
Beware that open_files_limit (2505) variable
should be greater than table_open_cache (300)
Performance schema should be activated for better diagnostics
Consider installing Sys schema from https://github.com/mysql/mysql-sys for MySQL
Consider installing Sys schema from https://github.com/FromDual/mariadb-sys for MariaDB
Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: https://bit.ly/2TcGgtU
Ensure that all table(s) are InnoDB tables for Galera replication
Variables to adjust:
*** MySQL's maximum memory usage is dangerously high ***
*** Add RAM before increasing MySQL buffer variables ***
long_query_time (<= 10)
query_cache_size (=0)
query_cache_type (=0)
query_cache_size (> 50M)
join_buffer_size (> 50.0M, or always use indexes with JOINs)
table_open_cache (> 300)
table_definition_cache(400) > 639 or -1 (autosizing if supported)
performance_schema = ON enable PFS
innodb_buffer_pool_size (>= 118.3G) if possible.
innodb_log_file_size should be (=6G) if possible, so InnoDB total log files size equals to 25% of buffer pool size.
innodb_buffer_pool_instances(=52)
wsrep_slave_threads = 64
gcs.fc_limit= wsrep_slave_threads * 5
gcs.fc_limit= wsrep_slave_threads * 5
gcs.fc_factor=0.8
innodb_flush_log_at_trx_commit = 0
set up parameter wsrep_notify_cmd to be notify
set up parameter wsrep_sst_method to xtrabackup based parameter
What could be possible solutions.
Soluzione
Test
DELETE
FROM tmp_VendorRate_stage_1 v
WHERE EXISTS ( SELECT NULL
FROM tmp_VendorRate_stage_1_orig vd
WHERE v.RowOriginationCodeID = 0
AND v.RowCodeID = vd.RowCodeID
AND v.TimezonesID = vd.TimezonesID )
Ensure the indices by (RowCodeID, TimezonesID, RowOriginationCodeID) exists (in _orig table first 2 fields is enough).
Altri suggerimenti
Rate Per Second = RPS
Suggestions for your my.cnf [mysqld] section,
innodb_buffer_pool_size=50G # from 52G for 80% of 62G
sort_buffer_size=4M # from 52M for this per connection RAM request
join_buffer_size=1M # from 52M to support row pointers for join per connection
read_rnd_buffer_size=192K # from 52M to reduce handler_read_rnd_next RPS
query_cache_limit=1M # from 50M for reasonable limit of one query results
query_cache_min_res_unit=512 # from 4K to store more results in RAM
max_heap_table_size=620M # from ~9G for 1% of RAM
tmp_table_size=620M # from ~9G for 1% RAM
table_definition_cache=700 # to reduce opened_table_definitions count
You will find these configuration changes will REDUCE CPU busy significantly.
Indexes are needed A) to avoid 18 thousand queries per day taking more than 600 seconds and B) to avoid 6M joins per day with no indexes.
Many more Global Variables need to be adjusted.
Autorizzato sotto: CC-BY-SA insieme a attribuzione
Non affiliato a dba.stackexchange
