MySQL server has gone away when maual reindex catalog_product_flat

Question

In my Magento EE 14.0.1, when I try to reindex catalog_product_flat like this:

php indexer.php --reinex catalog_product_flat

After serveral minutes, it throws me this error:

Product Flat Data index process unknown error:
exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /var/www/magento/lib/Zend/Db/Statement/Pdo.php:228
Stack trace:
#0 /var/www/magento/lib/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array)
#1 /var/www/magento/lib/Varien/Db/Statement/Pdo/Mysql.php(110): Zend_Db_Statement_Pdo->_execute(Array)
#2 /var/www/magento/app/code/core/Zend/Db/Statement.php(291): Varien_Db_Statement_Pdo_Mysql->_execute(Array)
#3 /var/www/magento/lib/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
#4 /var/www/magento/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('SELECT `enterpr...', Array)
#5 /var/www/magento/lib/Varien/Db/Adapter/Pdo/Mysql.php(428): Zend_Db_Adapter_Pdo_Abstract->query('SELECT `enterpr...', Array)
#6 /var/www/magento/lib/Zend/Db/Adapter/Abstract.php(753): Varien_Db_Adapter_Pdo_Mysql->query(Object(Varien_Db_Select), Array)
#7 /var/www/magento/app/code/core/Mage/Core/Model/Resource/Db/Abstract.php(380): Zend_Db_Adapter_Abstract->fetchRow(Object(Varien_Db_Select))
#8 /var/www/magento/app/code/core/Mage/Core/Model/Abstract.php(225): Mage_Core_Model_Resource_Db_Abstract->load(Object(Enterprise_Mview_Model_Metadata), '10', NULL)
#9 /var/www/magento/app/code/core/Enterprise/Index/Model/Action/Abstract.php(167): Mage_Core_Model_Abstract->load('10')
#10 /var/www/magento/app/code/core/Enterprise/Catalog/Model/Index/Action/Product/Flat/Refresh.php(136): Enterprise_Index_Model_Action_Abstract->_setChangelogValid()
#11 /var/www/magento/app/code/core/Enterprise/Mview/Model/Client.php(109): Enterprise_Catalog_Model_Index_Action_Product_Flat_Refresh->execute()
#12 /var/www/magento/app/code/core/Enterprise/Catalog/Model/Index/Observer/Flat.php(78): Enterprise_Mview_Model_Client->execute('enterprise_cata...')
#13 /var/www/magento/app/code/core/Mage/Core/Model/App.php(1338): Enterprise_Catalog_Model_Index_Observer_Flat->processShellProductReindexEvent(Object(Varien_Event_Observer))
#14 /var/www/magento/app/code/core/Mage/Core/Model/App.php(1317): Mage_Core_Model_App->_callObserverMethod(Object(Enterprise_Catalog_Model_Index_Observer_Flat), 'processShellPro...', Object(Varien_Event_Observer))
#15 /var/www/magento/app/Mage.php(451): Mage_Core_Model_App->dispatchEvent('catalog_product...', Array)
#16 /var/www/magento/shell/indexer.php(167): Mage::dispatchEvent('catalog_product...')
#17 /var/www/magento/shell/indexer.php(212): Mage_Shell_Compiler->run()
#18 {main}

Next exception 'Zend_Db_Statement_Exception' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /var/www/magento/lib/Zend/Db/Statement/Pdo.php:234
Stack trace:
#0 /var/www/magento/lib/Varien/Db/Statement/Pdo/Mysql.php(110): Zend_Db_Statement_Pdo->_execute(Array)
#1 /var/www/magento/app/code/core/Zend/Db/Statement.php(291): Varien_Db_Statement_Pdo_Mysql->_execute(Array)
#2 /var/www/magento/lib/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
#3 /var/www/magento/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('SELECT `enterpr...', Array)
#4 /var/www/magento/lib/Varien/Db/Adapter/Pdo/Mysql.php(428): Zend_Db_Adapter_Pdo_Abstract->query('SELECT `enterpr...', Array)
#5 /var/www/magento/lib/Zend/Db/Adapter/Abstract.php(753): Varien_Db_Adapter_Pdo_Mysql->query(Object(Varien_Db_Select), Array)
#6 /var/www/magento/app/code/core/Mage/Core/Model/Resource/Db/Abstract.php(380): Zend_Db_Adapter_Abstract->fetchRow(Object(Varien_Db_Select))
#7 /var/www/magento/app/code/core/Mage/Core/Model/Abstract.php(225): Mage_Core_Model_Resource_Db_Abstract->load(Object(Enterprise_Mview_Model_Metadata), '10', NULL)
#8 /var/www/magento/app/code/core/Enterprise/Index/Model/Action/Abstract.php(167): Mage_Core_Model_Abstract->load('10')
#9 /var/www/magento/app/code/core/Enterprise/Catalog/Model/Index/Action/Product/Flat/Refresh.php(136): Enterprise_Index_Model_Action_Abstract->_setChangelogValid()
#10 /var/www/magento/app/code/core/Enterprise/Mview/Model/Client.php(109): Enterprise_Catalog_Model_Index_Action_Product_Flat_Refresh->execute()
#11 /var/www/magento/app/code/core/Enterprise/Catalog/Model/Index/Observer/Flat.php(78): Enterprise_Mview_Model_Client->execute('enterprise_cata...')
#12 /var/www/magento/app/code/core/Mage/Core/Model/App.php(1338): Enterprise_Catalog_Model_Index_Observer_Flat->processShellProductReindexEvent(Object(Varien_Event_Observer))
#13 /var/www/magento/app/code/core/Mage/Core/Model/App.php(1317): Mage_Core_Model_App->_callObserverMethod(Object(Enterprise_Catalog_Model_Index_Observer_Flat), 'processShellPro...', Object(Varien_Event_Observer))
#14 /var/www/magento/app/Mage.php(451): Mage_Core_Model_App->dispatchEvent('catalog_product...', Array)
#15 /var/www/magento/shell/indexer.php(167): Mage::dispatchEvent('catalog_product...')
#16 /var/www/magento/shell/indexer.php(212): Mage_Shell_Compiler->run()
#17 {main}

Next exception 'Enterprise_Index_Model_Action_Exception' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /var/www/magento/app/code/core/Enterprise/Catalog/Model/Index/Action/Product/Flat/Refresh.php:140
Stack trace:
#0 /var/www/magento/app/code/core/Enterprise/Mview/Model/Client.php(109): Enterprise_Catalog_Model_Index_Action_Product_Flat_Refresh->execute()
#1 /var/www/magento/app/code/core/Enterprise/Catalog/Model/Index/Observer/Flat.php(78): Enterprise_Mview_Model_Client->execute('enterprise_cata...')
#2 /var/www/magento/app/code/core/Mage/Core/Model/App.php(1338): Enterprise_Catalog_Model_Index_Observer_Flat->processShellProductReindexEvent(Object(Varien_Event_Observer))
#3 /var/www/magento/app/code/core/Mage/Core/Model/App.php(1317): Mage_Core_Model_App->_callObserverMethod(Object(Enterprise_Catalog_Model_Index_Observer_Flat), 'processShellPro...', Object(Varien_Event_Observer))
#4 /var/www/magento/app/Mage.php(451): Mage_Core_Model_App->dispatchEvent('catalog_product...', Array)
#5 /var/www/magento/shell/indexer.php(167): Mage::dispatchEvent('catalog_product...')
#6 /var/www/magento/shell/indexer.php(212): Mage_Shell_Compiler->run()
#7 {main}
PHP Fatal error:  Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /var/www/magento/lib/Zend/Db/Statement/Pdo.php:228
Stack trace:
#0 /var/www/magento/lib/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array)
#1 /var/www/magento/lib/Varien/Db/Statement/Pdo/Mysql.php(110): Zend_Db_Statement_Pdo->_execute(Array)
#2 /var/www/magento/app/code/core/Zend/Db/Statement.php(291): Varien_Db_Statement_Pdo_Mysql->_execute(Array)
#3 /var/www/magento/lib/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
#4 /var/www/magento/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('SELECT RELEASE_...', Array)
#5 /var/www/magento/lib/Varien/Db/Adapter/Pdo/Mysql.php(428): Zend_Db_Adapter_Pdo_Abstract->query('SELECT RELEASE_...', Array)
#6 /var/www/magento/app/code/core/Enterprise/Index/Model/Resource/Helper/Mysql4.php(56): Varien_Db_Adapter_Pdo_Mysql->query('SELECT RELEASE_...', Array)
#7 /var/www/magento/app in /var/www/magento/lib/Zend/Db/Statement/Pdo.php on line 234
PHP Fatal error:  Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /var/www/magento/lib/Zend/Db/Statement/Pdo.php:228
Stack trace:
#0 /var/www/magento/lib/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array)
#1 /var/www/magento/lib/Varien/Db/Statement/Pdo/Mysql.php(110): Zend_Db_Statement_Pdo->_execute(Array)
#2 /var/www/magento/app/code/core/Zend/Db/Statement.php(291): Varien_Db_Statement_Pdo_Mysql->_execute(Array)
#3 /var/www/magento/lib/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
#4 /var/www/magento/lib/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('SELECT RELEASE_...', Array)
#5 /var/www/magento/lib/Varien/Db/Adapter/Pdo/Mysql.php(428): Zend_Db_Adapter_Pdo_Abstract->query('SELECT RELEASE_...', Array)
#6 /var/www/magento/app/code/core/Enterprise/Index/Model/Resource/Helper/Mysql4.php(56): Varien_Db_Adapter_Pdo_Mysql->query('SELECT RELEASE_...', Array)
#7 /var/www/magento/app in /var/www/magento/lib/Zend/Db/Statement/Pdo.php on line 234

No additional error in system.log, exception.log or php error log. In MySQL Error Log, I find this Warning, that is print some seconds before:

Aborted connection 23157178 to db: 'magento_magento' user: 'usert' host: 'xx.xx.xx.xx' (Got timeout reading communication packets)

I tried setting max_allowed_packet as 64 MB in MySQL Configuration, but i isn't result.

Any idea, please?

EDIT1:

I tried increasing the wait_timeout to 3600 and max_allowed_packet to 999999488 (Bytes) but the error persists.

As additional, if I run some other index as catalog_url_product, catalogsearch_fulltext or catalog_product_attribute, these executed successfully.

I do not know if it has to do, but the database has replication. I have one Master and one Slave, and the error "Got timeout reading communication packets" appears in the log of the Master.

Answer 1

Well, I do not know whether the error is 100 % resolved or not. But based on what you told me here, and in this post : https://dba.stackexchange.com/questions/40899/mysql-error-reading-communication-packets, I make this changes in the MySQL Config File:

max-allowed-packet              = 1024M
wait-timeout                    = 7200
innodb-log-file-size            = 2047M
innodb-log-buffer-size         = 8M
innodb-log-buffer-size         = 32M

And this in my MySQL:

SET GLOBAL wait_timeout = 7200;
SET GLOBAL max_allowed_packet = 1000000000;
SET GLOBAL net_buffer_length = 1000000;

And after that, I could reindex all successfully. I don't know if in fact innodb-log-buffer-size and innodb-log-buffer-size were the trick.

I will continue monitoring, thanks.

Answer 2

This is mostly due to any of the below two reasons

1. Server timed out and closed the connection.
   fix: try increasing the wait_timeout variable in your mysqld’s my.cnf/my.ini configuration file.
2. Server dropped an incorrect or too large packet.
   fix : increase the maximal packet size limit by increasing the value of max_allowed_packet in my.cnf/my.ini file.

You have already increased max_allowed_packet to 64 MB, try to increase it to 128 or more as well, if it doesn't work.
It seems to be resources issue only.
Also check if indexation is not breaking due to any other reason as well, just to make sure if indexation process is fine.

Answer 3

Try to increase max_allowed_packet to more until it fix your issue. and restart mysql after changing this setting.