How to make long running bulk update

Question

I have a php script for making catalog updates of prices and inventory. This script might take quite a long time to run, because I need to update around 3k skus.

The site is on a shared hosting, so not all php.ini changes might be possible.

I have 3 problems / questions:

1. Should I shut down the site or can I avoid it?
2. If I can avoid shutting down the server, I need to preserve some cpu in order to let people access the site with decent reponse times
3. How can I avoid the max_execution_time error? Is there a suggested strategy for breaking the task into smaller chunks?

Answer 1

To answer your questions:

1. It is not necessary to shut down your server/site. You didn't specify if you have a supplier or you are managing your own inventory. In either scenario if a file is provided daily to run with the script, setup a cron to run it on "off" hours.

2. If you run the script on non-peak times, there shouldn't be a problem.

3. If there is no possibility of modifying the appropriate files, consider separating your script into 2. 1 for inventory updates and the other for price updates.

You should also have a quick run through of the following links and implement those that are applicable:

http://www.gxjansen.com/101-ways-to-speed-up-your-magento-e-commerce-website/ http://www.magentocommerce.com/boards/viewthread/36225/

Note: 3k products should be easily handled by a shared hosting environment. If you find your server too slow, contact your service provider and ask to be moved to another server.

Good luck,

SR

Answer 2

Bulk product updating process runs a quite long time, because of product model updating ($product->save()) fires a lot of observers. If you can add some coding to the script try add this logic given below.
1.Switch index process to "Manual indexing" before updating runs, after updating I revert to auto indexing:

//swich indexer to "manual update" mode
$pCollection = Mage::getSingleton('index/indexer')->getProcessesCollection();
foreach ($pCollection as $process) {
   $process->setMode(Mage_Index_Model_Process::MODE_MANUAL)->save();
}

//Your script starts.
//Here is your codes
//which updates collection of products within the loop.
//foreach($collection as $product){
//   ...
//   $product->save();
//   ...
//} 
//
//End your custom update script.

//swich indexer to "update on save" mode
$pCollection = Mage::getSingleton('index/indexer')->getProcessesCollection();
foreach ($pCollection as $process) {
   $process->setMode(Mage_Index_Model_Process::MODE_REAL_TIME)->save();
}

After sucessfully updating all products you should re-index all indexers via Magento admin panel.

2. If you want update some attributes of products (but not inventory attributes), use this code also:

Mage::getSingleton('catalog/product_action')
   ->updateAttributes($arrayOfProductIds, array('some_attribute' =>'some_value'), Mage::app()->getStore()->getId());

Two snippets given above really help your script to run fast.