Magento 2.3 - what is the recommended way of looping through a large product collection
-
16-04-2021 - |
Frage
When working with large product collections (50,000 products +) in Magento 1 I ended up using something similar to the following.
public function getProductCollection()
{
$productsCollection = Mage::getModel('catalog/product')
->getCollection()
->addAttributeToSelect(array('name', 'price', 'visibility'));
$productsCollection->setPageSize(50);
$pages = $productsCollection->getLastPageNumber();
$currentPage = 1;
do {
$productsCollection->setCurPage($currentPage);
$productsCollection->load();
foreach ($productsCollection as $_product) {
echo $_product->getId() . "\n";
echo $_product->getSku() . "\n";
echo $_product->getName() . "\n";
/* do stuff */
}
$currentPage++;
//clear collection and free memory
$productsCollection->clear();
} while ($currentPage <= $pages);
}
Does anyone have experience on working with very large product collections in Magento2? What is the recommended approach?
Lösung
Someone on the magento forum suggested something like this:
public function __construct(
\Magento\Catalog\Model\ResourceModel\Product\CollectionFactory $collectionFactory
) {
$this->collectionFactory = $collectionFactory;
}
public function yourMethod()
{
$collection = $this->collectionFactory->create();
$collection->addAttributeToSelect(['name','price','visibility']);
$collection->setPageSize(50);
$pages = $collection->getLastPageNumber();
for ($pageNum = 1; $pageNum<=$pages; $pageNum++) {
$collection->setCurPage($pageNum);
foreach ($collection as $item) {
//Do something.
}
$collection->clear();
}
}
Andere Tipps
Also I've come across this:
https://magento.stackexchange.com/a/213059/70343
In theory sound although I've not tested.
The recent comment reminded me I later came back to this and wrote two versions of an export script borrowing some logic from core for iterator
Uses iterator
https://github.com/DominicWatts/ProductCsvExport/blob/1.0.2/Console/Command/Product.php#L297
Uses collection
https://github.com/DominicWatts/ProductCsvExport/blob/1.0.1/Console/Command/Product.php#L238
Iterator handles memory usage but is much slower approach. If you've got a collections with thousands of products it might be the only suitable approach
Anyway hope helps someone