Magento 2 import new and update current products with a CSV file and cron job

Question

I previously used Danslo ApiImport in several of my Magento 1.9.X projects in combination with a cron job to import new products and update existing products with CSV files automatically/periodically.

I'm wondering if there is a similar free extension/module for Magento 2.X that offers the same basic import/update functionality of default Magento 2 product attributes (preferably with basic documentation on how to use it)?

Or if someone could help with an example script on how to load data from a CSV file and update product qty and price or a script to create new simple products?

Answer 1

I made 2 scripts for updating products and importing new products, based on a few similar questions/answers I found on SME and my own research.

The solution is tested and works on Magento 2.2 (but probably works on 2.1.X as well).

The scripts work together with a manually set up cron job which calls them (for example every 5 mins -> this depends on specific shop needs).

For my needs, I set up the cron job to call a shell script every 5 mins, which calls these php scripts and archives the CSV files in a seperate folder for potential debugging purposes (which might be needed later) and deletes the archive files if they're older than 30 days.

Example CSV for updating price and qty

"sku","price","qty"
"test_sku1","100","10"
"test_sku2","123","12"

Script for updating product qty and price

$file = fopen('var/import/update.csv', 'r', '"'); // set path to the CSV file
if ($file !== false) {

    require __DIR__ . '/app/bootstrap.php';
    $bootstrap = \Magento\Framework\App\Bootstrap::create(BP, $_SERVER);

    $objectManager = $bootstrap->getObjectManager();

    $state = $objectManager->get('Magento\Framework\App\State');
    $state->setAreaCode('adminhtml');

    // used for updating product info
    $productRepository = $objectManager->get('Magento\Catalog\Model\ProductRepository');

    // used for updating product stock
    $stockRegistry = $objectManager->get('Magento\CatalogInventory\Api\StockRegistryInterface');

    // add logging capability
    $writer = new \Zend\Log\Writer\Stream(BP . '/var/log/import-update.log');
    $logger = new \Zend\Log\Logger();
    $logger->addWriter($writer);

    // enter the number of data fields you require the product row inside the CSV file to contain
    $required_data_fields = 3;

    $header = fgetcsv($file); // get data headers and skip 1st row

    while ( $row = fgetcsv($file, 3000, ",") ) {

        $data_count = count($row);
        if ($data_count < 1) {
            continue;
        }

        $data = array();
        $data = array_combine($header, $row);

        $sku = $data['sku'];
        if ($data_count < $required_data_fields) {
            $logger->info("Skipping product sku " . $sku . ". Not enough data to import.");
            continue;
        }

        $qty = trim($data['qty']);
        $price = trim($data['price']);

        echo 'Updating product SKU: '.$sku.', with Qty: '.$qty.' and Price:'.$price.'<br />';

        try {
            $product = $productRepository->get($sku);
        } 
        catch (\Exception $e) {
            $logger->info("Invalid product SKU: ".$sku);
            continue;
        }

        // You can set other product data with $product->setAttributeName() if you want to update more data
        if ($product->getPrice() != $price) {
            $product->setPrice($price) 
                    ->setStoreId(0) // this is needed because if you have multiple store views, each individual store view will get "Use default value" unchecked for multiple attributes - which causes issues.
                    ->save();
        }

        try {
            $stockItem = $stockRegistry->getStockItemBySku($sku);
        } 
        catch (\Exception $e) {
            $logger->info("Invalid stock for product SKU: ".$sku);
            continue;
        }

        if ($stockItem->getQty() != $qty) {
            $stockItem->setQty($qty);
            if ($qty > 0) {
                $stockItem->setIsInStock(1);
            }
            $stockRegistry->updateStockItemBySku($sku, $stockItem);
        }
    }
    fclose($file);
}

Example CSV for importing new products

"sku","name","description","short_description","price","qty"
"test-sku1","Test product name1","Test product description1","Test product short description1","100","20"
"test-sku2","Test product name2","Test product description2","Test product short description2","10","30"

Script for importing new products

$file = fopen('var/import/new.csv', 'r', '"'); // set path to the CSV file
if ($file !== false) {

    require __DIR__ . '/app/bootstrap.php';
    $bootstrap = \Magento\Framework\App\Bootstrap::create(BP, $_SERVER);

    $objectManager = $bootstrap->getObjectManager();

    $state = $objectManager->get('Magento\Framework\App\State');
    $state->setAreaCode('adminhtml');

    // used for updating product stock - and it's important that it's inside the while loop
    $stockRegistry = $objectManager->get('Magento\CatalogInventory\Api\StockRegistryInterface');

    // add logging capability
    $writer = new \Zend\Log\Writer\Stream(BP . '/var/log/import-new.log');
    $logger = new \Zend\Log\Logger();
    $logger->addWriter($writer);

    $header = fgetcsv($file); // get data headers and skip 1st row

    // enter the min number of data fields you require that the new product will have (only if you want to standardize the import)
    $required_data_fields = 3;

    while ( $row = fgetcsv($file, 3000, ",") ) {

        $data_count = count($row);
        if ($data_count < 1) {
            continue;
        }

        // used for setting the new product data
        $product = $objectManager->create('Magento\Catalog\Model\Product');         
        $data = array();
        $data = array_combine($header, $row);

        $sku = $data['sku'];
        if ($data_count < $required_data_fields) {
            $logger->info("Skipping product sku " . $sku . ", not all required fields are present to create the product.");
            continue;
        }

        $name = $data['name'];
        $description = $data['description'];
        $shortDescription = $data['short_description'];
        $qty = trim($data['qty']);
        $price = trim($data['price']);

        try {
            $product->setTypeId('simple') // type of product you're importing
                    ->setStatus(1) // 1 = enabled
                    ->setAttributeSetId(4) // In Magento 2.2 attribute set id 4 is the Default attribute set (this may vary in other versions)
                    ->setName($name)
                    ->setSku($sku)
                    ->setPrice($price)
                    ->setTaxClassId(0) // 0 = None
                    ->setCategoryIds(array(2, 3)) // array of category IDs, 2 = Default Category
                    ->setDescription($description)
                    ->setShortDescription($shortDescription)
                    ->setUrlKey($url_key) // you don't need to set it, because Magento does this by default, but you can if you need to
                    ->setWebsiteIds(array(1)) // Default Website ID
                    ->setStoreId(0) // Default store ID
                    ->setVisibility(4) // 4 = Catalog & Search
                    ->save();

        } catch (\Exception $e) {
            $logger->info('Error importing product sku: '.$sku.'. '.$e->getMessage());
            continue;
        }

        try {
            $stockItem = $stockRegistry->getStockItemBySku($sku);

            if ($stockItem->getQty() != $qty) {
                $stockItem->setQty($qty);
                if ($qty > 0) {
                    $stockItem->setIsInStock(1);
                }
                $stockRegistry->updateStockItemBySku($sku, $stockItem);
            }
        } catch (\Exception $e) {
            $logger->info('Error importing stock for product sku: '.$sku.'. '.$e->getMessage());
            continue;
        }
        unset($product);
    }
    fclose($file);
}

These are quick and "dirty" solutions which work well for simple shops and imports, but they would probably have to be adjusted according to the specific needs of each project.

For example for more complex (multistore) shops which import custom (non-default) product data - but the scripts are easily adjustible.

You can get any CSV data with $data['custom_attribute'] and set it for the product by using $product->setCustomAttribute($data['custom_attribute']).