magento 2- showing special price products on the product list page not working with flat catalog tables

Question

I have created functionality to show all the special price products on the product list page by extending the core class \Magento\Catalog\Model\Layer. This functionality is working fine with default database structure but, not working with Catalog flat table Below are the error showing on frontend:

Exception #0 (Zend_Db_Statement_Exception): SQLSTATE[42S22]: Column not found: 1054 Unknown column 'e.special_from_date' in 'where clause', query was: SELECT FLOOR((ROUND((e.min_price) * 1, 2)) / 100) + 1 AS range, COUNT(*) AS count FROM catalog_product_index_price AS e INNER JOIN catalog_category_product_index_store3 AS cat_index ON cat_index.product_id=e.entity_id AND cat_index.store_id=3 AND cat_index.visibility IN(2, 4) AND cat_index.category_id=2 INNER JOIN (SELECT product.entity_id AS entity_pk_value, rt.avg_percent FROM catalog_product_entity AS product LEFT JOIN (SELECT rova.entity_pk_value, avg(percent_approved) AS avg_percent FROM rating_option_vote_aggregated AS rova WHERE (rova.store_id = 3) GROUP BY rova.entity_pk_value) AS rt ON product.entity_id = rt.entity_pk_value GROUP BY product.entity_id) AS rt ON e.entity_id = rt.entity_pk_value LEFT JOIN review_entity_summary AS review_summary ON e.entity_id = review_summary.entity_pk_value AND review_summary.store_id = 3 AND review_summary.entity_type = (SELECT review_entity.entity_id FROM review_entity WHERE (entity_code = 'product')) INNER JOIN cataloginventory_stock_status AS stock_status_index ON e.entity_id = stock_status_index.product_id WHERE (((((e.special_from_date <= '2021-05-10 23:59:59') OR (e.special_from_date IS null))))) AND (((((e.special_to_date >= '2021-05-10 00:00:00') OR (e.special_to_date IS null))))) AND (e.special_price > '0.1') AND (e.price > '0.1') AND (stock_status_index.stock_status = 1) AND ( e.website_id = '1' ) AND ( e.customer_group_id = 0) AND (e.min_price IS NOT NULL) GROUP BY FLOOR((ROUND((e.min_price) * 1, 2)) / 100) + 1 ORDER BY (FLOOR((ROUND((e.min_price) * 1, 2)) / 100) + 1) ASC

Below is the code to filter the product collection:

namespace Vendor\Offer\Model;

use Magento\Catalog\Api\CategoryRepositoryInterface; use Magento\Catalog\Model\ResourceModel\Product\Attribute\CollectionFactory as AttributeCollectionFactory; use Magento\Catalog\Model\ResourceModel\Product\CollectionFactory;

class Layer extends \Magento\Catalog\Model\Layer { protected $productCollectionFactory;

public function __construct(
    \Magento\Catalog\Model\Layer\ContextInterface $context,
    \Magento\Catalog\Model\Layer\StateFactory $layerStateFactory,
    AttributeCollectionFactory $attributeCollectionFactory,
    \Magento\Catalog\Model\ResourceModel\Product $catalogProduct,
    \Magento\Store\Model\StoreManagerInterface $storeManager,
    \Magento\Framework\Registry $registry,
    CategoryRepositoryInterface $categoryRepository,
    CollectionFactory $productCollectionFactory,
    array $data = []
) {
    $this->productCollectionFactory = $productCollectionFactory;
    parent::__construct(
        $context,
        $layerStateFactory,
        $attributeCollectionFactory,
        $catalogProduct,
        $storeManager,
        $registry,
        $categoryRepository,
        $data
    );
}

public function getProductCollection()
{
    if (isset($this->_productCollections['xigen_custom'])) {
        $collection = $this->_productCollections['xigen_custom'];
    } else {
        $collection = $this->productCollectionFactory->create();
        $date = new \Zend_Date();

        $collection->addAttributeToFilter(
            'special_from_date',
            [
                'or' => [
                    0 => [
                        'date' => true,
                        'to' => $date->get('YYYY-MM-dd') . ' 23:59:59'
                    ],
                    1 => [
                        'is' => new \Zend_Db_Expr('null')
                    ],
                ]
            ],
            'left'
        );

        $collection->addAttributeToFilter(
            'special_to_date',
            [
                'or' => [
                    0 => [
                        'date' => true,
                        'from' => $date->get('YYYY-MM-dd') . ' 00:00:00'
                    ],
                    1 => [
                        'is' => new \Zend_Db_Expr('null')
                    ],
                ]
            ],
            'left'
        );
        $collection->addAttributeToFilter('special_price', ['gt' => '0.1']);
        $collection->addAttributeToFilter('price', ['gt' => '0.1']);
        // $collection->addAttributeToFilter('special_price', ['lt' => new \Zend_Db_Expr('at_price.value')]);
        $this->prepareProductCollection($collection);
        $this->_productCollections['xigen_custom'] = $collection;
    }
    return $collection;
}

}

Also, I change my approach to fix this issue, but still there some problem.

public function getProductCollection() {
    if (isset($this->_productCollections['xigen_custom'])) {
        $collection = $this->_productCollections['xigen_custom'];
    } else {
        $collection = $this->productCollectionFactory->create();

        $date = new \Zend_Date();
        $collection->addAttributeToSelect('*');
        $priceExpression = $collection->isEnabledFlat() ? new \Zend_Db_Expr('e.special_price') : new \Zend_Db_Expr('at_price.value');
        $collection->addAttributeToFilter('special_price', ['lt', $priceExpression]);
        $collection->addAttributeToFilter('special_to_date', ['neq' => ''])
                ->addAttributeToFilter('special_from_date', ['neq' => '']);
        $todayStartOnDate = $date->get('YYYY-MM-dd') . ' 00:00:00';
        $collection->getSelect()->where(
                (
                "("
                . "IF(at_special_to_date_default.special_from_date IS NOT NULL AND e.special_to_date IS NOT NULL,"
                       . " at_special_to_date_default.special_from_date<= '" . $todayStartOnDate . "' AND e.special_to_date >= '" . $todayStartOnDate . "',"
                       . "IF(at_special_to_date_default.special_from_date IS NOT NULL, at_special_to_date_default.special_from_date<= '" . $todayStartOnDate . "',"
                            . "IF(e.special_to_date IS NOT NULL, e.special_to_date >= '" . $todayStartOnDate . "','')"
                        . ")"
                    . ")"
                . ")"
                ));
        echo $collection->getSelect();exit;

        $this->prepareProductCollection($collection);
        $this->_productCollections['xigen_custom'] = $collection;
    }
    return $collection;
}

Please help me to fix thi sissue.

Answer 1

I think it is quite more complex than just extend that class because the value special_from_date and special_to_date are not part of the indexer table catalog_product_index_price. During the indexing process, Magento checks those dates and determines the final price value based on the current date.

I did some tests and I was not able to get a successful result using your approach, because catalog_product_index_price does not have special_from_date even not using the flat catalog directive. So, could you please share the SQL that is working for you? I am really wondering how you made it works.

My idea to achieve that goal would be trying to change the Magento price index behavior and include the data that you need over there. It is a huge SQL query but I am pretty sure it has all the information you need.

https://github.com/magento/magento2/blob/2.3/app/code/Magento/Catalog/Model/ResourceModel/Product/Indexer/Price/Query/BaseFinalPrice.php#L194

And you can also use the observer:

https://github.com/magento/magento2/blob/2.3/app/code/Magento/Catalog/Model/ResourceModel/Product/Indexer/Price/Query/BaseFinalPrice.php#L234

I did not check deeper but maybe you have to include some changes on the Elastic Search index. As far as I know, the products will come from ES on category and catalog search pages.

Further than that, I would recommend you change your class overwritten for the layer by a plugin over the method prepareProductCollection`:

https://github.com/magento/magento2/blob/2.3/app/code/Magento/Catalog/Model/Layer.php#L160

By doing that, you will have the benefit of using cache instead of search on the database for all requests.

I hope it helps you.