Magento 2.2 error: Unable to unserialize value

Question

After upgrade to the version 2.2 my modules which has serialized values does not work any more.

Here is some info from the Magento 2.2 release notes:

Significant enhancements in platform security and developer experience. Security improvements include the removal of unserialize calls and protection of this functionality to increase resilence against dangerous code execution attacks. We have also continued to review and improve our protection against Cross-Site Scripting (XSS) attacks.
...
In general, we’ve removed serialize/unserialize from most the code to improve protection against remote code execution attacks. We’ve enhanced protection of code where use of object serialization or unserialization was unavoidable. Additionally, we’ve increased our use of output escaping to protect against cross-site scripting (XSS) attacks.

Its seems ok, but how I can correctly convert a data when a customer updates a module to the new version? In my case update to the version 2.2 does brokes the site with the error:

Unable to unserialize value.

I've added the correct serializer (Magento\Framework\Serialize\Serializer\Serialize) to the my resource models, but seems it is not correct solution.

In the Magento\Framework\Flag class I seen interesting solution, but seems it's not good enough:

/**
 * Retrieve flag data
 *
 * @return mixed
 */
public function getFlagData()
{
    if ($this->hasFlagData()) {
        $flagData = $this->getData('flag_data');
        try {
            $data = $this->json->unserialize($flagData);
        } catch (\InvalidArgumentException $exception) {
            $data = $this->serialize->unserialize($flagData);
        }
        return $data;
    }
}

where:

• $this->json instance of Magento\Framework\Serialize\Serializer\Json
• $this->serialize instance of Magento\Framework\Serialize\Serializer\Serialize

So, the question is: what is the correct solution for this case? Should I write the UpgradeData script that does unserialize the old values and serializes them back using Json (re-save all models)?

PS: I've read this posts

• Notice: unserialize(); Error at offset in Magento 2.2
• Magento 2.2: Unable to unserialize value?
• Magento 2 Unable to unserialize value

but there is no answer for my question.

Answer 1

So, solution is write a UpdateData script in the own module which change the default serialized values to the JSON (like Magento does it in the CatalogRules module):

app/code/MageWorx/ShippingRules/Setup/UpgradeData.php

class UpgradeData implements UpgradeDataInterface
{
    /**
     * @var \Magento\Framework\App\ProductMetadata
     */
    protected $productMetadata;

    /**
     * @var MetadataPool
     */
    private $metadataPool;

    /**
     * @var \Magento\Framework\DB\AggregatedFieldDataConverter
     */
    private $aggregatedFieldConverter;

    /**
     * UpgradeData constructor.
     *
     * @param MetadataPool $metadataPool
     * @param ObjectManagerInterface $objectManager
     */
    public function __construct(
        MetadataPool $metadataPool,
        ObjectManagerInterface $objectManager,
        \Magento\Framework\App\ProductMetadata $productMetadata
    ) {
        $this->productMetadata = $productMetadata;
        $this->metadataPool = $metadataPool;
        if ($this->isUsedJsonSerializedValues()) {
            $this->aggregatedFieldConverter = $objectManager->get('Magento\Framework\DB\AggregatedFieldDataConverter');
        }
    }

    /**
     * @return bool
     */
    public function isUsedJsonSerializedValues()
    {
        $version = $this->productMetadata->getVersion();
        if (version_compare($version, '2.2.0', '>=') &&
            class_exists('\Magento\Framework\DB\AggregatedFieldDataConverter')
        ) {
            return true;
        }

        return false;
    }

Here we use the ObjectManager class because there is no such class \Magento\Framework\DB\AggregatedFieldDataConverter in the magento versions < 2.2.

Then initialize the update method:

/**
 * {@inheritdoc}
 */
public function upgrade(ModuleDataSetupInterface $setup, ModuleContextInterface $context)
{
    $setup->startSetup();

    // 2.0.2 - module version compatible with magento 2.2
    if (version_compare($context->getVersion(), '2.0.2', '<') && $this->aggregatedFieldConverter) {
        // Convert each entity values
        $this->convertRuleSerializedDataToJson($setup);
        $this->convertZoneSerializedDataToJson($setup);
    }

    $setup->endSetup();
}

and method for conversion:

/**
 * Convert Zone metadata from serialized to JSON format:
 *
 * @param ModuleDataSetupInterface $setup
 *
 * @return void
 */
protected function convertZoneSerializedDataToJson(ModuleDataSetupInterface $setup)
{
    $metadata = $this->metadataPool->getMetadata(ZoneInterface::class);
    $this->aggregatedFieldConverter->convert(
        [
            new FieldToConvert(
                SerializedToJson::class,
                $setup->getTable(Zone::ZONE_TABLE_NAME),
                $metadata->getLinkField(),
                'conditions_serialized'
            ),
        ],
        $setup->getConnection()
    );
}

where:

• ZoneInterface - entity interface, usually located at the Module/Vendor/Api/Data/
• Zone::ZONE_TABLE_NAME - corresponding table name, like 'mageworx_shippingrules_zone' string
• conditions_serialized - name of the field having a serialized data, which should be converted to JSON

To make it work it is necessary to have few important things:

• Your interface should have corresponding model in the etc/di.xml:

  <preference for="MageWorx\ShippingRules\Api\Data\RuleInterface" type="MageWorx\ShippingRules\Model\Rule" />
  

• Corresponding entity repository should be defined in the RepositoryFactory in the etc/di.xml (and this repository should exist):

  <type name="Magento\Framework\Model\Entity\RepositoryFactory">
      <arguments>
          <argument name="entities" xsi:type="array">
              <item name="MageWorx\ShippingRules\Api\Data\RuleInterface" xsi:type="string">MageWorx\ShippingRules\Api\RuleRepositoryInterface</item>
              <item name="MageWorx\ShippingRules\Api\Data\ZoneInterface" xsi:type="string">MageWorx\ShippingRules\Api\ZoneRepositoryInterface</item>
          </argument>
      </arguments>
  </type>
  

• Your entity shoul be defined in the MetadataPool (in etc/di.xml):

  <type name="Magento\Framework\EntityManager\MetadataPool">
      <arguments>
          <argument name="metadata" xsi:type="array">
              <item name="MageWorx\ShippingRules\Api\Data\RuleInterface" xsi:type="array">
                  <item name="entityTableName" xsi:type="string">mageworx_shippingrules</item>
                  <item name="identifierField" xsi:type="string">rule_id</item>
              </item>
              <item name="MageWorx\ShippingRules\Api\Data\ZoneInterface" xsi:type="array">
                  <item name="entityTableName" xsi:type="string">mageworx_shippingrules_zone</item>
                  <item name="identifierField" xsi:type="string">entity_id</item>
              </item>
          </argument>
      </arguments>
  </type>
  

• In corresponding resource model default serializer (class) should not be defined or should be instance of Magento\Framework\Serialize\Serializer\Json (as default). It's stored in the $this->serializer attribute of the resource model.

If something goes wrong during the load process of your model I'll recomend you to start debugging from resource model, especially from the magento/framework/Model/ResourceModel/AbstractResource.php class from method protected function _unserializeField(DataObject $object, $field, $defaultValue = null) where:

• $object - should be instance of your model
• $field - serialized field, like conditions_serialized

The following example is taken from the MageWorx Shipping Suite extension.

Answer 2

The problem is in /vendor/magento/framework/Serialize/Serializer/Json.php there is a function unserialize($string) which gives you a syntax error if the string is already serialized.

There is a workaround - you can check if string is serialized and then use serialize($string). Change unserialize to:

public function unserialize($string)
{
    /* Workaround: serialize first if is serialized */
    if($this->is_serialized($string))
    {
        $string = $this->serialize($string);
    }
    $result = json_decode($string, true);
    if (json_last_error() !== JSON_ERROR_NONE) {
         throw new \InvalidArgumentException('Unable to unserialize value.');

    }
    return $result;
}

and add function to check if string is serialized:

function is_serialized($value, &$result = null)
{
    // Bit of a give away this one
    if (!is_string($value))
    {
        return false;
    }
    // Serialized false, return true. unserialize() returns false on an
    // invalid string or it could return false if the string is serialized
    // false, eliminate that possibility.
    if ($value === 'b:0;')
    {
        $result = false;
        return true;
    }
    $length = strlen($value);
    $end    = '';
    switch ($value[0])
    {
        case 's':
            if ($value[$length - 2] !== '"')
            {
                return false;
            }
        case 'b':
        case 'i':
        case 'd':
            // This looks odd but it is quicker than isset()ing
            $end .= ';';
        case 'a':
        case 'O':
            $end .= '}';
            if ($value[1] !== ':')
            {
                return false;
            }
            switch ($value[2])
            {
                case 0:
                case 1:
                case 2:
                case 3:
                case 4:
                case 5:
                case 6:
                case 7:
                case 8:
                case 9:
                    break;
                default:
                    return false;
            }
        case 'N':
            $end .= ';';
            if ($value[$length - 1] !== $end[0])
            {
                return false;
            }
            break;
        default:
            return false;
    }
    if (($result = @unserialize($value)) === false)
    {
        $result = null;
        return false;
    }
    return true;
}

After save fe. category without problem, You can restore class to default and there wont be such problem in future.

Answer 3

In my case flushing redis cache redis-cli flushall the solved issue.

Answer 4

@Edmund is correct, Json.php expects the value is json encoded but it's not, it's serialized and thus a syntax error exception is thrown.

Whilst some may find the overriden Json.php a possibly good solution, I prefer to not mess with core code. I fixed my problem quite simply as follows:

1. Add debug code to the exception to output the actual erroneous value, i.e. print_r($string, true) in Json.php.
2. Refreshed the page to get the debug logged.
3. Unserialized the data.
4. json encoded the data.
5. Manually replace the data in the db with json encoded value.
6. Refreshed the checkout and got no errors.
7. Undid the debug code from Json.php.

In my case it was the Amasty Payment Restrictions module, with the rules table containing serialized rule data instead of json encoded. It is more than likely the migration or the module upgrade did not convert the data appropriately.

Moral of the story, hunt down the offending data, add debug statements, get to the bottom of the real issue.

This to me is a preferred approach, fixing corrupt data which turned out to be the core issue, rather than the code.

I recommend that the Magento team take appropriate steps in rewriting the unserialize() function, to first check if the value is already serialized before attempting to json decode it, and release this fix in a new version.

PS: I'm using Magento 2.3.5-p1 so it was quite apparent it's not code related, but rather data/3rd party module related.