Is it safe to disable Magento Message Queues ( Magento_Amqp )?

Question

Upon checking my server logs I see that in the system.log I have a lot of records with this error: main.INFO: Consumer "async.operations.all" skipped as required connection "amqp" is not configured. Unknown connection name amqp [] []

Could someone please confirm if I can disable this module I am not able to understand what it is even used for. I've read this article from the official Magento docs: Manage message queues but still I am not able to understand do I have to leave it enabled or I can disable it. If someone could please explain what this module does would be great as well.

I am using Magento 2.3.2 on a VPS.

Answer 1

I keep bumping in this so I read up a bit more and now feel there is a need for better answers

Solutions that tell you to do cron_run = 'false" will disable other queues you do want, like bulk attribute update or export file generation.

Option 1 disable package

php bin/magento module:disable Magento_WebapiAsync
php bin/magento module:disable Magento_Amqp

They are useful for large site for bulk update and integrations, but for smaller sites totally fine without

Option 2 Tweak your queue cron, dont disable it

1. List your existing queues

   bin/magento queue:consumers:list

2. Add all of them (actually, those that make sense, read update below) except async to your cron consumer, in env.php. If there is functionality you never use, skip that one (but remember you did...)

    'cron_consumers_runner' => [
       'cron_run' => true,
       'max_messages' => 2,
       'single_thread' => true,
        'consumers' => [
        'product_action_attribute.update',
        'product_action_attribute.website.update',
        'exportProcessor'
        ]
    ],

Most places have a higher max_messages but if you are not having rabbitmq you are possibly on a low budget host too and short queues more often are better. Up the number as you want.

UPDATE: these queues can end up staying up forever and there are a lot of them so if your server is not very busy and not very meaty that can take a permanent slice of memory off for no valuable use. Also if you don't set 'single thread' you end up with multiple queues waiting for the same thing.

It is useful to REMOVE any of the queues that are not relevant to your usage, put them in a slower cron with very low message numbers or run them manually should you need them once. Eg: I don't use coupon generation so removed codegenerator. I also don't do mass inventory updates often, so just run it manually as bin/magento queue:consumers:start --single-thread --max-messages=1 inventory.mass.update

UPDATE: there is a "process whats there don't wait around" option now, though it cannot be set via command line, or only at installation time. It is called queue/consumers_wait_for_messages and technically should mean your queue runner clears the queue then exists, not waiting around. This will use a bit more CPU but free memory

I have tried to add it to my env.php in two places and it seems to have worked.

'queue' => [
    'consumers_wait_for_messages' => 0,
],
'cron_consumers_runner' => [
    'cron_run' => true,
    'max_messages' => 2,
    'single_thread' => true,
    'consumers-wait-for-messages' => 0,
    'consumers' => [
        'product_action_attribute.update',
        'product_action_attribute.website.update',
        'exportProcessor',
        'inventory.source.items.cleanup',
        'inventory.mass.update',
        'inventory.reservations.cleanup',
        'inventory.reservations.update'
    ]
],

ALSO: queue names have changed and there are some new ones in 5.4

product_action_attribute.update
product_action_attribute.website.update
exportProcessor
inventory.source.items.cleanup
inventory.mass.update
inventory.reservations.cleanup
inventory.reservations.update
media.storage.catalog.image.resize
codegeneratorProcessor  (coupon codes)
inventory.reservations.updateSalabilityStatus
inventory.indexer.sourceItem
inventory.indexer.stock
media.content.synchronization
media.gallery.synchronization
async.operations.all (meant for a real queue system with third party integrations)

** option 3 - out of internal cron and manually run or individually in cron**

The command to run them individually in cron on on command line is

bin/magento queue:consumers:start --single-thread --max-messages=1 inventory.source.items.cleanup

Answer 2

The answer is yes.You can disable module Magento_Amqp.

Magento have mechanism asynchronous that avoid many heavy task (export file, indexing table..) execute in same time. Just imagine in back-end,you have multiple admin user export products into csv file in the same time. It will really make your server stressful. So mechanism asynchronous allow when admin user export file, this task(message) will be pushed into a queue, it will be processed one by one.Your admin user may not see the export file right away, but after a few minutes the export file will be shown.

If you're using Magento_Amqp,you have to use third-party software like RabitMQ and configured at `app/etc/env.php` file to implement that mechanism. Its advantage is nearly real-time.The admin user will see the export file as soon as possible.

If you don't use Magento_Amqp,it's okay,you can disable it. The Magento has already Magento_MysqlMq module. It also implements asynchronous mechanism in database. Its disadvantage is not real-time.It depend on how period the cron you setup. Your admin user may have to wait

I hope you got the ideal. Sorry for my bad English

Answer 3

It is optional yes.

Message queuing allows different components to communicate by sending messages to each other. The message queue provides temporary message storage when the destination component is busy or not responding.

Although remove it via composer

https://github.com/yireo/magento2-replace-core/blob/magento-2.3.2/composer.json

    "magento/module-amqp": "*"

That said you might use third party extensions which require it

Answer 4

Magento_Amqp :- Magento_Amqp module provides functionality to publish/consume messages with the Advanced Message Queuing Protocol (AMQP).

And more information read this link :-

https://devdocs.magento.com/guides/v2.3/mrg/ce/Amqp.html

If you don't want to use the Magento Message Queues, you can just disable it.

php bin/magento module:disable Magento_Amqp

If you want to use it, you might modify your app/etc/env.php

<?php
return [

// ...

'cron_consumers_runner' => [
    'cron_run' => false,
    'max_messages' => 1000,
    'consumers' => [
    'async.operations.all',
    ]
]

]; // return closing

Reference: https://devdocs.magento.com/guides/v2.3/config-guide/mq/manage-message-queues.html#configuration