Simple queue with Celery and RabbitMQ
Question
I'm trying to implement a simple queue that performs one task at a time. Offloading tasks off the main thread using Celery and setting concurrency=1 in the Celery config works fine, but I might want to use more concurrent workers for other tasks.
Is there a way to tell Celery or RabbitMQ to not use multiple concurrent workers for a task (except by forcing concurrency=1)? I can't find anything in the documentation but maybe these tools are not designed for a linear queue?
Thanks!
Solution
I think what you need is a separate queue for each type of task. Create separate workers that consume from each queue, with concurrency set to 1.
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow