I need a django-celery daemon to listen to specific rabbitmq channel

Question

In fact I have few django applications with celery tasks. I need each task to be executed within particular channel so that I can control the load. For example I may have 3 servers listen to channel_for_app_1 and two to channel_for_app_2. My question is how can I run celery daemon and specify the channel? Any other ways to do that?

Answer 1

Please review this page: http://docs.celeryproject.org/en/latest/userguide/routing.html.

Your celery just should start with -Q, --queue settings, which define which queue will be used for fetching tasks

Answer 2

In your django settings, you can specify CELERY_QUEUES:

CELERY_QUEUES = {
"worker": {
    "exchange": "worker",
    "binding_key": "worker" 
},

like so. Each key is a queue name, and you can change the exchange and binding key if you want to get fancy (multiple exchanges, etc), but I've never needed to.

When you define a task, you can

@task(queue="worker", etc)

The last step is to specify the queue names when you run celery - either via your celery daemon configuration, or on the command line when you run it. The result of all of this is that celery tasks will go to the queues specified by the task definition, and only on the boxes running the specified queue.

So I'm not sure if you mean something explicitly different when you say "channels", but I've always used multiple per-task queues to do exactly what you're describing.