Celery: Start task from Django management command

Question

I'm trying to write a custom management command that starts one or several Celery tasks.

For this, I defined a helper function:

from celery import chord
from front import models
from worker.tasks import process_account, notify  # The celery tasks

def register_check(account_ids, user):
    if not account_ids:
        return
    tasks = [process_account.si(user.pk, acc) for acc in account_ids]
    maintask = chord(tasks)(notify.s())
    maintask.track_started = True
    kwargs = {
        'task_id': maintask.id,
        'user': user,
        'accounts': map(int, account_ids),
    }
    user_job = models.UserJob.objects.create(**kwargs)
    return user_job

Everything works nice when starting tasks this way from a view, but when calling the same function from a custom management command, it doesn't work:

import sys
from django.core.management.base import BaseCommand
from front.utils import register_check
from front.models import User

class Command(BaseCommand):
    help = 'Run recurring checks. Use --daily, --weekly and/or --monthly options.'

    def handle(self, *args, **options):
        users = User.objects.filter(condition='value')
        for user in users:
            # fetch account_ids
            user_job = register_check(account_ids=account_ids, user=user)
        sys.stdout.write('Done, processed {} users.\n'.format(users.count()))

Instead, I get the following traceback:

  # (...)
  File "/home/danilo/.virtualenvs/radar/lib/python2.7/site-packages/amqp/connection.py", line 136, in __init__
    self.transport = create_transport(host, connect_timeout, ssl)
  File "/home/danilo/.virtualenvs/radar/lib/python2.7/site-packages/amqp/transport.py", line 264, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/home/danilo/.virtualenvs/radar/lib/python2.7/site-packages/amqp/transport.py", line 99, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused

Do I somehow have to initialize Celery in the management command? Or is there some setting I need to enable to get Celery tasks to work from a custom management command?

(I don't think it's relevant, but we're using Redis and Postgres as broker/backend.)

Answer 1

You could check where on your application you're calling import djcelery djcelery.setup_loader(), because maybe that is not being called when you run the management command.