Demonized Celery for django project

Question

I have already wasted a lot of time figuring out how to execute task in django project with demonized celery

VERSIONS
django version is 1.3
celery version is 3.1

my directory structure is

project/
   ./manage.py
   ./settings.py

   ./celeryapp/
       ./celeryqueue.py
       ./tasks.py

i have been able to run celery in demonized mode by creating "/etc/init.d/celeryd" as specified at here

also copied config file for demonized celery specified at here

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Where to chdir at start.
CELERYD_CHDIR="/opt/project/"

# Extra arguments to celeryd
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# Name of the celery config module.
CELERY_CONFIG_MODULE="celeryconfig"

# %n will be replaced with the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
CELERYD_USER="celery"
CELERYD_GROUP="celery"

My celeryconfig.py is

BROKER_URL = 'amqp://'

CELERY_RESULT_BACKEND = 'amqp://'

CELERY_IMPORTS = ("celeryapp.tasks",)

I have created celery user and given required privileges by executing

rabbitmqctl add_user celery celery  
rabbitmqctl add_vhost celeryvhost  
rabbitmqctl set_permissions -p celeryvhost celery ".*" ".*"

Still i am getting KeyError saying

Received unregistered task of type "mytaskname"

This is the traceback

ERROR/MainProcess] Received unregistered task of type 'process_transcoding'.  
The message has been ignored and discarded.  
Did you remember to import the module containing this task?  
Or maybe you are using relative imports?  
Please see http://bit.ly/gLye1c for more information.  
The full contents of the message body was:  
{'utc': True, 'chord': None, 'args': [2202L,u'/opt/ossite/media/videos/Screenshot_from_2013-09-24_150354_1.png', u'/opt/ossite/media/thumbnails/Screenshot_from_2013-09-24_150354_1.jpeg', 'thumbnail'], 'retries': 0, 'expires': None, 'task': 'process_transcoding', 'callbacks': None, 'errbacks': None, 'timelimit': (None, None), 'taskset': None, 'kwargs': {}, 'eta': None, 'id': 'f8468f52-ade3-4405-96d6-21a187583c77'} (381b)  
Traceback (most recent call last):  
File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 457, in  on_task_received  
strategies[name](message, body,
KeyError: 'process_transcoding'

Desperately waiting for help.

Thanks in advance.

Answer 1

Finally My boss got the bandwidth to sit with me, and he found the exact problem.

And the problem was relative imports...

I just needed to append "project." for every project related import for all the imports in tasks.py and in all the imports in imported files too. e.g

tasks.py  
from celeryapp.celeryqueue import app # replaced by  
from project.celeryapp.celeryqueue import app  # and  

from abc import xyz  # is replced by  
from project.abc import xyz  

xyz.py  
from pqr import stu # is replced by
from project.pqr import stu  

and so on...

Here is the link as daniula suggested relative imports

Answer 2

Can you add "--loglevel=DEBUG" to configs:

# Extra arguments to celeryd
CELERYD_OPTS="--time-limit=300 --concurrency=8 --loglevel=DEBUG"

Check the log files and edit your question with whatever unusual you see there. The problem you described can be due to many reasons.