Cannot get django celeryd to work on Windows
-
28-04-2021 - |
Question
I have a django project which runs just fine with runserver located in:
D:\EverTabs\work\evertabs-website\src\evertabs\
My settings.py and manage.py reside in the above path.
I open the trusty cmd shell, cd to that directory and run manage.py celeryd
, I get an exception with the following error:
ImportError: Could not import settings 'evertabs.settings' (Is it on sys.path?): No module named evertabs.settings
fair, enough. Google and StackOverflow to the rescue and I find an answer. I need to provide --settings=settings
to the command. Strange, as runserver does not need --settings=settings
, but such is life. I'm willing to compromise. I continue to get an error:
Traceback (most recent call last):
File "C:\Progra~1\Python26\lib\multiprocessing\process.py", line 231, in _bootstrap
self.run()
File "C:\Progra~1\Python26\lib\multiprocessing\process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "c:\program files\python26\lib\site-packages\celery-2.4.6-py2.6.egg\celery\concurrency\processes\pool.py", line 173, in worker
initializer(*initargs)
File "c:\program files\python26\lib\site-packages\celery-2.4.6-py2.6.egg\celery\worker\__init__.py", line 64, in process_initializer
app.loader.init_worker()
File "c:\program files\python26\lib\site-packages\celery-2.4.6-py2.6.egg\celery\loaders\base.py", line 100, in init_worker
self.on_worker_init()
File "c:\program files\python26\lib\site-packages\django_celery-2.4.2-py2.6.egg\djcelery\loaders.py", line 92, in on_worker_init
autodiscover()
File "c:\program files\python26\lib\site-packages\django_celery-2.4.2-py2.6.egg\djcelery\loaders.py", line 114, in autodiscover
for app in settings.INSTALLED_APPS])
File "c:\program files\python26\lib\site-packages\django_celery-2.4.2-py2.6.egg\djcelery\loaders.py", line 124, in find_related_module
app_path = importlib.import_module(app).__path__
File "build\bdist.win32\egg\importlib\__init__.py", line 37, in import_module
ImportError: No module named evertabs
So I scratch my head vigorously and decide that maybe if I feed this guy a different pythonpath, he will place nicely. My next try is:
manage.py celeryd --pythonpath=D:\EverTabs\work\evertabs-website\src\
Now we're getting somewhere! No errors. However, my celeryconfig.py, which is in the same directory as settings.py, is not getting processed. None of my tasks are loaded. I went as far as putting a syntax error into celeryconfg.py to prove to myself it is not being loaded.
The next step I did was to use a manage.py from django 1.4. This manage.py is placed in D:\EverTabs\work\evertabs-website\src\
as opposed to D:\EverTabs\work\evertabs-website\src\evertabs\
. At this point, I can execute celeryd with the simple syntax and not have to add to the pythonpath. Still, my celeryconfig.py is left untouched:
D:\EverTabs\work\evertabs-website\src>manage.py celeryd
Further research shows that if I add a tasks.py to my top level dir 9where setting.py is located). Celery will auto-discover them. This has worked. however, celeryconfig.py is still not being processed so I cannot do more intricate config in there. For now, this is not a concern, as my tasks are finally found. However, it bothers me greatly that celeryconfig.py is not being processed.
It is now dawning on me that maybe modern celery is not looking for celeryconfig.py after all. I can simply put these settings in my settings.py and celery will pick them up. Things are starting to look good.
Solution
django-celery uses settings.py as a celery configuration file.
The other main difference is that configuration values are stored in your Django projects’ settings.py module rather than in celeryconfig.py.
OTHER TIPS
Short summary of things I did to get celery to work:
Use either
--pythonpath
to point to the directory above your project directory so that celery can import project.settings or move your manage.py up one level and use the manage.py which django 1.4 generates.Create a tasks.py in your top level of your project (where settings.py is located) and celery will auto-discover them. If you want to use another module, add
CELERY_IMPORTS = ("yourproject.anothermodule", )
to your settings.py so that celery will look inside ofanothermodule
for tasks.Forget about
celeryconfig.py
which is mentioned in some documentation. Just put all the settings right in yoursettings.py