I have stubbled with this and there isn't a single "best" way to do this. It's important to keep in mind a few things.
- Celery is run on production as a separate executable. This seems obvious but that means that sharing log files with other stuff (like django) won't work nicely (See CELERYD_HIJACK_ROOT_LOGGER) so don't do it..
- Keeping things running in a similar manner on both production and your development environment is really critical so things don't go awry when you move to procution (See CELERY_ALWAYS_EAGER and use it)
- Celery when run from manage.py will reference your
settings.py
file for configuration if you use settings
With that in mind here is how I would suggest you set up your logging to meet your needs. First make sure that you have the logging level for your modules appropriately set. See my other post on how to set up logging. Here is a condensed version to meet your needs
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': "[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s",
'datefmt': "%d/%b/%Y %H:%M:%S"
},
},
'handlers': {
'console': {
'level': 'DEBUG',
'()': 'logutils.colorize.ColorizingStreamHandler',
'formatter': 'standard',
'stream': sys.stdout
},
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler',
'include_html': True,
},
},
'loggers': {
'django': {
'handlers': ['mail_admins'],
'propagate': True,
'level': 'ERROR',
},
'mymodule': {
'handlers': ['console', 'mail_admins'],
'propagate': True,
'level': 'DEBUG'
},
}
}
Then run it like this.
./manage.py celeryd --event --beat --settings=dev
What this does is sets two different handlers. The first one is the console handler which reports out anything DEBUG and above. The second is the mail handler which only looks at ERRORS and above. Next is what gets reported from the loggers. All django.* loggers will get reported to the any ERROR level handler (mail_admins). And any of our modules get push DEBUG and above to both.
I think this is what you are after.