Why does Django-Celery schedule long ETA?

Question

I have a simple Django Celery task like:

@task(name="upload_file_to_cdn", max_retries=10)
def upload_file_to_cdn(file_id):
    stuff

When I launch this via a Django shell:

./manage shell
from myapp.tasks import upload_file_to_cdn
upload_file_to_cdn.apply_async(args=(123,))

Celery reports it runs immediately:

tail -f /var/log/celeryd.log
[2014-01-02 22:54:18,694: INFO/MainProcess] Received task: tasks.upload_file_to_cdn[6e58b72b-8832-427d-a6b8-574f71ac9baa]
[2014-01-02 22:54:18,698: DEBUG/MainProcess] Task accepted: tasks.upload_file_to_cdn[6e58b72b-8832-427d-a6b8-574f71ac9baa] pid:5007
[2014-01-02 22:54:18,714: WARNING/Worker-1] Uploading file 123...

However, when I run the exact same line upload_file_to_cdn.apply_async(args=(123,)) from within a Django view, all I see in the celeryd.log is:

[2014-01-02 22:47:04,006: INFO/MainProcess] Received task: tasks.upload_file_to_cdn[cb8be827-4604-4aa8-81b0-879657827e63] eta:[2014-01-03 03:47:19.004160-05:00]

As you can see, not only doesn't it run the task right away, it sets an ETA for approximately 6 hours later! Why is this?

Answer 1

The problem ended up being caused by different versions in the django-celery/celery Python packages installed on the separate hosts launching and executing the task. One was using celery==3.1.7 whereas the other was using celery==3.1.1 and that difference was enough to trigger the behavior. Syncing the versions fixed the problem.