django-nose unit testing a celery task … missing database data

Question

I'm writing unit tests for a celery task using django-nose. It's fairly typical; a blank test database (REUSE_DB=0) that is pre-populated via a fixture at test time.

The problem I have is that even though the TestCase is loading the fixture and I can access the objects from the test method, the same query fails when executed within an async celery task.

I've checked that the settings.DATABASES["default"]["name"] are the same both in the test method and the task under test. I've also validated the that the task that's under test behaves correctly when invoked as a regular method call.

And that's about where I'm out of ideas.

Here's a sample:

class MyTest(TestCase):
    fixtures = ['test_data.json']

    def setUp(self):
        settings.CELERY_ALWAYS_EAGER = True # seems to be required; if not I get socket errors for Rabbit
        settings.CELERY_EAGER_PROPAGATES_EXCEPTIONS = True # exposes errors in the code under test.

    def test_city(self):
        self.assertIsNotNone(City.objects.get(name='brisbane'))
        myTask.delay(city_name='brisbane').get()
        # The following works fine: myTask('brisbane')

from celery.task import task

@task()
def myTask(city_name):
    c = City.objects.count() # gives 0
    my_city = City.objects.get(name=city_name) # raises DoesNotExist exception
    return

Answer 1

This sounds a lot like a bug in django-celery 2.5 which was fixed in 2.5.2: https://github.com/celery/django-celery/pull/116

The brief description of the bug is that the django-celery loader was closing the DB connection prior to executing the task even eager tasks. Since the tests run inside a transaction the new connection for the task execution can't see the data created in the setUp.