Running Django unittests causes South migrations to duplicate tables

Question

How do you prevent Django unittests from running South migrations?

I have a custom Django app, myapp, that I'm trying to test with manage.py test myapp but when I run it I get the error:

django.db.utils.OperationalError: table "myapp_mymodel" already exists

and sure enough, the traceback shows that South is being executed:

File "/usr/local/myproject/.env/local/lib/python2.7/site-packages/south/management/commands/test.py", line 8, in handle
super(Command, self).handle(*args, **kwargs)

However, in my settings, I've specified:

SOUTH_TESTS_MIGRATE = 0
SKIP_SOUTH_TESTS = 1

which I believe should prevent Django's test framework from executing any South components.

What am I doing wrong?

Edit: I worked around this by simply removing south with:

if 'test' in sys.argv:
    INSTALLED_APPS.remove('south')

However, then I got:

ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the NAME value.

For my test database settings, I was using:

DATABASES = {
    'default':{
        'ENGINE': 'django.db.backends.sqlite3'
    }
}

which worked fine in Django 1.4. Now I'm using Django 1.5, and I guess that's not kosher. However, no NAME value I see it to fixes it. They all report none of my tables exist. I've tried:

DATABASES = {
    'default':{
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': '/dev/shm/test.db',
        'TEST_NAME': '/dev/shm/test.db',
    }
}


DATABASES = {
    'default':{
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': ':memory:',
        'TEST_NAME': ':memory:',
    }
}

DATABASES = {
    'default':{
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(os.path.dirname(__file__), 'test.db'),
        'TEST_NAME': os.path.join(os.path.dirname(__file__), 'test.db'),
    }
}

each seems to create a physical test.db file, which I don't understand, because unittests should be run in-memory. It should never save anything to disk. Presumably, it's failing to run syncdb after creating the file but before it executes the actual unittest. How do I fix this?

Edit: I discovered that, in one of my forms, I was populating field choices by directly querying a model (whereas I should have been doing that inside the form's init), so when Django's test framework imported my model, it was trying to read the table before the sqlite3 database had been created. I fixed that, but now I'm getting the error:

DatabaseError: table "myapp_mythroughmodel" already exists

so I'm back to square one, even though it's throwing a different exception type than initially.

Edit: I had a duplicate through model defined, causing Django to attempt to create it twice, resulting in the error.

Answer 1

This error was the result of several problems. I'll summarize them here to help others who may have stumbled across this.

1. Ensure your settings.DATABASES is set correctly. Django's docs mention using TEST_NAME, but for clarity, I find it easier to check for the test command and override everything. e.g. at the bottom of my settings.py, I have:

   if 'test' in sys.argv:
       DATABASES = {
           'default':{
               'ENGINE': 'django.db.backends.sqlite3',
               'NAME': ':memory:',
           },
       }
   

   Unless you have a good reason, always use :memory: to ensure it runs in memory and doesn't create a physical file that will be bogged down on disk. For some odd reason, a lot of other answers on SO recommend specifying a literal path to a test.db file for testing. This is horrible advice.

2. Unless you want to test South and/or your South migrations, disable South, because it'll only complicate things:

   SOUTH_TESTS_MIGRATE = False
   SKIP_SOUTH_TESTS = True
   

3. Don't be dumb like me and try to access your models before they're created. This mostly means don't directly refer to models from the fields of other models or forms. e.g.

   class MyForm(forms.Form):
   
       somefield = forms.ChoiceField(
           required=True,
           choices=[(_.id, _.name) for _ in OtherModel.objects.filter(criteria=blah)],
       )
   

   This might work in code where your database already exists, but it'll break Django's unittest framework when it tries to load your tests, which load your models.py and forms.py, causing it to read a table that doesn't exist. Instead, set the choices value in the form's __init__().

Answer 2

This also happened to me with a legacy code but for another reason.

I had two models with db_table referencing the same db table. I know that is stupid, but it's not my fault )

And I never found anything on the internet that could help me. I was saved by verbosity set to 3 (manage.py test -v 3) Hope this helps anyone.

class Bla1(Model):
    some_column = ...
    class Meta:
        db_table = 'some_table'

class Bla2(Model):
    some_column = ...
    class Meta:
        db_table = 'some_table'