For a django unit test, why do some test runners take into account the production database, and others do not?

Question

As part of bringing up the django tutorial app, I noticed that certain test runners were going out to the production database when running unit tests, while other test runners appeared to ignore it.

I pared down a django app to the most basic features, with three tests:

• Model instantiation (a control test)
• Request on an empty database
• Request on a database with exactly one element

I added a single element to the production database, and ran them through the test runners that Eclipse PyDev uses.

Code is available on my GitHub site.

The django test runner passes (claims to be creating a test database?):

(django)kenners@elendil:~/my_first_app$ python manage.py test demo
Creating test database for alias 'default'...
...
----------------------------------------------------------------------
Ran 3 tests in 0.135s

OK
Destroying test database for alias 'default'...

The pytest runner passes (without any such claim, though it might be hidden):

(django)kenners@elendil:~/my_first_app$ python -m pytest demo/tests.py
=============================================================================== test session starts ================================================================================
platform linux2 -- Python 2.7.3 -- pytest-2.4.2
plugins: django
collected 3 items

demo/tests.py ...

============================================================================= 3 passed in 1.29 seconds =============================================================================

The nose runner fails (it's combining the production database with the tests):

(django)kenners@elendil:~/my_first_app$ python -m nose demo/tests.py
FF.
======================================================================
FAIL: test_no_available_things (demo.tests.DemoDatabaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kenners/my_first_app/demo/tests.py", line 23, in test_no_available_things
    self.assertEquals(0, pull_count_from_body(response))
AssertionError: 0 != 1

======================================================================
FAIL: test_one_available_things (demo.tests.DemoDatabaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kenners/my_first_app/demo/tests.py", line 30, in test_one_available_things
    self.assertEquals(1, pull_count_from_body(response))
AssertionError: 1 != 2

----------------------------------------------------------------------
Ran 3 tests in 0.334s

FAILED (failures=2)

The unittest2 runner fails (for the same reason):

(django)kenners@elendil:~/my_first_app$ python -m unittest2 demo.tests
FF.
======================================================================
FAIL: test_no_available_things (demo.tests.DemoDatabaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "demo/tests.py", line 23, in test_no_available_things
    self.assertEquals(0, pull_count_from_body(response))
AssertionError: 0 != 1

======================================================================
FAIL: test_one_available_things (demo.tests.DemoDatabaseTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "demo/tests.py", line 30, in test_one_available_things
    self.assertEquals(1, pull_count_from_body(response))
AssertionError: 1 != 2

----------------------------------------------------------------------
Ran 3 tests in 0.348s

FAILED (failures=2)

What part of nose / unittest2 causes these tests to fail? Why is pytest working?

Answer 1

Touching production database while running unit-tests is absolutely inappropriate. Unit tests should generally work on a mock database. Integration tests should work on a real, but test database. But production database? Why would you want to risk your real data?

Django documentation claims "the test runner takes care of creating its own test database".

In django-nose documentation it can be clearly seen that it is supposed to run tests on a test database.