Paramaterize unit tests in python

Question

I work on a set of python unit tests that are currently built using pythons built in testing framework. I would like to write paramaterized tests that will execute multiple times based on the set of data I give it.

ie. if my data set is [1,2,3,4] my test function would run four times using the input in my data set.

def test(data):
    if data > 0:
       #Pass the test

From my understanding this isn't possible currently in the built in framework, unless I put a loop in my test function. I don't want to do this because I need the test to continue executing even if one input fails.

I've seen that it's possible to do using nose, or pyTest. Which is the best framework to use? Is there another framework I could use that would be better than either of these?

Thanks in advance!

Answer 1

Note that this is precisely one of the most common uses of the recent addition of funcargs in py.test.

In your case you'd get:

def pytest_generate_tests(metafunc):
    if 'data' in metafunc.funcargnames:
        metafunc.parametrize('data', [1,2,3,4])

def test_data(data):
    assert data > 0

[EDIT] I should probably add that you can also do that as simply as

@pytest.mark.parametrize('data', [1,2,3,4])
def test_data(data):
    assert data > 0

So I'd say that py.test is a great framework for parameterized unit testing...

Answer 2

You can create tests dynamically based on your data set in the following way:

import unittest

data_set = [1,2,3,4]

class TestFunctions(unittest.TestCase):
    pass  # all your non-dynamic tests here as normal

for i in data_set:
    test_name = "test_number_%s" % i # a valid unittest test name starting with "test_"
    def dynamic_test(self, i=i):
        self.assertTrue(i % 2)
    setattr(TestFunctions, test_name, dynamic_test)

if __name__ == '__main__':
    unittest.main()

The question Python unittest: Generate multiple tests programmatically? has more discussion of this, including another approach that achieves the same thing by dynamically creating multiple instances of the test case into a test suite.