Can you polymorph a python unittest test case?

Question

I need to test the combination of two parameters, each having only a few possibilities. So I want to have a class hierarchy, where each subclass has its own fixed parameter A, while Parameter B can be tested based on that common A. I want to polymorph the test methods in the test case in terms of a shifting Parameter A, so that I don't have to rewrite the test methods for each subclass I test, but still can have variations in the subclasses' tests.

However, I found that if I declare the attribute in setUp(self). the attribute self.A does not get subclassed. Meaning that all my subclasses all have the same self.A value from the first test case (parent class). Also all my testSomething(self) methods are not virtual! Meaning with the following hierarchy:

class baseTest(unittest.TestCase):
    def setUp(self):
        print('base setup')
        self.A = 100

    def testSomething(self):
        print('base test')
        # Do something with self.A

class subTest(baseTest):
    def setUp(self):
        print('sub setup')
        self.A = 999

    def testSomething(self):
        print('sub test')
        # Do something with self.A

EDIT: I put both classes in one file, and relied on unittest.main() launched in if __name__ == "__main__"

Running this file gives me:

base setup
base test
base setup
base test

I thought all python methods are virtual, but this does not seem the case in unittest.

What should I do?

Answer 1

UPDATE

I implemented the plan below and it works as expected. See below:

class baseTest(unittest.TestCase):
    def setUp(self):
        print('base setup')
        self.A = 100

    def testSomething(self):
        print('base test')
        self._DoTestSomething(self)

    def _DoTestSomething(self)
        # Do something with self.A

class subTest(baseTest):
    def setUp(self):
        print('sub setup')
        self.A = 999

    def _DoTestSomething(self)
        # Do something with self.A

OLD

Ok, I have a plan now. I can write the test implementations in non testSomething() methods, which should be virtual, then delegate the testSomething() to these virtual methods.

I haven't tested this idea, but I think it should work.

Answer 2

Not sure what you're running into, but it works fine for me...?

Using your code, and then adding this:

a = baseTest()
b = subTest()

a.setUp()
a.testSomething()
b.setUp()
b.testSomething()

Here's my output:

base setup
base test
sub setup
sub test

Answer 3

The testscenarios library is designed to take the test methods you write, and some data scenarios, to produce individual test cases for each combination of (test-method × scenario).

For instance:

import unittest
import testscenarios

from .. import system_under_test

class foo_TestCase(testscenarios.WithScenarios, unittest.TestCase):
    """ Test cases for `foo` function. """

    scenarios = [
            ('purple': {
                'wibble': "purple",
                'expected_count': 2,
                'expected_output': "Purple",
                }),
            ('orange': {
                'wibble': "orange",
                'expected_count': 3,
                'expected_output': "Orange",
                }),
            ('red': {
                'wibble': "red",
                'expected_count': 1,
                'expected_output': "Red",
                }),
            ]

    def test_has_expected_vowel_count(self):
        """ Should give the expected count of vowels. """
        vowel_count = system_under_test.foo(self.wibble)
        self.assertEqual(self.expected_count, vowel_count)

    def test_sets_expected_output(self):
        """ Should set the output to the expected value. """
        system_under_test.foo(self.wibble)
        self.assertEqual(self.expected_output, system_under_test.output)

The foo_TestCase class will, when test discovery occurs, generate 6 tests for the test runner. They will have names annotated with the scenario names, so the resulting test report distinguishes each specific test case:

foo_TestCase.test_has_expected_vowel_count (purple)
foo_TestCase.test_has_expected_vowel_count (orange)
foo_TestCase.test_has_expected_vowel_count (red)
foo_TestCase.test_sets_expected_output (purple)
foo_TestCase.test_sets_expected_output (orange)
foo_TestCase.test_sets_expected_output (red)

More flexibility is available if you need it – see the testscenarios documentation – but the above demonstrates how I use it most of the time.