Generate Unit Tests from a scenario table where inputs and expected results are specified

Question

I'm currently trying to generate some post coding UT for a module. The module classifies input data to a DB, if a match is found, a proper value is returned.

After building a table with all possible input scenarios and their expected results, I discovered that I got over 50 tests to create. As all tests are basically the same, except the test name and the inputs, some kind of template seemed to fit here, reducing code mess.

What I originally imagined, was either a macro that creates such a template and expands to the tests with input data and the expected result OR using a structure representing the scenario table with some help from a macro to wrap it up. (I guess a C++ template is also adequate here).

However, I'm not sure this is a correct path to take and would like to know if anyone can give some recommendations of how to tackle it. There is also the question of how it fits a TDD approach..

Here is the idea using CppUTest:

#define GEN_TEST_MOD_CLASSIFIER(GROUP_NAME, TEST_NAME, PRIORITY, isCOMPRESS, isX, EXPECTING) \
TEST(GROUP_NAME, TEST_NAME) \
{ \
   int hit; \
   setupDB(PRIORITY, isCOMPRESS, isX); \
   hit = func(PRIORITY, isCOMPRESS, isX); \
   CHECK_EQUAL(EXPECTING, hit); \
}

Usage example:

GEN_TEST_MOD_CLASSIFIER(Classifier_Tests, LowPriority_NoCompress_NoX__HIT, PRIO_LOW, NOT_COMPRESS, NO_X, HIT_DB)

Answer 1

Well, w/o external tools the best you can do is most likely using macros. That's because you need to have TEST(GROUP_NAME, TEST_NAME) for each test case.

It may not be the case. You may be fine with not having separate test case for each scenario or CppUTest may have support for programmatic way of adding test cases. In this case, you can create a method which takes vector of input-output-testcasename tuples. And will add testcases / run tests on each tuple. No macros needed.

Something like that (pseudocode):

typedef std::tuple<std::string, std::string, PriorityType, CompressType, ExpectedValueType> TestInfo;

void RunTest(const TestInfo& testInfo)
{
// Assuming here you're OK with this kind of test cases separation
    std::cout << "Running test" << std::get<0>(testInfo) << ", " << std::get<1>(testInfo) << std::endl;

   int hit;
   setupDB(std::get<2>(testInfo), std::get<3>(testInfo), isX);
   hit = func(std::get<2>(testInfo), std::get<3>(testInfo), isX);
   CHECK_EQUAL(std::get<4>, hit);
}

void RunTests(const TestInfo& testInfo)
{
    std::for_each(testInfo.begin(), testInfo.end(), RunTest);
}

std::vector<TestInfo> tests = { test1, test2 };
RunTests(tests);

If that doesn't work, macros are actually fine, and you could go with table based approach there as well. Take a look at boost preprocessor (http://www.boost.org/doc/libs/1_54_0/libs/preprocessor/doc/index.html)

Pseudocode again:

#include <boost\preprocessor\seq.hpp>

#define GENERATE_TEST_CASE(_ignore1_, _ignore2_, _testCaseInfoSequence_) \
    TEST(BOOST_PP_SEQ_ELEM(0, testCaseInfoSequence), BOOST_PP_SEQ_ELEM(1, testCaseInfoSequence)) \
    { \
        int hit; \
        setupDB(BOOST_PP_SEQ_ELEM(2, testCaseInfoSequence), BOOST_PP_SEQ_ELEM(3, testCaseInfoSequence), BOOST_PP_SEQ_ELEM(4, testCaseInfoSequence)); \
        hit = func(BOOST_PP_SEQ_ELEM(2, testCaseInfoSequence), BOOST_PP_SEQ_ELEM(3, testCaseInfoSequence), BOOST_PP_SEQ_ELEM(4, testCaseInfoSequence)); \
        CHECK_EQUAL(BOOST_PP_SEQ_ELEM(5, testCaseInfoSequence), hit); \
    }

#define TESTCASES \
    ( \
        (Group1)(Test1)(prio1)(isCompress1)(isX1)(expectedVal1) \
        (Group1)(Test2)(prio2)(isCompress2)(isX2)(expectedVal2) \
    ) 

// This statement will generate all tests cases in defined in TESTCASES sequnce.
BOOST_PP_SEQ_FOREACH(GENERATE_TEST_CASE, _ignore_, TESTCASES);