Producing reference test files out of CPPunit test cases for a specification

Question

I'm writing a reference implementation for a specification that deals with data transformation.

I have a number of test cases that check that data is getting forward and backward transformed properly, something like :

void test(){
    int x[maxTest], y[maxTest], z[maxTest];
    fillRandomly(x, testSize);

    MyTransform trans = MyTransform(testsize);

    trans.apply(x, testSize, y);
    trans.remove(y, testSize, z);

    for(int i = 0; i < testSize; i++)
        CPPUNIT_ASSERT(abs(x[i] - z[i]) < 2);
}

Data types vary wildly between test cases and transforms.

What I would like to produce is files that contain the input/output and maybe the intermediary stage so further implementations can verify their results.

There are a few obvious ways I can make this work (e.g. name the output file and slap everything in it) but most of them imply going back to each test case and a whole bunch of copy pasting.

Coming from Java, introspection comes to mind with a wrapper to solve some of my issues (automatic naming of test output files ...), but not all and I've been recommended to stay away from it in C++ (opinions welcome).

Generic apply / remove methods also seem like a good candidate but won't be that easy as there is no abstract transform type and little conformity between transforms (might still be a good option as there aren't that many types of transforms).

So I'm wondering if any of you had to do something similar and how you solved it. I don't mind modifying all the test cases if it means later on I can have something clean.

Any input would be much appreciated.

Answer 1

What you've described is not really a unit test, and so CppUnit alone is probably not your best choice.

It looks like you're trying to do three independent things, but have combined them all together to do them at one time. You want to run unit tests, you want to generate test data, and you want to verify the program works with the generated data, and you're attempting to do it all within a single unit test framework. But because it's doing so much, you're finding it hard to reuse.

Consider simplifying it into three total testing apps: a CppUnit test suite that only executes your unit tests (and these tests should have nothing to do with files of test data); a test data generator; and a transformation tester. Every time you build you should run the automated unit tests. When you run the data generator, you should automate the execution of the tester on the data it outputted to ensure you've generated good data. Then you can deliver the known tested data and the tester app to your clients.