Writing unit tests for source-to-source compilers

Question

How do you test across different languages?

My compiler translates code from a source language to a target language. I want to make sure that when the (source language) input and (target language) output are executed, they produce exactly the same result.

Right now I have a script that executes the source input, runs it through the compiler, executes the output, then string-compares the results. It feels unsatisfactory (the slightest semantic difference causes tests to fail; "1.0" is not equal to "1" so that test fails, etc.), so are there any better ways?

Answer 1

This isn't really be any different to TDD on a standard application (especially one that interfaces to a database etc...). You can unit test the components that are composed to make your application in isolation. Then for the test you mention above, you just have to use your best judgement to make a test that still allows the code to be flexible, but also still thoroughly tests the desired behaviour.

One idea would be to still test the resultant code comes out correctly, but isolate the checks behind a builder object that builds up the expected output source code to assert against. e.g.

    With(int number)
    {    
       _sourceString += number.ToString(<Assign formatting here>);
       Return this;
    }

This way if your outputs do change, you only have to change them in the one builder method.

Answer 2

The testing pattern you've mentioned is not unit testing. You are testing the behaviour of the whole application, so it's rather black-box testing. @grrrrrrrrrrrrr correctly pointed out that unit tests should test individual methods of individual classes. Then you can verify that, for example, a small piece of code containing literal "1.0" will be correctly transformed into "1.0f" or anything acceptable by one method that does this exactly one thing (i.e. transforms literals or something).