Automating Acceptance Testing for ETL

Question

We have a java application which essentially performs ETL - reading from and writing to files/databases with transformation rules applied in the middle.

I've started looking into automating acceptance testing for the application however am struggling to apply the frameworks i've looked at so far (concordion, cucumber etc). They seem very easy to implement for simple applications like those shown on their tutorials, but I basically have to have tests saying "I have this input file and expect this output file (or result in db table)" - with each file having 100's of fields.

I could fake it so that the input values are read from a html table (as per concordion tutorial) however that isnt really a true test.

Has anyone come across a framework that could help? or been able to use concordion for such a purpose?

Many thanks

Answer 1

Who is the audience of the test? If this is purely a technical exercise and there are no non technical business owners that need to interact with the test then just doing it with your favorite unit testing framework is fine. Fitnesse works best when there is collaboration for the acceptance criteria with non techies.

So no, just "file input 'a' produces file output 'b'" probably is not enough to warrant the overhead of fitnesse. I would only move it to tables if someone was going to change it on a regular basis and that person was not comfortable editing a file directly.

Answer 2

Did you try JBehave ?

read more information about it at http://www.qatestingtools.com/jbehave

Answer 3

At a leading bank in the Netherlands we have setup testautomation with Fitnesse and ETL fixtures. It is an Agile project and for our ETL solution we use Informatica Powercenter and Oracle DB. For us our test automation/specification in Fitnesse is of great value now. We have SLIM fixtures for truncating tables, inserting records into tables, checking table records with expected values, updating records and calling our Fitnesse workflows.