Data-driven tests with jUnit

Question

What do you use for writing data-driven tests in jUnit?

(My definition of) a data-driven test is a test that reads data from some external source (file, database, ...), executes one test per line/file/whatever, and displays the results in a test runner as if you had separate tests - the result of each run is displayed separately, not in one huge aggregate.

Answer 1

In JUnit4 you can use the Parameterized testrunner to do data driven tests.

It's not terribly well documented, but the basic idea is to create a static method (annotated with @Parameters) that returns a Collection of Object arrays. Each of these arrays are used as the arguments for the test class constructor, and then the usual test methods can be run using fields set in the constructor.

You can write code to read and parse an external text file in the @Parameters method (or get data from another external source), and then you'd be able to add new tests by editing this file without recompiling the tests.

Answer 2

This is where TestNG, with its @DataSource, shines. That's one reason why I prefer it to JUnit; the others are dependencies and parallel threaded tests.

Answer 3

I use an in-memory database such as hsqldb so that I can either pre-populate the database with a "production-style" set of data or I can start with an empty hsqldb database and populate it with rows that I need to perform my testing. On top of that I will write my tests using JUnit and Mockito.

Answer 4

I use combination of dbUnit, jMock and jUnit 4. Then you can ether run it as suite or separately

Answer 5

You are better off extending TestCase with a "DataDrivenTestCase" that suits your needs. Here is working example: http://mrlalonde.blogspot.ca/2012/08/data-driven-tests-with-junit.html

Unlike parameterized tests, it allows for nicely named test cases.

Answer 6

I'm with @DroidIn.net, that is exactly what I am doing, however to answer your question literally "and displays the results in a test runner as if you had separate tests," you have to look at the JUnit4 Parameterized runner. DBUnit doesn't do that. If you have to do a lot of this, honestly TestNG is more flexible, but you can absolutely get it done in JUnit.

You can also look at the JUnit Theories runner, but my recollection is that it isn't great for data driven datasets, which kind of makes sense because JUnit isn't about working with large amounts of external data.

Answer 7

Even though this is quite an old topic, i still thought of contributing my share. I feel JUnit's support for data driven testing is to less and too unfriendly. for eg. in order to use parameterized, we need to write our constructor. With Theories runner we do not have control over the set of test data that is passed to the test method.

There are more drawbacks as identified in this blog post series: http://www.kumaranuj.com/2012/08/junits-parameterized-runner-and-data.html

There is now a comprehensive solution coming along pretty nicely in the form of EasyTest which is a a framework extended out of JUnit and is meant to give a lot of functionality to its users. Its primary focus is to perform Data Driven Testing using JUnit, although you are not required to actually depend on JUnit anymore. Here is the github project for refernece: https://github.com/anujgandharv/easytest

If anyone is interested in contributing their thoughts/code/suggestions then this is the time. You can simply go to the github repository and create issues.

Answer 8

Typically data driven tests use a small testable component to handle the data. (File reading object, or mock objects) For databases, and resources outside of the application mocks are used to similate other systems. (Web services, and databases etc). Typically I see is that there are external data files that handle the data and the output. This way the data file can be added to the VCS.

Answer 9

We currently have a props file with our ID numbers in it. This is horribly brittle, but is easy to get something going. Our plan is to initially have these ID numbers overridable by -D properties in our ant builds.

Our environment uses a legacy DB with horribly intertwined data that is not loadable before a run (e.g. by dbUnit). Eventually we would like to get to where a unit test would query the DB to find an ID with the property under test, then use that ID in the unit test. It would be slow and is more properly called integration testing, not "unit testing", but we would be testing against real data to avoid the situation where our app runs perfectly against test data but fails with real data.

Answer 10

Some tests will lend themselves to being interface driven.

If the database/file reads are retrieved by an interface call then simply get your unit test to implement the interface and the unit test class can return whatever data you want.