Unit testing several implementation of the same trait/interface

Question

I program mostly in scala and java, using scalatest in scala and junit for unit testing. I would like to apply the very same tests to several implementations of the same interface/trait. The idea is to verify that the interface contract is enforced and to check Liskov substitution principle.

For instance, when testing implementations of lists, tests could include:

• An instance should be empty, if and only if and only if it has zero size.
• After calling clear, the size should be zero.
• Adding an element in the middle of a list, will increment by one the index of rhs elements.
• etc.

What are the best practices ?

Answer 1

This sounds like it could be a job for shared tests. Shared tests are tests that are shared by different fixture objects. I.e., the same test code is run on different data. ScalaTest does have support for that. Search for "shared tests" in the documentation of your favorite style trait that represents tests as functions (Spec, WordSpec, FunSuite, FlatSpec, etc.). An example is the syntax for FlatSpec:

it should behave like emptyList

See Sharing Tests in the FlatSpec documentation

Answer 2

In Java/JUnit, I generally handle this by having an abstract testcase from which tests for the specific test class inherit all the tests and have a setup method instantiating the implementation. I can't watch the video abyx posted right now, but I suspect it's this general idea.

Another interesting possibility if you don't mind introducing yet another testing framework would be to use JDave Specification classes.

I haven't tried using either of these with Scalatest or with Scala traits and implementations, but it should be possible to do something similar.

Answer 3

Contract tests are easy to do with JUnit 4, here's a video by Ben Rady.

Answer 4

For Scala, strongly consider ScalaCheck. All of those contracts are expressible as one-line specifications in ScalaCheck. When run, ScalaCheck will generate a configurable number of sample inputs randomly, and check that all of the specifications hold. It's about the most semantically dense way possible to create unit tests.