Hierarchy of JUnit tests

Question

I need the following test

@runwith(cache, memory)
class CollectionA is -- this is a suite (aka folder)
  class Cache {   -- this is a sub-suite (aka folder)
    @test testCache1()  -- this is a method (aka file)
    @test testCache2()
    @test testCache3()
  }
  class RAM {  -- this is a sub-suite (aka folder)
    @test testRAM1()
    @test testRAM2()
  }
  @test testIO()
  @test testKeyboard()
  @test testMouse()
  @test testMonitor()
  @test testPower()
  @test testBoot()

Please note that only Cache and RAM need to be grouped. The hierarchy helps to fight the complexity and run related tests, e.g. Cache subsystem, alone, when necessary. The problem that is as soon I use @runwith to do that grouping, all the single test methods besides the RAM and Cache collections are ignored by JUnit. It seems that you cannot have sibling files and folders in JUnit design. The comments in the official example of grouping also hints that

@RunWith(Suite.class)
@Suite.SuiteClasses({
  TestA.class,
  TestA.class
})

public class FeatureTestSuite {
  // the class remains empty,
  // used only as a holder for the above annotations
  // HEY!!! WHAT ABOUT MY @Tests HERE?
}

The answers say that whether I need to wrap every single test, e.g. testPower into their singletone suit or flatten the suite - get rid if the hierarchy completely.

So, is it right that JUnit is designed to disallow mixing single files (@test methods) with the folders (@runwith suites)? Why? How can this be worked around? Might be there is an alternative to @runwith.Suite?

Answer 1

What you like to create is a mixin type, which is not supported by the JUnit runner. So yes, you are right, it is not possible out of the box.

For this purpose I created an add-on that can be used to create hierarchical contexts for your test. In my point of view this is a missing feature in JUnit and I also stay in contact to get this included into the JUnit core.

The add-on provides an HierarchicalContextRunner which allows to use inner class to group your tests into contexts. Each context can contain tests or other contexts. It also allows to have @Before, @After, @Rule methods and fields, as well as other feature like @Ignore of the standard Runner. :-)

Example:

@RunWith(HierarchicalContextRunner.class)
public class CollectionA {
    public class Cache {
        @Test testCache1() {...}
        @Test testCache2() {...}
        @Test testCache3() {...}
    }
    public class RAM {
        @Test testRAM1() {...}
        @Test testRAM2() {...}
    }
    @Test testIO() {...}
    @Test testKeyboard() {...}
    @Test Mouse() {...}
    @Test testMonitor() {...}
    @Test testPower() {...}
    @Test testBoot() {...}
}

Give it a try: https://github.com/bechte/junit-hierarchicalcontextrunner/wiki

Votes and feedback are highly appreciated. :)

Answer 2

Your design should be something like this:

// folder com.myco.project
SuiteX.java
TestA.java
TestB.java


// contents of TestA.java
public class TestA{
   @Test
   public void someTestInA(){...}
}

// contents of TestB.java
public class TestB{
   @Test
   public void someTestInB(){...}
}

// contents of SuiteX.java
@RunWith(Suite.class)
@Suite.SuiteClasses({
  TestA.class,
  TestB.class
})
public class FeatureTestSuite {
  // the class remains empty,
  // used only as a holder for the above annotations
}

As I state in my comment, use separate java files for each test class. Do not use inner classes.