Code coverage in Java with EclEmma not scanning expecting exception methods

Question

I'm trying to get my code coverage in java, using Eclipse and EclEmma.

My tests are using JUnit 4 and I've got some tests looking like this :

    @Test(expected = IllegalArgumentException.class)
    public void createTime_withInvalidMinuteUnder0_throws(){
    //Arrange
    ...
    //Act
    Something triggering IllegalArgumentException Here       
}

And EclEmma says that the test fails because there's an IllegalArgumentException being thrown. So it drops my code coverage indicator even though it's supposed to throw something. Is there an option to make it see that JUnit expected exception tag ?

edit: I've found out that if you add the throw to the declaration of the test aswell, it works!

Answer 1

No, there is no way to get EclEmma to notice the expected clause. They acknowledge this fact here.

Why are JUnit4 test cases with expected exceptions shown as not covered?

JUnit4 test cases with expected exceptions are shown as not covered even though they were executed. The reason for this is that underlying JaCoCo code coverage library only considers code as executed when certain probes are executed. For successful test cases marked with @Test{expected=...} this is not the case.

Personally, I wouldn't worry too much about it. Coverage of test cases is the least interesting thing EclEmma can tell you; I always completely ignore those metrics and focus on the coverage of my production code.

Answer 2

Can't comment yet, but I wanted to point out, in relation to the accepted answer, that there's a very good reason to pay attention to coverage of your test code.

It's just way to easy with JUnit to screw up the test method naming convention or forget the @Test annotation, depending on your JUnit version. Do that, and you could easily be fooled into thinking that your solid green bar means that the nice test you just added passed, when in fact it never ran. Coverage coloring will show this very prominently. Except of course, as this thread points out, some of your tests that throw exceptions might look like they didn't run when they did.

Answer 3

I faced the same issue, and offered a pull request dealing with the most prominent cause of this annoyance. Instead of adding way too many probes, I only deal with the opcodes of method invocations. In other words, the problem of (previously) wrong coverage results caused by an exception thrown by a method invocation is solved, but not if the exception is thrown by, for example, a division by zero, a wrong cast, or an array index problem.

https://github.com/jacoco/jacoco/pull/261

Feel free to comment on the pull request or participate in the discussion I started on the JaCoCo mailing list.