EMMA with Junit, should I be testing with the instrumented classes or java classes?

Question

I've been going through this tutorial for my ANT Script for emma/junit, it states

In the <classpath> element of <junit>, change the locations of your real classes, so that you are pointing at the ${instr.dir} attribute instead of your normal target/classes folder.

Here is my emma section from of my build.xml for Apache Ant (pastebin link to full build.xml)

<!-- =================================================================== -->
<!-- Run the tests with EMMA
* depends from  compile                                                                                                                        -->
<!-- =================================================================== -->
<target name="emmatest" depends="compile" description="Tests the project with Emma Query" >
    <!-- ======================================================================= -->
    <!-- EMMA INSTRUMENTATION -->
    <!-- ======================================================================= -->
    <emma enabled="${emma.enabled}" >
        <instr instrpathref="emma.coverage.classes"
             destdir="${instr.dir}"
             metadatafile="${coverage.dir}/metadata.emma"
             merge="true"
             >
            <filter value="${emma.filter}" />
        </instr>
    </emma> 
    <junit fork="true" forkmode="once" >
         <test name="test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest" todir="${coverage.dir}">
          <formatter type="xml"/>
          </test>
         <classpath>
                <path refid="emma.lib" />
                <pathelement location="${instr.dir}"/>
                <fileset dir="${lib.dir}">
                   <include name="**/*.jar"/>
                </fileset>
        </classpath>
         <jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.emma" />
         <jvmarg value="-Demma.coverage.out.merge=false" />
    </junit>
</target>

The Build Section Results for emma: JUNIT Fail Result, pointing to ${instr.dir}

 emmatest:
 [instr] processing instrumentation path ...
 [instr] instrumentation path processed in 109 ms
 [instr] [6 class(es) instrumented, 0 resource(s) copied]
 [instr] metadata merged into [C:\Users\Michael\workspace\log4jassignment.s06005586\build\emma-reports\metadata.emma] {in 42 ms}
 [junit] Test test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest FAILED

The Build Section results for junitreport: JUNIT Pass Result, pointing to real classes ${build.classes}

test:
[junit] Running test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.504 sec
[junit] Running test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.StressTest
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.526 sec

I was wondering why is my JUNIT Test AppenderLayoutTest failing with emma but passing with junit. I'm also pretty confident my coverage file isn't being created because the JUNIT for AppenderLayoutTest is failing.

Am I testing with the wrong set of classes? or what I missed?

Update: 8/28/2012 6:21 PM

I then changed the

<test 
name="test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest"
todir="${coverage.dir}">

with

<batchtest>
    <fileset dir="${instr.dir}" includes="**/*Test.class" />
</batchtest>

And the updated Build Section Results for emma: JUNIT Fail Result, pointing to ${instr.dir} where..

emmatest:
[instr] processing instrumentation path ...
[instr] instrumentation path processed in 105 ms
[instr] [6 class(es) instrumented, 0 resource(s) copied]
[instr] metadata merged into [C:\Users\Michael\workspace\log4jassignment.s06005586\build\emma-reports\metadata.emma] {in 89 ms}
[junit] TEST test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest FAILED
[junit] TEST test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.StressTest FAILED
[junit] Tests FAILED

Update: 7:02 PM

I added <formatter type="plain" usefile="false" />, And this is the output

emmatest:
[instr] [EMMA v2.1, build 5320 (stable) (2005/06/20 22:08:27)]
[instr] instrumentation path:
[instr] {
[instr]   C:\Users\Michael\workspace\log4jassignment.s06005586\build\classes
[instr] }
[instr] instrumentation output mode: copy
[instr] metadata output file: C:\Users\Michael\workspace\log4jassignment.s06005586\build\emma-reports\metadata.emma
[instr] metadata output merge mode: true
[instr] processing dir path entry [C:\Users\Michael\workspace\log4jassignment.s06005586\build\classes] ...
[instr] instrumentation path processed in 147 ms
[instr] [6 class(es) instrumented, 0 resource(s) copied]
[instr] metadata contains 6 entries
[instr] metadata merged into [C:\Users\Michael\workspace\log4jassignment.s06005586\build\emma-reports\metadata.emma] {in 49 ms}
[junit] Running test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest
[junit] Testsuite: test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit]     Caused an ERROR
[junit] Illegal local variable table length 17 in method test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest.<init>()V
[junit] java.lang.ClassFormatError: Illegal local variable table length 17 in method test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest.<init>()V
[junit]     at java.lang.Class.forName0(Native Method)
[junit]     at java.lang.Class.forName(Unknown Source)
[junit] TEST test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest FAILED
[junit] Running test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.StressTest
[junit] Testsuite: test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.StressTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit]     Caused an ERROR
[junit] Expecting a stackmap frame at branch target 11 in method test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.StressTest.<init>()V at offset 4
[junit] java.lang.VerifyError: Expecting a stackmap frame at branch target 11 in method test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.StressTest.<init>()V at offset 4
[junit]     at java.lang.Class.forName0(Native Method)
[junit]     at java.lang.Class.forName(Unknown Source)
[junit] TEST test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.StressTest FAILED
[junit] Tests FAILED

what is [junit] java.lang.ClassFormatError: Illegal local variable table length 17 in method test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest.<init>()V

Answer 1

Can you try replacing

<test 
name="test.nz.ac.massey.cs.sdc.log4jassignment.s06005586.AppenderLayoutTest"
todir="${coverage.dir}">

with

<batchtest>
    <fileset dir="${instr.dir}" includes="**/*Test.class" />
</batchtest>

Lastly I would ask you to put some sysout in your test and see what is going wrong in the test. This might something to do with configuration I feel.

Answer 2

Looking at the latest trace you shared it seems can use

-XX:-UseSplitVerifier

as VM argument to get rid of the error related to JDK7.

or Check this out