Cobertura code coverage is wrong

Question

I am have been trying to setup code coverage on my local Jenkins server with Maven. It would seem that my tests are not being included in the coverage reports.

I have the following in my pom.xml:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
        <artifactId>sonar-maven-plugin</artifactId>
        <version>2.0</version>
</plugin>
<plugin>
    <groupId>org.codehaus.mojo</groupId>
        <artifactId>cobertura-maven-plugin</artifactId>
        <version>2.5.2</version>
        <configuration>
        <check>
        <haltOnFailure>false</haltOnFailure>
            <totalLineRate>85</totalLineRate>
        </check>
        <formats>
            <format>xml</format>
            <format>html</format>
        </formats>
        </configuration>
        <executions>
            <execution>
                <id>clean</id>
                <phase>pre-site</phase>
                <goals>
                   <goal>clean</goal>
                </goals>
            </execution>
            <execution>
                <id>instrument</id>
                <phase>site</phase>
                <goals>
                    <goal>instrument</goal>
                    <goal>cobertura</goal>
                </goals>
            </execution>
        </executions>
</plugin>

and here is where I define the tests to be run

<plugin>
    <!-- Separates the unit tests from the integration tests. -->
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <skip>true</skip>
        <trimStackTrace>false</trimStackTrace>
        </configuration>
    <executions>
        <execution>
            <id>unit-tests</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <skip>false</skip>
                <includes>
                    <include>**/*Test*.java</include>
                </includes>
                <excludes>
                    <exclude>**/*IT.java</exclude>
                </excludes>
            </configuration>
        </execution>

        <execution>
            <id>integration-tests</id>
            <phase>integration-test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <skip>false</skip>
                <includes>
                    <include>**/*IT.java</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>

And on my Jenkins server I am have the following: Jenkins Configuration: Sonar Sonar name:sonar local 3.5.1 server url: http://local host:9000

Project level: publish Cobertura Coverate Report: xml report pattern: **/target/site.covertura/coverage.xml

But when I run either the maven task "mvn cobertura:cobertura" or view the sonar reports in Jenkins, all the coverage is at 0%, which makes me think my unit and integration tests are not being seen.

Is this a problem with the way my pom separates unit and integration tests?

Answer 1

cobertura can't look into integration test to see code coverage.

Try a look at code coverage by integration tests with sonar and at separating integration and unit tests

To make a short story : you must launch the jacoco maven plugin who will set two properties, one for unit tests, the another for it tests, and launch surefire and failsafe with these properties.

And after, you can show result in sonar in separated applet.

In my own pom file, you will se a part for enabled jacoco :

            <properties>
               <!-- Coverage and report -->
               <jacoco.destFile.unit>${project.build.directory}/jacoco-unit.target</jacoco.destFile.unit>
               <jacoco.destFile.it>${project.build.directory}/jacoco-it.target</jacoco.destFile.it>
               <coverage.reports.dir>${project.build.directory}/coverage-reports</coverage.reports.dir>
               <version.surefire>2.12.4</version.surefire>

               <!-- Sonar -->
               <sonar.jacoco.itReportPath>${jacoco.destFile.it}</sonar.jacoco.itReportPath>
               <sonar.jacoco.reportPath>${jacoco.destFile.unit}</sonar.jacoco.reportPath>
               <sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
               <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
            </properties>


            <build>
            ....
            <plugins>
                <!-- Jacoco configuration -->
                <plugin>
                    <groupId>org.jacoco</groupId>
                    <artifactId>jacoco-maven-plugin</artifactId>
                    <version>0.6.0.201210061924</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>prepare-agent</goal>
                            </goals>
                            <configuration>
                                <propertyName>jacoco.argLine.unit</propertyName>
                                <destFile>${jacoco.destFile.unit}</destFile>
                            </configuration>
                        </execution>
                        <execution>
                            <id>pre-integration-test</id>
                            <phase>pre-integration-test</phase>
                            <goals>
                                <goal>prepare-agent</goal>
                            </goals>
                            <configuration>
                                <propertyName>jacoco.argLine.it</propertyName>
                                <destFile>${jacoco.destFile.it}</destFile>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
                <!-- And in the surefire and failsafe plugins you need to enable jacoco like this ->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>${version.surefire}</version>
                    <dependencies>
                        <dependency>
                            <groupId>org.apache.maven.surefire</groupId>
                            <artifactId>surefire-junit47</artifactId>
                            <version>${version.surefire}</version>
                        </dependency>
                    </dependencies>
                    <configuration>
                        <argLine>${jacoco.argLine.unit}
                            -Dfile.encoding=${project.build.sourceEncoding} -Xmx512m</argLine>
                        <forkMode>always</forkMode>
                        <parallel>classes</parallel>
                    </configuration>
                </plugin-->

                <!-- Play the /src/test/java/**IT.java with goal verify -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <version>${version.surefire}</version>
                    <dependencies>
                        <dependency>
                            <groupId>org.apache.maven.surefire</groupId>
                            <artifactId>surefire-junit47</artifactId>
                            <version>${version.surefire}</version>
                        </dependency>
                    </dependencies>
                    <configuration>
                        <forkMode>pertest</forkMode>
                        <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
                        <argLine>${jacoco.argLine.it}
                            -Dfile.encoding=${project.build.sourceEncoding} -Xmx512m</argLine>
                    </configuration>
                    <executions>
                        <execution>
                            <id>integration-test</id>
                            <phase>integration-test</phase>
                            <goals>
                                <goal>integration-test</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>verify</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>verify</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>

In Jenkins, you must setup the "postbuild" JaCoCo plugins with "**/**.target" for path to exec files

Hope this will help you.