how do JaCoCo and sonar work?

Question

I am trying to understand how JaCoCo and Sonar work together using the sonar ut-ant-jacoco-runTests example: https://github.com/SonarSource/sonar-examples/blob/master/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/build.xml

<?xml version="1.0" encoding="UTF-8"?>
<project name="UT coverage with Ant and JaCoCo running tests" default="all" basedir="." xmlns:sonar="antlib:org.sonar.ant">

        <!-- ========= Define the main properties of this project ========= -->
        <property name="src.dir" value="src" />
        <property name="test.dir" value="test" />
        <property name="lib.junit.dir" value="lib" />
        <property name="build.dir" value="target" />
        <property name="classes.dir" value="${build.dir}/classes" />
        <property name="reports.dir" value="${build.dir}/reports" />
        <property name="reports.junit.xml.dir" value="${reports.dir}/junit" />

        <!-- Define the SonarQube properties -->
        <property name="sonar.projectKey" value="org.codehaus.sonar:example-ut-ant-jacoco-runTests" />
        <property name="sonar.projectName" value="UT coverage with Ant and JaCoCo running tests" />
        <property name="sonar.projectVersion" value="1.0" />
        <property name="sonar.language" value="java" />
        <property name="sonar.sources" value="${src.dir}" />
        <property name="sonar.tests" value="${test.dir}" />
        <property name="sonar.binaries" value="${classes.dir}" />
        <property name="sonar.sourceEncoding" value="UTF-8" />
        <property name="sonar.junit.reportsPath" value="${reports.junit.xml.dir}" />   

        <!-- The following properties are required to use JaCoCo: -->
        <property name="sonar.dynamicAnalysis" value="reuseReports" />
        <property name="sonar.java.coveragePlugin" value="jacoco" />
        <property name="sonar.jacoco.reportPath" value="target/jacoco.exec" />

        <!-- Add your basic SonarQube configuration below: sonar.jdbc.url, sonar.jdbc.username, etc. properties -->
        <!--
          <property name="sonar.jdbc.url" value="jdbc:..." />
          <property name="sonar.jdbc.username" value="..." />
          <property name="sonar.jdbc.password" value="..." />
        -->

        <!-- ========= Define "regular" targets: clean, compile, test, ... ========= -->
        <target name="clean">
                <delete dir=".sonar" />
                <delete dir="${build.dir}" />
                <delete dir="${reports.dir}" />
        </target>

        <target name="init">
                <mkdir dir="${build.dir}" />
                <mkdir dir="${classes.dir}" />
                <mkdir dir="${reports.dir}" />
                <mkdir dir="${reports.junit.xml.dir}" />
        </target>

        <target name="compile" depends="init">
                <javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath" fork="true" debug="true" includeAntRuntime="false" />
                <javac srcdir="${test.dir}" destdir="${classes.dir}" classpathref="classpath" fork="true" debug="true" includeAntRuntime="false" />
        </target>


        <path id="classpath">
                <fileset dir="${lib.junit.dir}" includes="*.jar"/>
        </path>

        <target name="test" depends="compile">

                <taskdef name="junit" classname="org.apache.tools.ant.taskdefs.optional.junit.JUnitTask">
                        <classpath>
                                <path refid="classpath"/>
                        </classpath>
                </taskdef>

                <!-- Import the JaCoCo Ant Task -->
                <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
                        <!-- Update the following line, or put the "jacocoant.jar" file in your "$HOME/.ant/lib" folder -->
                        <classpath path="path/to/jacoco/ant/task/lib/jacocoant.jar" />
                </taskdef>

                <!-- Run your unit tests, adding the JaCoCo agent -->
                <jacoco:coverage destfile="target/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">
                  <junit fork="yes" dir="${basedir}" failureProperty="test.failed">
                        <classpath location="${classes.dir}" />
                        <classpath refid="classpath" />

                        <formatter type="xml" />
                        <batchtest todir="${reports.junit.xml.dir}">
                          <fileset dir="${test.dir}">
                                <include name="**/*Test.java" />
                          </fileset>
                        </batchtest>
                  </junit>
                </jacoco:coverage>        

        </target>

        <!-- ========= Define SonarQube target ========= -->
        <target name="sonar" depends="compile">
                <taskdef uri="antlib:org.sonar.ant" resource="org/sonar/ant/antlib.xml">
                        <!-- Update the following line, or put the "sonar-ant-task-*.jar" file in your "$HOME/.ant/lib" folder -->
                        <classpath path="path/to/sonar/ant/task/lib/sonar-ant-task-*.jar" />
                </taskdef>

                <!-- Execute SonarQube -->
                <sonar:sonar />
        </target>

        <!-- ========= The main target "all" ========= -->
        <target name="all" depends="clean,compile,test,sonar" />

</project>

I install sonar in my local machines, default configuration. then I run the example as follows:

$ git clone https://github.com/SonarSource/sonar-examples.git
$ cd sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests

# get the two missing libraries:
$ wget http://repository.codehaus.org/org/codehaus/sonar-plugins/sonar-ant-task/2.1/sonar-ant-task-2.1.jar
$ wget -Ojacoco-0.6.4.201312101107.zip http://search.maven.org/remotecontent?filepath=org/jacoco/jacoco/0.6.4.201312101107/jacoco-0.6.4.201312101107.zip
$ unzip -p jacoco-0.6.4.201312101107.zip lib/jacocoant.jar >jacocoanto-0.6.4.201312101107.jar

#edit build.xml and replace:
   <classpath path="path/to/jacoco/ant/task/lib/jacocoant.jar" />
   <classpath path="path/to/sonar/ant/task/lib/sonar-ant-task-*.jar" />
by
   <classpath path="./jacocoanto-0.6.4.201312101107.jar" />
   <classpath path="./sonar-ant-task-2.1.jar" />


$ ant clean compile
# this creates:
./target/classes/One.class
./target/classes/OneTest.class


$ ant test
[jacoco:coverage] Enhancing junit with coverage
    [junit] objc[50244]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.7.0_40.jdk/Contents/Home/jre/bin/java and /Library/Java/JavaVirtualMachines/jdk1.7.0_40.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.

# Why I get this warning? is there something misconfigured?

# this creates:
./target/jacoco.exec
./target/reports/junit/TEST-OneTest.xml


$ ant sonar
# this generates these two files, which are just a list of pmd rules (taken from the local sonar installation I guess). in any case, they are not a computation result.
.sonar//pmd-unit-tests.xml
.sonar//pmd.xml

# and sends the results to sonar.
with Unit tests success: 100% (1 tests), Unit Tests coverage: 60.0%

so, how do JaCoCo and Sonar work together?

what does JaCoCo do exactly?

what does this ./target/jacoco.exec file contain exactly?

how does Sonar use this file, and what does Sonar do exactly?

where is the 60.00% coverage result computed? during ant test or during ant sonar?

Full log of ant sonar:

$ ant sonar
Buildfile: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/build.xml

init:

compile:

sonar:
[sonar:sonar] Apache Ant(TM) version 1.9.3 compiled on December 23 2013
[sonar:sonar] Sonar Ant Task version: 2.1
[sonar:sonar] Loaded from: file:/private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/sonar-ant-task-2.1.jar
[sonar:sonar] INFO: Default locale: "en_US", source code encoding: "UTF-8"
[sonar:sonar] INFO: Work directory: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/.sonar
[sonar:sonar] INFO: Sonar Server 4.0
[sonar:sonar] 15:49:08.182 INFO  - Load batch settings
[sonar:sonar] 15:49:08.259 INFO  - User cache: /Users/david/.sonar/cache
[sonar:sonar] 15:49:08.266 INFO  - Install plugins
[sonar:sonar] 15:49:08.523 INFO  - Install JDBC driver
[sonar:sonar] 15:49:08.529 WARN  - H2 database should be used for evaluation purpose only
[sonar:sonar] 15:49:08.529 INFO  - Create JDBC datasource for jdbc:h2:tcp://localhost/sonar
[sonar:sonar] 15:49:09.365 INFO  - Initializing Hibernate
[sonar:sonar] 15:49:11.269 INFO  - Load project settings
[sonar:sonar] 15:49:11.293 INFO  - Apply project exclusions
[sonar:sonar] 15:49:11.405 INFO  - -------------  Scan UT coverage with Ant and JaCoCo running tests
[sonar:sonar] 15:49:11.408 INFO  - Load module settings
[sonar:sonar] 15:49:11.732 INFO  - Quality profile : [name=Sonar way,language=java]
[sonar:sonar] 15:49:11.744 INFO  - Excluded tests: 
[sonar:sonar] 15:49:11.744 INFO  -   **/package-info.java
[sonar:sonar] 15:49:11.809 INFO  - Index files
[sonar:sonar] 15:49:11.827 INFO  - 2 files indexed
[sonar:sonar] 15:49:11.836 INFO  - Loading technical debt model...
[sonar:sonar] 15:49:12.599 INFO  - Loading technical debt model done: 763 ms
[sonar:sonar] 15:49:12.615 INFO  - Configure Maven plugins
[sonar:sonar] 15:49:12.975 INFO  - Compare to previous analysis (2014-01-20)
[sonar:sonar] 15:49:12.990 INFO  - Compare over 30 days (2013-12-21, analysis of 2014-01-16 12:11:07.227)
[sonar:sonar] 15:49:13.142 INFO  - Base dir: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests
[sonar:sonar] 15:49:13.142 INFO  - Working dir: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/.sonar
[sonar:sonar] 15:49:13.143 INFO  - Source dirs: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/src
[sonar:sonar] 15:49:13.143 INFO  - Test dirs: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/test
[sonar:sonar] 15:49:13.143 INFO  - Binary dirs: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/target/classes
[sonar:sonar] 15:49:13.143 INFO  - Source encoding: UTF-8, default locale: en_US
[sonar:sonar] 15:49:13.152 INFO  - Sensor JavaSourceImporter...
[sonar:sonar] 15:49:13.171 INFO  - Sensor JavaSourceImporter done: 19 ms
[sonar:sonar] 15:49:13.171 INFO  - Sensor JavaSquidSensor...
[sonar:sonar] 15:49:13.237 INFO  - Java AST scan...
[sonar:sonar] 15:49:13.242 INFO  - 1 source files to be analyzed
[sonar:sonar] 15:49:13.347 INFO  - 1/1 source files analyzed
[sonar:sonar] 15:49:13.348 INFO  - Java AST scan done: 111 ms
[sonar:sonar] 15:49:13.350 INFO  - Java bytecode scan...
[sonar:sonar] 15:49:13.374 INFO  - Java bytecode scan done: 24 ms
[sonar:sonar] 15:49:13.383 INFO  - Package design analysis...
[sonar:sonar] 15:49:13.394 INFO  - Package design analysis done: 11 ms
[sonar:sonar] 15:49:13.399 INFO  - Sensor JavaSquidSensor done: 228 ms
[sonar:sonar] 15:49:13.399 INFO  - Sensor SurefireSensor...
[sonar:sonar] 15:49:13.400 INFO  - parsing /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/target/reports/junit
[sonar:sonar] 15:49:13.464 INFO  - Sensor SurefireSensor done: 65 ms
[sonar:sonar] 15:49:13.464 INFO  - Sensor CpdSensor...
[sonar:sonar] 15:49:13.464 INFO  - SonarEngine is used
[sonar:sonar] 15:49:13.465 INFO  - Cross-project analysis disabled
[sonar:sonar] 15:49:13.483 INFO  - Sensor CpdSensor done: 19 ms
[sonar:sonar] 15:49:13.483 INFO  - Sensor PmdSensor...
[sonar:sonar] 15:49:13.484 INFO  - Execute PMD 4.3...
[sonar:sonar] 15:49:13.492 INFO  - Java version: 1.5
[sonar:sonar] 15:49:13.512 INFO  - PMD configuration: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/.sonar/pmd.xml
[sonar:sonar] 15:49:14.061 INFO  - PMD configuration: /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/.sonar/pmd-unit-tests.xml
[sonar:sonar] 15:49:14.062 INFO  - Execute PMD 4.3 done: 578 ms
[sonar:sonar] 15:49:14.062 INFO  - Sensor PmdSensor done: 579 ms
[sonar:sonar] 15:49:14.062 INFO  - Sensor InitialOpenIssuesSensor...
[sonar:sonar] 15:49:14.070 INFO  - Sensor InitialOpenIssuesSensor done: 8 ms
[sonar:sonar] 15:49:14.071 INFO  - Sensor ProfileSensor...
[sonar:sonar] 15:49:14.175 INFO  - Sensor ProfileSensor done: 104 ms
[sonar:sonar] 15:49:14.175 INFO  - Sensor ProfileEventsSensor...
[sonar:sonar] 15:49:14.194 INFO  - Sensor ProfileEventsSensor done: 19 ms
[sonar:sonar] 15:49:14.195 INFO  - Sensor ProjectLinksSensor...
[sonar:sonar] 15:49:14.201 INFO  - Sensor ProjectLinksSensor done: 6 ms
[sonar:sonar] 15:49:14.201 INFO  - Sensor VersionEventsSensor...
[sonar:sonar] 15:49:14.213 INFO  - Sensor VersionEventsSensor done: 12 ms
[sonar:sonar] 15:49:14.214 INFO  - Sensor FileHashSensor...
[sonar:sonar] 15:49:14.215 INFO  - Sensor FileHashSensor done: 1 ms
[sonar:sonar] 15:49:14.215 INFO  - Sensor JaCoCoSensor...
[sonar:sonar] 15:49:14.220 INFO  - Analysing /private/tmp/aa/sonar-examples/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests/target/jacoco.exec
[sonar:sonar] 15:49:14.289 INFO  - No information about coverage per test.
[sonar:sonar] 15:49:14.289 INFO  - Sensor JaCoCoSensor done: 74 ms
[sonar:sonar] 15:49:14.794 INFO  - Execute decorators...
[sonar:sonar] 15:49:15.196 INFO  - Store results in database
[sonar:sonar] 15:49:15.238 INFO  - ANALYSIS SUCCESSFUL, you can browse http://localhost:9000/dashboard/index/org.codehaus.sonar:example-ut-ant-jacoco-runTests
[sonar:sonar] 15:49:15.263 INFO  - Executing post-job class org.sonar.plugins.core.issue.notification.SendIssueNotificationsPostJob
[sonar:sonar] 15:49:15.264 INFO  - Executing post-job class org.sonar.plugins.core.batch.IndexProjectPostJob
[sonar:sonar] 15:49:15.284 INFO  - Executing post-job class org.sonar.plugins.dbcleaner.ProjectPurgePostJob
[sonar:sonar] 15:49:15.295 INFO  - -> Keep one snapshot per day between 2013-12-23 and 2014-01-19
[sonar:sonar] 15:49:15.296 INFO  - -> Keep one snapshot per week between 2013-01-21 and 2013-12-23
[sonar:sonar] 15:49:15.297 INFO  - -> Keep one snapshot per month between 2009-01-26 and 2013-01-21
[sonar:sonar] 15:49:15.297 INFO  - -> Delete data prior to: 2009-01-26
[sonar:sonar] 15:49:15.299 INFO  - -> Clean UT coverage with Ant and JaCoCo running tests [id=1018]
[sonar:sonar] 15:49:15.303 INFO  - <- Clean snapshot 18068

Answer 1

JaCoCo is used during the <junit/> task itself. When executed, JaCoCo instruments the class files used in the test with the coverage information. This produces a jacaco.exec file. This file is then used to generate the code coverage report.

There's usually a <jacoco:report/> task that takes the jacoco.exec file and generates either a text file, a HTML page, or an XML file with the coverage information in it. This is probably what Sonar uses for its analysis.

Of course, it's hard without seeing the build.xml file how things work.