Producing CTest results in Jenkins (xUnit >= 1.58)

Question 1

Here is a small example that demonstrates how to have xUnit pick up a CTest generated XML test result file. The example consists of a single C++ source file main.cpp

#include <cstdlib>
int main() {
    std::exit(-1);
}

and an accompanying CMakeLists.txt:

cmake_minimum_required(VERSION 2.8)
project(JenkinsExample)
enable_testing()
add_executable(main main.cpp)
add_test(test1 main)
add_test(test2 main)
set_tests_properties(test2 PROPERTIES WILL_FAIL ON)

The CMake list file generates an executable main and runs this executable from two CMake tests, where for demonstration purposes one will always fail and the other one will always succeed.

Using Jenkins, set up a new job and a add new cmakebuilder build step. Point the CMake source directory to the folder that contains the example project on your local machine. The CMake build directory should be set to build. This will make Jenkins create a CMake build folder in the job's workspace directory. It's a good idea to set the Clean Build flag to make the job always start with a clean state.

Then, assuming you are running Unix, add an Execute shell job step and enter the following shell script in the Command box:

cd build
/usr/local/bin/ctest --no-compress-output -T Test || /usr/bin/true

Running ctest with the option -T Test will make CTest generate an XML output file in a sub-folder Testing inside the build folder, which can be picked up by the xUnit plug-in in a post-build action then. The || /usr/bin/true is necessary to prevent Jenkins from aborting the build prematurely (without running the xUnit plug-in) if some tests fail.

If you are using Windows, set up a similar Execute Windows batch command job step instead:

cd build
"C:\Program Files (x86)\CMake 2.8\bin\ctest.exe" --no-compress-output -T Test || verify > NUL

Finally the xUnit plug-in must be configured in the following way:

Add a Publish xUnit test result report post-build action and then use the plug-in's Add button to create a CTest-Version test result report. In the CTest-Version (default) Pattern enter the file pattern build/Testing/**/Test.xml.

Question 2

For those wanting to parse CTest output in a Jenkins Declarative Pipeline, you can now do this quite easily using the xUnit plugin, since it can parse CTest output directly.

Add a stage in your Jenkinsfile to run ctest in the build directory, and add a post stage to process the output with xUnit. Here's a skeleton example:

pipeline {
  agent any
  stages {
    stage('Configure') {
      steps {
        dir('build') {
          sh 'cmake ../'
        }
      }
    }
    stage('Build') {
      steps {
        dir('build') {
          sh 'cmake --build .'
        }
      }
    }  
    stage('Test') {
      steps {
        dir('build') {
          sh 'ctest -T test --no-compress-output'
        }
      }
    }
  }
  post {
    always {
      // Archive the CTest xml output
      archiveArtifacts (
        artifacts: 'build/Testing/**/*.xml',
        fingerprint: true
      )

      // Process the CTest xml output with the xUnit plugin
      xunit (
        testTimeMargin: '3000',
        thresholdMode: 1,
        thresholds: [
          skipped(failureThreshold: '0'),
          failed(failureThreshold: '0')
        ],
      tools: [CTest(
          pattern: 'build/Testing/**/*.xml',
          deleteOutputFiles: true,
          failIfNotNew: false,
          skipNoTestFiles: true,
          stopProcessingIfError: true
        )]
      )

      // Clear the source and build dirs before next run
      deleteDir()
    }
  }
}

For examples of how to configure the xUnit plugin in a Declarative Pipeline, the Jenkins Snippets Generator is a very helpful resource.

Question 3

Since CMake 3.21.0 there is also a new option in CTest:

$ man ctest
[...]
  --output-junit <file>        = Output test results to JUnit XML file.

Reference:

ctest: Add support for generating xUnit/JUnit XML output