CMake: setting an environmental variable for ctest (or otherwise getting failed test output from ctest/make test automatically)

Question

I want ctest to show me the failed tests output by default. That is, I want to run:

$ make all test

and see any output of failed tests without having to cat Testing/Temporary/LastTest.log.

It appears that there are two ways of doing this:

(1) Setting the CTEST_OUTPUT_ON_FAILURE environmental variable:

 $ CTEST_OUTPUT_ON_FAILURE=1 make all test
 $ # or CTEST_OUTPUT_ON_FAILURE=1 ctest

(2) Specifying the --output-on-failure flag to the ctest invocation:

 $ ctest --output-on-failure

Is there a way to write a CMakeLists.txt file such that ctests dumps failed tests output by default on a normal "make all test" invocation WITHOUT resorting to exporting the environmental variable globally in the session or resorting to a custom target like make check (as described here)?

I am aware of the SET_TESTS_PROPERTIES() command, but trying it out like this:

CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
PROJECT(testenv CXX)
ENABLE_TESTING()
ADD_EXECUTABLE(hello hello.cpp)
ADD_TEST(testhello hello)

# Following sets the environment variable for the shell in which the test
# progoram 'hello' is run, but not the shell in which ctest is run
SET_TESTS_PROPERTIES(testhello
    PROPERTIES ENVIRONMENT "CTEST_OUTPUT_ON_FAILURE=1")

and experimenting shows that the environmental variable is set in the shell that the test program is executed in, but not in the shell that ctest is executed in.

Answer 1

The built-in test target cannot be modified, but you can add a custom check target which invokes ctest with the --output-on-failure switch in the following way:

if (CMAKE_CONFIGURATION_TYPES)
    add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND} 
        --force-new-ctest-process --output-on-failure 
        --build-config "$<CONFIGURATION>")
else()
    add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND} 
        --force-new-ctest-process --output-on-failure)
endif()

The custom target has to be set up differently for single build type and multi-configuration builds. In the latter case, the active build configuration has to be passed on to the ctest invocation using the --build-config flag. The --force-new-ctest-process is used by the built-in test target by default.

Answer 2

There is now the list variable CMAKE_CTEST_ARGUMENTS in which you can set ctest arguments.

https://cmake.org/cmake/help/latest/variable/CMAKE_CTEST_ARGUMENTS.html