Is there a way to run C++ Unit Tests tests in parallel?

Question

I'm using Boost Test for a long time now and I ends up having my tests running too slowly. As each test is highly parallel, I want them to run concurrently with all my cores.

Is there a way to do that using the Boost Test Library ? I didn't found any solution. I tried to look a how to write custom test runner, but I didn't much documentation on that point :(

If there is no way, does someone know a good C++ Test Framework to achieve that goal ? I was thinking that Google Test would do the job but apparently it cannot run test in parallel either. Even if the framework has less features than other more known framework, it is not a problem, I just need simple assertions and multi-threaded execution.

Thanks

Answer 1

You could use CTest for this.

CTest is the test driver which accompanies CMake (the build system generator), so you'd need to use CMake to create the build system using your existing files and tests, and in doing so you would then be able to use CTest to run the test executables.

I haven't personally used Boost.Test with CMake (we use GoogleTest), but this question goes into a little more detail on the process.

Once you have the tests added in your CMakeLists file, you can make use of CTest's -j argument to specify how many jobs to run in parallel.

Answer 2

What google is hinting at in the gtest documentation is test sharding - letting multiple machines run the tests by just using command line parameters and environment variables. You could run them all on one machines in separated processes, where you set the GTEST_SHARD_INDEX and GTEST_TOTAL_SHARDS environment variables appropriately.

In principle, nothing is preventing you from starting multiple processes of the test executable with a different filtering parameter (Boost.test, gtest)

Update 2014: https://github.com/google/gtest-parallel

Answer 3

Split the suite to consist of multiple smaller sets each launched with individual binary, and add .PHONY target test to your build sysem depending on all of them. Run as (assuming you are using make) make -jN test

Answer 4

Given that the third bullet point on the open issues list is currently thread safety, I don't believe there's a way to tell Boost test to run tests in multiple threads.

Instead, you will need to find an external test runner that supports running tests in parallel (I would expect this to work by forking off new processes).