Performance testing utilities for C++ [closed]

Question

I was curious whether anyone has any suggestions for performance testing libraries and frontends that will produce nice graphical charts for C++ (like how gcov produces coverage data and there are frontends out there for viewing the code coverage data). Being able to produce charts like:

http://download.eclipse.org/eclipse/downloads/drops/S-3.7M4-201012081300/performance/performance.php?fp_type=0

would be pretty slick. We use cppunit right now for unit testing, so maybe there is something that integrates with that.

Some more info: We're compiling on Linux (we use Ubuntu Lucid/Maverick) on Intel x86-64 machines.

Answer 1

Some suggestions:

• The googletest C++ framework is capable of producing JUnit-compatible reports.
• Hudson can be used to run your tests. It only requires that your C++ application can be run as a console application.
• The Hudson Performance Plugin can generate graphical charts from JUnit reports.
• There are many other plugins.

Answer 2

On linux you can try the valgrind toolkit. Valgrind includes the callgrind tool that can profile your code. KCacheGrind visualizes the output of cachegrind very nicely.

Answer 3

Some hopefully relevant notes from my experience of this sort of thing in answers here and here.

There's no reason your existing CppUnit setup combined with something like the scoped_timer (see second of the above) and a bit of postprocessing to some charts maybe shouldn't satisfy your needs. While I believe CppUnit includes a plugin ("Clocker" ?) which will log out unittest times, I don't think it's actually that useful for this sort of performance testing in practice because you invariably end up wanting to time just part of each test and not all the setup code.

Answer 4

There are two performance testing frameworks can help you.

• SkyPat (​https://skypat.skymizer.com)
• Hayai (​https://github.com/nickbruun/hayai)

Both of them are inspired by Google Test framework and provide interface to support performance testing, so it’s easy to transport your original googletest to Hayai or SkyPat.

SkyPat combines unit tests and perf_evnet. It extends the concept of Google Test and provides an interface to access PMU. perf_event gives SkyPat accurate cycle counts that are useful for tools that are sensitive to the variance of timing, such as compilers. SkyPat can also profile a piece of code by PMU events (eq: cycles, instructions, cache reference, cache miss, etc.).

Hayai supports several performance measuring methods of different OS (Windows: QueryPerformanceCounter(), Linux: gethrtime(), Apple(OSX, iOS): mach_absolute_time()) into its performance testing framework.