Timing a block of code with C++ and Java

Question

I am trying to compare the accuracy of timing methods with C++ and Java.

With C++ I usually use CLOCKS_PER_SEC, I run the block of code I want to time for a certain amount of time and then calculate how long it took, based on how many times the block was executed.

With Java I usually use System.nanoTime().

Which one is more accurate, the one I use for C++ or the one I use for Java? Is there any other way to time in C++ so I don't have to repeat the piece of code to get a proper measurement? Basically, is there a System.nanoTime() method for C++?

I am aware that both use system calls which cause considerable latencies. How does this distort the real value of the timing? Is there any way to prevent this?

Answer 1

Every method has errors. Before you spend a great deal of time on this question, you have to ask yourself "how accurate do I need my answer to be"? Usually the solution is to run a loop / piece of code a number of times, and keep track of the mean / standard deviation of the measurement. This is a good way to get a handle on the repeatability of your measurement. After that, assume that latency is "comparable" between the "start time" and "stop time" calls (regardless of what function you used), and you have a framework to understand the issues.

Bottom line: clock() function typically gives microsecond accuracy.

See https://stackoverflow.com/a/20497193/1967396 for an example of how to go about this in C (in that instance, using a usec precision clock). There's the ability to use ns timing - see for example the answer to clock_gettime() still not monotonic - alternatives? which uses clock_gettime(CLOCK_MONOTONIC_RAW, &tSpec);

Note that you have to extract seconds and nanoseconds separately from that structure.

Answer 2

Be careful using System.nanoTime() as it is still limited by the resolution that the machine you are running on can give you.

Also there are complications timing Java as the first few times through a function will be a lot slower until they get optimized for your system.

Virtually all modern systems use pre-emptive multi threading and multiple cores, etc - so all timings will vary from run to run. (For example if control gets switched away from your thread while it in the method).

To get reliable timings you need to

1. Warm up the system by running around the thing you are timing a few hundred times before starting.
2. Run the code for a good number of times and average the results.

The reliability issues are the same for any language so apply just as well to C as to Java so C may not need the warm-up loop but you will still need to take a lot of samples and average them.