JVM warmup with ahead-of-time compilation

Question

I'm using Leiningen to make a project in Clojure. I have to benchmark this project. I came to the understanding that I have to warm up my JVM. I already to ahead-of-time compilation, does this mean I do not have to warm up my JVM?

I understand that calling function f for example 100 times will create a hotspot in the JIT.

My benchmarking function load will run the code once with given parameters including a filepath parameter. At the moment I run my function load 1000 times with the same parameters but I pass it a different file path to store its results to. After these 1000 calls are done I execute the function once more with the correct file path.

The problem is that my benchmark has side-effects (writing measurements to a file). So I would rather not execute it 1000 times before I do the real run once. At the moment I just give it a dummy file so it still writes to the file but I discard those results.

Could somebody elaborate please what I have to pay attention to?

Answer 1

Nope, AOT does not mean you don't have to warm up the JIT. AOT refers to Clojure compilation, not optimization.

Try to separate side effects into different functions and run the warmup with pure parts before running the benchmark.

UPD: You can also use /dev/null as the file for test runs.

Answer 2

I recommend using Criterium for benchmarking Clojure code. It takes care of the issues you asked about. It will take longer to run a benchmark, because Criterium does the JVM warmup, etc., and then runs the code multiple times, giving you averages and other statistics. But you can feel fairly confident about the results.