Why do we oppose AOT and JIT compilation. Can they be complementary?

Question

I'm just not sure as to why JIT (Just-in-time) and AOT (Ahead-of-time) are often presented in contradiction to another.

If we do not care about about portability, it feels to me that a program could very well be AOT compiled and then, at runtime the JIT could be used to re-optimized the hot parts.

What are some known implementations using this scheme. If there are none, why so?

Answer 1

You're talking about profile-guided optimization. The Scala native AOT compilers I've seen employ PGO do so in two stages. Basically, the first pass instruments the code to generate a profile file during runtime. You then run that version through a test suite that exercises the typical use cases of your product. Then that profile file is used as an input to a second build pass that actually applies the optimization to generate a native executable.

By pregenerating the runtime profile at build time, you lose the potential benefits of optimizations based on unusual usage patterns of your code, but you also lose the runtime profiling overhead. I've seen this demoed at conferences and don't remember the exact numbers, but the run time was comparable and there was an impressive RAM and startup time reduction over the HotSpot JVM which does essentially the same PGO in a JIT manner.

Answer 2

You seem to be missing a critical aspect of JIT: platform independence. As soon as you compile down to machine code AOT, your program isn't platform-independent anymore.

That said, the .NET ecosystem has provided this capability for quite some time now. You can read about that in this article.

I'm not sure what you mean by re-optimizing hot spots using JIT. By definition, once you compile AOT, your program is, well, already compiled.

Answer 3

Yes. It's possible and quite common for a program to be recompiled three times:

1. At the developer's machine, the code is compiled to an intermediate language code. This is a form of the code that's faster to parse than the source code, but is targeted for a generic virtual machine or a generic version of the target architecture. The purpose of the compilation here is mainly for portability rather than optimisation, but there are many simple optimisations that can happen at this step, like dead code elimination, constant folding, method inlining, etc.

2. When the program is installed or loaded for the first time, the intermediate language code can be compiled to the machine's architecture native code. This compilation step allows the program to take advantage of the knowledge of the particular instruction set that it is running on. Rather than compiling for a generic architecture, like "most x86 CPUs", compiling at installation time allows the compiler to take advantage of the knowledge that it is running on an "AMD Ryzen 3rd Gen Threadripper 3960X", which means the program can take advantage of the latest CPU instruction set for that particular machine, without having to worry about making the code unsuitable in other x86 machines.

3. At runtime, a JIT may trace that the execution path of the code to find out that the file the user is working on doesn't use feature Foo, so that means that a hot loop that calls method A, it is always followed by calls to method C, D, and H and there is a number of if-condition in those methods that have never been observed to be true as they only ever is needed for files that uses Foo, so let's recompile C, D, and H into a new method C_D_H that removes those if-blocks and put a guard up in method A to return to the old code if suddenly the user opened another file that do use feature Foo. JIT compilation has the best vantage point to produce the fastest code, but it's also a risky place because they need to instrument the code to trace where the code is going, and they need to do these analysis and recompilations in real time, which mean they have the potential to actually slow down execution compared to just running the unoptimised code.

Languages that often does take advantage of all three compilation is JavaScript, Web Assembly, and Android.