JIT vs. NGen for cold and warm startup

Question

I am trying to understand the difference between JIT and NGen, please help me. Does JIT cache binary code? If yes, it means NGen can improve only cold run performance. But I have read some articles which say the contrary: NGen improves significantly only warm startup.

Answer 1

It is the exact opposite. Cold starts take longer than warm starts because the assemblies are not in the file system cache and need to be found on disk. Which can be quite slow on mechanical disk drives. NGen creates an extra file on disk for every ngen-ed assembly, a .ni.dll file that contains the pre-jitted machine code. This file needs to be found and loaded in addition to the original assembly, roughly doubling the cost of a cold start for that particular assembly.

So by design, NGen can only reliably improve the warm start of your program, it worsens the part of a cold start that makes it slow. You will only be ahead in the cold start case if the time taken by the jitter at startup is more than the time needed to find the .ni.dll file. Which of course is very hard to predict since it depends so much on disk perf and very hard to measure since you only have one shot at this before starts turn warm.

You'll have to experiment. A rough guide is that the assembly should be "substantial" with extra reasons to ngen when it is stored in the GAC and used in multiple processes since that permits loading the pre-jitted code into RAM only once for multiple processes.

Answer 2

The .NET JIT does not cache machine code across processes (I suspect for security and correctness reasons - assemblies can change and the stored machine code can be edited. Those are hard problems to solve). Every process creates the machine code anew.

Because of that, NGEN improves load time for cold and for hot startup.