In short my question is:
What do forked warmups offer that within-fork warmups do not? What is the thing that makes executing forked warmups useful?
JMH conveniently allows you to fork a new JVM instance in which your benchmarks are run. This is useful because as explained in this and this question, it allows you to run your benchmakrs with a clean slate: a fresh OS process where the state of both the JVM and the OS/system has no previous decisions/observations for the process which runs the benchmarks.
In addition, JMH allows you to perform a number of initial "warmup" runs of your benchmarks in the forked process, the results of which are discared. These are useful to let the system "settle" with decisions made during execution (OS makes memory paging decisions, JIT compiles VM code to native, etc).
There are actually though two ways this can be done: using
@Warmup annotation and using the
warmups parameter of the
@Fork annotation. As explained in what is the difference between warmup attribute in Fork and Warmup annotation in jmh? the former executes some initial iterations within each fork, whereas the latter executes some entire forks that are discarded.
I am having trouble understanding why the forked warmups were introduced as a feature. A forked process presents a fresh challenge to the the OS/JVM removing any bias that may arise from previously executed code in the existing process. Having the forked process execute enough warmup iterations also means that any "initial optimizations" the OS/JVM perform, are given time to "settle".
So then, why would I want to run some entire forks that are discarded all together by adding
@Fork(warmups = N)? I'm guessing it has to do with the OS/system being given time to settle on something (as the JVM does not stick around after each fork to use the information from the previous warm-up forks) but what is it about a process being created/torn-down a few times that helps improve results? And why would a generous
@Warmup annotation not achieve the same if a single fork were to be used?