Whole program optimizations with JIT

[ladeak]

@stephentoub mentions:

Native AOT is capable of doing whole program optimization, where it can look at everything in the program and optimize based on the totality of everything involved (in contrast, a JIT’d .NET application may load additional .NET libraries at any point).

Could the compiler have a flag for building JITted application (non-NativeAOT) indicating that the compiler sees the whole application? Meaning, that the application does not load assemblies that are not part of the build.

Would that enable some optimizations for the JITted apps? (like knowing all the types or if a type is used or not etc., devirtualization etc.)

[huoyaoyuan]

Could the compiler have a flag for building JITted application (non-NativeAOT) indicating that the compiler sees the whole application? Meaning, that the application does not load assemblies that are not part of the build.

No, because JIT compilation is incremental. Any untouched code is seen as newly loaded.
And also, it requires to turn off all the dynamic loading APIs. Currently only the AOT runtime is built in such way. The coreclr runtime for JIT always load things lazily.

Would that enable some optimizations for the JITted apps? (like knowing all the types or if a type is used or not etc., devirtualization etc.)

It isn't feasible at JIT time due to its cost. Presumably some metadata information can be gathered at build time and helps the optimization with JIT, but turning the runtime to run in AOT way is a strong prerequisite.

[ladeak]

I meant a flag that can be set at c# compile time already, can be validated by c# compiler and ca be respected by the JIT through metadata. And just like NativeAOT, I would expect dynamic assembly to loading, etc. to fail.

And it would be up to the developer to decide to set the flag or not to.

[huoyaoyuan]

Flags aren't magical. They can't change contracts about mechanisms influecing many components.

C# compiler can't validate any whole program optimization because it doesn't has view of the whole program. It has totally no idea about what's happening inside referenced assembly, and it's specifically designed so to make compilation optimal and flexible.

[ladeak]

My thinking is that the C# compiler could leverage the NativeAOT toolchain to gain the understanding of the whole program, but at the end still output IL.

[huoyaoyuan]

It's nothing to do with C#. Even if there is such pre-optization tool, it will work after C# compiler, like how ILLink works.

[AndyAyersMS]

I believe ILC (which drives the jit in NAOT compilation) already does some of this sort of analysis, eg sealing types that are not further derived from.

There has been some preliminary discussion about approaches to building inteprocedural capabilities in the JIT for NAOT and (perhaps as hints) for the full runtime. With any luck we'll have something we can share before too long.

[am11]

There is also an experiment by @MichalStrehovsky to delegate scanning / whole-program-view responsibility to RyuJIT #83021.