[Proposal] Partitioned R2R WASM modules - #135544

I do not see anything in the design that makes it single-threaded only. If there is some detail that is specific to single threading, I would call it out inline and delete the general note here.

It is fine to say that the initial implementation will be for single threaded wasm only in the implementation section.

You may want to say that this is going to target composite mode only. Is that right?

I removed this line from the overview and added a note in the Implementation phases section that V1 focuses on single-threaded, composite mode support.

This check is on the hot path through the interpreter. We really want this check to be very cheap (ideally, one test + branch) unless there is a chance we may be able to activate new method (ie the code was downloaded). You may want to call it out explicitly.

Two MethodDefEntryPoints and InstanceMethodEntryPoints exist, one in the primary module

Assuming we are targeting the composite mode, there should be one MethodDefEntryPoints per assembly.

I updated the AuxiliaryModuleMetadata section to include a MethodDefEntryPointsInfoRVA field which allows the runtime to find an auxiliary MethodDefEntryPoints table for each assembly. The data pointed to is structured and ordered like the ComponentAssemblies section (an array of 2 section pointers per entry) and points to zeroed memory allocated for the auxiliary MethodDefEntryPoints sections.

I also updated the AuxiliaryEntryPointsModule to include a MethodDefEntryPoints section per assembly. Activation populates the space allocated in the primary module for each auxiliary MethodDefEntryPoints.

I think this makes an assumption that there is only one auxiliary module. It is fine simplifying assumption to make for v1 design. If you make that assumption for v1 design (and assuming that we are targeting composite mode only for v1 design), this can be just simple global flag. No need to store anything in the InterpreterData.

(And for the eventual design with multiple aux modules, I do not think we know at this point which aux module is the method going to show up in, so it would have to be more complicated than this. You need to worry about that for now.)

I think we should only need a single AuxiliaryEntryPointsModule for a logical R2R image, even if there are many auxiliary modules. Do you see a blocker?

We also should be able to know which aux module the method belongs to once we have its RuntimeFunction index. If we lay out the RuntimeFunctions in contiguous ranges starting with the primary module and order the aux modules the same as the AuxiliaryModuleMetadata section, we can do a bounds check on the RuntimeFunctions index to determine which Auxiliary module it belongs to.

II think we should only need a single AuxiliaryEntryPointsModule for a logical R2R image, even if there are many auxiliary modules.

Right.

I do not see how you can get away with a single EntryPointsModuleActivationStatus flag shared by many auxiliary modules.

I do not think it matters a whole lot. I would focus on a single auxiliary module only in the initial implementation, and worry about extending it to multiple auxiliary modules later.

(Also, it looks odd for EntryPointsModuleActivationStatus to be in a Header. The module headers are typically readonly and part of image payload.)

it looks odd for EntryPointsModuleActivationStatus to be in a Header. The module headers are typically readonly and part of image payload.

If you mean the PE / WebCIL headers, the flag isn't there. I think I wrote poorly modeled pseudocode. The flag is in the "header" within the AuxiliaryModuleMetadata section's data, not the Module Headers. It may look clearer written like primary.AuxiliaryModuleMetadataSection.Header.EntryPointsModuleActivationStatus.

I do not think it matters a whole lot. I would focus on a single auxiliary module only in the initial implementation, and worry about extending it to multiple auxiliary modules later.

Good point, if 1 auxiliary module is enough this all becomes moot.

Term "cold module" is confusing to me. Because we say app "cold start" and speak about primary module to be eagerly downloaded for it. Can we use "auxiliary module" consistently ? They could also be called lazy or late loaded shards. In javascript bundlers they are called "chunks".

We can trigger fetch - download browser API from synchronous method, as soon as your trigger activates. But the API is async and you will be only able to instantiate the wasm module and activate it's functions after you yielded all the way back up to browser event loop.

In short, you can't synchronously block and wait for the download to finish (as opposed to loading a .DLL on normal OS synchronously).

We will only benefit from lazy loaded aux module "on the next user click", not within same synchronous busy loop as is usual with JIT and OSR.

I am aware of that, but I don't have a good sense for how often we return to the browser event loop - I had assumed it wasn't particularly infrequent and we wouldn't have to wait too long once the auxiliary module is downloaded. It sounds like maybe my assumption is wrong, though?

Web pages are expected to yield to browser event loop often, otherwise the UI doesn't re-render either. So long running synchronous CPU bound jobs are atypical in our apps.

Sometimes you can't avoid it, for example deserializing 10 MB of json. If that happens without R2R of the JSON parser being already downloaded/installed, the UI will be frozen while interpreter is slowly processing that JSON. For that reason span/utf8/json/simd methods need to be "forced" into the PGO/eager set.

Advanced users would be willing to use streaming JSON parser or decompose their CPU bound jobs into multiple Tasks with await Tasks.Yield() in between. Ultimate answer is support for MT.

Multi-threaded WASM module instantiation is really complex. It needs to be downloaded or transfered asynchronously to a WebWorker (thread) and then installed into each of them. The table0 is private to each WebWorker. That means that passing function pointer that could lead into aux module from one thread to another could point to function index that is not yet installed on the target/calling thread.
Also any new emscripten pthread would not be aware that it needs to install all existing aux modules first, before handing over the execution to managed code.

Dependency sets are interesting because they should inform which methods should be emitted into the same aux module (shard). Especially when direct calls would be supported.
The delay/latency of lazy async download and instantiation (say 100ms) could be prohibitive when done separately step by step on each recursive call. (when called methods are often in different file)

How do we know which aux module (shard) to download when we want to start activating some method lazily ?

We will also need to map it from crossgen file name suffix into full download URL, with fingerprint. That's WASM SDK and should probably work as is.

Once the AuxiliaryEntryPointsModule is downloaded and activated, we can look up the RuntimeFunctions index for the method. The RuntimeFunctions will be laid out so that each auxiliary module has a contiguous range which starts at sum(<previous modules' MethodCount>) and ends at start + MethodCount where MethodCount is found in the AuxiliaryModuleMetadata section.

The FilenameRVA field in the AuxiliaryModuleMetadata section has the file name, would that be enough to construct the URL from?