I plan to start with, "Exports of globals from Wasm are imported as WebAssembly.Global objects, not Numbers", tracked in #8145 . If you want to take on a particular task, please note it in a comment below, to avoid duplication.

Here are a few points I have in mind:

Convert the remaining .wasm tests to .wat for easier development

Some are intentially binary to test against various LEB128 encoding, but I have some on my side too, I think we could just remove/move them from here.

A WebAssembly.Table, Memory or Global instance, or exported Wasm function, exported JS -> Wasm -> JS, comes out with the same identity

What does identity mean here? The same function object instance?

We already discussed about that because it's an optimization, we could directly wire up the function in JS and avoid calling in wasm entirely.

Variables imported JS -> Wasm are snapshotted (whether you overwrite a bare Number or function, or an entire Memory, Global or Table object which isn't replaced when the variable containing it is overwritten)

Isn't that already handle by WebAssembly's instantiation which does a copy of the values? Versus being a live-binding.

Circular Wasm modules result in an error before any start function executes

Which means an error at compilation because the start function is executed in the same tick than the instantiation.

The same Wasm module can be imported multiple times (both from JS and Wasm) and only one copy of it is made (1 start function evaluation, 1 memory/table/global identity, etc)

Modules are identified by their URL, right?

I wouldn't mind help here, cc @Ms2ger @xtuc

As I said, I will be able to work on this, and before the end of the year.

• Imports of Numbers become only immutable, not mutable, Global objects,
  and undergo rounding based on the declared type

Does this mean?

• A WebAssembly.Table, Memory or Global instance, or exported Wasm
  function, exported JS -> Wasm -> JS, comes out with the same identity
• A cycle Wasm <-> JS, with Wasm on top, leads to a TDZ error for
  Memory, Global or Table, but hoisted functions work, and Wasm things are
  all initialized by the time the JS top-level statement runs

We will fail these, because it's not possible to import table or memory from JS into WASM

• Memory, Table and mutable Global can be mutated by JS code that's
  called out from Wasm code, and the mutations are observed from Wasm

Do the current WASM implementation already support importing mutable globals? This was not part of the MVP, was it?

• Circular Wasm modules result in an error before any start function
  executes

This seems like an unnecessary restriction to me and is not the case of ESM. We will fail this because this is allowed.

• The return value of import() for a Wasm module is a
  module namespace exotic object, similar to that of native modules
  (WebPack will need to take shortcuts here)

In webpack it's not so exotic. It's an object with Symbol.toStringTag = "Module"... The same for ESM.

• Exports of globals from Wasm are imported as WebAssembly.Global
  objects, not Numbers (this looks likely to be an existing spec violation)

We pass along the value of the property in WebAssembly.Instance.exports.

Seem like this was a recent change in the spec. The JS API now exports WebAssembly.Global for globals. I actually wonder why this change was made because it's a breaking change for non-mutable globals. We may need to update our test suite, when this update has been made to Node.js. WebAssembly/spec@d8f775e#diff-ef3ad1622710e5adc119fbe2af30971cR376

• Imports of Numbers become only immutable, not mutable, Global objects,
  and undergo rounding based on the declared type

Sounds like importing a number as mutable global is allowed, but mutation the global will lead to an runtime error inside the wasm.

Here we also divert from the spec. We use live-bindings for functions instead of a snapshot, and we don't error for globals. Errors for imported globals will make it easier for us, as we currently need to convert imported globals into mutable globals and write the value at time of evaluation.

We should be able to change this by snapshotting all imported functions on initialization.

Here we also divert. We apply the same rules as for wasm imports <- JS exports. Seems like the spec expects a different behavior here for reexports. We need a special-case here and resolve reexports. Seem like this is only possible with ESM as it statically declares reexports. When the same module is transpiled to CJS it would break.

There is no special case for JS imports <- wasm re-exports <- JS exports so I assume functions are snapshotted here at the time of wasm evaluation.

cc @linclark

The spec is not clear here:

• Did the init expression for globals already run at the time of JS evaluation or does it run when the start function runs?
• Same question for the data and elem segment?
• Is reading globals, memory and tables allowed in JS evaluation or will it lead to a TDZ Error?

The current webpack implementation does it this way:

• data and elem run at instantiation, which happens before the JS evaluation
• not-imported globals and globals not referencing imported globals (get_global in init) also run at instantiation
• imported globals and globals referencing imported globals run at evaluation

This is a bit inconsistent and can not change since it's a kind of workaround for importing values from JS into WASM. Once this is no longer allowed all global init can run at instantiation (and we connect wasm-to-wasm global imports directly).

Note: When init, data and elem should run at evaluation, they would be uninitialized in JS evaluation. It would be very in-efficient for webpack to delay the data section until the evaluation phase. A TDZ error would be possible. Using a TDZ Error would make it irrelevant when these sections run since it's no longer visible from outside.

The spec often claims that functions are already initialized before evaluation. This is not necessary true for all functions exported by JS. It's only true for FunctionDeclarations. It's not true for i. e. export const fn = () => {}. Since the spec snapshots exports in the Instantiation phase these exports would result in errors.

It might be better to snapshot functions in the Evaluation phase where these exports are available. But this is more difficult to implement with the current WASM JS API (trampoline functions).

Other option is to use live bindings for functions, which also results in trampoline functions.

The spec also claims that all other exports have the value undefined which is also not true. They can also be in TDZ state. i. e. export const a = 1. Reading it will result in a TDZ Error. This results in the same behavior, but should be mentioned.

This seem to be needed because functions are snapshotted. When functions are treated as live bindings this would be possible.

@xtuc

Some are intentially binary to test against various LEB128 encoding, but I have some on my side too, I think we could just remove/move them from here.

Seems like my test plan was poorly worded. If that's what you were testing for, that's great, no particular reason to remove it. I'd just like to get full test coverage of the things discussed above in text

A WebAssembly.Table, Memory or Global instance, or exported Wasm function, exported JS -> Wasm -> JS, comes out with the same identity

What does identity mean here? The same function object instance?

We already discussed about that because it's an optimization, we could directly wire up the function in JS and avoid calling in wasm entirely.

To be clear, the "same identity" I'm talking about for functions just applies if we're talking about an exported Wasm function, which is then re-imported and re-exported. If you import a JS function into WebAssembly and then export it again, its behavior changes in that arguments are coerced to Wasm types according to how the function is imported. According to the Wasm-JS API specification, this isn't quite an optimization but rather exactly how the behavior is defined.

Isn't that already handle by WebAssembly's instantiation which does a copy of the values? Versus being a live-binding.

Yes, it is (as is the above identity). The idea is just to write a test that verifies that everything went well.

Circular Wasm modules result in an error before any start function executes

Which means an error at compilation because the start function is executed in the same tick than the instantiation.

An error at compilation sounds like a good solution to me. Is there a test for this yet?

Modules are identified by their URL, right?

Yes, AFAIK (that's HTML's responsibility). I don't know the details of how paths different URLs/redirects might be identified with each other; presumably this is the kind of thing which might reasonably differ between webpack and the Web (given you already add extensions, which it doesn't look like the web will do).

@sokra

Does this mean?

Yes, in particular: If you pass in a Number, it will become a const Global, per step 4.5 of the JS instantiate algorithm. Then, in the WebAssembly instantiate algorithm, step 4, it's checked whether the types match, which will not be the case if a mutable global is expected.

We will fail these, because it's not possible to import table or memory from JS into WASM

I'm wondering, is there anything blocking webpack from supporting importing table or memory from JS into Wasm?

Do the current WASM implementation already support importing mutable globals? This was not part of the MVP, was it?

It wasn't part of the initial MVP, but it's part of the 1.0 specification (rushed to minimize the compat risk), and shipping in Firefox and Chrome.

This seems like an unnecessary restriction to me and is not the case of ESM. We will fail this because this is allowed.

This seems like an unnecessary restriction to me and is not the case of ESM. We will fail this because this is allowed.

Indeed, it looks like you put in a ton of work to make circularities between multiple WebAssembly modules work. @rossberg, @lukewager and @linclark can fill in more details, but my understanding is that one big reason to not support circularities is in relation to the gc proposal, which adds a way for modules to import types from other modules, and which would not map very well to circularities. (However, in JS, circular modules with classes extending each other don't really work out well either, as @concavelenz and others have pointed out... I don't quite understand why types are worse.) I have a feeling we'll need to spend some more time on this circularity, at the very least to document the motivation for this prohibition better, but possibly also to consider alternatives.

In webpack it's not so exotic. It's an object with Symbol.toStringTag = "Module"... The same for ESM.

That sounds like a good solution to me.

((I'll follow up soon with responses to the later comments on the thread.))

see also WebAssembly/esm-integration#15

Re cyclic linking: there are at least four features in JavaScript that its cyclic linking ability depends on: (1) runtime checks, (2) TDZ failures/undefined values, (3) non-local "hoisting" of functions, (4) the complete absence of type definitions. These are costs that JS has already sunk, but for other languages, including Wasm, they are not necessarily feasible or acceptable (and even JS doesn't always swallow them, consider mutually recursive classes).

For that reason, you shouldn't assume that linking between JS and other languages can be as liberal as linking between JS and JS. I believe Allan WB has pointed out something similar in an earlier thread on the subject over on the TC39 github.

Probably it's not time yet to start landing changes to webpack based on the proposed specification. It looks like a number of @sokra's comments above are based on the version at master, while I'm betting on this PR to land.

Some more thoughts on cyclic Wasm modules at WebAssembly/esm-integration#17 .

I'm betting against that PR. It makes a fundamental change that make it in my opinion incompatible with the JS API. Snapshoting can't happen in the evaluation phase, because when using the JS API the importsObject need to be passed on instantiate and this is an async function. Evaluation phase is running sync in a single microtask. One could change the ESM spec, but I don't think it's worth a breaking change here.

We actually implemented it this way in the first prototype using sync instantiate while evaluation, but using sync instantiate was not acceptable. There is also an issue about that by Lin.

Snapshoting can't happen in the evaluation phase, because when using the JS API the importsObject need to be passed on instantiate and this is an async function. Evaluation phase is running sync in a single microtask.

This is an interesting point that I hadn't considered. But I don't understand--why do you expect evaluation to happen all in a single microtask?

@littledan that's how it works currently, the instantiation until the start func gets evaluated is done in a single microtask.

But I don't understand--why do you expect evaluation to happen all in a single microtask?

1. That's how I read the ESM spec: https://tc39.github.io/ecma262/#sec-moduleevaluation
   Promises are not mentioned here. Everything happens in a sync way.

2. Adding async behavior would slow down the evaluation and allow other microtasks while evaluating modules. This seems to be very weird to me since it leaves the application in an intermediate state.

3. As the evaluation happens sequentially, wasm instantiation would also happen sequentially, which could be potentially bad for performance.

Note that async evaluate would not only affect wasm modules. The "async" would leak into every module in the chain to the wasm module, since now imports need to be internally awaited.

This is a webpack specific issue: Currently it's possible to require a ESM module. This would no longer be possible when it referencing any wasm module, since require is sync, but evaluation async.

When wasm instantiation happens in the instantiation phase, implementations are free to do some instantiations in parallel, since this is not visible at runtime. Performance benefit.

see also #6433

OK, we'll have to think more about this issue about whether it would be OK to do Wasm instantiation asynchronously during the ESM evaluation phase. To respond to your points:

1. You're right. If we go down this path, we'll have to make further JS spec changes to allow asynchronous-ness in the Evaluation step. These changes will be needed for top-level await as well, so we should figure out how to do them in a common way. cc @MylesBorins @GeorgNeis

2. We might even allow other tasks, not just microtasks, in the middle, if compilation happens during evaluation and it takes significant time. What's the problem, though? What sorts of application logic would be broken by this? (Presumably this brokenness would also affect top-level await similarly.)

3. I'm not sure the performance impact would be as bad as you're implying here:

When wasm instantiation happens in the instantiation phase, implementations are free to do some instantiations in parallel, since this is not visible at runtime. Performance benefit.

Some speculation from someone who has never worked on a Wasm implementation:

Even if Wasm instantiation happens in the evaluation phase, implementations may still be able to do some parts of starting up the Wasm module, especially compilation, in parallel in practice. Some parallelism could be based on all dependencies being satisfied (in which case, everything can proceed completely correctly until the start function, which would have to be run at the appropriate linear time); other parallelism could be speculative and possibly invalidated by later modules (e.g., if memory is imported and the module specialization depends on the memory type; the speculative compilation could even be kicked off in the instantiation phase).

I'll discuss this with implementers next week at TPAC to see what they think.

2. What sorts of application logic would be broken by this?

I could imaging a module connecting to a (web)socket and emitting events for each message incoming. When used this way:

import socket from "./socket";
import method from "other-unrelated-module";

socket.on("message", msg => console.log(msg));

Code is working fine so far. Now when other-unrelated-module introduce some top-level-await or wasm which is async at evaluation time, you may loose some messages, because there is something happening in-between.

2. Presumably this brokenness would also affect top-level await similarly.

yes, I'm also against that.

Even if Wasm instantiation happens in the evaluation phase, implementations may still be able to do some parts of starting up the Wasm module, especially compilation, in parallel in practice. Some parallelism could be based on all dependencies being satisfied (in which case, everything can proceed completely correctly until the start function, which would have to be run at the appropriate linear time); other parallelism could be speculative and possibly invalidated by later modules (e.g., if memory is imported and the module specialization depends on the memory type; the speculative compilation could even be kicked off in the instantiation phase).

This is something browser might be able to do, but not something we (as webpack) can do with the current WASM JS API.

For what it's worth, I posted an early draft of a specification for the semantics we've been talking about at https://littledan.github.io/esm-integration/#esm-integration .

This is something browser might be able to do, but not something we (as webpack) can do with the current WASM JS API.

That's right, it's not possible directly, but I wonder if there might be some kind of transform strategy to achieve similar semantics. (Sorry, I don't have a particular suggestion.)

I'd like to revive this topic. I've been working on the documentation and specification text for WebAssembly/ESM integration, and I plan to propose it for Phase 2 at an upcoming WebAssembly CG meeting. I'd be interested in more feedback from webpack here. I've expanded on the motivation for these semantics, and added in the detail about the relationship with top-level await (which you can also see reflected in the proposed spec text). See https://github.com/WebAssembly/esm-integration/blob/master/proposals/esm-integration/README.md for more details.

This issue had no activity for at least half a year.

It's subject to automatic issue closing if there is no activity in the next 15 days.

bump

This issue had no activity for at least three months.

It's subject to automatic issue closing if there is no activity in the next 15 days.

I want to note: Unfortunately, I'm really sorry to report that it's not certain whether browsers will implement what webpack has done here. Specifically, browsers might require import sites to mark WebAssembly modules as such. See tc39/proposal-import-attributes#19 for further discussion--your thoughts and comments there may be welcome.

I propose that we close this issue until we have a concret plan.

Issue was closed because of inactivity.

If you think this is still a valid issue, please file a new issue with additional information.