Skip to content

Commit e3f177c

Browse files
feat(esm): widen analyzable output to HMR, hashed and function chunk filenames (#21653)
* feat: emit analyzable ESM imports under hot module replacement The gate refused the analyzable form for any compilation with HMR applied. The hot require special-cased `ensureChunk` (`.e`) to block an update on a chunk load in flight and copied everything else through, so `.ei` escaped that wrapper — and it defined `.e` even when the runtime had none. Wrap `.ei` the same way and define each wrapper only when it exists. Lazy compilation generates a proxy between builds rather than from changed source, so one already loaded outlives the build that required `ensureChunk`; keep it in the runtime there instead of refusing the analyzable form everywhere. * feat: bake a url public path into the analyzable wasm binary reference `supportsAnalyzableWasm` chained through `supportsAnalyzableEsmUrl`, so it inherited an `auto` public path requirement it does not have: a public path that is a complete URL bakes just as well, because `new URL(...)` ignores its base for an absolute specifier. A root- or directory-relative one still cannot — the runtime form resolves it against the document and a baked one against the chunk — and neither can a loader that hands the result to `readFile`, which takes a `file:` URL alone. Those keep addressing the binary relative to their chunk, as they already did. * perf: drop the runtime chunk loader from analyzable css and hinted chunks A chunk carrying css or resource-hint children asked for `ensureChunk` just to get the `.f` map its handlers attach to, and that pulled in the js chunk loader and the chunk id to filename table behind it — the runtime the analyzable form exists to replace. `.ei` dispatches every handler but `.f.j` itself, so ask for the map alone and let `EnsureChunkRuntimeModule` emit `.e` only where it is used. The hot runtime registers its own handler on the map and force-loads through the others by bare chunk id, so HMR keeps both the map and the loader — which an all-analyzable graph had silently been dropping since HMR started emitting the analyzable form. * feat: support a function output.chunkFilename in analyzable ESM output A function was refused on its type alone, without asking what it returns. Call it for its template instead, and apply the same hash test to the answer. No hash is knowable at this point — during code generation `chunk.contentHash` is still empty and `chunk.hash` unset — so a function reading one would name a file that is never emitted. A second call against a stand-in chunk carrying sentinel hashes says whether the answer depends on them; one that does keeps the runtime form, as does a template with a hash placeholder in it. * feat: bake a hashed chunk filename into the analyzable specifier A hashed name is settled long after the code referencing it is generated, so the specifier is emitted as a stand-in and filled in once the hash exists. That covers `output.chunkFilename` carrying any hash, and so `[fullhash]` in a WebAssembly filename too — the last names the analyzable form could not address. The stand-in carries the prefix and chunk id rather than an index into per-build state: a module restored from the persistent cache is never generated again, and one that outlived its registry would reach the bundle unreplaced. Substituting rewrites a chunk after its own content hash was taken, so the form is kept only where `optimization.realContentHash` brings the two back in line. * fix: resolve a deferred chunk specifier with the runtime as well The stand-in was filled in from a path missing `runtime`, which the call naming the asset passes. A `[runtime]` in the template then resolved to `_` rather than the runtime's name, so the specifier addressed a file that is never emitted. Cover every placeholder a chunk filename can carry, asserting the baked name is one on disk instead of a string fixed in the test. * feat: defer a hash-dependent function chunk filename instead of refusing it The probe telling whether a function's answer depends on a hash meant "fall back", from when nothing could resolve one later. The deferred pass asks the function again with the hashes settled, so it now means "defer" — `null` stays for a function that throws, which no later call rescues. Both hash-bearing shapes bake as a result: a template carrying a placeholder, and a name built from `pathData` directly. * refactor: drop the unreachable chunk id guard from the analyzable import `blockPromise` drops any chunk without an id before it calls, so the check never fired — measured across the config and stats suites, where the operand was evaluated 296 times and was true none of them. The one in `_getAnalyzableChunkSpecifier` stays: `WorkerDependency` reaches that without the same filter, and an id-less stand-in would reach the bundle verbatim instead of falling back. Says so now. * fix(esm): probe every content-hash type when a chunk filename is a function A function `output.chunkFilename` reading `contentHash.css` saw the same missing value in both calls, so the name looked constant and `undefined` was baked into the specifier. Probe with two distinct hashes exposed for every type instead. Also skip the analyzable HMR case where ESM output cannot build `require`, and reject a stand-in payload that is not a chunk reference. * fix(esm): resolve an analyzable chunk specifier with the runtime as well `[runtime]` and `pathData.runtime` were absent where the specifier is baked, so both read back empty and the name looked settled — `src_lazy_js.undefined.mjs` was emitted against a `src_lazy_js.main.mjs` on disk. Hand the runtime to every site that resolves a chunk filename, as the deferred pass already does. * chore: bring the changeset entry under the length limit * fix(esm): defer a chunk name built by enumerating the content hashes Which content hashes a chunk carries is settled no earlier than their values are, so a filename function reading `Object.keys(chunk.contentHash)` saw an empty set in both probes and baked `src_lazy_js.none.mjs` against a `src_lazy_js.javascript-css.mjs` on disk. The probes now disagree about which hashes exist, so asking defers just as reading one does. * chore: tighten two comments in the wasm analyzable url case * perf(esm): keep the runtime chunk loader out of HMR builds without async chunks The hot runtime force-loads through the handler map by bare chunk id, so the analyzable path keeps `.f.j` around — but only where there is an async chunk to force-load. Requiring it unconditionally pulled the handler map, the chunk-filename helper and the whole `import()` loader into every `output.module` HMR build: +2.80 KiB on configCases/module/import-meta-hot-priority.
1 parent 23642ae commit e3f177c

36 files changed

Lines changed: 930 additions & 65 deletions

‎.changeset/021-analyzable-import-coverage.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
"webpack": minor
33
---
44

5-
Widen analyzable ESM output and fix chunk and WebAssembly loading in ES module builds.
5+
Widen analyzable ESM output to HMR, hashed and function chunk names, and wasm.

‎lib/RuntimePlugin.js‎

Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ const lazyModule = require("./util/lazyModule");
1515

1616
/** @typedef {import("../declarations/WebpackOptions").LibraryOptions} LibraryOptions */
1717
/** @typedef {import("./Chunk")} Chunk */
18+
/** @typedef {import("./Module").RuntimeRequirements} RuntimeRequirements */
1819
/** @typedef {import("./Compiler")} Compiler */
1920
/** @typedef {import("./RuntimeModule")} RuntimeModule */
2021

@@ -573,20 +574,39 @@ class RuntimePlugin {
573574
);
574575
return true;
575576
});
577+
/** @type {WeakSet<Chunk>} */
578+
const ensureChunkModuleAdded = new WeakSet();
579+
/**
580+
* The module carries the handler map as well as `ensureChunk`, and an
581+
* analyzable `import()` needs the map without the function around it.
582+
* @param {Chunk} chunk the chunk
583+
* @param {RuntimeRequirements} set the tree's requirements, filled in place
584+
*/
585+
const addEnsureChunkRuntimeModule = (chunk, set) => {
586+
if (ensureChunkModuleAdded.has(chunk)) return;
587+
ensureChunkModuleAdded.add(chunk);
588+
compilation.addLazyRuntimeModule(
589+
chunk,
590+
getEnsureChunkRuntimeModule,
591+
(Ctor) => new Ctor(set)
592+
);
593+
};
576594
compilation.hooks.runtimeRequirementInTree
577595
.for(RuntimeGlobals.ensureChunk)
578596
.tap(PLUGIN_NAME, (chunk, set) => {
579597
const hasAsyncChunks = chunk.hasAsyncChunks();
580598
if (hasAsyncChunks) {
581599
set.add(RuntimeGlobals.ensureChunkHandlers);
582600
}
583-
compilation.addLazyRuntimeModule(
584-
chunk,
585-
getEnsureChunkRuntimeModule,
586-
(Ctor) => new Ctor(set)
587-
);
601+
addEnsureChunkRuntimeModule(chunk, set);
588602
return true;
589603
});
604+
compilation.hooks.runtimeRequirementInTree
605+
.for(RuntimeGlobals.ensureChunkHandlers)
606+
.tap(PLUGIN_NAME, (chunk, set) => {
607+
// No bail: every other plugin attaching a handler taps this same key.
608+
addEnsureChunkRuntimeModule(chunk, set);
609+
});
590610
compilation.hooks.runtimeRequirementInTree
591611
.for(RuntimeGlobals.ensureChunkIncludeEntries)
592612
.tap(PLUGIN_NAME, (chunk, set) => {

‎lib/RuntimeTemplate.js‎

Lines changed: 181 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -35,16 +35,17 @@ const {
3535
getOptimizedDeferredModule
3636
} = require("./runtime/MakeDeferredNamespaceObjectRuntime");
3737
const { equals } = require("./util/ArrayHelpers");
38+
const { getScheme } = require("./util/URLAbsoluteSpecifier");
3839
const compileBooleanMatcher = require("./util/compileBooleanMatcher");
3940
const { getUndoPath, toJsStringLiteral } = require("./util/identifier");
4041
const memoize = require("./util/memoize");
4142
const { propertyAccess, propertyName } = require("./util/property");
4243
const { forEachRuntime, subtractRuntime } = require("./util/runtime");
4344

44-
const getModuleHotAcceptDependency = memoize(() =>
45-
require("./dependencies/ModuleHotAcceptDependency")
46-
);
4745
const getTemplatedPathPlugin = memoize(() => require("./TemplatedPathPlugin"));
46+
const getAnalyzableChunkHashPlugin = memoize(() =>
47+
require("./esm/AnalyzableChunkHashPlugin")
48+
);
4849

4950
// Module-federation module types whose chunks load through the federation runtime.
5051
/** @type {Set<string>} */
@@ -59,11 +60,18 @@ const FEDERATION_MODULE_TYPES = new Set([
5960
// or `:<digest>[:<length>]` suffix (e.g. `[contenthash:base64:8]`). Its value is only
6061
// resolved after code generation, so a filename using one can't be an inline literal.
6162
const HASH_IN_FILENAME = /\[(?:full|chunk|content)?hash(?::[^\]]+)?\]/;
63+
// Two stand-ins for every hash a filename function might read: a name built from one
64+
// is hash-dependent exactly when the two calls disagree.
65+
const HASH_PROBE = "0123456789abcdef0123";
66+
// Reversed, so the two differ at every position and no slice of one equals the other.
67+
const HASH_PROBE_ALTERNATE = "3210fedcba9876543210";
6268
const HASH_IN_FILENAME_GLOBAL = /\[(?:full|chunk|content)?hash(?::[^\]]+)?\]/g;
6369

6470
/** @typedef {import("./config/defaults").OutputNormalizedWithDefaults} OutputOptions */
6571
/** @typedef {import("./AsyncDependenciesBlock")} AsyncDependenciesBlock */
6672
/** @typedef {import("./Chunk")} Chunk */
73+
/** @typedef {import("./Chunk").ChunkFilenameTemplate} ChunkFilenameTemplate */
74+
/** @typedef {import("./Chunk").ChunkId} ChunkId */
6775
/** @typedef {import("./ChunkGraph")} ChunkGraph */
6876
/** @typedef {import("./Compilation")} Compilation */
6977
/** @typedef {import("./Dependency")} Dependency */
@@ -76,6 +84,41 @@ const HASH_IN_FILENAME_GLOBAL = /\[(?:full|chunk|content)?hash(?::[^\]]+)?\]/g;
7684
/** @typedef {import("./dependencies/ImportPhase").ImportPhaseType} ImportPhaseType */
7785
/** @typedef {import("./NormalModuleFactory").ModuleDependency} ModuleDependency */
7886

87+
/**
88+
* A stand-in whose every hash reads back as `hash`. Which content hashes a chunk
89+
* carries is settled after code generation, so `present` makes the two probes disagree
90+
* about that too — a name built by enumerating them then reads as hash-dependent.
91+
* Shadows the chunk rather than mutating it, so a filename function still reads every
92+
* other field and method straight off it.
93+
* @param {Chunk} chunk the chunk being referenced
94+
* @param {string} hash the stand-in hash
95+
* @param {boolean} present whether the chunk is claimed to carry content hashes
96+
* @returns {Chunk} the stand-in chunk
97+
*/
98+
const createHashProbeChunk = (chunk, hash, present) => {
99+
const probeChunk = Object.create(chunk);
100+
probeChunk.hash = hash;
101+
probeChunk.renderedHash = hash;
102+
probeChunk.contentHash = new Proxy(
103+
present ? { [JAVASCRIPT_TYPE]: hash } : {},
104+
{
105+
// Any content-hash type, not only the ones this chunk happens to carry.
106+
get: (target, key) => (typeof key === "string" ? hash : undefined),
107+
has: () => present,
108+
getOwnPropertyDescriptor: () =>
109+
present
110+
? {
111+
value: hash,
112+
writable: true,
113+
enumerable: true,
114+
configurable: true
115+
}
116+
: undefined
117+
}
118+
);
119+
return probeChunk;
120+
};
121+
79122
/**
80123
* No module id error message.
81124
* @param {Module} module the module
@@ -322,7 +365,8 @@ class RuntimeTemplate {
322365
*/
323366
supportsAnalyzableWasm() {
324367
return (
325-
this.supportsAnalyzableEsmUrl() &&
368+
this.supportsAnalyzableEsm() &&
369+
(this.outputOptions.publicPath === "auto" || this._hasUrlPublicPath()) &&
326370
// `[fullhash]` is only known after code generation.
327371
!getTemplatedPathPlugin()
328372
.getPresentKinds(
@@ -332,6 +376,23 @@ class RuntimeTemplate {
332376
);
333377
}
334378

379+
/**
380+
* Whether `output.publicPath` is a complete URL of its own. Baking one keeps the
381+
* same target, because `new URL(...)` ignores its base for an absolute specifier.
382+
* A root- or directory-relative public path does not: the runtime form resolves it
383+
* against the document and a baked one against the chunk, which differ exactly
384+
* where a public path is worth setting.
385+
* @returns {boolean} true for a static, absolute-URL public path
386+
*/
387+
_hasUrlPublicPath() {
388+
const { publicPath } = this.outputOptions;
389+
return (
390+
typeof publicPath === "string" &&
391+
!publicPath.includes("[") &&
392+
(publicPath.startsWith("//") || getScheme(publicPath) !== undefined)
393+
);
394+
}
395+
335396
supportTemplateLiteral() {
336397
return this.outputOptions.environment.templateLiteral;
337398
}
@@ -1669,26 +1730,18 @@ class RuntimeTemplate {
16691730
chunkGraph
16701731
) {
16711732
const { outputOptions } = this;
1733+
// `blockPromise` has already dropped any chunk without an id.
16721734
if (
16731735
!outputOptions.module ||
16741736
outputOptions.chunkFormat !== "module" ||
16751737
outputOptions.importFunctionName !== "import" ||
1676-
chunk.id === null ||
16771738
!originModule ||
16781739
!chunkGraph
16791740
) {
16801741
return null;
16811742
}
1682-
// HMR wraps `ensureChunk` (`.e`) to track in-flight loads and relies on it being
1683-
// present in the runtime; the analyzable `import()` (`.ei`) replaces `.e` and would
1684-
// leave a lazy-compilation proxy's own `.e` call undefined, so keep the runtime form.
1685-
// A runtime `__webpack_public_path__` override likewise can't reach a baked literal.
1686-
if (
1687-
this.compilation.dependencyFactories.has(
1688-
getModuleHotAcceptDependency()
1689-
) ||
1690-
APIPlugin.usesRuntimePublicPathOverride(this.compilation)
1691-
) {
1743+
// A runtime `__webpack_public_path__` override can't reach a baked literal.
1744+
if (APIPlugin.usesRuntimePublicPathOverride(this.compilation)) {
16921745
return null;
16931746
}
16941747
// A worker loading its own chunks some other way keeps that runtime; one on
@@ -1743,8 +1796,9 @@ class RuntimeTemplate {
17431796
: `"./${specifier.slice(1)}`;
17441797
runtimeRequirements.add(RuntimeGlobals.analyzableChunkImport);
17451798
if (needsHintHandlers || hasNonJsTypes) {
1746-
// `ensureChunk` is what creates the `.f` map the handlers attach to.
1747-
runtimeRequirements.add(RuntimeGlobals.ensureChunk);
1799+
// Those handlers attach to the `.f` map, which has to exist — but the runtime
1800+
// `ensureChunk` around it does not, since `.ei` dispatches them itself.
1801+
runtimeRequirements.add(RuntimeGlobals.ensureChunkHandlers);
17481802
}
17491803
// Drop-in for `ensureChunk(id)`: the literal `import()` can be statically
17501804
// followed, the helper keeps webpack's install timing and deduplication.
@@ -1779,7 +1833,7 @@ class RuntimeTemplate {
17791833
typeof template === "string"
17801834
? template.replace(HASH_IN_FILENAME_GLOBAL, "x")
17811835
: template,
1782-
{ chunk, contentHashType: JAVASCRIPT_TYPE }
1836+
{ chunk, runtime: chunk.runtime, contentHashType: JAVASCRIPT_TYPE }
17831837
);
17841838
const undo = getUndoPath(chunkName, outputPath, true);
17851839
if (!found) {
@@ -1817,34 +1871,63 @@ class RuntimeTemplate {
18171871
chunk,
18181872
outputOptions
18191873
);
1874+
const template = this._resolveChunkFilenameTemplate(
1875+
filenameTemplate,
1876+
chunk
1877+
);
1878+
if (template === null) return null;
1879+
// A hashed name is settled long after this code is generated, so a stand-in is
1880+
// emitted and filled in once the hash exists.
1881+
const deferred = template === undefined || HASH_IN_FILENAME.test(template);
18201882
if (
1821-
typeof filenameTemplate !== "string" ||
1822-
HASH_IN_FILENAME.test(filenameTemplate)
1883+
deferred &&
1884+
// An id names the chunk in the stand-in, and one nothing could resolve would
1885+
// reach the bundle verbatim.
1886+
(chunk.id === null ||
1887+
!getAnalyzableChunkHashPlugin().canDeferSpecifier(compilation))
18231888
) {
18241889
return null;
18251890
}
1826-
const filename = compilation.getPath(filenameTemplate, {
1827-
chunk,
1828-
contentHashType: JAVASCRIPT_TYPE
1829-
});
1891+
const filename = deferred
1892+
? ""
1893+
: compilation.getPath(/** @type {string} */ (template), {
1894+
chunk,
1895+
// Matches what names the asset, or a placeholder resolved here would
1896+
// not be the one on disk.
1897+
runtime: chunk.runtime,
1898+
contentHashType: JAVASCRIPT_TYPE
1899+
});
1900+
/**
1901+
* @param {string} prefix what goes in front of the chunk's filename
1902+
* @returns {string} the specifier, already quoted
1903+
*/
1904+
const specifier = (prefix) =>
1905+
toJsStringLiteral(
1906+
deferred
1907+
? getAnalyzableChunkHashPlugin().reserveSpecifier(
1908+
prefix,
1909+
/** @type {ChunkId} */ (chunk.id)
1910+
)
1911+
: prefix + filename
1912+
);
18301913
if (overridePublicPath) {
18311914
return overridePublicPath.includes("[")
18321915
? null
1833-
: toJsStringLiteral(overridePublicPath + filename);
1916+
: specifier(overridePublicPath);
18341917
}
18351918
const { publicPath } = outputOptions;
18361919
if (publicPath === "auto") {
18371920
const undo = this._getModuleUndoPath(consumingModule, chunkGraph);
1838-
if (undo !== null) return toJsStringLiteral(undo + filename);
1921+
if (undo !== null) return specifier(undo);
18391922
// Different depths — no single relative literal. No bundler follows a
18401923
// concatenation, but a `new URL(...)` caller still sheds the `.u(id)` lookup
18411924
// this way; `import()` needs a static specifier, so it gets nothing.
1842-
if (!runtimeRequirements) return null;
1925+
if (deferred || !runtimeRequirements) return null;
18431926
runtimeRequirements.add(RuntimeGlobals.publicPath);
18441927
return `${RuntimeGlobals.publicPath} + ${toJsStringLiteral(filename)}`;
18451928
}
18461929
if (typeof publicPath === "string" && !publicPath.includes("[")) {
1847-
return toJsStringLiteral(publicPath + filename);
1930+
return specifier(publicPath);
18481931
}
18491932
return null;
18501933
}
@@ -1864,6 +1947,19 @@ class RuntimeTemplate {
18641947
/** @type {string} */ (this.outputOptions.webassemblyModuleFilename),
18651948
{ module, runtime, chunkGraph }
18661949
);
1950+
// Only a chunk that fetches may carry the public path into the literal: `readFile`
1951+
// takes a `file:` URL alone, and those loaders address the binary relative to the
1952+
// chunk anyway, ignoring the public path exactly as the runtime form does.
1953+
if (
1954+
this.outputOptions.publicPath !== "auto" &&
1955+
this._wasmChunksFetch(module, chunkGraph)
1956+
) {
1957+
return this.importMetaUrl(
1958+
toJsStringLiteral(
1959+
/** @type {string} */ (this.outputOptions.publicPath) + filename
1960+
)
1961+
);
1962+
}
18671963
const undo = this._getModuleUndoPath(module, chunkGraph);
18681964
/** @type {string} */
18691965
let specifier;
@@ -1878,6 +1974,63 @@ class RuntimeTemplate {
18781974
return this.importMetaUrl(specifier);
18791975
}
18801976

1977+
/**
1978+
* The chunk filename template as a plain string. A function is called twice, with
1979+
* a different stand-in hash each time; disagreeing answers mean the name depends on
1980+
* a hash, which is not knowable during code generation, so it is left to the
1981+
* deferred pass to ask again once the hashes are settled.
1982+
* @param {ChunkFilenameTemplate} filenameTemplate the configured template
1983+
* @param {Chunk} chunk the chunk being referenced
1984+
* @returns {string | undefined | null} the template, `undefined` to defer, or
1985+
* `null` to fall back
1986+
*/
1987+
_resolveChunkFilenameTemplate(filenameTemplate, chunk) {
1988+
if (typeof filenameTemplate === "string") return filenameTemplate;
1989+
try {
1990+
// Everything the naming call is given except the hashes, so only a hash can
1991+
// make the two answers differ.
1992+
const template = filenameTemplate({
1993+
chunk: createHashProbeChunk(chunk, HASH_PROBE, true),
1994+
runtime: chunk.runtime,
1995+
contentHashType: JAVASCRIPT_TYPE,
1996+
hash: HASH_PROBE,
1997+
contentHash: HASH_PROBE
1998+
});
1999+
const probe = filenameTemplate({
2000+
chunk: createHashProbeChunk(chunk, HASH_PROBE_ALTERNATE, false),
2001+
runtime: chunk.runtime,
2002+
contentHashType: JAVASCRIPT_TYPE,
2003+
hash: HASH_PROBE_ALTERNATE,
2004+
contentHash: HASH_PROBE_ALTERNATE
2005+
});
2006+
return template === probe ? template : undefined;
2007+
} catch (_error) {
2008+
// A template that needs more than this chunk can't be resolved here.
2009+
return null;
2010+
}
2011+
}
2012+
2013+
/**
2014+
* Whether every chunk holding `module` loads WebAssembly through `fetch`, which is
2015+
* what makes an absolute URL usable — the `async-node` and `universal` loaders hand
2016+
* the result to `readFile`, and that accepts a `file:` URL alone.
2017+
* @param {Module} module the async wasm module
2018+
* @param {ChunkGraph} chunkGraph the chunk graph
2019+
* @returns {boolean} true when every holding chunk fetches
2020+
*/
2021+
_wasmChunksFetch(module, chunkGraph) {
2022+
const { outputOptions } = this;
2023+
for (const chunk of chunkGraph.getModuleChunksIterable(module)) {
2024+
const entryOptions = chunk.getEntryOptions();
2025+
const wasmLoading =
2026+
entryOptions && entryOptions.wasmLoading !== undefined
2027+
? entryOptions.wasmLoading
2028+
: outputOptions.wasmLoading;
2029+
if (wasmLoading !== "fetch") return false;
2030+
}
2031+
return true;
2032+
}
2033+
18812034
/**
18822035
* Async module factory.
18832036
* @param {object} options options

0 commit comments

Comments
 (0)