Skip to content

Commit f4c2fda

Browse files
alexander-akaitalexander-akait
committed
feat: add a processResultAsync hook, and rename the asset it rewrote
Rewriting an asset's bytes and emitting it under a new name was only possible from a loader. A plugin could not do it: minification is async and processResult is synchronous, and by the time an asset reaches processAssets its URL is already written into the bundle, so renaming there moves the file and leaves every reference pointing at the old one — with no error. processResultAsync runs after processResult, over the same result, and can await. It is skipped entirely when nothing taps it, so a build that does not use it keeps both the synchronous path and the exact stack depth anything thrown below it is reported at — converting processResult itself would have put tapable's own frames into every parse error, since CssModulesPlugin taps it and experiments.css is on by default. Renaming needs a channel that outlives the build. matchResource says the right thing but belongs to resolution and is not serialized, so a rename set while building is lost on a cache hit and the restored module's sourceFilename then conflicts with the cached codegen. buildInfo is written with the module, so the new assetResource lives there. Both the emitted name and the media type an inlined asset is written as now read the same resource, so an image re-encoded as another format cannot be emitted as pic.png or inlined as image/png while holding webp bytes.
1 parent 1f3264e commit f4c2fda

9 files changed

Lines changed: 237 additions & 8 deletions

File tree

‎.changeset/010-async-process-result-and-asset-rename.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": minor
3+
---
4+
5+
Add `NormalModule`'s `processResultAsync` hook, and let a tap rename the asset it transforms through `buildInfo.assetResource`, so re-encoding an image no longer needs a loader.

‎lib/NormalModule.js‎

Lines changed: 43 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
const querystring = require("querystring");
99
const {
1010
AsyncSeriesBailHook,
11+
AsyncSeriesWaterfallHook,
1112
HookMap,
1213
SyncHook,
1314
SyncWaterfallHook
@@ -184,7 +185,9 @@ const recordSideEffectsBailout = (mod, moduleGraph, dep) => {
184185
.getOptimizationBailout(mod)
185186
.push(
186187
() =>
187-
`Dependency (${dep.type}) with side effects at ${formatLocation(dep.loc)}`
188+
`Dependency (${dep.type}) with side effects at ${formatLocation(
189+
dep.loc
190+
)}`
188191
);
189192
};
190193

@@ -546,6 +549,16 @@ const createCompilationHooks = () => {
546549
* @since 5.99.0
547550
*/
548551
processResult: new SyncWaterfallHook(["result", "module"]),
552+
/**
553+
* The same, for a tap that has to await work of its own — minifying an
554+
* asset, say, which before only a loader could do. Runs after
555+
* `processResult` and is skipped entirely when nothing taps it, so a
556+
* build that does not use it keeps the synchronous path, and the stacks
557+
* of errors thrown below it stay free of the hook's own frames.
558+
* @type {AsyncSeriesWaterfallHook<[Result, NormalModule]>}
559+
* @since 5.110.0
560+
*/
561+
processResultAsync: new AsyncSeriesWaterfallHook(["result", "module"]),
549562
/**
550563
* @type {AsyncSeriesBailHook<[NormalModule, NeedBuildContext], boolean>}
551564
* @since 5.49.0
@@ -1052,7 +1065,9 @@ class NormalModule extends Module {
10521065
(parseJson(options));
10531066
} catch (err) {
10541067
throw new Error(
1055-
`Cannot parse string options: ${/** @type {Error} */ (err).message}`,
1068+
`Cannot parse string options: ${
1069+
/** @type {Error} */ (err).message
1070+
}`,
10561071
{ cause: err }
10571072
);
10581073
}
@@ -1441,7 +1456,9 @@ class NormalModule extends Module {
14411456
})
14421457
).catch((err) => {
14431458
throw new Error(
1444-
`Failed to parse source map. ${/** @type {Error} */ (err).message}`
1459+
`Failed to parse source map. ${
1460+
/** @type {Error} */ (err).message
1461+
}`
14451462
);
14461463
})
14471464
);
@@ -1503,7 +1520,29 @@ class NormalModule extends Module {
15031520
if (result.notCacheableReasons.length > 0) {
15041521
buildInfo.notCacheableReasons = result.notCacheableReasons;
15051522
}
1506-
processResult(err, result.result);
1523+
// Untapped, nothing can change: call straight through, so a build
1524+
// that does not use it keeps both the synchronous path and the
1525+
// exact stack depth anything thrown below it is reported at.
1526+
if (hooks.processResultAsync.taps.length === 0) {
1527+
return processResult(err, result.result);
1528+
}
1529+
1530+
hooks.processResultAsync.callAsync(
1531+
/** @type {Result} */
1532+
(result.result),
1533+
this,
1534+
(hookErr, asyncResult) => {
1535+
if (hookErr) {
1536+
return processResult(
1537+
hookErr instanceof Error
1538+
? hookErr
1539+
: new (getNonErrorEmittedError())(hookErr)
1540+
);
1541+
}
1542+
1543+
processResult(null, /** @type {Result} */ (asyncResult));
1544+
}
1545+
);
15071546
}
15081547
);
15091548
}

‎lib/asset/AssetGenerator.js‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -250,12 +250,29 @@ class AssetGenerator extends Generator {
250250
static getSourceFileName(module, runtimeTemplate) {
251251
return makePathsRelative(
252252
runtimeTemplate.compilation.compiler.context,
253-
/** @type {string} */
254-
(module.getResource()),
253+
AssetGenerator.getAssetResource(module),
255254
runtimeTemplate.compilation.compiler.root
256255
).replace(/^\.\//, "");
257256
}
258257

258+
/**
259+
* The resource this asset is named after: the one it was read from, unless
260+
* something rewrote its content into another format and said so. Both the
261+
* name it is emitted under and the media type it is inlined as follow from
262+
* this, so they cannot disagree about what the bytes are.
263+
* @param {NormalModule} module module
264+
* @returns {string} the resource
265+
*/
266+
static getAssetResource(module) {
267+
const buildInfo = /** @type {AssetModuleBuildInfo | undefined} */ (
268+
module.buildInfo
269+
);
270+
271+
return /** @type {string} */ (
272+
(buildInfo && buildInfo.assetResource) || module.getResource()
273+
);
274+
}
275+
259276
/**
260277
* Gets the source file name as seen by a filename template. With
261278
* `experiments.futureDefaults` (webpack 6 default) `[path]` and `[file]` of a
@@ -504,9 +521,13 @@ class AssetGenerator extends Generator {
504521
/** @type {AssetGeneratorDataUrlOptions} */
505522
(this.dataUrlOptions).mimetype;
506523
if (mimeType === undefined) {
524+
// The renamed resource, minus a query, the way `nameForCondition` reads
525+
// the original: an asset re-encoded as another format is inlined as
526+
// what it now is, not as what it was read from.
527+
const resource = AssetGenerator.getAssetResource(module);
528+
const queryIndex = resource.indexOf("?");
507529
const ext = path.extname(
508-
/** @type {NameForCondition} */
509-
(module.nameForCondition())
530+
queryIndex >= 0 ? resource.slice(0, queryIndex) : resource
510531
);
511532
if (
512533
module.resourceResolveData &&

‎lib/asset/AssetModule.js‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,12 @@ const makeSerializable = require("../util/makeSerializable");
2222
* @property {string=} filename
2323
* @property {AssetInfo=} assetInfo
2424
* @property {string=} fullContentHash
25+
* @property {string=} assetResource the resource this asset is named after, when
26+
* something changed it — a `processResultAsync` tap that re-encoded the image
27+
* as another format, say. `matchResource` says the same thing for a request,
28+
* but belongs to resolution and is not serialized, so a rename set while
29+
* building would be lost to the persistent cache; this is written with the
30+
* module.
2531
*/
2632

2733
/** @typedef {NormalModuleBuildInfo & KnownAssetModuleBuildInfo} AssetModuleBuildInfo */

‎test/configCases/asset-modules/process-result-rename-inline/index.js‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
import url from "../_images/file.png";
2+
3+
it("should inline the asset as the format the hook made it", () => {
4+
expect(url).toMatch(/^data:image\/webp;base64,/);
5+
});
6+
7+
it("should inline what the hook returned", () => {
8+
expect(Buffer.from(url.split(",")[1], "base64").toString()).toBe(
9+
"webp-bytes"
10+
);
11+
});

‎test/configCases/asset-modules/process-result-rename-inline/webpack.config.js‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
"use strict";
2+
3+
const NormalModule = require("../../../../lib/NormalModule");
4+
5+
// The inline half of the same question: an asset re-encoded as another format
6+
// has no file name to change, but the media type it is inlined as is the same
7+
// claim about what the bytes are, so it has to follow too.
8+
class RenameOnProcessResult {
9+
/**
10+
* @param {import("../../../../lib/Compiler")} compiler compiler
11+
*/
12+
apply(compiler) {
13+
compiler.hooks.compilation.tap("RenameOnProcessResult", (compilation) => {
14+
NormalModule.getCompilationHooks(
15+
compilation
16+
).processResultAsync.tapPromise(
17+
"RenameOnProcessResult",
18+
async (result, module) => {
19+
if (!/\.png$/.test(module.resource)) return result;
20+
await Promise.resolve();
21+
/** @type {import("../../../../lib/asset/AssetModule").AssetModuleBuildInfo} */
22+
(module.buildInfo).assetResource = module.resource.replace(
23+
/\.png$/,
24+
".webp"
25+
);
26+
return [Buffer.from("webp-bytes"), result[1], result[2]];
27+
}
28+
);
29+
});
30+
}
31+
}
32+
33+
/** @type {import("../../../../").Configuration} */
34+
module.exports = {
35+
mode: "development",
36+
module: {
37+
rules: [
38+
{
39+
test: /\.(png|webp)$/,
40+
type: "asset/inline"
41+
}
42+
]
43+
},
44+
plugins: [new RenameOnProcessResult()]
45+
};

‎test/configCases/asset-modules/process-result-rename/index.js‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
import fs from "fs";
2+
import path from "path";
3+
import url from "../_images/file.png";
4+
5+
it("should emit the asset under the name the hook asked for", () => {
6+
expect(url).toBe("file.webp");
7+
expect(fs.existsSync(path.join(__STATS__.outputPath, "file.webp"))).toBe(
8+
true
9+
);
10+
expect(fs.existsSync(path.join(__STATS__.outputPath, "file.png"))).toBe(
11+
false
12+
);
13+
});
14+
15+
it("should emit what the hook returned", () => {
16+
expect(
17+
fs.readFileSync(path.join(__STATS__.outputPath, url), "utf8")
18+
).toBe("webp-bytes");
19+
});

‎test/configCases/asset-modules/process-result-rename/webpack.config.js‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
"use strict";
2+
3+
const NormalModule = require("../../../../lib/NormalModule");
4+
5+
// A `processResult` tap standing in for an image minimizer: it awaits work of
6+
// its own and re-encodes the asset as another format, so the name it is
7+
// emitted under has to change with the bytes.
8+
class RenameOnProcessResult {
9+
/**
10+
* @param {import("../../../../lib/Compiler")} compiler compiler
11+
*/
12+
apply(compiler) {
13+
compiler.hooks.compilation.tap("RenameOnProcessResult", (compilation) => {
14+
NormalModule.getCompilationHooks(
15+
compilation
16+
).processResultAsync.tapPromise(
17+
"RenameOnProcessResult",
18+
async (result, module) => {
19+
if (!/\.png$/.test(module.resource)) return result;
20+
// Nothing here needs to wait; awaiting is what the sync hook
21+
// could not do, so the case is only meaningful if it does.
22+
await Promise.resolve();
23+
/** @type {import("../../../../lib/asset/AssetModule").AssetModuleBuildInfo} */
24+
(module.buildInfo).assetResource = module.resource.replace(
25+
/\.png$/,
26+
".webp"
27+
);
28+
return [Buffer.from("webp-bytes"), result[1], result[2]];
29+
}
30+
);
31+
});
32+
}
33+
}
34+
35+
/** @type {import("../../../../").Configuration} */
36+
module.exports = {
37+
mode: "development",
38+
output: {
39+
assetModuleFilename: "[name][ext]"
40+
},
41+
module: {
42+
rules: [
43+
{
44+
test: /\.(png|webp)$/,
45+
type: "asset/resource"
46+
}
47+
]
48+
},
49+
plugins: [new RenameOnProcessResult()]
50+
};

‎types.d.ts‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14946,6 +14946,16 @@ declare interface KnownAssetModuleBuildInfo {
1494614946
filename?: string;
1494714947
assetInfo?: AssetInfo;
1494814948
fullContentHash?: string;
14949+
14950+
/**
14951+
* the resource this asset is named after, when
14952+
* something changed it — a `processResultAsync` tap that re-encoded the image
14953+
* as another format, say. `matchResource` says the same thing for a request,
14954+
* but belongs to resolution and is not serialized, so a rename set while
14955+
* building would be lost to the persistent cache; this is written with the
14956+
* module.
14957+
*/
14958+
assetResource?: string;
1494914959
}
1495014960
declare interface KnownBuildInfo {
1495114961
cacheable?: boolean;
@@ -18970,6 +18980,29 @@ declare class NormalModule extends Module {
1897018980
undefined | PreparsedAst
1897118981
]
1897218982
>;
18983+
/**
18984+
* The same, for a tap that has to await work of its own — minifying an
18985+
* asset, say, which before only a loader could do. Runs after
18986+
* `processResult` and is skipped entirely when nothing taps it, so a
18987+
* build that does not use it keeps the synchronous path, and the stacks
18988+
* of errors thrown below it stay free of the hook's own frames.
18989+
* @since 5.110.0
18990+
*/
18991+
processResultAsync: AsyncSeriesWaterfallHook<
18992+
[
18993+
[
18994+
string | Buffer,
18995+
undefined | string | RawSourceMap,
18996+
undefined | PreparsedAst
18997+
],
18998+
NormalModule
18999+
],
19000+
[
19001+
string | Buffer,
19002+
undefined | string | RawSourceMap,
19003+
undefined | PreparsedAst
19004+
]
19005+
>;
1897319006
/**
1897419007
* @since 5.49.0
1897519008
*/

0 commit comments

Comments
 (0)