Skip to content

Commit 3947b8c

Browse files
feat: let a processResult tap await, and rename the asset it rewrote
Rewriting an asset's bytes and emitting it under a new name was only possible from a loader. A plugin could not do it: minification is async and processResult was synchronous, and by the time an asset reaches processAssets its URL is already written into the bundle, so renaming there moves the file and leaves every reference pointing at the old one — with no error. processResult becomes an AsyncSeriesWaterfallHook. A tap written for the synchronous hook keeps working, and costs nothing: tapable runs sync taps inline and calls back without a tick, and an untapped hook is skipped entirely rather than paying one per module. Renaming needs a channel that outlives the build. matchResource says the right thing but belongs to resolution and is not serialized, so a rename set while building is lost on a cache hit and the restored module's sourceFilename then conflicts with the cached codegen. buildInfo is written with the module, so the new assetResource lives there. Both the emitted name and the media type an inlined asset is written as now read the same resource, so an image re-encoded as another format cannot be emitted as pic.png or inlined as image/png while holding webp bytes.
1 parent 1f3264e commit 3947b8c

9 files changed

Lines changed: 215 additions & 14 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": minor
3+
---
4+
5+
Make `NormalModule`'s `processResult` hook async, and let a tap rename the asset it transforms through `buildInfo.assetResource`, so re-encoding an image no longer needs a loader.

‎lib/NormalModule.js‎

Lines changed: 45 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@
88
const querystring = require("querystring");
99
const {
1010
AsyncSeriesBailHook,
11+
AsyncSeriesWaterfallHook,
1112
HookMap,
12-
SyncHook,
13-
SyncWaterfallHook
13+
SyncHook
1414
} = require("tapable");
1515
const {
1616
CachedSource,
@@ -184,7 +184,9 @@ const recordSideEffectsBailout = (mod, moduleGraph, dep) => {
184184
.getOptimizationBailout(mod)
185185
.push(
186186
() =>
187-
`Dependency (${dep.type}) with side effects at ${formatLocation(dep.loc)}`
187+
`Dependency (${dep.type}) with side effects at ${formatLocation(
188+
dep.loc
189+
)}`
188190
);
189191
};
190192

@@ -542,10 +544,14 @@ const createCompilationHooks = () => {
542544
*/
543545
beforeSnapshot: new SyncHook(["module"]),
544546
/**
545-
* @type {SyncWaterfallHook<[Result, NormalModule]>}
547+
* Called with what the loaders produced, before it is parsed. Return the
548+
* (possibly transformed) result. Async since 5.110.0, so a tap can await
549+
* work of its own — minifying an asset, say — where before only a loader
550+
* could; a `tap` written for the synchronous hook keeps working.
551+
* @type {AsyncSeriesWaterfallHook<[Result, NormalModule]>}
546552
* @since 5.99.0
547553
*/
548-
processResult: new SyncWaterfallHook(["result", "module"]),
554+
processResult: new AsyncSeriesWaterfallHook(["result", "module"]),
549555
/**
550556
* @type {AsyncSeriesBailHook<[NormalModule, NeedBuildContext], boolean>}
551557
* @since 5.49.0
@@ -1052,7 +1058,9 @@ class NormalModule extends Module {
10521058
(parseJson(options));
10531059
} catch (err) {
10541060
throw new Error(
1055-
`Cannot parse string options: ${/** @type {Error} */ (err).message}`,
1061+
`Cannot parse string options: ${
1062+
/** @type {Error} */ (err).message
1063+
}`,
10561064
{ cause: err }
10571065
);
10581066
}
@@ -1305,11 +1313,37 @@ class NormalModule extends Module {
13051313
});
13061314
return callback(error);
13071315
}
1308-
const result = hooks.processResult.call(
1316+
// Untapped, nothing can change: stay on the synchronous path rather
1317+
// than paying a tick for every module a build has.
1318+
if (hooks.processResult.taps.length === 0) {
1319+
return processedResult(/** @type {Result} */ (result_));
1320+
}
1321+
1322+
hooks.processResult.callAsync(
13091323
/** @type {Result} */
13101324
(result_),
1311-
this
1325+
this,
1326+
(hookErr, result) => {
1327+
if (hookErr) {
1328+
return callback(
1329+
new (getModuleBuildError())(
1330+
hookErr instanceof Error
1331+
? hookErr
1332+
: new (getNonErrorEmittedError())(hookErr)
1333+
)
1334+
);
1335+
}
1336+
1337+
processedResult(/** @type {Result} */ (result));
1338+
}
13121339
);
1340+
};
1341+
1342+
/**
1343+
* @param {Result} result what the loaders produced, past the hook
1344+
* @returns {void}
1345+
*/
1346+
const processedResult = (result) => {
13131347
const source = result[0];
13141348
const sourceMap = result.length >= 1 ? result[1] : null;
13151349
const extraInfo = result.length >= 2 ? result[2] : null;
@@ -1441,7 +1475,9 @@ class NormalModule extends Module {
14411475
})
14421476
).catch((err) => {
14431477
throw new Error(
1444-
`Failed to parse source map. ${/** @type {Error} */ (err).message}`
1478+
`Failed to parse source map. ${
1479+
/** @type {Error} */ (err).message
1480+
}`
14451481
);
14461482
})
14471483
);

‎lib/asset/AssetGenerator.js‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -250,12 +250,29 @@ class AssetGenerator extends Generator {
250250
static getSourceFileName(module, runtimeTemplate) {
251251
return makePathsRelative(
252252
runtimeTemplate.compilation.compiler.context,
253-
/** @type {string} */
254-
(module.getResource()),
253+
AssetGenerator.getAssetResource(module),
255254
runtimeTemplate.compilation.compiler.root
256255
).replace(/^\.\//, "");
257256
}
258257

258+
/**
259+
* The resource this asset is named after: the one it was read from, unless
260+
* something rewrote its content into another format and said so. Both the
261+
* name it is emitted under and the media type it is inlined as follow from
262+
* this, so they cannot disagree about what the bytes are.
263+
* @param {NormalModule} module module
264+
* @returns {string} the resource
265+
*/
266+
static getAssetResource(module) {
267+
const buildInfo = /** @type {AssetModuleBuildInfo | undefined} */ (
268+
module.buildInfo
269+
);
270+
271+
return /** @type {string} */ (
272+
(buildInfo && buildInfo.assetResource) || module.getResource()
273+
);
274+
}
275+
259276
/**
260277
* Gets the source file name as seen by a filename template. With
261278
* `experiments.futureDefaults` (webpack 6 default) `[path]` and `[file]` of a
@@ -504,9 +521,13 @@ class AssetGenerator extends Generator {
504521
/** @type {AssetGeneratorDataUrlOptions} */
505522
(this.dataUrlOptions).mimetype;
506523
if (mimeType === undefined) {
524+
// The renamed resource, minus a query, the way `nameForCondition` reads
525+
// the original: an asset re-encoded as another format is inlined as
526+
// what it now is, not as what it was read from.
527+
const resource = AssetGenerator.getAssetResource(module);
528+
const queryIndex = resource.indexOf("?");
507529
const ext = path.extname(
508-
/** @type {NameForCondition} */
509-
(module.nameForCondition())
530+
queryIndex >= 0 ? resource.slice(0, queryIndex) : resource
510531
);
511532
if (
512533
module.resourceResolveData &&

‎lib/asset/AssetModule.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,11 @@ const makeSerializable = require("../util/makeSerializable");
2222
* @property {string=} filename
2323
* @property {AssetInfo=} assetInfo
2424
* @property {string=} fullContentHash
25+
* @property {string=} assetResource the resource this asset is named after, when
26+
* something changed it — a `processResult` tap that re-encoded the image as
27+
* another format, say. `matchResource` says the same thing for a request, but
28+
* belongs to resolution and is not serialized, so a rename set while building
29+
* would be lost to the persistent cache; this is written with the module.
2530
*/
2631

2732
/** @typedef {NormalModuleBuildInfo & KnownAssetModuleBuildInfo} AssetModuleBuildInfo */
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
import url from "../_images/file.png";
2+
3+
it("should inline the asset as the format the hook made it", () => {
4+
expect(url).toMatch(/^data:image\/webp;base64,/);
5+
});
6+
7+
it("should inline what the hook returned", () => {
8+
expect(Buffer.from(url.split(",")[1], "base64").toString()).toBe(
9+
"webp-bytes"
10+
);
11+
});
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
"use strict";
2+
3+
const NormalModule = require("../../../../lib/NormalModule");
4+
5+
// The inline half of the same question: an asset re-encoded as another format
6+
// has no file name to change, but the media type it is inlined as is the same
7+
// claim about what the bytes are, so it has to follow too.
8+
class RenameOnProcessResult {
9+
/**
10+
* @param {import("../../../../lib/Compiler")} compiler compiler
11+
*/
12+
apply(compiler) {
13+
compiler.hooks.compilation.tap("RenameOnProcessResult", (compilation) => {
14+
NormalModule.getCompilationHooks(compilation).processResult.tapPromise(
15+
"RenameOnProcessResult",
16+
async (result, module) => {
17+
if (!/\.png$/.test(module.resource)) return result;
18+
await Promise.resolve();
19+
/** @type {import("../../../../lib/asset/AssetModule").AssetModuleBuildInfo} */
20+
(module.buildInfo).assetResource = module.resource.replace(
21+
/\.png$/,
22+
".webp"
23+
);
24+
return [Buffer.from("webp-bytes"), result[1], result[2]];
25+
}
26+
);
27+
});
28+
}
29+
}
30+
31+
/** @type {import("../../../../").Configuration} */
32+
module.exports = {
33+
mode: "development",
34+
module: {
35+
rules: [
36+
{
37+
test: /\.(png|webp)$/,
38+
type: "asset/inline"
39+
}
40+
]
41+
},
42+
plugins: [new RenameOnProcessResult()]
43+
};
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
import fs from "fs";
2+
import path from "path";
3+
import url from "../_images/file.png";
4+
5+
it("should emit the asset under the name the hook asked for", () => {
6+
expect(url).toBe("file.webp");
7+
expect(fs.existsSync(path.join(__STATS__.outputPath, "file.webp"))).toBe(
8+
true
9+
);
10+
expect(fs.existsSync(path.join(__STATS__.outputPath, "file.png"))).toBe(
11+
false
12+
);
13+
});
14+
15+
it("should emit what the hook returned", () => {
16+
expect(
17+
fs.readFileSync(path.join(__STATS__.outputPath, url), "utf8")
18+
).toBe("webp-bytes");
19+
});
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
"use strict";
2+
3+
const NormalModule = require("../../../../lib/NormalModule");
4+
5+
// A `processResult` tap standing in for an image minimizer: it awaits work of
6+
// its own and re-encodes the asset as another format, so the name it is
7+
// emitted under has to change with the bytes.
8+
class RenameOnProcessResult {
9+
/**
10+
* @param {import("../../../../lib/Compiler")} compiler compiler
11+
*/
12+
apply(compiler) {
13+
compiler.hooks.compilation.tap("RenameOnProcessResult", (compilation) => {
14+
NormalModule.getCompilationHooks(compilation).processResult.tapPromise(
15+
"RenameOnProcessResult",
16+
async (result, module) => {
17+
if (!/\.png$/.test(module.resource)) return result;
18+
// Nothing here needs to wait; awaiting is what the sync hook
19+
// could not do, so the case is only meaningful if it does.
20+
await Promise.resolve();
21+
/** @type {import("../../../../lib/asset/AssetModule").AssetModuleBuildInfo} */
22+
(module.buildInfo).assetResource = module.resource.replace(
23+
/\.png$/,
24+
".webp"
25+
);
26+
return [Buffer.from("webp-bytes"), result[1], result[2]];
27+
}
28+
);
29+
});
30+
}
31+
}
32+
33+
/** @type {import("../../../../").Configuration} */
34+
module.exports = {
35+
mode: "development",
36+
output: {
37+
assetModuleFilename: "[name][ext]"
38+
},
39+
module: {
40+
rules: [
41+
{
42+
test: /\.(png|webp)$/,
43+
type: "asset/resource"
44+
}
45+
]
46+
},
47+
plugins: [new RenameOnProcessResult()]
48+
};

‎types.d.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14946,6 +14946,15 @@ declare interface KnownAssetModuleBuildInfo {
1494614946
filename?: string;
1494714947
assetInfo?: AssetInfo;
1494814948
fullContentHash?: string;
14949+
14950+
/**
14951+
* the resource this asset is named after, when
14952+
* something changed it — a `processResult` tap that re-encoded the image as
14953+
* another format, say. `matchResource` says the same thing for a request, but
14954+
* belongs to resolution and is not serialized, so a rename set while building
14955+
* would be lost to the persistent cache; this is written with the module.
14956+
*/
14957+
assetResource?: string;
1494914958
}
1495014959
declare interface KnownBuildInfo {
1495114960
cacheable?: boolean;
@@ -18953,9 +18962,13 @@ declare class NormalModule extends Module {
1895318962
*/
1895418963
beforeSnapshot: SyncHook<[NormalModule]>;
1895518964
/**
18965+
* Called with what the loaders produced, before it is parsed. Return the
18966+
* (possibly transformed) result. Async since 5.110.0, so a tap can await
18967+
* work of its own — minifying an asset, say — where before only a loader
18968+
* could; a `tap` written for the synchronous hook keeps working.
1895618969
* @since 5.99.0
1895718970
*/
18958-
processResult: SyncWaterfallHook<
18971+
processResult: AsyncSeriesWaterfallHook<
1895918972
[
1896018973
[
1896118974
string | Buffer,

0 commit comments

Comments
 (0)