Skip to content

Commit e954b81

Browse files
build(html): sweep the HTML printer for spelling-dependent output (#22097)
* build(html): sweep the HTML printer for spelling-dependent output `yarn check:html-invariants` holds the printer to two properties of its own, which is the class of defect `benchmark:html-tools` cannot see: being a few bytes off webpack's own best is not being worse than another tool. Minifying an already-minified page must change nothing, and a value's spelling must not decide what it minifies to. Respellings are lexical and checked against webpack's own tokenizer before use, so a mutation that moved the document is dropped rather than reported; each finding is bisected to the attributes carrying it and re-run on the enclosing tag alone, then grouped by that repro. Over `test/**/*.html` it names 112 findings in under two seconds, 40 of which #22095 fixes. * build(html): report a printer that throws on its own output The second minify call sat outside a handler, so a printer that reads its own output back and throws would end the sweep with a stack trace — losing every later document, and hiding the strongest finding the sweep can make. It is a report now, and the respellings still answer: they never read the second pass. A grouped finding also counted one document once per preset, so a defect both presets reach read as two documents and sorted above one that reached more pages. * test(html): keep the sweep's own test from passing vacuously Restoring an env var that was unset stored the string "undefined", which reads as a filter matching nothing in whatever loads the module next in that worker — so the filters are deleted rather than reassigned. A filter naming a fixture that was since renamed would leave the corpus empty and every assertion after it vacuous, so the corpus is asserted to still hold what the filter names. * build: sweep the CSS and HTML printers from the comparison itself The invariant sweep moves into `compare-html-tools.js` as an `--invariants` mode, `compare-css-tools.js` gains the same over `test/**/*.css`, and the relation both hold is `minify(minify(source)) === minify(source)`. Their shared half — bisection, the finding report, the relation itself — lives in `compare-tools-harness.js`, so one script per language owns that language. Every printing row in both comparisons also carries a `2nd` column now: what a second pass over the tool's own output moved, which is the same question asked of the ecosystem's tools rather than only of webpack's. The standalone script and its unit test are gone with it: what catches a defect here is a tooling script, not a test beside it. * build: report the printer's own invariants from a plain comparison run The relations need no install, so each comparison opens with them rather than hiding them behind a script of its own: what a run costs and what it owes are one report. `--invariants` still prints that section and stops, for a caller that does not want the ten minutes behind it. The sweep reads the comparison's built fixtures too — tag soup, web components, a component page, a table report — at a size the bisection can still cut a repro out of.
1 parent da4ad13 commit e954b81

5 files changed

Lines changed: 1076 additions & 59 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ The directory listings below are the canonical map of the repository. **Whenever
9292
- `lib/wasm/`, `lib/wasm-async/`, `lib/wasm-sync/` — WebAssembly module support.
9393
- `hot/` — Runtime code shipped to browsers for HMR (browser-side, not Node tooling).
9494
- `bin/` — `webpack` CLI entry point.
95-
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-tools.js` / `compare-html-tools.js` / `compare-js-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`, `yarn benchmark:js-tools`), which share the measuring harness in `compare-tools-harness.js`. Those three need no arguments and no reading of their source: each runs what webpack owns for that language and the ecosystem's equivalents over popular framework stylesheets, real documents and shipped JavaScript bundles, printing **three tables per fixture** — parsing alone, parsing and printing readably (`beautify`), and parsing and printing minified (`minify`) — so what a parse costs is separated from what the printing and the transforms on top of it cost. Every table reports best-of-3 wall and cpu ms and the worker's own peak RSS (each tool × fixture measured in its own process, so the numbers are attributable; read from `/proc/self/status`, since Linux carries `maxRSS` across `fork`+`exec` and a worker asking for its own would report the parent's). **Read wall and cpu together**: cpu above wall is a tool using more than one core (V8's background threads, a native thread pool), cpu below it is a tool waiting rather than computing. A tool that works in a service process of its own says so in its name and reports `-` for cpu and peak, which are spent where nothing here can see them — esbuild is the one today. The two printing tables add output size raw and under gzip/brotli/zstd (the `test:size` settings) and whether the output lost classes / changed the DOM / stopped naming a property ("rejects it" rows mean the tool errored on that input). The JavaScript one is shaped by webpack owning a parser there and no printer: webpack has a row in the parse table only, and that table's last column holds every ESTree parser against acorn's tree and names the first node where the two differ — which is how a divergence in `lib/javascript/syntax.js` is caught — while a parser answering in a dialect of its own reads as `own dialect` rather than as a disagreement. A round-trip printer that reformats nothing — postcss for CSS, parse5 for HTML — sits in the `beautify` table as the floor the others are read against. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take upwards of ten. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand.
95+
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-tools.js` / `compare-html-tools.js` / `compare-js-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`, `yarn benchmark:js-tools`), which share the measuring harness in `compare-tools-harness.js`. Those three need no arguments and no reading of their source: each runs what webpack owns for that language and the ecosystem's equivalents over popular framework stylesheets, real documents and shipped JavaScript bundles, printing **three tables per fixture** — parsing alone, parsing and printing readably (`beautify`), and parsing and printing minified (`minify`) — so what a parse costs is separated from what the printing and the transforms on top of it cost. Every table reports best-of-3 wall and cpu ms and the worker's own peak RSS (each tool × fixture measured in its own process, so the numbers are attributable; read from `/proc/self/status`, since Linux carries `maxRSS` across `fork`+`exec` and a worker asking for its own would report the parent's). **Read wall and cpu together**: cpu above wall is a tool using more than one core (V8's background threads, a native thread pool), cpu below it is a tool waiting rather than computing. A tool that works in a service process of its own says so in its name and reports `-` for cpu and peak, which are spent where nothing here can see them — esbuild is the one today. The two printing tables add output size raw and under gzip/brotli/zstd (the `test:size` settings) and whether the output lost classes / changed the DOM / stopped naming a property ("rejects it" rows mean the tool errored on that input), plus what printing the output a second time moved (`2nd`). The JavaScript one is shaped by webpack owning a parser there and no printer: webpack has a row in the parse table only, and that table's last column holds every ESTree parser against acorn's tree and names the first node where the two differ — which is how a divergence in `lib/javascript/syntax.js` is caught — while a parser answering in a dialect of its own reads as `own dialect` rather than as a disagreement. A round-trip printer that reformats nothing — postcss for CSS, parse5 for HTML — sits in the `beautify` table as the floor the others are read against. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take upwards of ten. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand. **The CSS and HTML comparisons open with a section no one has to ask for**: before the install, each holds webpack's own printer to what it owes whatever it printed, over `test/**/*.css` / `test/**/*.html` plus its own fixtures and whatever the cache already holds, in seconds rather than minutes. `--invariants` runs that section and stops, for a caller that wants the relations without the comparison behind them. Two relations, under each option set the script's own webpack rows are measured with: minifying an already-minified source changes nothing, and (HTML) how a value was spelled does not decide what it minifies to. This is the shape of defect a comparison cannot see, because being a few bytes off its own best is not being worse than another tool: `method=GET` folding only where the source quoted it went unnoticed until this sweep, which is what #22095 fixed. A respelling is lexical (a delimiter, a character reference, the case of a name) and is checked against webpack's own tokenizer before use, so a mutation that moved the document is dropped rather than reported; whitespace inside a tag is deliberately not one, since a tag nothing beats is echoed as written. Each finding is bisected to what carries it and re-run on the enclosing tag alone, so the report names a repro, and groups by it — one printer defect reaches hundreds of pages. `FIXTURE=`, `RELATION=`, `PRESET=` and `SPELLING=` narrow a run. `--invariants` exits non-zero while any finding stands and findings stand today, so read it rather than gating on it, and no embedded body is minified: the CSS minifier has a fixed point of its own and reaching it here would report every page holding a `<style>`. The same question is asked of every other tool in the comparison's own tables, where the `2nd` column is what a second pass over the tool's own output moved — `-` being the answer a printer that is done printing gives.
9696
- `assembly/` — WebAssembly source for the hash function.
9797
- `setup/` — One-time setup. `setup.js` (`yarn setup`) is the only entry point and picks its own path: a contributor at a terminal gets the interactive one, which installs yarn when it is missing and links through yarn's registry, while everything else gets the non-interactive one — it verifies the lockfile instead of rewriting it, installs no global yarn, and links the checkout in as `node_modules/webpack` without touching yarn's machine-global registry. What it reads is both streams being a terminal plus an unset `CI`, never a list of vendor variables, so an agent nobody has heard of yet takes the safe path — but one that allocates a PTY reads as a contributor, and should set `WEBPACK_SETUP=automated`, which with `interactive` forces either path. Safe to re-run.
9898

‎cspell.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -380,6 +380,7 @@
380380
"repos",
381381
"repr",
382382
"resizer",
383+
"respellings",
383384
"return'development",
384385
"returnfalse",
385386
"revparse",

‎tooling/compare-css-tools.js‎

Lines changed: 185 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@
1616
// The two printing tables add what the output weighs — raw and under the
1717
// encodings a CDN serves — and whether it still matches every class it did.
1818

19+
// The run opens with the invariants webpack's own printer owes its output,
20+
// which need no install; `--invariants` prints that section and stops.
21+
1922
// `FIXTURE=`, `TOOL=` and `STAGE=` narrow the run to rows whose name contains
2023
// what they name, so one cell is re-measured without the whole matrix.
2124

@@ -33,13 +36,23 @@ const MODERN_BROWSERS = require("browserslist")(
3336
"last 1 chrome version, last 1 firefox version, last 1 safari version, last 1 edge version"
3437
);
3538
const cssMinify = require("../lib/css/cssMinify");
36-
const { SourceProcessor } = require("../lib/css/syntax");
39+
const {
40+
SourceProcessor,
41+
TT_EOF,
42+
TT_WHITESPACE,
43+
TokenStream,
44+
pickTransforms
45+
} = require("../lib/css/syntax");
3746
const {
3847
STAGES,
48+
collectFiles,
3949
compress,
4050
exists,
4151
filterFrom,
52+
findingGroups,
4253
formatCost,
54+
formatSecond,
55+
idempotence,
4356
installPackages,
4457
kb,
4558
loaderFor,
@@ -212,6 +225,29 @@ const fixtures = () => [
212225
)
213226
];
214227

228+
/** @typedef {import("../lib/css/syntax").CssPrintOptions} CssPrintOptions */
229+
230+
/** @type {CssPrintOptions} */
231+
const DEFAULT_OPTIONS = {};
232+
233+
/** @type {CssPrintOptions} */
234+
const TARGET_OPTIONS = { environment: { browsers: MODERN_BROWSERS } };
235+
236+
/** @type {CssPrintOptions} */
237+
const TARGET_VARS_OPTIONS = {
238+
environment: { browsers: MODERN_BROWSERS },
239+
rewriteCustomProperties: true
240+
};
241+
242+
// The option sets the invariants are held over, which are the ones the
243+
// comparison's own webpack rows are measured with.
244+
/** @type {[string, CssPrintOptions][]} */
245+
const PRESETS = [
246+
["default", DEFAULT_OPTIONS],
247+
["target", TARGET_OPTIONS],
248+
["target+vars", TARGET_VARS_OPTIONS]
249+
];
250+
215251
// Each entry builds its callable on demand, so the measuring worker loads only
216252
// the one tool it measures — anything else would land in that tool's peak RSS.
217253
/** @type {import("./compare-tools-harness").Tool[]} */
@@ -366,32 +402,25 @@ const TOOLS = [
366402
{
367403
name: "webpack",
368404
stage: "minify",
369-
create: () => async (css) => (await cssMinify({ "input.css": css })).code
405+
create: () => async (css) =>
406+
(await cssMinify({ "input.css": css }, undefined, DEFAULT_OPTIONS)).code
370407
},
371408
{
372409
// The rivals strip the spellings a modern engine makes dead; webpack, told
373410
// nothing, keeps them. The two rows say what the target is worth.
374411
name: "webpack+target",
375412
stage: "minify",
376413
create: () => async (css) =>
377-
(
378-
await cssMinify({ "input.css": css }, undefined, {
379-
environment: { browsers: MODERN_BROWSERS }
380-
})
381-
).code
414+
(await cssMinify({ "input.css": css }, undefined, TARGET_OPTIONS)).code
382415
},
383416
{
384417
// The rivals shorten a custom property's value the way they shorten any
385418
// other; webpack holds off unless told, since `getPropertyValue()` reads it.
386419
name: "webpack+target+vars",
387420
stage: "minify",
388421
create: () => async (css) =>
389-
(
390-
await cssMinify({ "input.css": css }, undefined, {
391-
environment: { browsers: MODERN_BROWSERS },
392-
rewriteCustomProperties: true
393-
})
394-
).code
422+
(await cssMinify({ "input.css": css }, undefined, TARGET_VARS_OPTIONS))
423+
.code
395424
},
396425
{
397426
name: "esbuild (service)",
@@ -574,7 +603,129 @@ const wantedFixture = filterFrom("FIXTURE");
574603
const wantedTool = filterFrom("TOOL");
575604
const wantedStage = filterFrom("STAGE");
576605

606+
// --- Invariants -------------------------------------------------------------
607+
608+
// What a comparison cannot see, because being a few bytes off its own best is
609+
// not being worse than another tool: a second pass that moves the stylesheet.
610+
611+
// A stylesheet the repo ships is a shape the printer exists for, so the sweep
612+
// reads `test/**/*.css` rather than the handful of fixtures timings need.
613+
614+
// A web-platform-tests checkout, which the html5lib job alone fetches: its
615+
// thousands of stylesheets are a corpus of their own rather than a default run.
616+
const SKIPPED_FIXTURE_DIRS = new Set(["node_modules", "wpt"]);
617+
618+
/**
619+
* @param {CssPrintOptions} options one preset's minimizer options
620+
* @returns {(css: string) => string} the printer those options name
621+
*/
622+
const printerFor = (options) => {
623+
const printOptions = {
624+
mode: /** @type {"minify"} */ ("minify"),
625+
transforms: pickTransforms(options),
626+
...options
627+
};
628+
return (css) =>
629+
/** @type {{ code: string }} */ (
630+
new SourceProcessor().process(css, printOptions)
631+
).code;
632+
};
633+
634+
/**
635+
* What a stylesheet says, as a flat token stream: whitespace runs collapse to
636+
* one marker (dropping a comment leaves the run either side of it), so only a
637+
* token the printer actually rewrote separates two outputs. A respelling is one,
638+
* which is why `#FFF` -> `#fff` between two passes reads as `differs` here.
639+
* @param {string} css a stylesheet
640+
* @returns {string} its stream, as the JSON of every token in order
641+
*/
642+
const tokenStream = (css) => {
643+
const stream = new TokenStream(css);
644+
/** @type {string[]} */
645+
const out = [];
646+
for (;;) {
647+
const token = stream.consume();
648+
if (token.type === TT_EOF) break;
649+
if (token.type === TT_WHITESPACE) {
650+
if (out[out.length - 1] !== " ") out.push(" ");
651+
continue;
652+
}
653+
out.push(`${token.type}:${css.slice(token.start, token.end)}`);
654+
}
655+
return JSON.stringify(out);
656+
};
657+
658+
const wantedRelation = filterFrom("RELATION");
659+
const wantedPreset = filterFrom("PRESET");
660+
661+
/**
662+
* The stylesheets the invariants are swept over: the repo's own fixtures, plus
663+
* whatever framework stylesheets the comparison installed.
664+
* @returns {[string, string][]} `[label, css]` for every stylesheet
665+
*/
666+
const invariantFixtures = () => {
667+
/** @type {[string, string][]} */
668+
const out = [];
669+
for (const file of collectFiles(
670+
path.join(ROOT, "test"),
671+
".css",
672+
SKIPPED_FIXTURE_DIRS
673+
)) {
674+
out.push([
675+
path.relative(ROOT, file).replace(/\\/g, "/"),
676+
fs.readFileSync(file, "utf8")
677+
]);
678+
}
679+
for (const [label, file] of fixtures()) {
680+
if (fs.existsSync(file)) out.push([label, fs.readFileSync(file, "utf8")]);
681+
}
682+
return out;
683+
};
684+
685+
/**
686+
* Sweep mode: hold the printer to its own invariants and report what it breaks.
687+
* Nothing is installed and nothing is compared to, so this is the cheap half of
688+
* the script and the one a check can be gated on.
689+
* @param {(text: string) => void} write receives the report
690+
* @returns {number} how many distinct findings it named
691+
*/
692+
const invariants = (write) => {
693+
const corpus = invariantFixtures().filter(([label]) => wantedFixture(label));
694+
const presets = PRESETS.filter(([name]) => wantedPreset(name));
695+
if (!wantedRelation("idempotence")) return 0;
696+
log(
697+
`sweeping ${corpus.length} stylesheets under ${presets.length} presets …`
698+
);
699+
const groups = findingGroups();
700+
for (const [label, css] of corpus) {
701+
for (const [preset, options] of presets) {
702+
const { reports } = idempotence({
703+
minify: printerFor(options),
704+
source: css,
705+
says: tokenStream
706+
});
707+
for (const report of reports) groups.add(report, preset, label);
708+
}
709+
}
710+
return groups.write(write);
711+
};
712+
713+
/**
714+
* The sweep as a section of the comparison's own report, so a run that asks
715+
* what the output costs is told what it owes as well.
716+
* @returns {number} how many distinct findings it named
717+
*/
718+
const reportInvariants = () => {
719+
process.stdout.write("\ninvariants — what the printer owes its own output\n");
720+
const found = invariants((text) => process.stdout.write(text));
721+
process.stdout.write(`\n${found} finding${found === 1 ? "" : "s"}\n`);
722+
return found;
723+
};
724+
577725
const main = async () => {
726+
// Before the install: the relations are webpack's own, so they answer in
727+
// seconds whether or not there is anything to compare against yet.
728+
reportInvariants();
578729
await setup();
579730
const postcss = load("postcss");
580731
const selectorParser = load("postcss-selector-parser");
@@ -610,7 +761,7 @@ const main = async () => {
610761
9
611762
)}${"ms".padStart(7)}${"cpu".padStart(6)}${"peak".padStart(
612763
8
613-
)} lost\n`
764+
)}${"2nd".padStart(7)} lost\n`
614765
);
615766
for (const tool of tools) {
616767
const result = await measureInWorker(__filename, stage, tool.name, css);
@@ -647,7 +798,8 @@ const main = async () => {
647798
kb(out.zstd).padStart(9) +
648799
cost.wall.padStart(7) +
649800
cost.cpu.padStart(6) +
650-
cost.peak.padStart(8)
801+
cost.peak.padStart(8) +
802+
formatSecond(result.second).padStart(7)
651803
} ${
652804
lost.length === 0
653805
? "-"
@@ -665,16 +817,24 @@ if (require.main === module) {
665817
// `--setup` installs the fixtures and builds nothing else, so a consumer
666818
// that only reads them does not run the comparison to get them.
667819
const mode = process.argv[2];
668-
const started =
669-
mode === "--measure"
670-
? measure(TOOLS)
671-
: mode === "--setup"
672-
? setup()
673-
: main();
674-
started.catch((error) => {
675-
log(String(error && error.stack ? error.stack : error));
676-
process.exitCode = 1;
677-
});
820+
// The sweep alone, for a caller that wants the relations without the ten
821+
// minutes the comparison costs; a full run prints the same section.
822+
if (mode === "--invariants") {
823+
// Non-zero while any finding stands, and findings stand today: read it
824+
// rather than gating on it until they are gone.
825+
process.exitCode = reportInvariants() > 0 ? 1 : 0;
826+
} else {
827+
const started =
828+
mode === "--measure"
829+
? measure(TOOLS)
830+
: mode === "--setup"
831+
? setup()
832+
: main();
833+
started.catch((error) => {
834+
log(String(error && error.stack ? error.stack : error));
835+
process.exitCode = 1;
836+
});
837+
}
678838
}
679839

680840
// Where the cache holds each fixture, for a reader that is not this script.

0 commit comments

Comments
 (0)