Skip to content

Commit 184eef9

Browse files
build: report the printer's own invariants from a plain comparison run
The relations need no install, so each comparison opens with them rather than hiding them behind a script of its own: what a run costs and what it owes are one report. `--invariants` still prints that section and stops, for a caller that does not want the ten minutes behind it. The sweep reads the comparison's built fixtures too — tag soup, web components, a component page, a table report — at a size the bisection can still cut a repro out of.
1 parent 029be89 commit 184eef9

4 files changed

Lines changed: 49 additions & 9 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ The directory listings below are the canonical map of the repository. **Whenever
9292
- `lib/wasm/`, `lib/wasm-async/`, `lib/wasm-sync/` — WebAssembly module support.
9393
- `hot/` — Runtime code shipped to browsers for HMR (browser-side, not Node tooling).
9494
- `bin/` — `webpack` CLI entry point.
95-
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-tools.js` / `compare-html-tools.js` / `compare-js-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`, `yarn benchmark:js-tools`), which share the measuring harness in `compare-tools-harness.js`. Those three need no arguments and no reading of their source: each runs what webpack owns for that language and the ecosystem's equivalents over popular framework stylesheets, real documents and shipped JavaScript bundles, printing **three tables per fixture** — parsing alone, parsing and printing readably (`beautify`), and parsing and printing minified (`minify`) — so what a parse costs is separated from what the printing and the transforms on top of it cost. Every table reports best-of-3 wall and cpu ms and the worker's own peak RSS (each tool × fixture measured in its own process, so the numbers are attributable; read from `/proc/self/status`, since Linux carries `maxRSS` across `fork`+`exec` and a worker asking for its own would report the parent's). **Read wall and cpu together**: cpu above wall is a tool using more than one core (V8's background threads, a native thread pool), cpu below it is a tool waiting rather than computing. A tool that works in a service process of its own says so in its name and reports `-` for cpu and peak, which are spent where nothing here can see them — esbuild is the one today. The two printing tables add output size raw and under gzip/brotli/zstd (the `test:size` settings) and whether the output lost classes / changed the DOM / stopped naming a property ("rejects it" rows mean the tool errored on that input), plus what printing the output a second time moved (`2nd`). The JavaScript one is shaped by webpack owning a parser there and no printer: webpack has a row in the parse table only, and that table's last column holds every ESTree parser against acorn's tree and names the first node where the two differ — which is how a divergence in `lib/javascript/syntax.js` is caught — while a parser answering in a dialect of its own reads as `own dialect` rather than as a disagreement. A round-trip printer that reformats nothing — postcss for CSS, parse5 for HTML — sits in the `beautify` table as the floor the others are read against. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take upwards of ten. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand. **The CSS and HTML comparisons each carry a second mode, `--invariants` (`yarn check:css-invariants`, `yarn check:html-invariants`), which installs nothing and is the cheap half of the script**: it holds webpack's own printer to what it owes whatever it printed, over `test/**/*.css` / `test/**/*.html` plus whatever the comparison's cache already holds, in seconds rather than minutes. Two relations, under each option set the script's own webpack rows are measured with: minifying an already-minified source changes nothing, and (HTML) how a value was spelled does not decide what it minifies to. This is the shape of defect a comparison cannot see, because being a few bytes off its own best is not being worse than another tool: `method=GET` folding only where the source quoted it went unnoticed until this sweep, which is what #22095 fixed. A respelling is lexical (a delimiter, a character reference, the case of a name) and is checked against webpack's own tokenizer before use, so a mutation that moved the document is dropped rather than reported; whitespace inside a tag is deliberately not one, since a tag nothing beats is echoed as written. Each finding is bisected to what carries it and re-run on the enclosing tag alone, so the report names a repro, and groups by it — one printer defect reaches hundreds of pages. `FIXTURE=`, `RELATION=`, `PRESET=` and `SPELLING=` narrow a run. Each exits non-zero while any finding stands and findings stand today, so read them rather than gating on them, and no embedded body is minified: the CSS minifier has a fixed point of its own and reaching it here would report every page holding a `<style>`. The same question is asked of every other tool in the comparison's own tables, where the `2nd` column is what a second pass over the tool's own output moved — `-` being the answer a printer that is done printing gives.
95+
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-tools.js` / `compare-html-tools.js` / `compare-js-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`, `yarn benchmark:js-tools`), which share the measuring harness in `compare-tools-harness.js`. Those three need no arguments and no reading of their source: each runs what webpack owns for that language and the ecosystem's equivalents over popular framework stylesheets, real documents and shipped JavaScript bundles, printing **three tables per fixture** — parsing alone, parsing and printing readably (`beautify`), and parsing and printing minified (`minify`) — so what a parse costs is separated from what the printing and the transforms on top of it cost. Every table reports best-of-3 wall and cpu ms and the worker's own peak RSS (each tool × fixture measured in its own process, so the numbers are attributable; read from `/proc/self/status`, since Linux carries `maxRSS` across `fork`+`exec` and a worker asking for its own would report the parent's). **Read wall and cpu together**: cpu above wall is a tool using more than one core (V8's background threads, a native thread pool), cpu below it is a tool waiting rather than computing. A tool that works in a service process of its own says so in its name and reports `-` for cpu and peak, which are spent where nothing here can see them — esbuild is the one today. The two printing tables add output size raw and under gzip/brotli/zstd (the `test:size` settings) and whether the output lost classes / changed the DOM / stopped naming a property ("rejects it" rows mean the tool errored on that input), plus what printing the output a second time moved (`2nd`). The JavaScript one is shaped by webpack owning a parser there and no printer: webpack has a row in the parse table only, and that table's last column holds every ESTree parser against acorn's tree and names the first node where the two differ — which is how a divergence in `lib/javascript/syntax.js` is caught — while a parser answering in a dialect of its own reads as `own dialect` rather than as a disagreement. A round-trip printer that reformats nothing — postcss for CSS, parse5 for HTML — sits in the `beautify` table as the floor the others are read against. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take upwards of ten. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand. **The CSS and HTML comparisons open with a section no one has to ask for**: before the install, each holds webpack's own printer to what it owes whatever it printed, over `test/**/*.css` / `test/**/*.html` plus its own fixtures and whatever the cache already holds, in seconds rather than minutes. `--invariants` runs that section and stops, for a caller that wants the relations without the comparison behind them. Two relations, under each option set the script's own webpack rows are measured with: minifying an already-minified source changes nothing, and (HTML) how a value was spelled does not decide what it minifies to. This is the shape of defect a comparison cannot see, because being a few bytes off its own best is not being worse than another tool: `method=GET` folding only where the source quoted it went unnoticed until this sweep, which is what #22095 fixed. A respelling is lexical (a delimiter, a character reference, the case of a name) and is checked against webpack's own tokenizer before use, so a mutation that moved the document is dropped rather than reported; whitespace inside a tag is deliberately not one, since a tag nothing beats is echoed as written. Each finding is bisected to what carries it and re-run on the enclosing tag alone, so the report names a repro, and groups by it — one printer defect reaches hundreds of pages. `FIXTURE=`, `RELATION=`, `PRESET=` and `SPELLING=` narrow a run. `--invariants` exits non-zero while any finding stands and findings stand today, so read it rather than gating on it, and no embedded body is minified: the CSS minifier has a fixed point of its own and reaching it here would report every page holding a `<style>`. The same question is asked of every other tool in the comparison's own tables, where the `2nd` column is what a second pass over the tool's own output moved — `-` being the answer a printer that is done printing gives.
9696
- `assembly/` — WebAssembly source for the hash function.
9797
- `setup/` — One-time setup. `setup.js` (`yarn setup`) is the only entry point and picks its own path: a contributor at a terminal gets the interactive one, which installs yarn when it is missing and links through yarn's registry, while everything else gets the non-interactive one — it verifies the lockfile instead of rewriting it, installs no global yarn, and links the checkout in as `node_modules/webpack` without touching yarn's machine-global registry. What it reads is both streams being a terminal plus an unset `CI`, never a list of vendor variables, so an agent nobody has heard of yet takes the safe path — but one that allocates a PTY reads as a contributor, and should set `WEBPACK_SETUP=automated`, which with `interactive` forces either path. Safe to re-run.
9898

‎package.json‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,8 +59,6 @@
5959
"benchmark:html-tools:setup": "node tooling/compare-html-tools.js --setup",
6060
"benchmark:js-tools": "node tooling/compare-js-tools.js",
6161
"measure:color-agreement": "node tooling/measure-color-agreement.js",
62-
"check:css-invariants": "node tooling/compare-css-tools.js --invariants",
63-
"check:html-invariants": "node tooling/compare-html-tools.js --invariants",
6462
"pretest": "yarn lint",
6563
"test": "yarn test:base",
6664
"test:base": "node --expose-gc --max-old-space-size=4096 --experimental-vm-modules --trace-deprecation node_modules/jest-cli/bin/jest --workerIdleMemoryLimit=512MB --logHeapUsage",

‎tooling/compare-css-tools.js‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@
1616
// The two printing tables add what the output weighs — raw and under the
1717
// encodings a CDN serves — and whether it still matches every class it did.
1818

19+
// The run opens with the invariants webpack's own printer owes its output,
20+
// which need no install; `--invariants` prints that section and stops.
21+
1922
// `FIXTURE=`, `TOOL=` and `STAGE=` narrow the run to rows whose name contains
2023
// what they name, so one cell is re-measured without the whole matrix.
2124

@@ -707,7 +710,22 @@ const invariants = (write) => {
707710
return groups.write(write);
708711
};
709712

713+
/**
714+
* The sweep as a section of the comparison's own report, so a run that asks
715+
* what the output costs is told what it owes as well.
716+
* @returns {number} how many distinct findings it named
717+
*/
718+
const reportInvariants = () => {
719+
process.stdout.write("\ninvariants — what the printer owes its own output\n");
720+
const found = invariants((text) => process.stdout.write(text));
721+
process.stdout.write(`\n${found} finding${found === 1 ? "" : "s"}\n`);
722+
return found;
723+
};
724+
710725
const main = async () => {
726+
// Before the install: the relations are webpack's own, so they answer in
727+
// seconds whether or not there is anything to compare against yet.
728+
reportInvariants();
711729
await setup();
712730
const postcss = load("postcss");
713731
const selectorParser = load("postcss-selector-parser");
@@ -799,12 +817,12 @@ if (require.main === module) {
799817
// `--setup` installs the fixtures and builds nothing else, so a consumer
800818
// that only reads them does not run the comparison to get them.
801819
const mode = process.argv[2];
820+
// The sweep alone, for a caller that wants the relations without the ten
821+
// minutes the comparison costs; a full run prints the same section.
802822
if (mode === "--invariants") {
803-
const found = invariants((text) => process.stdout.write(text));
804-
log(`\n${found} finding${found === 1 ? "" : "s"}`);
805823
// Non-zero while any finding stands, and findings stand today: read it
806824
// rather than gating on it until they are gone.
807-
process.exitCode = found > 0 ? 1 : 0;
825+
process.exitCode = reportInvariants() > 0 ? 1 : 0;
808826
} else {
809827
const started =
810828
mode === "--measure"

‎tooling/compare-html-tools.js‎

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@
1616
// The two printing tables add what the output weighs and whether the DOM it
1717
// parses back to still says what the input's did.
1818

19+
// The run opens with the invariants webpack's own printer owes its output,
20+
// which need no install; `--invariants` prints that section and stops.
21+
1922
// `FIXTURE=`, `TOOL=` and `STAGE=` narrow the run to rows whose name contains
2023
// what they name, so one cell is re-measured without the whole matrix.
2124

@@ -1322,6 +1325,12 @@ const invariantFixtures = () => {
13221325
]);
13231326
}
13241327
out.push(["App shell (inline critical CSS)", APP_SHELL]);
1328+
// The shapes the comparison builds rather than installs, at a size the
1329+
// bisection can still cut down: what they carry is the shape, not the bulk.
1330+
out.push(["Component library page", componentPage(8)]);
1331+
out.push(["Table report", tablePage(10)]);
1332+
out.push(["Tag soup", TAG_SOUP]);
1333+
out.push(["Web components", WEB_COMPONENTS]);
13251334
for (const [label, file] of INSTALLED_DOCUMENTS) {
13261335
const full = path.join(MODULES, file);
13271336
if (fs.existsSync(full)) out.push([label, fs.readFileSync(full, "utf8")]);
@@ -1357,7 +1366,22 @@ const invariants = (write) => {
13571366
return groups.write(write);
13581367
};
13591368

1369+
/**
1370+
* The sweep as a section of the comparison's own report, so a run that asks
1371+
* what the output costs is told what it owes as well.
1372+
* @returns {number} how many distinct findings it named
1373+
*/
1374+
const reportInvariants = () => {
1375+
process.stdout.write("\ninvariants — what the printer owes its own output\n");
1376+
const found = invariants((text) => process.stdout.write(text));
1377+
process.stdout.write(`\n${found} finding${found === 1 ? "" : "s"}\n`);
1378+
return found;
1379+
};
1380+
13601381
const main = async () => {
1382+
// Before the install: the relations are webpack's own, so they answer in
1383+
// seconds whether or not there is anything to compare against yet.
1384+
reportInvariants();
13611385
await setup();
13621386
const parse5 = /** @type {Parse5} */ (load("parse5"));
13631387
for (const [label, html] of (await fixtures()).filter(([name]) =>
@@ -1450,12 +1474,12 @@ if (require.main === module) {
14501474
// `--setup` installs the fixtures and builds nothing else, so a consumer
14511475
// that only reads them does not run the comparison to get them.
14521476
const mode = process.argv[2];
1477+
// The sweep alone, for a caller that wants the relations without the ten
1478+
// minutes the comparison costs; a full run prints the same section.
14531479
if (mode === "--invariants") {
1454-
const found = invariants((text) => process.stdout.write(text));
1455-
log(`\n${found} finding${found === 1 ? "" : "s"}`);
14561480
// Non-zero while any finding stands, and findings stand today: read it
14571481
// rather than gating on it until they are gone.
1458-
process.exitCode = found > 0 ? 1 : 0;
1482+
process.exitCode = reportInvariants() > 0 ? 1 : 0;
14591483
} else {
14601484
const started =
14611485
mode === "--measure"

0 commit comments

Comments
 (0)