Skip to content

Commit 15beefc

Browse files
build(tooling): compare CSS and HTML parsers and printers, not just minifiers (#22007)
* build(tooling): compare CSS and HTML parsers and printers, not just minifiers `compare-css-minifiers.js` and `compare-html-minifiers.js` become `compare-css-tools.js` and `compare-html-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`), sharing the measuring harness they duplicated. Each fixture now prints three tables rather than one: parsing alone, parsing and printing readably, and parsing and printing minified. What a parse costs is then separated from what printing and the transforms on top of it cost, and a parser that ships no minifier is measured for the first time — postcss, css-tree, stylis, @adobe/css-tools and crass on the CSS side, parse5, htmlparser2, node-html-parser, linkedom and angular-html-parser on the HTML one, plus prettier and js-beautify as printers. A round-trip printer that reformats nothing sits in each beautify table as the floor the rest are read against: postcss and parse5 both hit it. Two measurement bugs came out of the rewrite. The peak-memory column reported the parent's high-water mark, not the tool's: Linux carries `maxRSS` across `fork`+`exec`, so every worker inherited whatever the harness had reached, and the rows climbed in lockstep with it. It reads `/proc/self/status` now. And loading a comparison package by joined path skips its `exports` map, so a package shipping one and no `main` read as missing; they resolve as bare specifiers from the cache instead. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. * build(tooling): say when a tool spends its cost in a service process esbuild transforms in a child process of its own, so `cpuUsage` and `VmHWM` both miss it: 25 ms of wall time reported 1.6 ms of cpu and a peak that was only this worker's. Those two cells now read `-`, and the row says why. * build(tooling): name the tool when a worker's output will not parse Parsing straight into `resolve` threw inside the close listener, so a tool that wrote anything of its own to stdout ended the whole run with a syntax error naming no tool, no fixture and no stage. * test(tooling): cover what the comparison harness does with a bad worker Both messages are asserted whole rather than by substring: 200 characters are a substring of 500, so a truncation that never happened would pass either way. * test(tooling): drive the comparison harness itself, not only its worker Requiring the harness from a test instruments it, so its own lines became a patch codecov measures — and a spawned worker earns none of them back, since coverage does not follow a child. `measure` runs in this process instead. What is left uncovered cannot be reached from a unit test: the npm install a warm cache skips, and the fallback for a platform without `/proc`. * test(tooling): reach the install the comparison cache skips when it is warm An `npm` of our own first on `PATH` stands in for the real one, so the orchestration is exercised without fetching anything: the cache is created, the installer is run, and the package list is recorded only after it exits cleanly — a list written any earlier would let a broken cache pass for warm. * docs(tooling): say what the peak measurement actually returns It never answers 0: where `/proc/self/status` is not there to read, the platform's own `maxRSS` is, and that is what comes back. * test(tooling): keep the install cases away from the real npm The stand-in npm was put first on `PATH`, but jest hands a test file a copy of `process.env`, and `child_process` reads the real one when no env is given — so both install cases spawned the real npm and installed over the network. `run` now hands the child the environment this process holds, the stand-in leaves a mark in the cache it was called for, and each case asserts whether that mark is there. The caches are also cleared before the run, so one left behind by an interrupted run cannot send the cold case down the warm path. * test(tooling): measure a peak where there is no status file to read The fallback to what the platform accounts for was the one line the suite never reached, so a machine without `/proc/self/status` was covered by nothing. Both ways of not getting a high-water mark out of it are driven now: the file missing, and the file naming no `VmHWM`. * test(tooling): name the command a failed spawn could not find Bun answers a missing executable with "Executable not found in $PATH" and node with the errno, so asserting `ENOENT` failed the bun runtime job. The name is in both messages.
1 parent b2815d2 commit 15beefc

8 files changed

Lines changed: 1489 additions & 694 deletions

‎AGENTS.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ The directory listings below are the canonical map of the repository. **Whenever
9292
- `lib/wasm/`, `lib/wasm-async/`, `lib/wasm-sync/` — WebAssembly module support.
9393
- `hot/` — Runtime code shipped to browsers for HMR (browser-side, not Node tooling).
9494
- `bin/` — `webpack` CLI entry point.
95-
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-minifiers.js` / `compare-html-minifiers.js` (`yarn benchmark:css-minifiers`, `yarn benchmark:html-minifiers`). Those two need no arguments and no reading of their source: each runs webpack's CSS/HTML minifier and the ecosystem's over popular framework stylesheets and real documents, printing one table per fixture — output size raw and under gzip/brotli/zstd (the `test:size` settings), best-of-3 wall and cpu ms, peak RSS (each minifier × fixture measured in its own worker process, so the numbers are attributable), and whether the output lost classes / changed the DOM ("rejects it" rows mean the tool errored on that input). They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take a few. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand.
95+
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-tools.js` / `compare-html-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`), which share the measuring harness in `compare-tools-harness.js`. Those two need no arguments and no reading of their source: each runs webpack's CSS/HTML parser and printer and the ecosystem's over popular framework stylesheets and real documents, printing **three tables per fixture** — parsing alone, parsing and printing readably (`beautify`), and parsing and printing minified (`minify`) — so what a parse costs is separated from what the printing and the transforms on top of it cost. Every table reports best-of-3 wall and cpu ms and the worker's own peak RSS (each tool × fixture measured in its own process, so the numbers are attributable; read from `/proc/self/status`, since Linux carries `maxRSS` across `fork`+`exec` and a worker asking for its own would report the parent's). **Read wall and cpu together**: cpu above wall is a tool using more than one core (V8's background threads, a native thread pool), cpu below it is a tool waiting rather than computing. A tool that works in a service process of its own says so in its name and reports `-` for cpu and peak, which are spent where nothing here can see them — esbuild is the one today. The two printing tables add output size raw and under gzip/brotli/zstd (the `test:size` settings) and whether the output lost classes / changed the DOM ("rejects it" rows mean the tool errored on that input). A round-trip printer that reformats nothing — postcss for CSS, parse5 for HTML — sits in the `beautify` table as the floor the others are read against. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take upwards of ten. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand.
9696
- `assembly/` — WebAssembly source for the hash function.
9797
- `setup/` — One-time setup. `setup.js` (`yarn setup`) is the only entry point and picks its own path: a contributor at a terminal gets the interactive one, which installs yarn when it is missing and links through yarn's registry, while everything else gets the non-interactive one — it verifies the lockfile instead of rewriting it, installs no global yarn, and links the checkout in as `node_modules/webpack` without touching yarn's machine-global registry. What it reads is both streams being a terminal plus an unset `CI`, never a list of vendor variables, so an agent nobody has heard of yet takes the safe path — but one that allocates a PTY reads as a contributor, and should set `WEBPACK_SETUP=automated`, which with `interactive` forces either path. Safe to re-run.
9898

@@ -283,7 +283,7 @@ A perf/memory claim needs evidence, and the cheap kinds are the trustworthy ones
283283

284284
`FILTER="<case-name>" yarn benchmark` drives the repo's own cases; `test/benchmarkCases/` is the fixture set.
285285

286-
A claim about **webpack's CSS or HTML minifier versus the ecosystem's** (size, speed, memory, or safety) is already harnessed: run `yarn benchmark:css-minifiers` / `yarn benchmark:html-minifiers` and read the tables — see the `tooling/` entry in [Architecture](#architecture) for what they report — rather than hand-rolling a comparison.
286+
A claim about **webpack's CSS or HTML minifier versus the ecosystem's** (size, speed, memory, or safety) is already harnessed: run `yarn benchmark:css-tools` / `yarn benchmark:html-tools` and read the tables — see the `tooling/` entry in [Architecture](#architecture) for what they report — rather than hand-rolling a comparison.
287287

288288
A claim about the **size of what webpack emits** is the counting kind, and `yarn test:size` is how it is counted: it builds every `configCases/` case with the defaults a user gets and reports the raw/gzip/brotli/zstd size of every asset, so a change to `lib/runtime/` or to a dependency template shows up as bytes on the wire. Compare two runs with `--baseline <report>`; the `Code Size` CI job does the same against the report `main` last uploaded and comments the diff on the pull request.
289289

@@ -292,7 +292,7 @@ A claim about the **size of what webpack emits** is the counting kind, and `yarn
292292
The two columns disagree along one seam, and knowing which side a change is on saves measuring it:
293293

294294
- **Removing information** — a dead rule, a duplicate declaration, a longhand its shorthand implies, whitespace, a comment — shrinks raw and normally the compressed columns with it, so it needs no routine check. Not a guarantee: a small removal can delete a run a later match pointed back to, so read the compressed column when the removal is small.
295-
- **Re-encoding it** — swapping one spelling for a shorter one — is where they part, because a compressor rewards repetition and a short-but-novel token widens the literal alphabet. **A re-encoding has to show a compressed win to be worth taking.** Both of the refusals webpack ships on size grounds rather than correctness ones are this shape: `output.environment.convertLengthUnits` (`16px` → `1pc`) is off by default because it earns nothing compressed, and a string's hex escape is left as written because writing the character it names saves 6422 raw bytes and costs 595 gzip ones over the `benchmark:css-minifiers` fixtures — the same trade that makes a rival 3.8 KB smaller raw on Font Awesome and 0.3 KB bigger gzipped.
295+
- **Re-encoding it** — swapping one spelling for a shorter one — is where they part, because a compressor rewards repetition and a short-but-novel token widens the literal alphabet. **A re-encoding has to show a compressed win to be worth taking.** Both of the refusals webpack ships on size grounds rather than correctness ones are this shape: `output.environment.convertLengthUnits` (`16px` → `1pc`) is off by default because it earns nothing compressed, and a string's hex escape is left as written because writing the character it names saves 6422 raw bytes and costs 595 gzip ones over the `benchmark:css-tools` fixtures — the same trade that makes a rival 3.8 KB smaller raw on Font Awesome and 0.3 KB bigger gzipped.
296296

297297
This is a rule for which number a change is accepted on at review time, not something a printer can consult: no minifier runs a compressor in its inner loop, and neither should ours.
298298

‎cspell.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,7 @@
223223
"libuv",
224224
"lightgoldenrodyellow",
225225
"lightningcss",
226+
"linkedom",
226227
"lintunit",
227228
"loadername",
228229
"loglevel",
@@ -437,6 +438,7 @@
437438
"strtok",
438439
"styleset",
439440
"stylesheet",
441+
"stylis",
440442
"subdir",
441443
"subfolder",
442444
"subgrid",

‎eslint.config.mjs‎

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -226,12 +226,12 @@ export default defineConfig([
226226
}
227227
},
228228
{
229-
// The minifier comparisons run on a developer's or CI's modern Node.js,
230-
// not on webpack's runtime baseline: they measure with post-10 APIs
231-
// (fs.promises, zstd, resourceUsage).
229+
// The tool comparisons run on modern Node, not webpack's runtime baseline:
230+
// they measure with post-10 APIs (fs.promises, zstd, resourceUsage).
232231
files: [
233-
"tooling/compare-css-minifiers.js",
234-
"tooling/compare-html-minifiers.js"
232+
"tooling/compare-css-tools.js",
233+
"tooling/compare-html-tools.js",
234+
"tooling/compare-tools-harness.js"
235235
],
236236
rules: {
237237
"n/no-unsupported-features/node-builtins": "off"
@@ -242,10 +242,7 @@ export default defineConfig([
242242
// through a dynamic import rather than a `require` no jest `vm` supports.
243243
// `html-minifier-next` is ESM only, and the HTML comparison imports it the
244244
// same way.
245-
files: [
246-
"tooling/generate-css-data.js",
247-
"tooling/compare-html-minifiers.js"
248-
],
245+
files: ["tooling/generate-css-data.js", "tooling/compare-html-tools.js"],
249246
rules: {
250247
"n/no-unsupported-features/es-syntax": [
251248
"error",

‎package.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,8 @@
5353
"fix:serializables": "node tooling/generate-internal-serializables.js --write",
5454
"build:examples": "cd examples && node buildAll.js",
5555
"benchmark": "node --max-old-space-size=6144 --experimental-vm-modules --trace-deprecation --hash-seed=1 --random-seed=1 --no-opt --predictable --predictable-gc-schedule --interpreted-frames-native-stack --allow-natives-syntax --expose-gc --no-concurrent-sweeping ./test/BenchmarkTestCases.benchmark.mjs",
56-
"benchmark:css-minifiers": "node tooling/compare-css-minifiers.js",
57-
"benchmark:html-minifiers": "node tooling/compare-html-minifiers.js",
56+
"benchmark:css-tools": "node tooling/compare-css-tools.js",
57+
"benchmark:html-tools": "node tooling/compare-html-tools.js",
5858
"measure:color-agreement": "node tooling/measure-color-agreement.js",
5959
"pretest": "yarn lint",
6060
"test": "yarn test:base",

0 commit comments

Comments
 (0)