Skip to content

Commit af8bf0b

Browse files
build(tooling): compare JavaScript parsers and printers, and widen the CSS and HTML fixtures (#22010)
* build(tooling): compare JavaScript parsers and printers, and hold them to acorn The CSS and HTML comparisons already weigh what webpack's own parser and printer cost against the ecosystem's. This is the same three tables for JavaScript, over fourteen bundles as they ship — jQuery through the 8.9 MB TypeScript one, ES5 and modern, script and module. webpack owns a parser there and no printer, so it has a row in the parse table only, and that table's last column is what the comparison is for: every ESTree parser is held against acorn's tree and the first node where the two differ is named, so a divergence in `lib/javascript/syntax.js` is a line of output rather than a bug someone hits later. A parser answering in a dialect of its own reads as `own dialect` instead of as a disagreement. * build(tooling): widen what the comparisons are measured over Every fixture so far was source as its author wrote it, which is not the shape most of what a build reads is in. The JavaScript comparison gains minified bundles (jQuery, React DOM and Vue as they ship to production, plus Swagger UI's own bundle), a modern ESM one written in classes and private fields (pdf.js), the smallest module worth timing (Preact) and two more that dwarf the rest (ECharts, Immutable). CSS gains the shapes the framework sheets do not have: an icon font's thousands of one-declaration rules, a sheet that is custom properties and nothing else, and four more component and widget sheets. HTML gains the parts of §13.2 nothing there reached — a document that is all tables, one that leans on every recovery rule the spec writes down, one built out of templates and a declarative shadow root, and an inlined icon sprite for foreign content. * build(tooling): read what oxc refused rather than what it recovered oxc does not throw on a file it cannot parse: it answers with whatever tree or text it salvaged and the diagnostics beside it. The parse row then read a refusal as a tree that merely disagreed with acorn's, and the minify row read one as an empty output — which is how `oxc-minify` first came out at 0.0 KB with every property name lost. Both rows raise the first diagnostic now, so a refusal reads as one.
1 parent 1861d25 commit af8bf0b

7 files changed

Lines changed: 941 additions & 2 deletions

File tree

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ The directory listings below are the canonical map of the repository. **Whenever
9292
- `lib/wasm/`, `lib/wasm-async/`, `lib/wasm-sync/` — WebAssembly module support.
9393
- `hot/` — Runtime code shipped to browsers for HMR (browser-side, not Node tooling).
9494
- `bin/` — `webpack` CLI entry point.
95-
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-tools.js` / `compare-html-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`), which share the measuring harness in `compare-tools-harness.js`. Those two need no arguments and no reading of their source: each runs webpack's CSS/HTML parser and printer and the ecosystem's over popular framework stylesheets and real documents, printing **three tables per fixture** — parsing alone, parsing and printing readably (`beautify`), and parsing and printing minified (`minify`) — so what a parse costs is separated from what the printing and the transforms on top of it cost. Every table reports best-of-3 wall and cpu ms and the worker's own peak RSS (each tool × fixture measured in its own process, so the numbers are attributable; read from `/proc/self/status`, since Linux carries `maxRSS` across `fork`+`exec` and a worker asking for its own would report the parent's). **Read wall and cpu together**: cpu above wall is a tool using more than one core (V8's background threads, a native thread pool), cpu below it is a tool waiting rather than computing. A tool that works in a service process of its own says so in its name and reports `-` for cpu and peak, which are spent where nothing here can see them — esbuild is the one today. The two printing tables add output size raw and under gzip/brotli/zstd (the `test:size` settings) and whether the output lost classes / changed the DOM ("rejects it" rows mean the tool errored on that input). A round-trip printer that reformats nothing — postcss for CSS, parse5 for HTML — sits in the `beautify` table as the floor the others are read against. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take upwards of ten. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand.
95+
- `tooling/` — Repo-internal scripts: build/codegen (runtime/wasm generators, hash-debug tool) invoked by `yarn fix:special`, plus standalone analysis tools such as `compare-css-tools.js` / `compare-html-tools.js` / `compare-js-tools.js` (`yarn benchmark:css-tools`, `yarn benchmark:html-tools`, `yarn benchmark:js-tools`), which share the measuring harness in `compare-tools-harness.js`. Those three need no arguments and no reading of their source: each runs what webpack owns for that language and the ecosystem's equivalents over popular framework stylesheets, real documents and shipped JavaScript bundles, printing **three tables per fixture** — parsing alone, parsing and printing readably (`beautify`), and parsing and printing minified (`minify`) — so what a parse costs is separated from what the printing and the transforms on top of it cost. Every table reports best-of-3 wall and cpu ms and the worker's own peak RSS (each tool × fixture measured in its own process, so the numbers are attributable; read from `/proc/self/status`, since Linux carries `maxRSS` across `fork`+`exec` and a worker asking for its own would report the parent's). **Read wall and cpu together**: cpu above wall is a tool using more than one core (V8's background threads, a native thread pool), cpu below it is a tool waiting rather than computing. A tool that works in a service process of its own says so in its name and reports `-` for cpu and peak, which are spent where nothing here can see them — esbuild is the one today. The two printing tables add output size raw and under gzip/brotli/zstd (the `test:size` settings) and whether the output lost classes / changed the DOM / stopped naming a property ("rejects it" rows mean the tool errored on that input). The JavaScript one is shaped by webpack owning a parser there and no printer: webpack has a row in the parse table only, and that table's last column holds every ESTree parser against acorn's tree and names the first node where the two differ — which is how a divergence in `lib/javascript/syntax.js` is caught — while a parser answering in a dialect of its own reads as `own dialect` rather than as a disagreement. A round-trip printer that reformats nothing — postcss for CSS, parse5 for HTML — sits in the `beautify` table as the floor the others are read against. `FIXTURE=`, `TOOL=` and `STAGE=` narrow a run to matching rows. They install the packages they compare against into `node_modules/.cache/` on first run rather than into webpack's dependencies; expect the first run to install for a minute and every full run to take upwards of ten. `check-comment-length.js` (`yarn lint:comments`) reports the plain comments a change grows past the two-line limit of [Code comments](#code-comments). `measure-color-agreement.js` (`yarn measure:color-agreement`) is the third: it asks a real browser for its own color conversions rather than for a pixel, and prints how far they sit from webpack's — which is where the rounding margins in `lib/css/syntax.js` and the list of spaces an engine reads through another transfer come from. Re-run it (a few seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand.
9696
- `assembly/` — WebAssembly source for the hash function.
9797
- `setup/` — One-time setup. `setup.js` (`yarn setup`) is the only entry point and picks its own path: a contributor at a terminal gets the interactive one, which installs yarn when it is missing and links through yarn's registry, while everything else gets the non-interactive one — it verifies the lockfile instead of rewriting it, installs no global yarn, and links the checkout in as `node_modules/webpack` without touching yarn's machine-global registry. What it reads is both streams being a terminal plus an unset `CI`, never a list of vendor variables, so an agent nobody has heard of yet takes the safe path — but one that allocates a PTY reads as a contributor, and should set `WEBPACK_SETUP=automated`, which with `interactive` forces either path. Safe to re-run.
9898

@@ -283,7 +283,7 @@ A perf/memory claim needs evidence, and the cheap kinds are the trustworthy ones
283283

284284
`FILTER="<case-name>" yarn benchmark` drives the repo's own cases; `test/benchmarkCases/` is the fixture set.
285285

286-
A claim about **webpack's CSS or HTML minifier versus the ecosystem's** (size, speed, memory, or safety) is already harnessed: run `yarn benchmark:css-tools` / `yarn benchmark:html-tools` and read the tables — see the `tooling/` entry in [Architecture](#architecture) for what they report — rather than hand-rolling a comparison.
286+
A claim about **webpack's CSS or HTML minifier, or its JavaScript parser, versus the ecosystem's** (size, speed, memory, or safety) is already harnessed: run `yarn benchmark:css-tools` / `yarn benchmark:html-tools` / `yarn benchmark:js-tools` and read the tables — see the `tooling/` entry in [Architecture](#architecture) for what they report — rather than hand-rolling a comparison.
287287

288288
A claim about the **size of what webpack emits** is the counting kind, and `yarn test:size` is how it is counted: it builds every `configCases/` case with the defaults a user gets and reports the raw/gzip/brotli/zstd size of every asset, so a change to `lib/runtime/` or to a dependency template shows up as bytes on the wire. Compare two runs with `--baseline <report>`; the `Code Size` CI job does the same against the report `main` last uploaded and comments the diff on the pull request.
289289

‎cspell.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
"argjson",
1919
"asmjs",
2020
"assemblyscript",
21+
"astring",
2122
"asyncloader",
2223
"atlaskit",
2324
"autocrlf",
@@ -207,6 +208,7 @@
207208
"Junya",
208209
"jwalton",
209210
"kaios",
211+
"katex",
210212
"Kees",
211213
"khtml",
212214
"kibibytes",
@@ -269,6 +271,7 @@
269271
"moji",
270272
"MONEI",
271273
"mpath",
274+
"mrow",
272275
"mult",
273276
"multiplicator",
274277
"mycompany",

‎eslint.config.mjs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,7 @@ export default defineConfig([
231231
files: [
232232
"tooling/compare-css-tools.js",
233233
"tooling/compare-html-tools.js",
234+
"tooling/compare-js-tools.js",
234235
"tooling/compare-tools-harness.js"
235236
],
236237
rules: {

‎package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@
5555
"benchmark": "node --max-old-space-size=6144 --experimental-vm-modules --trace-deprecation --hash-seed=1 --random-seed=1 --no-opt --predictable --predictable-gc-schedule --interpreted-frames-native-stack --allow-natives-syntax --expose-gc --no-concurrent-sweeping ./test/BenchmarkTestCases.benchmark.mjs",
5656
"benchmark:css-tools": "node tooling/compare-css-tools.js",
5757
"benchmark:html-tools": "node tooling/compare-html-tools.js",
58+
"benchmark:js-tools": "node tooling/compare-js-tools.js",
5859
"measure:color-agreement": "node tooling/measure-color-agreement.js",
5960
"pretest": "yarn lint",
6061
"test": "yarn test:base",

‎tooling/compare-css-tools.js‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ const PACKAGES = [
6161
"animate.css@4",
6262
"beercss@5",
6363
"bootstrap@5",
64+
"bootstrap-icons@1",
6465
"bulma@1",
6566
"clean-css@5",
6667
@@ -73,21 +74,26 @@ const PACKAGES = [
7374
"@fortawesome/fontawesome-free@6",
7475
"fomantic-ui-css@2",
7576
"foundation-sites@6",
77+
7678
"lightningcss@1",
7779
"materialize-css@1",
7880
"milligram@1",
81+
"nes.css@2",
7982
"normalize.css@8",
83+
"open-props@1",
8084
"@patternfly/patternfly@6",
8185
"@picocss/pico@2",
8286
"postcss@8",
8387
"postcss-selector-parser@7",
8488
"prettier@3",
8589
"@primer/css@21",
8690
"purecss@3",
91+
"@radix-ui/themes@3",
8792
"sanitize.css@13",
8893
"@shoelace-style/shoelace@2",
8994
9095
"stylis@4",
96+
"swiper@14",
9197
9298
"semantic-ui-css@2",
9399
"@tabler/core@1",
@@ -96,6 +102,7 @@ const PACKAGES = [
96102
"@tdewolff/minify@2",
97103
"@tailwindcss/cli@4",
98104
"uikit@3",
105+
"video.js@8",
99106
"water.css@2"
100107
];
101108

@@ -151,31 +158,41 @@ const setup = async () => {
151158

152159
// Every installed stylesheet the comparison runs on: component frameworks, the
153160
// classless ones, and icon/animation/reset sheets that look nothing like them.
161+
162+
// The odd shapes earn their place: an icon font is thousands of one-declaration
163+
// rules, and Open Props is custom properties and almost nothing else.
154164
/** @type {[string, string][]} */
155165
const INSTALLED_FIXTURES = [
156166
["98.css", "98.css/dist/98.css"],
157167
["Animate.css 4", "animate.css/animate.css"],
158168
["Beer CSS 5", "beercss/dist/cdn/beer.css"],
159169
["Bootstrap 5 (full)", "bootstrap/dist/css/bootstrap.css"],
160170
["Bootstrap 5 (grid)", "bootstrap/dist/css/bootstrap-grid.css"],
171+
["Bootstrap Icons 1", "bootstrap-icons/font/bootstrap-icons.css"],
161172
["Bulma 1", "bulma/css/bulma.css"],
162173
["Fomantic-UI 2", "fomantic-ui-css/semantic.css"],
163174
["Font Awesome 6", "@fortawesome/fontawesome-free/css/all.css"],
164175
["Foundation 6", "foundation-sites/dist/css/foundation.css"],
176+
["KaTeX 0.18", "katex/dist/katex.css"],
165177
["Materialize 1", "materialize-css/dist/css/materialize.css"],
166178
["Milligram 1", "milligram/dist/milligram.css"],
179+
["NES.css 2", "nes.css/css/nes.css"],
167180
["normalize.css 8", "normalize.css/normalize.css"],
181+
["Open Props 1 (minified)", "open-props/open-props.min.css"],
168182
["PatternFly 6", "@patternfly/patternfly/patternfly-base.css"],
169183
["Pico 2", "@picocss/pico/css/pico.css"],
170184
["Primer 21", "@primer/css/dist/primer.css"],
171185
["Pure 3", "purecss/build/pure.css"],
186+
["Radix Themes 3 (components)", "@radix-ui/themes/components.css"],
172187
["sanitize.css 13", "sanitize.css/sanitize.css"],
173188
["Semantic UI 2", "semantic-ui-css/semantic.css"],
174189
["Shoelace 2 (light)", "@shoelace-style/shoelace/dist/themes/light.css"],
175190
["Spectre 0.5", "spectre.css/dist/spectre.css"],
191+
["Swiper 14", "swiper/swiper-bundle.css"],
176192
["Tabler 1", "@tabler/core/dist/css/tabler.css"],
177193
["Tachyons 4", "tachyons/css/tachyons.css"],
178194
["UIkit 3", "uikit/dist/css/uikit.css"],
195+
["Video.js 8", "video.js/dist/video-js.css"],
179196
["Water.css 2", "water.css/out/water.css"]
180197
];
181198

‎tooling/compare-html-tools.js‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ const PACKAGES = [
5151
"angular-html-parser@9",
5252
"bootstrap@5",
5353
"dom-serializer@2",
54+
"@fortawesome/fontawesome-free@6",
5455
"htmlparser2@10",
5556
"js-beautify@1",
5657
"@minify-html/[email protected]",
@@ -231,6 +232,100 @@ ${css}
231232
</body>
232233
</html>`;
233234

235+
/**
236+
* A page of nothing but tables: rows a parser has to put in an implied
237+
* `<tbody>`, cells that span, and a `<caption>` and `<colgroup>` before them.
238+
* @param {number} rows how many rows to write
239+
* @returns {string} the page
240+
*/
241+
const tablePage = (rows) => {
242+
let body = "";
243+
for (let i = 0; i < rows; i++) {
244+
body += `\t\t<tr><th scope="row">Row ${i}</th><td>${i}</td><td colspan="2">${
245+
i * 2
246+
}</td><td><a href="/row/${i}">open</a></td></tr>\n`;
247+
}
248+
return `<!DOCTYPE html>
249+
<html lang="en">
250+
<head><meta charset="utf-8"><title>Report</title></head>
251+
<body>
252+
<table>
253+
\t<caption>Rows</caption>
254+
\t<colgroup><col><col span="2" class="numeric"><col></colgroup>
255+
\t<thead><tr><th scope="col">Name</th><th scope="col">A</th><th scope="col">B</th><th scope="col">Link</th></tr></thead>
256+
${body}\t<tfoot><tr><td colspan="4">${rows} rows</td></tr></tfoot>
257+
</table>
258+
</body>
259+
</html>`;
260+
};
261+
262+
// Every shape §13.2 has a recovery rule for: implied end tags, formatting
263+
// elements reopened across a block, and text a table fosters out of itself.
264+
const TAG_SOUP = `<!DOCTYPE html>
265+
<html>
266+
<head><title>Soup</title>
267+
<body>
268+
<p>One
269+
<p>Two<b>bold<i>both</b>italic</i>
270+
<ul>
271+
<li>first
272+
<li>second<div>block inside a list item
273+
<li>third
274+
</ul>
275+
<table>fostered text<tr><td>cell<td>next
276+
<tr><th>head
277+
</table>
278+
<div><span>unclosed
279+
</div>
280+
</p></span>
281+
<form><form><input name=a><button>go
282+
<select><option>a<option>b</select>
283+
<a href="/x"><a href="/y">nested anchors</a>
284+
<font size=3><p>font across a paragraph</font>
285+
</body>`;
286+
287+
// A page as a component library ships it: templates that are not rendered,
288+
// a declarative shadow root, and elements the parser knows nothing about.
289+
const WEB_COMPONENTS = `<!DOCTYPE html>
290+
<html lang="en">
291+
<head><meta charset="utf-8"><title>Elements</title></head>
292+
<body>
293+
<my-app theme="dark">
294+
\t<template shadowrootmode="open">
295+
\t\t<style>:host{display:block}::slotted(p){margin:0}</style>
296+
\t\t<slot name="header"></slot>
297+
\t\t<slot></slot>
298+
\t</template>
299+
\t<h1 slot="header">Title</h1>
300+
\t<p>Light DOM child</p>
301+
</my-app>
302+
<template id="row">
303+
\t<tr><td><slot name="cell"></slot></td></tr>
304+
</template>
305+
<ul is="sortable-list" data-sort="asc">
306+
\t<li><x-item value="1">One</x-item></li>
307+
\t<li><x-item value="2">Two</x-item></li>
308+
</ul>
309+
<script type="module">customElements.define("x-item", class extends HTMLElement {});</script>
310+
</body>
311+
</html>`;
312+
313+
/**
314+
* @param {string} sprite an SVG document
315+
* @returns {string} it inlined into a page, as an icon sprite is used
316+
*/
317+
const spritePage = (sprite) => `<!DOCTYPE html>
318+
<html lang="en">
319+
<head><meta charset="utf-8"><title>Icons</title></head>
320+
<body>
321+
<div hidden>
322+
${sprite}
323+
</div>
324+
<p><svg width="16" height="16" aria-hidden="true"><use href="#user"></use></svg> Inline reference</p>
325+
<math><mrow><mi>x</mi><mo>+</mo><mn>1</mn></mrow></math>
326+
</body>
327+
</html>`;
328+
234329
/**
235330
* Two kinds of real HTML: an app shell (attribute- and `<meta>`-heavy, little
236331
* text) and a document (mostly text, with the `<pre>` blocks whose whitespace no
@@ -270,10 +365,23 @@ const fixtures = async () => {
270365
}
271366
for (const [label, file] of [
272367
["webpack README (rendered)", path.join(ROOT, "README.md")],
368+
["webpack AGENTS (rendered)", path.join(ROOT, "AGENTS.md")],
273369
["webpack CHANGELOG (rendered)", path.join(ROOT, "CHANGELOG.md")]
274370
]) {
275371
out.push([label, marked.parse(await fs.promises.readFile(file, "utf8"))]);
276372
}
373+
out.push(["Table report (600 rows)", tablePage(600)]);
374+
out.push(["Tag soup", TAG_SOUP]);
375+
out.push(["Web components", WEB_COMPONENTS]);
376+
out.push([
377+
"Icon sprite (inlined SVG)",
378+
spritePage(
379+
await fs.promises.readFile(
380+
path.join(MODULES, "@fortawesome/fontawesome-free/sprites/solid.svg"),
381+
"utf8"
382+
)
383+
)
384+
]);
277385
return out;
278386
};
279387
// Each entry builds its callable on demand, so the measuring worker loads only

0 commit comments

Comments
 (0)