Skip to content

Commit 3ff0051

Browse files
refactor(html): split syntax into syntax-parser and syntax-printer (#22223)
* refactor(html): split syntax into syntax-parser and syntax-printer The tokenizer, tree construction and entity table now live in syntax-parser.js and the default node printer in syntax-printer.js, with lib/html/syntax.js keeping the public surface: it loads both, binds SourceProcessor to them and re-exports what it always did. Module state the walk writes and the printer reads moved into one initialized object, so the two files share it without a stale copy. generate-html-data.js writes the entity region into the parser file. * refactor(html): reach the halves through getters, as javascript does syntax.js now exposes `parser` and `printer` the way lib/javascript's facade does, and every name it published before is a getter onto the half that owns it. Nothing loads until a name is read: requiring the facade loads neither half, and a walk that only fires visitors never loads the printer. SourceProcessor asks for its printer on the first print rather than naming it up front — util/SourceProcessor takes that loader as a third argument and calls it once per process, never per node. HtmlParser and HtmlModulesPlugin now import the parser half directly. `html.syntax.printer` is the printer half now, as in javascript; the print function it holds is `printer.printer`. * test(html): cover the printer a processor asks for on its first print * test: prove a supplied printer is not re-asked for through the loader * test(html): assert the whole facade surface, not two names of it The surface test read `parseHtml` and `NodeType`, leaving the other 33 getters unexecuted. Walk every published name instead: each resolves to a defined value, and each one the parser half spells the same way is identical to it. * test(html): state each facade binding instead of counting them Deriving the expected names from the facade only checked how many there are: a renamed getter, an alias repointed at another export of the same module, and a printer-owned name bound to the parser all kept the counts at 35 and 25. Spell out the source of each of the 35.
1 parent 13ec116 commit 3ff0051

18 files changed

Lines changed: 15086 additions & 14308 deletions
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": patch
3+
---
4+
5+
Split the HTML syntax module into a parser file and a printer file.

‎AGENTS.md‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,12 @@ The directory listings below are the canonical map of the repository. **Whenever
103103
through in `lib/template/` — this holds the base classes every build has.
104104
- `lib/hmr/` — Hot Module Replacement: `HotModuleReplacementPlugin` and the runtime
105105
modules, lazy-compilation backend and helpers it drives.
106-
- `lib/html/` — Experimental HTML support.
106+
- `lib/html/` — Experimental HTML support. `syntax.js` names its two halves the
107+
way `javascript` does: `syntax-parser.js` reads a document — tokenizer, §13.2
108+
tree construction and the entity table — and `syntax-printer.js` writes one back
109+
out. `syntax.js` reaches both through a getter, as `javascript` does, so a walk
110+
that never prints never loads the printer; it holds `SourceProcessor` and every
111+
name it published before.
107112
- `lib/ids/` — Module/chunk id assignment plugins, and `RecordIdsPlugin`, which persists
108113
the assignment across builds through `recordsPath`.
109114
- `lib/javascript/` — JavaScript parsing (webpack's own ECMAScript parser, ported from acorn), generation, exports analysis. `syntax.js` is the pair `css` and `html` name the same way — `parser` and `printer` — and reaches each through a getter, so parsing never loads the printer and printing never loads the parser. `syntax-parser.js` is the parser a build reads source with: tokenizer, acorn-derived core and every production in one file, since a build that parses at all reaches the productions, and the struct-of-arrays rewrite ahead of it moves node creation through them. `regexp.js` (the pattern validator) is the one piece still loaded on demand, because only a pattern the host engine itself rejected reaches it — never `require` it from a path a build takes. `syntax-printer.js` is where JavaScript is printed back out: `jsMinify.js`, the `minify` function the default minimizer dispatches JavaScript to, goes through it rather than through terser's published entry point, because that loader reads terser's own sources — which is what lets a phase webpack implements itself replace the method terser installs. The name says where this is going: each phase webpack takes over is one less thing terser does. A phase states what it reads with `supports`, and a minifier that moved any of it, or a runtime that cannot import those sources, keeps its own. Add a phase to the `PHASES` list there and nowhere else, and hold it to writing byte-for-byte what it replaced.
@@ -737,10 +742,10 @@ These files are produced by `yarn fix:special` and must not be edited by hand:
737742
- `schemas/**/*.check.{js,d.ts}` — precompiled schema validators.
738743
- Generated runtime code under `lib/` (driven by `tooling/generate-runtime-code.js`).
739744
- `lib/css/data.js` — every table the CSS minifier looks a name up in, and the arithmetic its math-function descriptors bind to: derived from `mdn-data` + `color-name` (box shorthands, color-argument and math functions, named colors) plus the generator's `SUPPLEMENT` of spec-prose tables and math primitives, by `tooling/generate-css-data.js` — which also holds the value-definition-syntax parser those grammars are read with, and runs the generation only as the entry point so its tests can require it.
740-
- `lib/html/data.js` — every table the HTML parser and minifier look a name up in: the reflected-attribute tables distilled from webref's HTML IDL (the `@webref/idl` and `@webref/elements` packages), plus the generator's `SUPPLEMENT` and `PARSER_TABLES` of §13.2 tree-construction vocabulary, by `tooling/generate-html-data.js` — which also emits the `// #region html entities` block inside `lib/html/syntax.js` from the vendored `tooling/html-entities.json`, WHATWG's own named character references table.
745+
- `lib/html/data.js` — every table the HTML parser and minifier look a name up in: the reflected-attribute tables distilled from webref's HTML IDL (the `@webref/idl` and `@webref/elements` packages), plus the generator's `SUPPLEMENT` and `PARSER_TABLES` of §13.2 tree-construction vocabulary, by `tooling/generate-html-data.js` — which also emits the `// #region html entities` block inside `lib/html/syntax-parser.js` from the vendored `tooling/html-entities.json`, WHATWG's own named character references table.
741746
- `lib/javascript/data.js` — the Unicode tables the JavaScript parser classifies with, in one module: the run-length identifier ranges the tokenizer decodes on its first non-ASCII code point, and the per-edition `\p{...}` property names only a pattern the engine itself rejected reaches, by `tooling/generate-js-data.js` — which reads both back out of the pinned acorn devDependency, so bumping it is what moves them.
742747

743-
The three `syntax.js` files are algorithm only — a new lookup table belongs in the matching generator, not next to the code that reads it.
748+
A `syntax-parser.js` or `syntax-printer.js` is algorithm only — a new lookup table belongs in the matching generator, not next to the code that reads it. A region a generator writes itself, such as `// #region html entities`, is the exception; `syntax.js` is a facade rather than either.
744749

745750
**And in the generator, derive it — do not type it out.** Read the table out of a published dataset (`mdn-data`, `color-name`, `@webref/idl`) whenever it is derivable at all, _including by analyzing a grammar rather than by listing names_: the value-definition syntax states which properties take an `<integer>`, so that set is computed, never enumerated. A table already in `SUPPLEMENT` counts as a source too — cosine at each eighth turn is sine two eighths along, and each inverse trig table is its forward one read back, so one stated table can carry several.
746751

‎lib/html/HtmlModulesPlugin.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ const HtmlModule = require("./HtmlModule");
4444

4545
const getMimeTypes = memoize(() => require("../util/mimeTypes"));
4646

47-
const getHtmlSyntax = memoize(() => require("./syntax"));
47+
const getHtmlSyntax = memoize(() => require("./syntax-parser"));
4848

4949
// Lazy: this plugin is applied on every build (`experiments.html` defaults on),
5050
// but the HTML parser and tokenizer are only reached once HTML is in the graph.

‎lib/html/HtmlParser.js‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,24 +26,24 @@ const {
2626
} = require("../util/magicComment");
2727
const memoize = require("../util/memoize");
2828
const { JAVASCRIPT_SCRIPT_TYPES } = require("./data");
29+
const { SVG_TAG_ADJUST } = require("./data");
30+
const { SourceProcessor } = require("./syntax");
2931
const {
3032
NS_HTML,
3133
NS_SVG,
3234
NodeType,
33-
SVG_TAG_ADJUST,
34-
SourceProcessor,
3535
baseTag,
3636
buildHeadTags,
3737
decodeEntities,
3838
escapeAttribute,
3939
escapeText,
40-
isAsciiWhitespace,
40+
isSpace: isAsciiWhitespace,
4141
metaTag,
4242
parseCssUrls,
4343
parseMsapplicationTask,
4444
parseSrc,
4545
parseSrcset
46-
} = require("./syntax");
46+
} = require("./syntax-parser");
4747

4848
const getUnsupportedFeatureWarning = memoize(() =>
4949
require("../errors/UnsupportedFeatureWarning")

‎lib/html/htmlMinify.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
"use strict";
77

8-
/** @import { HtmlPrintOptions } from "../html/syntax" */
8+
/** @import { HtmlPrintOptions } from "../html/syntax-printer" */
99
/** @import { CssEnvironment } from "../css/syntax" */
1010

1111
/**

‎lib/html/syntax-parser.js‎

Lines changed: 13174 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)