Skip to content

Commit 55c9808

Browse files
refactor(js): own the ECMAScript parser and drop the acorn dependency (#22042)
* refactor(js): own the ECMAScript parser and drop the acorn dependency `lib/javascript/parser.js` is webpack's own port of acorn 8.18.0, so nothing under `lib/` requires acorn at runtime and the package ships one dependency fewer. acorn stays a devDependency: the parity suites and `tooling/generate-js-data.js` still read it. Owning the parser pays for itself at load: the `\p{...}` property tables move to `lib/javascript/regexpData.js` and are required only when a pattern the engine itself rejected reaches the validator, the identifier ranges decode lazily from run-length tables, and the destructuring-errors class is exported rather than captured by parsing through a probe at module load. * perf(js): load the regexp validator only when a pattern needs it `readRegexp` builds the literal's `RegExp` and lets the host engine's verdict stand, so the ported validator is reached only for a pattern that engine rejected. It moves to `lib/javascript/regexp.js` and is installed on the parser's prototype the first time that happens, taking its Unicode property tables and 1,700 lines of `lib/javascript/parser.js` off what every webpack process reads and compiles. * perf(js): install the grammar only when something asks to parse A webpack build is served entirely by `lib/javascript/syntax.js`, so the 187 base productions and the regexp validator are now separate modules copied onto the parser's prototype on first use. * perf(js): resolve a member expression once and walk it without closures The walker re-resolved a member expression when the walk and an evaluate met on the same node, and built three arrays, three closures and four objects per hit whether or not anything read them. Resolve into instance slots, keyed on the node and its scope, and build the public info object only for callers that ask for it. The binary and unary evaluators no longer allocate their helpers per evaluation. * perf(js): dispatch a member-chain hook only when one is registered The call, assignment and unhandled member-chain dispatches built their argument arrays and member ranges for every chain, and found a hook for fewer than one in ten. * perf(js): probe the word cache in one line and track parameter clashes in a Set The word cache kept its word, type and owner in three arrays, so a probe touched three cache lines and verified a hit through a builtin call; one interleaved array and a char loop serve it from one line. Parameter names were checked for clashes through hasOwnProperty on a null-prototype object; a pooled Set does the same lookup without dictionary mode. * perf(js): memoize a word's reserved kind so plain identifiers skip checkUnreserved Every identifier ran checkUnreserved's length and first-char gates and, for the short lowercase majority, a Map lookup. The word cache already classifies each word once; it now records the reserved kind beside the token type, and parseIdent skips the check when it is zero. * fix(js): give the `import` of import.meta its start location The owned parseExprImport built the meta identifier without the start position acorn passes along, so with locations on it carried only an end. The build never tracks locations, which is why the parity sweeps missed it. * fix(js): validate a regexp literal below ES2018 without property tables The base parser built the `\p{...}` name tables for every edition, and editions before 2018 have none, so any regexp literal threw at ecmaVersion 5 through 2017. acorn keeps them undefined there and never reads them; so does the port now. The new unit test holds the validator to acorn's verdict over a pattern corpus at every edition. * test(js): hold the base parser to acorn over a program corpus Drive Parser.parse, parseExpressionAt, tokenizer, extend and the line, identifier and option helpers against acorn across editions, source types and option sets, so the paths a build never reaches stay covered. * test(js): teach cspell the script names in the regexp corpus * test(js): malform the pattern in the flag-order regexp test With a valid pattern the assertions passed whichever check ran first; an unterminated group makes the flag error prove its precedence, and a valid flag set shows the pattern error surfacing afterwards. * test(js): reach the grammar's remaining branches from the parity corpus Explicit resource management, pattern and export edge cases, template and number tokens, the start-location option and the semicolon and trailing comma callbacks are now held to acorn as well, which leaves grammar.js and parser.js at 99.5% line coverage from the unit suites alone. * test(side-effects): probe symlink support at a per-process path ConfigTestCases, ConfigCacheTestCases and RoundTripConfigCases run this filter in separate workers at the same instant, and all three raced on one probe path between removing and recreating it, which surfaced as an EEXIST that failed the whole suite. * perf(js): hold the grammar module once it is installed The four grammar stubs resolved the module with require() on every call, so the tokenizer entry point paid the loader's lookup per token and read seven times slower than main. The module is kept after the first install and the stubs delegate through it. * perf(js): let the grammar take the stubs' slots once installed `install()` skips members the parser prototype already owns, so the four stubs stayed in place and every token still paid a stub frame, the install check and a `Function.prototype.call` on its way to the grammar's `getToken`, which read 10% slower than main on the tokenize benchmarks. The stubs now hand their slots to the grammar's own methods after the first install. * chore(js): drop the grammar's unreferenced typedef imports eslint-config-webpack 4.12.0 rejects import tags nothing references (webpack/no-unused-import-tag).
1 parent e1ab6f2 commit 55c9808

23 files changed

Lines changed: 13381 additions & 2314 deletions
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": patch
3+
---
4+
5+
Drop the `acorn` dependency in favor of webpack's own JavaScript parser.

‎.changeset/180-own-parser-outside-syntax.md‎

Lines changed: 0 additions & 5 deletions
This file was deleted.

‎AGENTS.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ The directory listings below are the canonical map of the repository. **Whenever
7272
- `lib/hmr/` — Hot Module Replacement plugins.
7373
- `lib/html/` — Experimental HTML support.
7474
- `lib/ids/` — Module/chunk id assignment plugins.
75-
- `lib/javascript/` — JavaScript parsing (acorn), generation, exports analysis.
75+
- `lib/javascript/` — JavaScript parsing (webpack's own ECMAScript parser, ported from acorn), generation, exports analysis. `syntax.js` serves every production a build reaches, so `grammar.js` (the full grammar) and `regexp.js` (the pattern validator) are installed onto the parser's prototype only when something asks for one — keep it that way and never `require` either from a path a build takes.
7676
- `lib/json/` — JSON modules.
7777
- `lib/library/` — UMD/AMD/ESM/CommonJS library output formats.
7878
- `lib/loaders/` — Loader execution runtime (vendored loader-runner): pitching/normal loader iteration and loader module loading.
@@ -573,8 +573,9 @@ These files are produced by `yarn fix:special` and must not be edited by hand:
573573
- Generated runtime code under `lib/` (driven by `tooling/generate-runtime-code.js`).
574574
- `lib/css/data.js` — every table the CSS minifier looks a name up in, and the arithmetic its math-function descriptors bind to: derived from `mdn-data` + `color-name` (box shorthands, color-argument and math functions, named colors) plus the generator's `SUPPLEMENT` of spec-prose tables and math primitives, by `tooling/generate-css-data.js` — which also holds the value-definition-syntax parser those grammars are read with, and runs the generation only as the entry point so its tests can require it.
575575
- `lib/html/data.js` — every table the HTML parser and minifier look a name up in: the reflected-attribute tables distilled from webref's HTML IDL (the `@webref/idl` and `@webref/elements` packages), plus the generator's `SUPPLEMENT` and `PARSER_TABLES` of §13.2 tree-construction vocabulary, by `tooling/generate-html-data.js` — which also emits the `// #region html entities` block inside `lib/html/syntax.js` from the vendored `tooling/html-entities.json`, WHATWG's own named character references table.
576+
- `lib/javascript/data.js` and `lib/javascript/regexpData.js` — the Unicode tables the JavaScript parser classifies with: the run-length identifier ranges the tokenizer decodes on its first non-ASCII code point, and the per-edition `\p{...}` property names only a pattern the engine itself rejected reaches, by `tooling/generate-js-data.js` — which reads both back out of the pinned acorn devDependency, so bumping it is what moves them.
576577

577-
Both `syntax.js` files are algorithm only — a new lookup table belongs in the matching generator, not next to the code that reads it.
578+
The three `syntax.js` files are algorithm only — a new lookup table belongs in the matching generator, not next to the code that reads it.
578579

579580
**And in the generator, derive it — do not type it out.** Read the table out of a published dataset (`mdn-data`, `color-name`, `@webref/idl`) whenever it is derivable at all, _including by analyzing a grammar rather than by listing names_: the value-definition syntax states which properties take an `<integer>`, so that set is computed, never enumerated. A table already in `SUPPLEMENT` counts as a source too — cosine at each eighth turn is sine two eighths along, and each inverse trig table is its forward one read back, so one stated table can carry several.
580581

@@ -586,7 +587,7 @@ Re-run `yarn fix:special` **before the next commit** whenever you touch:
586587

587588
- `schemas/**/*.json` — reshapes validators, declarations, and `types.d.ts`.
588589
- `lib/**/*.js` JSDoc on anything reachable from a public export — regenerates `types.d.ts`.
589-
- `tooling/generate-runtime-code.js`, `tooling/generate-wasm-code.js`, `tooling/generate-css-data.js`, `tooling/generate-html-data.js`, or any file they consume (including the `mdn-data` / `color-name` / `@webref/*` versions in `package.json` and the vendored `tooling/html-entities.json`).
590+
- `tooling/generate-runtime-code.js`, `tooling/generate-wasm-code.js`, `tooling/generate-css-data.js`, `tooling/generate-html-data.js`, `tooling/generate-js-data.js`, or any file they consume (including the `acorn` / `mdn-data` / `color-name` / `@webref/*` versions in `package.json` and the vendored `tooling/html-entities.json`).
590591

591592
CI's `lint` job verifies these outputs are up to date. The combined `yarn fix` script runs `fix:code` + `fix:special` + `fmt` in one go; prefer it as the final step.
592593

‎cspell.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -577,6 +577,7 @@
577577
"**/**/*.snap",
578578
"test/cases/json/weird-properties/globals.json",
579579
"test/JavascriptParser.unittest.js",
580+
"lib/javascript/regexpData.js",
580581
"tooling/html-entities.json",
581582
"test/test262-cases/**",
582583
"test/wpt/**",

‎eslint.config.mjs‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -305,6 +305,18 @@ export default defineConfig([
305305
"import/no-extraneous-dependencies": "off"
306306
}
307307
},
308+
{
309+
// The parser keeps acorn's method and property names, since
310+
// `lib/javascript/syntax.js` and any acorn plugin call them by those names.
311+
files: [
312+
"lib/javascript/grammar.js",
313+
"lib/javascript/parser.js",
314+
"lib/javascript/regexp.js"
315+
],
316+
rules: {
317+
camelcase: "off"
318+
}
319+
},
308320
{
309321
files: [".changeset/**/*.mjs"],
310322
languageOptions: {

0 commit comments

Comments
 (0)