Skip to content

Commit e6f1c14

Browse files
fix(javascript): read the walk off the path and print the preamble
The printer reads the tree and the nodes enclosing the one printed off the JavaScript path rather than a private export, writes format.preamble after the #! line as minify does, and the changeset fits one line. Claude-Session: https://claude.ai/code/session_01Tgxgyb4N35ndP6cC4KcrXC
1 parent c4b02ff commit e6f1c14

6 files changed

Lines changed: 104 additions & 53 deletions

File tree

‎.changeset/040-javascript-source-processor.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
"webpack": minor
33
---
44

5-
Add the JavaScript `SourceProcessor`, which prints a source minified as the CSS and HTML ones do, and parse JavaScript to minify faster.
5+
Add a JavaScript `SourceProcessor` that prints minified, and parse JS faster.

‎docs/syntax.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ Conventions every language keeps, and a new member follows:
122122

123123
## JavaScript
124124

125-
- `syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser for its `parse` phase, which reads an asset with it into a `SyntaxTree` the printer adopts. It is the only parser: a source it refuses is refused, and terser's parse options for what is not a program (`parse.expression`, `parse.strict`, `parse.experimental_typescript`) are refused too. `toEstreeObject(tree, node, fields?)` builds a `SyntaxTree` node's ESTree object, each call returning the same object: every field, or only those listed and the rest on a later call. It bridges hooks and plugins while the parser moves to that struct-of-arrays tree. The tree already reads as [the node path](#the-node-path) does, a node passed last: `type`, `start`, `end`, `range`, `name`, `value`, `field`, `fieldCount`, `flag` and `textOf` with the `NodeType`, `Field`, `Flag` and `Part` that `syntax.parser` exports. `SourceProcessor` walks the tree a parse writes as the CSS and HTML ones walk theirs, each visitor handed that path and the children taken in source order (a template's quasis and expressions in turn). Asked for `mode: "minify"`, the same walk prints: `syntax-printer.js`'s `printer` returns each node's text from its children's, as `minify` prints a source it neither compresses nor mangles, comments left out and terser's format options read from `format`. What terser's stream reads off the text before a node — `has_parens` before a function or object starting a statement, `!--`, `</script` — it reads off the tree instead, and each child's text is read once (`PrintContext#consume`), so a long one is made flat and its pieces let go as it is printed.
125+
- `syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser for its `parse` phase, which reads an asset with it into a `SyntaxTree` the printer adopts. It is the only parser: a source it refuses is refused, and terser's parse options for what is not a program (`parse.expression`, `parse.strict`, `parse.experimental_typescript`) are refused too. `toEstreeObject(tree, node, fields?)` builds a `SyntaxTree` node's ESTree object, each call returning the same object: every field, or only those listed and the rest on a later call. It bridges hooks and plugins while the parser moves to that struct-of-arrays tree. The tree already reads as [the node path](#the-node-path) does, a node passed last: `type`, `start`, `end`, `range`, `name`, `value`, `field`, `fieldCount`, `flag` and `textOf` with the `NodeType`, `Field`, `Flag` and `Part` that `syntax.parser` exports. `SourceProcessor` walks the tree a parse writes as the CSS and HTML ones walk theirs, each visitor handed that path and the children taken in source order (a template's quasis and expressions in turn). Asked for `mode: "minify"`, the same walk prints: `syntax-printer.js`'s `printer` returns each node's text from its children's, as `minify` prints a source it neither compresses nor mangles, comments left out and terser's format options read from `format`. What terser's stream reads off the text before a node — `has_parens` before a function or object starting a statement, `!--`, `</script` — it reads off the tree instead, through two members only the JavaScript path has: `tree`, the `SyntaxTree` walked, and `ancestor(level)`, a node enclosing the current one; and each child's text is read once (`PrintContext#consume`), so a long one is made flat and its pieces let go as it is printed.
126126
- `syntax-parser.js` — tokenizer, acorn-derived core and every production in one file (any parse reaches the productions). Given a `syntaxTree`, the lazy path writes each node into it as it finishes, an id in the tree's columns rather than an object; the few the parser reads back by name (a key, a module name, an exported declaration, a function before its body ends) stay objects until their parent is written, and an expression read as a pattern after all is retyped in place. Only the latest edition without parser plugins takes a tree, since acorn's own productions read their nodes back. `regexp.js` (pattern validator) is the one on-demand piece, reached only for a pattern the host engine rejected — never `require` it from a build path.
127127
- `syntax-printer.js` — webpack's JavaScript minifier, terser's design ported to write byte for byte what terser writes; `jsMinify.js` is the `minify` the default minimizer dispatches JS to under `experiments.futureDefaults`, and goes through it alone; without that flag the minimizer dispatches to `terserMinify` as `minimizer-webpack-plugin` publishes it, running the user's own terser since webpack lists it only as a devDependency. Neither needs an option to say which, and the printer itself never loads terser. `createModules` builds what terser's modules exported — the node factories (`createAst`), the parser's tables (`createParseTables`), the compressor and the output stream — and each entry of `PHASES` installs one part onto them: add phases there and nowhere else. A node is a plain object without methods, made by a factory `createAst` compiles from `NODE_KINDS` with its fields; its prototype is its kind's, holding its ESTree `type`, the values ESTree fixes for the kind (`prefix`, a name's `role`, a declaration's `kind`), the kind itself, which `ast.kindOf` reads, and its kind bits. A kind's prototype sits below a frozen prototype per terser class, chained as `NODE_KIND_ANCESTRY` gives each kind's classes, and `createAst` names a predicate per class, `is<Class>` (`isLambdaNode`), a bound `isPrototypeOf` of that class's prototype: the phases ask a node what it is through these, never `instanceof`. There are no node methods: every operation terser defined per class (`optimize`, `evaluate`, `has_side_effects`, `drop_side_effect_free`, `size`, scope analysis, …) is a function on `ast` dispatching on `node.type` through a table, a type absent reading the default, and walkers, transformers and the compressor are plain objects (`createWalker`, `createTransformer`, `createCompressor`), with `ast.walkNode` and `ast.transformNode` visiting children by type. A `pure_funcs` function is still handed a call under terser's field names (`args`, `expression`, `print_to_string`) through a read-only view. A parsed source is adopted from its `SyntaxTree`, reading the columns rather than calling a method per field (`ast.adoptTree`); an ESTree, which a `parse.spidermonkey` input is, through `ast.adoptEstree`, and `ast.toEstree` writes the SpiderMonkey JSON `format.spidermonkey` hands back, as terser's `to_mozilla_ast` did. `correct` wraps the methods with which terser writes a wrong program and changes only that output; `corrections.enabled` switches it off. `improve` writes less where terser's longer program provably runs the same — after compressing, a statement calling a function in place with nothing passed runs its body there, and a call of an exactly specified built-in on literals (`"abc".charAt(1)`, `Math.max(1, 2)`, `parseInt("ff", 16)`, `100 .toString(16)`, `[1].concat([2])`) is replaced by its value, an array without holes as an array literal, and a `Number` constant by its number where shorter, and an expression nobody reads that only reads a built-in global (`JSON`, `Math.PI`, `Object.prototype.toString`), calls one on literals without throwing (`new Set`, `Object.keys([])`, `new RegExp("a", "i")`) or reads a global a `typeof` proved declared (`typeof x != "undefined" && x`) is dropped, and a shorter spelling of the same is written: `Array` and `Object` constructed as literals (`new Array(1, 2)` as `[1,2]`), `new` dropped where the call constructs the same (`Error`, `Function`, `RegExp` passed no regular expression, …, by `CONSTRUCTED_WHEN_CALLED`), a spread array literal passed as its elements, `typeof x == "undefined"` as `typeof x > "u"`, a `+` beside a number literal in arithmetic dropped, `!(a == b)` as `a != b`, `!!` of a boolean dropped and `Boolean(x)` as `!!x`, `??` after a value never `null` or `undefined` dropped, an integer tested against zero as the integer, `===` between numbers as `==`, an array literal or a bigint concatenated as the string it makes, a string literal's character (`"abc"[1]`), `+[]`, `Number`, `String` and `BigInt` of a literal and `toString` of a boolean or a string as their value, `c ? a = x : a = y` as `a = c ? x : y` (a name no `with` reaches, an undeclared one in sloppy code only, a property of `this` where it is bound or of a name nothing reassigns), `(a = b) === null || a === void 0` as `null == (a = b)`, from ECMAScript 2020 `a == null ? void 0 : a.b` as `a?.b` (and `null == (a = b) ? void 0 : a.c` as `(a = b)?.c`), and from ECMAScript 2021 `a || (a = b)` as `a ||= b` (a name or a property of a name assigned, `&&` and `??` alike) and `a = a || b` as `a ||= b` (a `var`, `let` or parameter, which writing back to itself shows nothing), and `const` as `let` where nothing writes the name, below the top level and where no `eval` or `with` reaches (once names are mangled, so the letters mangling counts stay terser's); `drop_unused` also drops from a declared or parameter pattern the names nothing reads (an array's as holes but for the last, an object's where no rest follows and no getter could run: under `pure_getters`, or reading a literal), a declaration whose object pattern then names nothing and reads a literal (an array pattern still looks up its value's iterator), and from ECMAScript 2019 a `catch` binding nothing reads — taking built-ins as present and as the language defines them, as oxc and swc do — and `improvements.enabled` switches it off. The `parse` phase builds the tree from `syntax-parser.js`; `javascript/printer.unittest.js` holds the adopted tree to terser's own conversion, and `PARSED_DIFFERENTLY` in `javascript/printer.unittest.js` lists where the tree it builds prints otherwise than terser's. `test/specCases/minify-corpora.spectest.js` holds the whole printer to terser's output, a refusal matching any refusal, over each corpus in its `CORPORA` list (add a new one there) under the option sets in `OPTION_SETS`, and runs each output whose test states what its input prints (terser's `expect_stdout`, swc's `expected.stdout` and `exec.rs`) in terser's sandbox. A byte difference is accepted only where switching the `correct` and `improve` phases off gives terser's bytes back, and fails where `improve` writes more than without it, unless the program is shorter and only `const` written as `let` makes it gzip bigger (`improvements.lets.enabled` switches that one off to tell); what it fixes is listed in `CORRECTED` (printing right with the phase, as terser without), and an output misprinting as terser's by an option's own contract in `INHERITED`, each with the reason — an entry that stops matching fails until retired. Where the reference instead refuses a source the spec allows, the parse error it answers with is listed in `REFUSED_BY_REFERENCE` with what makes the source legal; where webpack refuses a source the reference reads, the source is listed in `REFUSED_BY_WEBPACK` with the rule it breaks (a test262 test expecting a parse error needs no entry); where it reads or prints one differently, the case is listed in `REFERENCE_MISPRINTS` with what it gets wrong — a listed one webpack must still run correctly, and one the reference stops differing on fails until retired. Each output is also sized against swc's recorded one, a fixture's `output.js` or, for one of terser's cases, what swc's port of it records (gzip decides, raw breaks a tie): one bigger fails unless `test/helpers/swcSmaller.js` lists it under what swc does that webpack does not yet, and a listed one no longer bigger fails until retired. Under the default minimizer's options each source of every corpus but test262 is also minified by `oxc-minify`, held the same way to `test/helpers/oxcSmaller.js`; an oxc output that misprints is no lead. `JS_MINIFY_REPORT=<file>` tallies where webpack's is worse, the same and better. terser's corpus submodule is pinned to the `terser` devDependency — bump both together, and regenerate `data.js`.
128128
- Printing and mangling read the compressed tree as it is held, by its ESTree types: no conversion step stands between the compressor and the printer, so a tree a caller passes in or `format.ast` hands back is printed and mangled as it is. Where the compressor holds a shape of its own the printer reads it directly — a template's `segments`, two side by side printing as one text; an import's `imported_name` and `imported_names`, an export's `exported_definition`, `exported_value` and `exported_names`, attributes as the object holding them; `import()`'s `args`; a key's or a property read's name as a symbol node; a literal's `role` for a regular expression or a bigint's digits — and a name prints as its `definition` names it (`mangled_name`, else the definition's own name), terser's or the mangler's alike, never renamed in place. Where nobody keeps the tree, `analyzeManglingScopes` and `printProgram` drop what only the compressor read (`COMPRESSOR_ONLY_FIELDS`) off each node they read; every field written is one its kind's factory made. The `estreeMangle` phase mangles that tree as terser's mangler did: `analyzeManglingScopes` resolves its scopes, writing each name's `definition`, `computeEstreeCharFrequency` orders the identifiers, `mangleManglingScopes` gives the definitions their `mangled_name` and `renameMangledEstree` renames `#private` names and properties in one walk, skipped where there are neither. `printEstree` (the `estreePrint` phase) dispatches on `node.type` through three tables — `codegen` (the text), `needsParens` and `sourceMapping` — and asks about a parent through the `kindOf` predicates, never `instanceof`; the stream (`MinifiedOutput`) reads comments through `NodeAccess`. Everything that prints goes through it: `minify`, the mangler's character frequency and the compressor's `printToString`.

‎lib/javascript/syntax-parser.js‎

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -20941,6 +20941,22 @@ const A = {
2094120941
walkStopped = true;
2094220942
walkHalted = getHaltVisitors().haltVisitors(walkVisitors);
2094320943
},
20944+
/**
20945+
* The tree walked, for a reader of many nodes that would rather not go
20946+
* through the path for each.
20947+
* @returns {SyntaxTree} the tree
20948+
*/
20949+
get tree() {
20950+
return /** @type {SyntaxTree} */ (walkTree);
20951+
},
20952+
/**
20953+
* @param {number} level how many levels further out than the parent
20954+
* @returns {number} a node enclosing the current one, 0 past the program
20955+
*/
20956+
ancestor(level) {
20957+
const at = walkParents.length - 1 - level;
20958+
return at >= 0 ? walkParents[at] : 0;
20959+
},
2094420960
/**
2094520961
* @param {number} type a `NodeType`
2094620962
* @returns {boolean} whether a node enclosing the current one has that type
@@ -21265,17 +21281,4 @@ module.exports.positionAt = positionAt;
2126521281
module.exports.releaseParserCaches = releaseParserCaches;
2126621282
module.exports.toEstreeObject = toEstreeObject;
2126721283
module.exports.A = A;
21268-
// What the JavaScript node printer reads the run it prints through, as the
21269-
// HTML printer reads `_S`: the tree, its source, the nodes holding the one printed.
21270-
module.exports._walkState = {
21271-
/** @returns {SyntaxTree | null} the tree walked */
21272-
get tree() {
21273-
return walkTree;
21274-
},
21275-
/** @returns {string} its source */
21276-
get input() {
21277-
return walkInput;
21278-
},
21279-
parents: walkParents
21280-
};
2128121284
module.exports.grammar = grammar;

0 commit comments

Comments
 (0)