Skip to content

Commit a786b43

Browse files
test(javascript): hold the printer and the parser's trees to the invariants sweep (#22648)
* fix(javascript): print what the printer reads back the same - Keep the parentheses an `import()`'s arguments and callee need, which terser drops: `import(a, (b, c))` gained an argument and `new (import(a))` stopped parsing. - Print a directive spelled with escapes as spelled. Encoding its text as a value doubled each backslash at every print, so `"a\\b"` became another string. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011CuRTB8fAv8PE7di49qDL4 * test(javascript): hold the printer and the parser's trees to the invariants sweep `compare-js-tools.js --invariants` now asks of the JavaScript printer what the CSS and HTML sweeps ask of theirs, over every fixture, acorn's corpus and the installed bundles: - idempotence under a build's default, a second compress pass, the printer alone and `beautify`, the output read back print-only; - respelling: a string's delimiter, a decimal integer as hex and an identifier escape must not move the output; - `acorn`: every fixture's tree equals acorn's, positions included. The tree comparator moves from `test/helpers/compareParserOutput.js` into the script, which the parser suites now import it from. The printer gains terser's `minify_sync` as `modules.minifySync`, so the sweep stays synchronous like the CSS and HTML ones. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011CuRTB8fAv8PE7di49qDL4 * fix(javascript): keep the line break after a comment whose code the minifier drops A kept comment moved off a dropped `;` or declaration lost the line break after it, so a license banner printed `*/(function(){` where terser prints `*/\n(function(){`, and printing the output again added the break back. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011CuRTB8fAv8PE7di49qDL4 * fix(javascript): spell an escaped directive safe for an inline script - A directive printed as spelled skipped the escaping `inline_script` gives every other string, so `"x\x20</script>"` could end the script element it sits in; it now gets the same `<\/script`, `\x3c!--` and `--\x3e`, each reading as what it replaces. - The JavaScript sweep asks only for the expectations a narrowed run can reach: a fixture or preset left out no longer reads as stale. - Say the parser's relations parse under a build's options, and drop the `EXPECTED_ANY` the test option objects did not need. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011CuRTB8fAv8PE7di49qDL4 * fix(javascript): print from a SyntaxTree the directives, import() parentheses and dropped comments printEstree prints The tree printer main gained in #22643 now spells an escaped directive as written (inline-script safe), parenthesizes an import() under new or around a sequence, and keeps the comments of an empty statement it drops. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011CuRTB8fAv8PE7di49qDL4 --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
1 parent e166297 commit a786b43

13 files changed

Lines changed: 847 additions & 205 deletions

‎.github/workflows/test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,7 @@ jobs:
140140
if: ${{ !cancelled() }}
141141
run: yarn test:invariants:html --require-corpus
142142

143-
- name: JavaScript parser invariants
143+
- name: JavaScript parser and printer invariants
144144
if: ${{ !cancelled() }}
145145
run: yarn test:invariants:js --require-corpus
146146

‎docs/architecture.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ This is the canonical repository map. **When you add, rename or remove a top-lev
6060
- `tooling/` — repo-internal scripts:
6161
- Codegen run by `yarn fix:special`: runtime/wasm generators, the hash-debug tool, `generate-schemas.js` — which derives each schema from the module declaring its options and reports what no longer derives — `precompile-schemas.js`, which compiles each schema into the standalone validator webpack validates with, and `generate-types.js`, which prints `types.d.ts` from the types the sources under `lib/` declare. Each names its own stale outputs.
6262
- `check-agents-md-size.js` / `check-class-names.js` (`yarn lint:agents` / `lint:class-names`, both inside `yarn lint`) — the two invariants nothing else enforces: AGENTS.md's character budget, and that no two classes under `lib/` share a name, since a collision makes `generate-types.js` invent one for the published `types.d.ts`.
63-
- `compare-css-tools.js` / `compare-html-tools.js` / `compare-js-tools.js` (`yarn benchmark:css-tools` / `:html-tools` / `:js-tools`) and the invariants gate every PR runs (`yarn test:invariants`) — how to read the tables, the pinned corpora and the five relations: [tooling/COMPARE_TOOLS.md](../tooling/COMPARE_TOOLS.md). Read it before claiming anything about webpack's CSS/HTML/JS implementation vs the ecosystem, or when `invariants` is red.
63+
- `compare-css-tools.js` / `compare-html-tools.js` / `compare-js-tools.js` (`yarn benchmark:css-tools` / `:html-tools` / `:js-tools`) and the invariants gate every PR runs (`yarn test:invariants`) — how to read the tables, the pinned corpora and the invariant relations: [tooling/COMPARE_TOOLS.md](../tooling/COMPARE_TOOLS.md). Read it before claiming anything about webpack's CSS/HTML/JS implementation vs the ecosystem, or when `invariants` is red.
6464
- `type-coverage.js` (`yarn types:cover`) — how much of `lib/` is precisely typed.
6565
- `find-deep-webpack-imports.js` (`yarn find-deep-imports`) — which `webpack/lib/…` paths published packages import directly; `--check` (offline, run by the `Deep Imports` job) reads `tooling/deep-webpack-imports.json`. How it collects, and its `removed` map: [tooling/DEEP_IMPORTS.md](../tooling/DEEP_IMPORTS.md).
6666
- `measure-color-agreement.js` (`yarn measure:color-agreement`) — asks a real browser for its own color conversions (not pixels) and prints how far webpack's sit from them; it is the source of the rounding margins in `lib/css/syntax-parser.js` and the list of spaces an engine reads through another transfer. Re-run it (seconds; `PUPPETEER_EXECUTABLE_PATH` picks the binary) rather than adjusting either by hand.

‎lib/javascript/syntax-printer.js‎

Lines changed: 95 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7984,6 +7984,8 @@ const installMinify = (modules) => {
79847984
} else {
79857985
const next = /** @type {Token} */ (tables.of(sorted[low]));
79867986
next.comments_before = [...important, ...next.comments_before];
7987+
// The line break after them moves too, as in `keepCommentsOf`.
7988+
if (/** @type {Token} */ (tables.of(token)).nlb) next.nlb = true;
79877989
}
79887990
for (const comment of important) {
79897991
comments.splice(comments.indexOf(comment), 1);
@@ -8634,6 +8636,20 @@ const installMinify = (modules) => {
86348636
} while (!step.done);
86358637
return step.value;
86368638
};
8639+
8640+
/**
8641+
* terser's `minify_sync`: the same minify, for a caller handing over no
8642+
* source map that reads asynchronously.
8643+
* @param {EXPECTED_ANY} files the sources, or a tree
8644+
* @param {EXPECTED_ANY} options the options
8645+
* @returns {EXPECTED_ANY} the result
8646+
*/
8647+
modules.minifySync = (files, options) => {
8648+
const driver = minifySyncOrAsync(files, options);
8649+
let step = driver.next();
8650+
while (!step.done) step = driver.next(step.value);
8651+
return step.value;
8652+
};
86378653
};
86388654

86398655
const KEY = 0b00001000;
@@ -12022,11 +12038,12 @@ const installEstree = (modules) => {
1202212038
startToken: statement.startToken,
1202312039
endToken: statement.endToken,
1202412040
directive,
12025-
// The literal spelling it, as terser held the directive's quote.
12041+
// The literal spelling it, as terser held the directive's quote; its
12042+
// value is what the spelling says, where `directive` keeps the spelling.
1202612043
expression: StringNode({
1202712044
startToken: statement.startToken,
1202812045
endToken: statement.endToken,
12029-
value: directive,
12046+
value: statement.expression.value,
1203012047
quote: raw === null ? statement.expression.quote : raw[0]
1203112048
})
1203212049
});
@@ -16841,6 +16858,30 @@ const installCorrect = (modules) => {
1684116858
parent.arguments.indexOf(node) > 0
1684216859
);
1684316860
};
16861+
// terser reads `import()` as a call, so it holds the parentheses a call's
16862+
// arguments and callee owe: without them `import(a, (b, c))` gains an
16863+
// argument and `new (import(a))` stops parsing.
16864+
const sequenceParens = parensByType.SequenceExpression;
16865+
parensByType.SequenceExpression = (node, output) => {
16866+
if (sequenceParens(node, output)) return true;
16867+
const parent = /** @type {PrintParent | undefined} */ (output.parent());
16868+
return (
16869+
corrections.enabled &&
16870+
parent !== undefined &&
16871+
parent.type === "ImportExpression"
16872+
);
16873+
};
16874+
const importParens = parensByType.ImportExpression;
16875+
parensByType.ImportExpression = (node, output) => {
16876+
if (importParens !== undefined && importParens(node, output)) return true;
16877+
const parent = /** @type {PrintParent | undefined} */ (output.parent());
16878+
return (
16879+
corrections.enabled &&
16880+
parent !== undefined &&
16881+
parent.type === "NewExpression" &&
16882+
parent.callee === node
16883+
);
16884+
};
1684416885
const printArrow =
1684516886
/** @type {NonNullable<EstreeGenerators["ArrowFunctionExpression"]>} */ (
1684616887
estreeCodegen.ArrowFunctionExpression
@@ -28430,6 +28471,9 @@ const createTightenBody = (modules) => {
2843028471
if (!from || !to || from.comments_before.length === 0) return;
2843128472
to.comments_before = [...from.comments_before, ...to.comments_before];
2843228473
from.comments_before = [];
28474+
// The line break after them moves too: a banner before a dropped `;`
28475+
// still ends its line, as terser prints it.
28476+
if (from.nlb) to.nlb = true;
2843328477
}
2843428478

2843528479
/**
@@ -38336,6 +38380,23 @@ const printedNameOf = (node) => {
3833638380
/** @typedef {EstreeTable<void>} EstreeGenerators per ESTree type, what prints a node of it */
3833738381
/** @typedef {EstreeTable<boolean | undefined>} EstreeParensTests per ESTree type, whether a node of it is parenthesized where it is printed */
3833838382

38383+
/**
38384+
* A string's spelling made safe in an inline `<script>`, as `encode_string`
38385+
* makes a value: each escape added reads as what it replaces.
38386+
* @param {string} spelling the text between a string's quotes
38387+
* @returns {string} the same string, spelled so no `</script` or `<!--` shows
38388+
*/
38389+
const inlineScriptSpelling = (spelling) =>
38390+
spelling
38391+
.replace(/<\u002F(script)([>/\t\n\f\r ])/gi, "<\\/$1$2")
38392+
// An escaped `\<` gives its backslash up to the `\x3c` taking its place.
38393+
.replace(
38394+
/(\\*)\u003C!--/g,
38395+
(_match, slashes) =>
38396+
`${slashes.length % 2 === 1 ? slashes.slice(1) : slashes}\\x3c!--`
38397+
)
38398+
.replace(/--\u003E/g, "--\\x3e");
38399+
3833938400
/**
3834038401
* Installs `printEstree`, the printer over the compressed tree as it is held:
3834138402
* a code generator and a test for its parentheses per ESTree type, each
@@ -39686,7 +39747,18 @@ const installEstreePrint = (modules) => {
3968639747
ExpressionStatement: (node, output) => {
3968739748
if (typeof node.directive === "string") {
3968839749
const expression = /** @type {PrintLiteral} */ (node.expression);
39689-
output.print_string(expression.value, expression.quote);
39750+
const value = /** @type {string} */ (expression.value);
39751+
// A directive spelled with escapes is printed as spelled: encoding
39752+
// its value instead could spell another directive, or none.
39753+
if (node.directive !== value) {
39754+
const quote = expression.quote || '"';
39755+
const spelled = output.option("inline_script")
39756+
? inlineScriptSpelling(node.directive)
39757+
: node.directive;
39758+
output.print(`${quote}${spelled}${quote}`);
39759+
} else {
39760+
output.print_string(value, expression.quote);
39761+
}
3969039762
} else {
3969139763
printEstree(node.expression, output);
3969239764
}
@@ -42607,7 +42679,17 @@ const installTreePrint = (modules) => {
4260742679
const directive = tree.textOf(Part.directive, node);
4260842680
const expression = childOf(Field.expression, node);
4260942681
if (directive !== null) {
42610-
output.print_string(directive, quoteOf(expression));
42682+
const value = /** @type {string} */ (tree.value(expression));
42683+
// Printed as spelled where it holds escapes, as `printEstree` prints it.
42684+
if (directive !== value) {
42685+
const quote = quoteOf(expression);
42686+
const spelled = output.option("inline_script")
42687+
? inlineScriptSpelling(directive)
42688+
: directive;
42689+
output.print(`${quote}${spelled}${quote}`);
42690+
} else {
42691+
output.print_string(value, quoteOf(expression));
42692+
}
4261142693
} else {
4261242694
printNode(expression);
4261342695
}
@@ -43508,6 +43590,7 @@ const installTreePrint = (modules) => {
4350843590
(type === NodeType.ForOfStatement &&
4350943591
node === childOf(Field.right, parent)) ||
4351043592
type === NodeType.YieldExpression ||
43593+
type === NodeType.ImportExpression ||
4351143594
type === NodeType.ExportNamedDeclaration ||
4351243595
type === NodeType.ExportDefaultDeclaration ||
4351343596
type === NodeType.ExportAllDeclaration
@@ -43575,6 +43658,14 @@ const installTreePrint = (modules) => {
4357543658
childOf(Field.left, grandparent) === parent
4357643659
);
4357743660
};
43661+
// As `printEstree` keeps them: `new import(a)` does not parse.
43662+
parensTests[NodeType.ImportExpression] = (node) => {
43663+
const parent = parentAt(0);
43664+
return (
43665+
typeOf(parent) === NodeType.NewExpression &&
43666+
childOf(Field.callee, parent) === node
43667+
);
43668+
};
4357843669
parensTests[NodeType.NewExpression] = (node) => {
4357943670
const parent = parentAt(0);
4358043671
const type = typeOf(parent);

‎test/helpers/compareParserOutput.js‎

Lines changed: 0 additions & 96 deletions
This file was deleted.

‎test/specCases/js-fuzz.spectest.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ const {
1414
parse
1515
} = require("../../lib/javascript/syntax").parser;
1616
const { decodeMappings } = require("../../lib/util/createMappings");
17-
const { firstDifference } = require("../helpers/compareParserOutput");
17+
const { firstDifference } = require("../../tooling/compare-js-tools");
1818
const {
1919
DEPENDENCY,
2020
createProgramGenerator,

‎test/specCases/test262-parser.spectest.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ const { parse } = require("../../lib/javascript/syntax").parser;
77
const {
88
firstDifference,
99
reportable
10-
} = require("../helpers/compareParserOutput");
10+
} = require("../../tooling/compare-js-tools");
1111

1212
/** @typedef {{ file: string, at: string, ours: string, acorn: string }} TreeDifference */
1313
/** @typedef {{ file: string, webpack: string, acorn: string }} VerdictDifference */

‎test/unitCases/JavascriptSyntaxTree.unittest.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ const {
99
SyntaxTree,
1010
WebpackParser
1111
} = require("../../lib/javascript/syntax-parser");
12-
const { firstDifference } = require("../helpers/compareParserOutput");
12+
const { firstDifference } = require("../../tooling/compare-js-tools");
1313

1414
/** @typedef {import("estree").Node} EstreeNode */
1515

‎test/unitCases/WebpackParser.unittest.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ const { Parser } = require("../../lib/javascript/syntax-parser");
1616
const {
1717
firstDifference,
1818
reportable
19-
} = require("../helpers/compareParserOutput");
19+
} = require("../../tooling/compare-js-tools");
2020

2121
/**
2222
* @param {string} code source code the parser mapped

0 commit comments

Comments
 (0)