Skip to content

Commit 91c28a1

Browse files
refactor(javascript): read sources with webpack's parser alone when minifying
Drop terser's lenient parser from the printer: webpack's parser is the only one, so a source it refuses is refused, and terser's `expression`, `strict` and `experimental_typescript` parse options are refused too.
1 parent b17fa7d commit 91c28a1

11 files changed

Lines changed: 570 additions & 5902 deletions

‎.changeset/098-minify-with-webpacks-printer.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
"webpack": patch
33
---
44

5-
Refuse a private field with no class around it when minifying with the printer.
5+
Read JavaScript with webpack's parser alone when minifying, refusing early errors.

‎docs/syntax.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Read this before touching the CSS, HTML or JavaScript parser, printer or minifie
2424
- `lib/css/data.js` — every table the CSS minifier looks names up in, plus the arithmetic its math-function descriptors bind to: from `mdn-data` + `color-name` (box shorthands, color-argument and math functions, named colors) and the generator's `SUPPLEMENT` of spec-prose tables and math primitives, by `tooling/generate-css-data.js` — which also holds the value-definition-syntax parser those grammars are read with, and runs generation only as the entry point so its tests can require it.
2525
- `lib/html/data.js` — every table the HTML parser and minifier look names up in: reflected-attribute tables from webref's HTML IDL (`@webref/idl`, `@webref/elements`) plus the generator's `SUPPLEMENT` and `PARSER_TABLES` of §13.2 tree-construction vocabulary, by `tooling/generate-html-data.js` — which also emits the `// #region html entities` block in `lib/html/syntax-parser.js` from the vendored `tooling/html-entities.json` (WHATWG's named character references).
2626
- `lib/javascript/data.js` — the JS parser's and printer's Unicode tables: run-length identifier ranges the tokenizer decodes at its first non-ASCII code point, and per-edition `\p{...}` property names (reached only for a pattern the engine rejected), which `tooling/generate-js-data.js` reads from the pinned acorn devDependency — bumping it moves them; plus the Unicode 8.0 identifier ranges the printer writes unescaped (so older engines read its output), read from terser's own frozen tables.
27-
- `lib/javascript/syntax-printer-data.js` — the tables only the printer reads, in their own module so a build that parses without minifying through the printer never loads them: the native-object tables of pure globals, calls, methods and static properties the compressor reads (`NATIVE_*`, built only when `nativeObjectTables()` is called, so loading the module allocates none), each name with its `unsafe` / `builtins_ecma >= N` conditions in terser's order, parsed with acorn out of terser's `lib/compress/native-objects.js`. Plus terser's node classes (`nodeClasses()`, built on call too): each class's base, the fields its constructor copies, the values on its prototype, and its children in the order its `_walk` and `_children_backwards` reach them, parsed out of terser's `lib/ast.js` — the printer builds every node class and its walk from them. Plus the tokenizer's and parser's tables (`parserTables()`), parsed out of terser's `lib/parse.js`, and the DOM property names property mangling keeps (`domProperties()`), out of its `tools/domprops.js`.
27+
- `lib/javascript/syntax-printer-data.js` — the tables only the printer reads, in their own module so a build that parses without minifying through the printer never loads them: the native-object tables of pure globals, calls, methods and static properties the compressor reads (`NATIVE_*`, built only when `nativeObjectTables()` is called, so loading the module allocates none), each name with its `unsafe` / `builtins_ecma >= N` conditions in terser's order, parsed with acorn out of terser's `lib/compress/native-objects.js`. Plus terser's node classes (`nodeClasses()`, built on call too): each class's base, the fields its constructor copies, the values on its prototype, and its children in the order its `_walk` and `_children_backwards` reach them, parsed out of terser's `lib/ast.js` — the printer builds every node class and its walk from them. Plus the keyword, reserved-word and operator-precedence tables the printer names tokens by (`parserTables()`), parsed out of terser's `lib/parse.js`, and the DOM property names property mangling keeps (`domProperties()`), out of its `tools/domprops.js`.
2828

2929
## CSS
3030

@@ -36,7 +36,7 @@ Read this before touching the CSS, HTML or JavaScript parser, printer or minifie
3636

3737
## JavaScript
3838

39-
- `syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser for its `parse` phase, which reads an asset with it — building the printer's tree as each statement finishes, rather than converting a tree. `parse.webpackParser` defaults to true and turns that off; terser's own parser reads what it reads its own way (`parse.expression`, `parse.strict`, TypeScript, a line opening an HTML comment) and whatever webpack's parser refuses.
39+
- `syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser for its `parse` phase, which reads an asset with it — building the printer's tree as each statement finishes, rather than converting a tree. It is the only parser: a source it refuses is refused, and terser's parse options for what is not a program (`parse.expression`, `parse.strict`, `parse.experimental_typescript`) are refused too.
4040
- `syntax-parser.js` — tokenizer, acorn-derived core and every production in one file (any parse reaches the productions, and the coming struct-of-arrays rewrite moves node creation through them). `regexp.js` (pattern validator) is the one on-demand piece, reached only for a pattern the host engine rejected — never `require` it from a build path.
41-
- `syntax-printer.js` — webpack's JavaScript minifier, terser's design ported to write byte for byte what terser writes; `jsMinify.js` is the `minify` the default minimizer dispatches JS to under `experiments.futureDefaults`, and goes through it alone; without that flag the minimizer dispatches to `terserMinify` as `minimizer-webpack-plugin` publishes it, running the user's own terser since webpack lists it only as a devDependency. Neither needs an option to say which, and the printer itself never loads terser. `createModules` builds what terser's modules exported — the node classes from `nodeClasses()` (`createAst`), the parser (`createLenientParser`), the compressor and the output stream — and each entry of `PHASES` installs one part onto them: add phases there and nowhere else. `correct` wraps the methods with which terser writes a wrong program and changes only that output; `corrections.enabled` switches it off. `improve` writes less where terser's longer program provably runs the same — after compressing, a statement calling a function in place with nothing passed runs its body there — and `improvements.enabled` switches it off. The `parse` phase builds the tree from `syntax-parser.js`, leaving to the lenient parser a source read unlike the spec; `syntax-printer-terser-parser.unittest.js` holds the ported parser to terser's own and `syntax-printer-estree.unittest.js` the ESTree conversion. `test/specCases/minify-corpora.spectest.js` holds the whole printer to terser's output, errors included, over each corpus in its `CORPORA` list (add a new one there) under the option sets in `OPTION_SETS`, and runs each output whose test states what its input prints (terser's `expect_stdout`, swc's `expected.stdout` and `exec.rs`) in terser's sandbox. A byte difference is accepted only where switching the `correct` and `improve` phases off gives terser's bytes back, and fails where `improve` writes more than without it; what it fixes is listed in `CORRECTED` (printing right with the phase, as terser without), and an output misprinting as terser's by an option's own contract in `INHERITED`, each with the reason — an entry that stops matching fails until retired. Where the reference instead refuses a source the spec allows, the parse error it answers with is listed in `REFUSED_BY_REFERENCE` with what makes the source legal; where it reads or prints one differently, the case is listed in `REFERENCE_MISPRINTS` with what it gets wrong — a listed one webpack must still run correctly, and one the reference stops differing on fails until retired. Each output is also sized against swc's recorded one, a fixture's `output.js` or, for one of terser's cases, what swc's port of it records (gzip decides, raw breaks a tie): one bigger fails unless `test/helpers/swcSmaller.js` lists it under what swc does that webpack does not yet, and a listed one no longer bigger fails until retired. Under the default minimizer's options each source of every corpus but test262 is also minified by `oxc-minify`, held the same way to `test/helpers/oxcSmaller.js`; an oxc output that misprints is no lead. `JS_MINIFY_REPORT=<file>` tallies where webpack's is worse, the same and better. terser's corpus submodule is pinned to the `terser` devDependency — bump both together, and regenerate `data.js`.
42-
- `yarn test:minify-corpora` — webpack's JS minifier vs the published one it replaces, byte for byte (errors included), over every JS corpus: terser's own `test/compress` and `test/input`, test262, swc's minifier tests and oxc's; where a test states what its input prints, the output is also run and must print it. Each output must also be no bigger than swc's recorded one unless `SWC_SMALLER` lists why, and every output outside test262 no bigger than `oxc-minify`'s unless `OXC_SMALLER` does; `JS_MINIFY_REPORT=<file>` tallies worse, the same and better.
41+
- `syntax-printer.js` — webpack's JavaScript minifier, terser's design ported to write byte for byte what terser writes; `jsMinify.js` is the `minify` the default minimizer dispatches JS to under `experiments.futureDefaults`, and goes through it alone; without that flag the minimizer dispatches to `terserMinify` as `minimizer-webpack-plugin` publishes it, running the user's own terser since webpack lists it only as a devDependency. Neither needs an option to say which, and the printer itself never loads terser. `createModules` builds what terser's modules exported — the node classes from `nodeClasses()` (`createAst`), the parser's tables (`createParseTables`), the compressor and the output stream — and each entry of `PHASES` installs one part onto them: add phases there and nowhere else. `correct` wraps the methods with which terser writes a wrong program and changes only that output; `corrections.enabled` switches it off. `improve` writes less where terser's longer program provably runs the same — after compressing, a statement calling a function in place with nothing passed runs its body there — and `improvements.enabled` switches it off. The `parse` phase builds the tree from `syntax-parser.js`; `syntax-printer-estree.unittest.js` holds the ESTree conversion to terser's, and `PARSED_DIFFERENTLY` in `syntax-printer.unittest.js` lists where the tree it builds prints otherwise than terser's. `test/specCases/minify-corpora.spectest.js` holds the whole printer to terser's output, a refusal matching any refusal, over each corpus in its `CORPORA` list (add a new one there) under the option sets in `OPTION_SETS`, and runs each output whose test states what its input prints (terser's `expect_stdout`, swc's `expected.stdout` and `exec.rs`) in terser's sandbox. A byte difference is accepted only where switching the `correct` and `improve` phases off gives terser's bytes back, and fails where `improve` writes more than without it; what it fixes is listed in `CORRECTED` (printing right with the phase, as terser without), and an output misprinting as terser's by an option's own contract in `INHERITED`, each with the reason — an entry that stops matching fails until retired. Where the reference instead refuses a source the spec allows, the parse error it answers with is listed in `REFUSED_BY_REFERENCE` with what makes the source legal; where webpack refuses a source the reference reads, the source is listed in `REFUSED_BY_WEBPACK` with the rule it breaks (a test262 test expecting a parse error needs no entry); where it reads or prints one differently, the case is listed in `REFERENCE_MISPRINTS` with what it gets wrong — a listed one webpack must still run correctly, and one the reference stops differing on fails until retired. Each output is also sized against swc's recorded one, a fixture's `output.js` or, for one of terser's cases, what swc's port of it records (gzip decides, raw breaks a tie): one bigger fails unless `test/helpers/swcSmaller.js` lists it under what swc does that webpack does not yet, and a listed one no longer bigger fails until retired. Under the default minimizer's options each source of every corpus but test262 is also minified by `oxc-minify`, held the same way to `test/helpers/oxcSmaller.js`; an oxc output that misprints is no lead. `JS_MINIFY_REPORT=<file>` tallies where webpack's is worse, the same and better. terser's corpus submodule is pinned to the `terser` devDependency — bump both together, and regenerate `data.js`.
42+
- `yarn test:minify-corpora` — webpack's JS minifier vs the published one it replaces, byte for byte (a refusal matching any refusal), over every JS corpus: terser's own `test/compress` and `test/input`, test262, swc's minifier tests and oxc's; where a test states what its input prints, the output is also run and must print it. Each output must also be no bigger than swc's recorded one unless `SWC_SMALLER` lists why, and every output outside test262 no bigger than `oxc-minify`'s unless `OXC_SMALLER` does; `JS_MINIFY_REPORT=<file>` tallies worse, the same and better.

‎lib/javascript/syntax-printer-data.js‎

Lines changed: 3 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -2182,8 +2182,9 @@ const nodeClasses = () => [
21822182
}
21832183
];
21842184

2185-
// terser's tokenizer and parser tables, read out of its `parse.js`: each a
2186-
// list of words, and `PRECEDENCE` the operators of each level, loosest first.
2185+
// terser's tables the printer names tokens by, read out of its `parse.js`:
2186+
// each a list of words, and `PRECEDENCE` the operators of each level, loosest
2187+
// first.
21872188
/**
21882189
* @returns {Record<string, string[] | string[][]>} the tables, fresh on each call
21892190
*/
@@ -2306,31 +2307,6 @@ const parserTables = () => ({
23062307
"while",
23072308
"with"
23082309
],
2309-
KEYWORDS_BEFORE_EXPRESSION: [
2310-
"await",
2311-
"case",
2312-
"delete",
2313-
"else",
2314-
"new",
2315-
"return",
2316-
"throw",
2317-
"yield"
2318-
],
2319-
OPERATOR_CHARS: [
2320-
"!",
2321-
"%",
2322-
"&",
2323-
"*",
2324-
"+",
2325-
"-",
2326-
"<",
2327-
"=",
2328-
">",
2329-
"?",
2330-
"^",
2331-
"|",
2332-
"~"
2333-
],
23342310
OPERATORS: [
23352311
"!",
23362312
"!=",
@@ -2383,58 +2359,6 @@ const parserTables = () => ({
23832359
"||=",
23842360
"~"
23852361
],
2386-
WHITESPACE_CHARS: [
2387-
"\t",
2388-
"\n",
2389-
"\u000B",
2390-
"\f",
2391-
"\r",
2392-
" ",
2393-
"\u00A0",
2394-
"\u2000",
2395-
"\u2001",
2396-
"\u2002",
2397-
"\u2003",
2398-
"\u2004",
2399-
"\u2005",
2400-
"\u2006",
2401-
"\u2007",
2402-
"\u2008",
2403-
"\u2009",
2404-
"\u200A",
2405-
"\u200B",
2406-
"\u2028",
2407-
"\u2029",
2408-
"\u202F",
2409-
"\u205F",
2410-
"\u3000",
2411-
"\uFEFF"
2412-
],
2413-
NEWLINE_CHARS: ["\n", "\r", "\u2028", "\u2029"],
2414-
PUNC_AFTER_EXPRESSION: [")", ",", ":", ";", "]"],
2415-
PUNC_BEFORE_EXPRESSION: ["(", ",", ":", ";", "[", "{"],
2416-
PUNC_CHARS: ["(", ")", ",", ":", ";", "[", "]", "{", "}"],
2417-
UNARY_PREFIX: ["!", "+", "++", "-", "--", "delete", "typeof", "void", "~"],
2418-
UNARY_POSTFIX: ["++", "--"],
2419-
ASSIGNMENT: [
2420-
"%=",
2421-
"&&=",
2422-
"&=",
2423-
"**=",
2424-
"*=",
2425-
"+=",
2426-
"-=",
2427-
"/=",
2428-
"<<=",
2429-
"=",
2430-
">>=",
2431-
">>>=",
2432-
"??=",
2433-
"^=",
2434-
"|=",
2435-
"||="
2436-
],
2437-
LOGICAL_ASSIGNMENT: ["&&=", "??=", "||="],
24382362
PRECEDENCE: [
24392363
["||"],
24402364
["??"],
@@ -2448,17 +2372,6 @@ const parserTables = () => ({
24482372
["+", "-"],
24492373
["*", "/", "%"],
24502374
["**"]
2451-
],
2452-
ATOMIC_START_TOKEN: ["atom", "big_int", "name", "num", "regexp", "string"],
2453-
TS_PREDEFINED_TYPES: [
2454-
"any",
2455-
"boolean",
2456-
"const",
2457-
"number",
2458-
"string",
2459-
"symbol",
2460-
"unknown",
2461-
"void"
24622375
]
24632376
});
24642377

0 commit comments

Comments
 (0)