You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 91c28a1
Browse filesBrowse the repository at this point in the historyBrowse files
refactor(javascript): read sources with webpack's parser alone when minifying
Drop terser's lenient parser from the printer: webpack's parser is the
only one, so a source it refuses is refused, and terser's `expression`,
`strict` and `experimental_typescript` parse options are refused too.
Copy file name to clipboardExpand all lines: docs/syntax.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,7 +24,7 @@ Read this before touching the CSS, HTML or JavaScript parser, printer or minifie
24
24
-`lib/css/data.js` — every table the CSS minifier looks names up in, plus the arithmetic its math-function descriptors bind to: from `mdn-data` + `color-name` (box shorthands, color-argument and math functions, named colors) and the generator's `SUPPLEMENT` of spec-prose tables and math primitives, by `tooling/generate-css-data.js` — which also holds the value-definition-syntax parser those grammars are read with, and runs generation only as the entry point so its tests can require it.
25
25
-`lib/html/data.js` — every table the HTML parser and minifier look names up in: reflected-attribute tables from webref's HTML IDL (`@webref/idl`, `@webref/elements`) plus the generator's `SUPPLEMENT` and `PARSER_TABLES` of §13.2 tree-construction vocabulary, by `tooling/generate-html-data.js` — which also emits the `// #region html entities` block in `lib/html/syntax-parser.js` from the vendored `tooling/html-entities.json` (WHATWG's named character references).
26
26
-`lib/javascript/data.js` — the JS parser's and printer's Unicode tables: run-length identifier ranges the tokenizer decodes at its first non-ASCII code point, and per-edition `\p{...}` property names (reached only for a pattern the engine rejected), which `tooling/generate-js-data.js` reads from the pinned acorn devDependency — bumping it moves them; plus the Unicode 8.0 identifier ranges the printer writes unescaped (so older engines read its output), read from terser's own frozen tables.
27
-
- `lib/javascript/syntax-printer-data.js` — the tables only the printer reads, in their own module so a build that parses without minifying through the printer never loads them: the native-object tables of pure globals, calls, methods and static properties the compressor reads (`NATIVE_*`, built only when `nativeObjectTables()` is called, so loading the module allocates none), each name with its `unsafe` / `builtins_ecma >= N` conditions in terser's order, parsed with acorn out of terser's `lib/compress/native-objects.js`. Plus terser's node classes (`nodeClasses()`, built on call too): each class's base, the fields its constructor copies, the values on its prototype, and its children in the order its `_walk` and `_children_backwards` reach them, parsed out of terser's `lib/ast.js` — the printer builds every node class and its walk from them. Plus the tokenizer's and parser's tables (`parserTables()`), parsed out of terser's `lib/parse.js`, and the DOM property names property mangling keeps (`domProperties()`), out of its `tools/domprops.js`.
27
+
- `lib/javascript/syntax-printer-data.js` — the tables only the printer reads, in their own module so a build that parses without minifying through the printer never loads them: the native-object tables of pure globals, calls, methods and static properties the compressor reads (`NATIVE_*`, built only when `nativeObjectTables()` is called, so loading the module allocates none), each name with its `unsafe` / `builtins_ecma >= N` conditions in terser's order, parsed with acorn out of terser's `lib/compress/native-objects.js`. Plus terser's node classes (`nodeClasses()`, built on call too): each class's base, the fields its constructor copies, the values on its prototype, and its children in the order its `_walk` and `_children_backwards` reach them, parsed out of terser's `lib/ast.js` — the printer builds every node class and its walk from them. Plus the keyword, reserved-word and operator-precedence tables the printer names tokens by (`parserTables()`), parsed out of terser's `lib/parse.js`, and the DOM property names property mangling keeps (`domProperties()`), out of its `tools/domprops.js`.
28
28
29
29
## CSS
30
30
@@ -36,7 +36,7 @@ Read this before touching the CSS, HTML or JavaScript parser, printer or minifie
36
36
37
37
## JavaScript
38
38
39
-
-`syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser for its `parse` phase, which reads an asset with it — building the printer's tree as each statement finishes, rather than converting a tree. `parse.webpackParser` defaults to true and turns that off; terser's own parser reads what it reads its own way (`parse.expression`, `parse.strict`, TypeScript, a line opening an HTML comment) and whatever webpack's parser refuses.
39
+
-`syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser for its `parse` phase, which reads an asset with it — building the printer's tree as each statement finishes, rather than converting a tree. It is the only parser: a source it refuses is refused, and terser's parse options for what is not a program (`parse.expression`, `parse.strict`, `parse.experimental_typescript`) are refused too.
40
40
-`syntax-parser.js` — tokenizer, acorn-derived core and every production in one file (any parse reaches the productions, and the coming struct-of-arrays rewrite moves node creation through them). `regexp.js` (pattern validator) is the one on-demand piece, reached only for a pattern the host engine rejected — never `require` it from a build path.
41
-
- `syntax-printer.js` — webpack's JavaScript minifier, terser's design ported to write byte for byte what terser writes; `jsMinify.js` is the `minify` the default minimizer dispatches JS to under `experiments.futureDefaults`, and goes through it alone; without that flag the minimizer dispatches to `terserMinify` as `minimizer-webpack-plugin` publishes it, running the user's own terser since webpack lists it only as a devDependency. Neither needs an option to say which, and the printer itself never loads terser. `createModules` builds what terser's modules exported — the node classes from `nodeClasses()` (`createAst`), the parser (`createLenientParser`), the compressor and the output stream — and each entry of `PHASES` installs one part onto them: add phases there and nowhere else. `correct` wraps the methods with which terser writes a wrong program and changes only that output; `corrections.enabled` switches it off. `improve` writes less where terser's longer program provably runs the same — after compressing, a statement calling a function in place with nothing passed runs its body there — and `improvements.enabled` switches it off. The `parse` phase builds the tree from `syntax-parser.js`, leaving to the lenient parser a source read unlike the spec; `syntax-printer-terser-parser.unittest.js` holds the ported parser to terser's own and `syntax-printer-estree.unittest.js` the ESTree conversion. `test/specCases/minify-corpora.spectest.js` holds the whole printer to terser's output, errors included, over each corpus in its `CORPORA` list (add a new one there) under the option sets in `OPTION_SETS`, and runs each output whose test states what its input prints (terser's `expect_stdout`, swc's `expected.stdout` and `exec.rs`) in terser's sandbox. A byte difference is accepted only where switching the `correct` and `improve` phases off gives terser's bytes back, and fails where `improve` writes more than without it; what it fixes is listed in `CORRECTED` (printing right with the phase, as terser without), and an output misprinting as terser's by an option's own contract in `INHERITED`, each with the reason — an entry that stops matching fails until retired. Where the reference instead refuses a source the spec allows, the parse error it answers with is listed in `REFUSED_BY_REFERENCE` with what makes the source legal; where it reads or prints one differently, the case is listed in `REFERENCE_MISPRINTS` with what it gets wrong — a listed one webpack must still run correctly, and one the reference stops differing on fails until retired. Each output is also sized against swc's recorded one, a fixture's `output.js` or, for one of terser's cases, what swc's port of it records (gzip decides, raw breaks a tie): one bigger fails unless `test/helpers/swcSmaller.js` lists it under what swc does that webpack does not yet, and a listed one no longer bigger fails until retired. Under the default minimizer's options each source of every corpus but test262 is also minified by `oxc-minify`, held the same way to `test/helpers/oxcSmaller.js`; an oxc output that misprints is no lead. `JS_MINIFY_REPORT=<file>` tallies where webpack's is worse, the same and better. terser's corpus submodule is pinned to the `terser` devDependency — bump both together, and regenerate `data.js`.
42
-
-`yarn test:minify-corpora` — webpack's JS minifier vs the published one it replaces, byte for byte (errors included), over every JS corpus: terser's own `test/compress` and `test/input`, test262, swc's minifier tests and oxc's; where a test states what its input prints, the output is also run and must print it. Each output must also be no bigger than swc's recorded one unless `SWC_SMALLER` lists why, and every output outside test262 no bigger than `oxc-minify`'s unless `OXC_SMALLER` does; `JS_MINIFY_REPORT=<file>` tallies worse, the same and better.
41
+
- `syntax-printer.js` — webpack's JavaScript minifier, terser's design ported to write byte for byte what terser writes; `jsMinify.js` is the `minify` the default minimizer dispatches JS to under `experiments.futureDefaults`, and goes through it alone; without that flag the minimizer dispatches to `terserMinify` as `minimizer-webpack-plugin` publishes it, running the user's own terser since webpack lists it only as a devDependency. Neither needs an option to say which, and the printer itself never loads terser. `createModules` builds what terser's modules exported — the node classes from `nodeClasses()` (`createAst`), the parser's tables (`createParseTables`), the compressor and the output stream — and each entry of `PHASES` installs one part onto them: add phases there and nowhere else. `correct` wraps the methods with which terser writes a wrong program and changes only that output; `corrections.enabled` switches it off. `improve` writes less where terser's longer program provably runs the same — after compressing, a statement calling a function in place with nothing passed runs its body there — and `improvements.enabled` switches it off. The `parse` phase builds the tree from `syntax-parser.js`; `syntax-printer-estree.unittest.js` holds the ESTree conversion to terser's, and `PARSED_DIFFERENTLY` in `syntax-printer.unittest.js` lists where the tree it builds prints otherwise than terser's. `test/specCases/minify-corpora.spectest.js` holds the whole printer to terser's output, a refusal matching any refusal, over each corpus in its `CORPORA` list (add a new one there) under the option sets in `OPTION_SETS`, and runs each output whose test states what its input prints (terser's `expect_stdout`, swc's `expected.stdout` and `exec.rs`) in terser's sandbox. A byte difference is accepted only where switching the `correct` and `improve` phases off gives terser's bytes back, and fails where `improve` writes more than without it; what it fixes is listed in `CORRECTED` (printing right with the phase, as terser without), and an output misprinting as terser's by an option's own contract in `INHERITED`, each with the reason — an entry that stops matching fails until retired. Where the reference instead refuses a source the spec allows, the parse error it answers with is listed in `REFUSED_BY_REFERENCE` with what makes the source legal; where webpack refuses a source the reference reads, the source is listed in `REFUSED_BY_WEBPACK` with the rule it breaks (a test262 test expecting a parse error needs no entry); where it reads or prints one differently, the case is listed in `REFERENCE_MISPRINTS` with what it gets wrong — a listed one webpack must still run correctly, and one the reference stops differing on fails until retired. Each output is also sized against swc's recorded one, a fixture's `output.js` or, for one of terser's cases, what swc's port of it records (gzip decides, raw breaks a tie): one bigger fails unless `test/helpers/swcSmaller.js` lists it under what swc does that webpack does not yet, and a listed one no longer bigger fails until retired. Under the default minimizer's options each source of every corpus but test262 is also minified by `oxc-minify`, held the same way to `test/helpers/oxcSmaller.js`; an oxc output that misprints is no lead. `JS_MINIFY_REPORT=<file>` tallies where webpack's is worse, the same and better. terser's corpus submodule is pinned to the `terser` devDependency — bump both together, and regenerate `data.js`.
42
+
-`yarn test:minify-corpora` — webpack's JS minifier vs the published one it replaces, byte for byte (a refusal matching any refusal), over every JS corpus: terser's own `test/compress` and `test/input`, test262, swc's minifier tests and oxc's; where a test states what its input prints, the output is also run and must print it. Each output must also be no bigger than swc's recorded one unless `SWC_SMALLER` lists why, and every output outside test262 no bigger than `oxc-minify`'s unless `OXC_SMALLER` does; `JS_MINIFY_REPORT=<file>` tallies worse, the same and better.
0 commit comments