Skip to content

Commit 5f281e9

Browse files
fix(javascript): find an HTML comment's line by scanning, not by pattern
A pattern spelling `<!--` and `-->` reads as a filter over HTML, which CodeQL reports as an incomplete one; this decides which parser reads a source, and filters nothing. A scan says the same thing without a pattern, and skips the line it would have walked where neither spelling occurs. Also say in docs/syntax.md what the printer's `parse` phase now does, and what `parse.webpackParser` turns off.
1 parent 755942b commit 5f281e9

2 files changed

Lines changed: 32 additions & 4 deletions

File tree

‎docs/syntax.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ Read this before touching the CSS, HTML or JavaScript parser, printer or minifie
3636

3737
## JavaScript
3838

39-
- `syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser only for its `parse` phase.
39+
- `syntax.js` reaches `parser` and `printer` through getters: parsing never loads the printer, and the printer loads the parser for its `parse` phase, which reads an asset with it — building the printer's tree as each statement finishes, rather than converting a tree. `parse.webpackParser` defaults to true and turns that off; terser's own parser reads what it reads its own way (`parse.expression`, `parse.strict`, TypeScript, a line opening an HTML comment) and whatever webpack's parser refuses.
4040
- `syntax-parser.js` — tokenizer, acorn-derived core and every production in one file (any parse reaches the productions, and the coming struct-of-arrays rewrite moves node creation through them). `regexp.js` (pattern validator) is the one on-demand piece, reached only for a pattern the host engine rejected — never `require` it from a build path.
4141
- `syntax-printer.js` — webpack's JavaScript minifier, terser's design ported to write byte for byte what terser writes; `jsMinify.js` (the `minify` the default minimizer dispatches JS to) goes through it when given `printer: true` — which `optimization.minimizeOptions.javascript` defaults to under `experiments.futureDefaults` — and through terser as published otherwise — the user's own, as `terser-webpack-plugin` brings it, since webpack lists terser only as a devDependency; the printer itself never loads terser. `createModules` builds what terser's modules exported — the node classes from `nodeClasses()` (`createAst`), the parser (`createTerserParser`), the compressor and the output stream — and each entry of `PHASES` installs one part onto them: add phases there and nowhere else. `correct` wraps the methods with which terser writes a wrong program and changes only that output; `corrections.enabled` switches it off. `syntax-printer-terser-parser.unittest.js` holds the ported parser to terser's over acorn's corpus and `syntax-printer-estree.unittest.js` the ESTree conversion. `test/specCases/minify-corpora.spectest.js` holds the whole printer to terser's output, errors included, over each corpus in its `CORPORA` list (add a new one there) under the option sets in `OPTION_SETS`, and runs each output whose test states what its input prints (terser's `expect_stdout`, swc's `expected.stdout` and `exec.rs`) in terser's sandbox. A byte difference is accepted only where switching the `correct` phase off gives terser's bytes back; what it fixes is listed in `CORRECTED` (printing right with the phase, as terser without), and an output misprinting as terser's by an option's own contract in `INHERITED`, each with the reason — an entry that stops matching fails until retired. `JS_MINIFY_REPORT=<file>` lists where swc's recorded output is smaller — leads for printing less, never a failure. terser's corpus submodule is pinned to the `terser` devDependency — bump both together, and regenerate `data.js`.
4242
- `yarn test:minify-corpora` — webpack's JS minifier vs the published one it replaces, byte for byte (errors included), over every JS corpus: terser's own `test/compress` and `test/input`, test262, and swc's minifier tests; where a test states what its input prints, the output is also run and must print it. `JS_MINIFY_REPORT=<file>` writes where swc's recorded output is smaller.

‎lib/javascript/syntax-printer.js‎

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16506,8 +16506,36 @@ const installFusedParse = (modules) => {
1650616506
// What `ends` holds for the token past the source, which no node ends at.
1650716507
const NO_END = 0x7fffffff;
1650816508

16509-
// A line opening an HTML comment, which only terser reads as one.
16510-
const HTML_COMMENT_LINE = /^[\t ]*(?:<!--|-->)/m;
16509+
// What terser reads as an HTML comment where a line opens with it.
16510+
const HTML_COMMENT_MARKERS = ["<!--", "-->"];
16511+
16512+
/**
16513+
* Whether a line of the source opens with what only terser reads as a
16514+
* comment. Written as a scan rather than a pattern: a pattern over these
16515+
* spellings reads as an HTML filter, which this is not.
16516+
* @param {string} text the source
16517+
* @returns {boolean} whether a line opens with one
16518+
*/
16519+
const opensHtmlComment = (text) => {
16520+
for (const marker of HTML_COMMENT_MARKERS) {
16521+
let at = text.indexOf(marker);
16522+
while (at !== -1) {
16523+
let opens = at;
16524+
while (opens > 0) {
16525+
const code = text.charCodeAt(opens - 1);
16526+
if (code !== 32 && code !== 9) break;
16527+
opens--;
16528+
}
16529+
if (opens === 0) return true;
16530+
const code = text.charCodeAt(opens - 1);
16531+
if (code === 10 || code === 13 || code === 0x2028 || code === 0x2029) {
16532+
return true;
16533+
}
16534+
at = text.indexOf(marker, at + 1);
16535+
}
16536+
}
16537+
return false;
16538+
};
1651116539

1651216540
// The list every token with no comments on either side shares. Nothing writes
1651316541
// into it: a gap gaining a comment takes a list of its own first.
@@ -16897,7 +16925,7 @@ const installFusedParse = (modules) => {
1689716925
// does. Either spelling is far more often written inside a string or a
1689816926
// comment, where it means nothing, so only a line opening with one goes
1689916927
// to terser; anywhere else, a source webpack's parser then refuses does.
16900-
(options.html5_comments !== false && HTML_COMMENT_LINE.test(text))
16928+
(options.html5_comments !== false && opensHtmlComment(text))
1690116929
) {
1690216930
return terserParse(text, givenOptions);
1690316931
}

0 commit comments

Comments
 (0)