Skip to content

Commit 12bd33f

Browse files
alexander-akaitclaude
authored andcommitted
feat(javascript): read a CommonJS module's body with sourceType: "commonjs"
Like acorn's `commonjs` goal: a function's body that a `#!` line may open, which `minify` compresses as a handler's but ends with its semicolon. A body's `#!` line and leading comments now survive the statement they sat on being dropped. Claude-Session: https://claude.ai/code/session_015gdnghKbLJ7VvJ5nNCNHhH
1 parent 14b99f3 commit 12bd33f

7 files changed

Lines changed: 112 additions & 43 deletions

File tree

‎.changeset/041-syntax-parse-error-option.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
"webpack": minor
33
---
44

5-
Add CSS and HTML `parseErrors`, `parseProgram`, and event handler `sourceType`.
5+
Add `parseErrors`, `parseProgram`, and CommonJS and event handler `sourceType`.

‎docs/syntax.md‎

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

‎lib/javascript/syntax-parser.js‎

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -20759,27 +20759,30 @@ const toEstreeObject = (tree, node, fields) => {
2075920759

2076020760
/**
2076120761
* Which production of JavaScript a source is: a module (ECMA-262 §16.2), a
20762-
* script (§16.1), or an event handler attribute's value, which HTML compiles
20763-
* into a function: a `FunctionBody` (§15.2).
20764-
* @typedef {"module" | "script" | "event-handler"} JavascriptSourceType
20762+
* script (§16.1), or a `FunctionBody` (§15.2), as a CommonJS module is in the
20763+
* function Node wraps it in and an event handler attribute's value in the one
20764+
* HTML compiles it into, which a `#!` line cannot open.
20765+
* @typedef {"module" | "script" | "commonjs" | "event-handler"} JavascriptSourceType
2076520766
*/
2076620767

2076720768
/**
2076820769
* The options webpack's parser reads a source as `sourceType` with.
2076920770
* @param {JavascriptSourceType=} sourceType what the source is, a module where not given
2077020771
* @returns {{ sourceType: "module" | "script" | "commonjs", allowHashBang: boolean }} acorn's goal, `commonjs` reading a program as a function's body, and whether a `#!` line opens it
2077120772
*/
20772-
const parserGoal = (sourceType) =>
20773-
sourceType === "event-handler"
20774-
? { sourceType: "commonjs", allowHashBang: false }
20775-
: {
20776-
sourceType: sourceType === "script" ? "script" : "module",
20777-
allowHashBang: true
20778-
};
20773+
const parserGoal = (sourceType) => ({
20774+
sourceType:
20775+
sourceType === "event-handler" || sourceType === "commonjs"
20776+
? "commonjs"
20777+
: sourceType === "script"
20778+
? "script"
20779+
: "module",
20780+
allowHashBang: sourceType !== "event-handler"
20781+
});
2077920782

2078020783
/**
2078120784
* @typedef {object} JavascriptProcessOptions what the JavaScript `SourceProcessor` reads its source as, and prints it with
20782-
* @property {JavascriptSourceType=} sourceType what the source is: a module (the default), a script, or an event handler's function body
20785+
* @property {JavascriptSourceType=} sourceType what the source is: a module (the default), a script, or a function's body, a CommonJS module's or an event handler's
2078320786
* @property {Partial<import("./syntax-printer").FormatSettings>=} format terser's format options a print reads, as `minify` reads them under `format`; `comments`, `beautify` and `source_map` are not read yet
2078420787
*/
2078520788
/** @typedef {import("../util/SourceProcessor").CompiledVisitorMap<JavascriptPath>} JavascriptVisitors */
@@ -21186,7 +21189,7 @@ const A = {
2118621189

2118721190
/**
2118821191
* @typedef {object} JavascriptParseOptions
21189-
* @property {JavascriptSourceType=} sourceType what the source is: a module (the default), a script, or an event handler's function body
21192+
* @property {JavascriptSourceType=} sourceType what the source is: a module (the default), a script, or a function's body, a CommonJS module's or an event handler's
2119021193
* @property {((input: string, code: string, start: number, end: number, severity: ParseErrorSeverity) => void)=} parseError optional callback for each parse error, with the CSS and HTML parsers' signature and Babel's `reasonCode` kebab-cased as `code`; `severity` is `"error"` for one that stops the parse and `"warning"` for one it reads past; without it, the errors read past go unreported
2119121194
*/
2119221195

‎lib/javascript/syntax-printer.js‎

Lines changed: 31 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -889,7 +889,7 @@ const NO_NODE = /** @type {Node} */ (/** @type {Node | null} */ (null));
889889
* @property {(self: Node, name: string) => Node} wrapCommonjs the toplevel wrapped as a CommonJS module
890890
* @property {(self: Node, argsValues: boolean | string) => Node} wrapEnclose the toplevel wrapped in a function called at once
891891
* @property {(self: Node) => Node} wrapFunctionBody a function's body, read as a toplevel, wrapped back in a function
892-
* @property {(self: Node) => Node} unwrapFunctionBody the toplevel `wrapFunctionBody` answered, holding that function's body again
892+
* @property {(self: Node, program: Node) => Node} unwrapFunctionBody the toplevel `wrapFunctionBody` answered, holding that function's body again
893893
* @property {(self: Node) => Node[]} argsAsNames the names a function's parameters bind
894894
* @property {(self: Node) => Node | false | undefined} isBraceless the expression an arrow's body is, where it has no braces
895895
* @property {(self: Node) => number} lengthProperty a function's `length`
@@ -3531,12 +3531,16 @@ const installNodes = (modules) => {
35313531
* The body `wrapFunctionBody` wrapped, compressed, taken out of its
35323532
* function: the toplevel keeps what the compressor left on it for printing.
35333533
* @param {Node} self the toplevel `wrapFunctionBody` answered, compressed
3534+
* @param {Node} program the toplevel it wrapped, whose tokens hold the comments before and after the body
35343535
* @returns {Node} that toplevel, holding the function's body
35353536
*/
3536-
ast.unwrapFunctionBody = function unwrapFunctionBody(self) {
3537+
ast.unwrapFunctionBody = function unwrapFunctionBody(self, program) {
35373538
// A top level not its own keeps the function, its one statement.
35383539
const [wrapper] = /** @type {Node[]} */ (self.body);
35393540
self.body = /** @type {Node} */ (wrapper.body).body;
3541+
// A `#!` line is read off the first token, which no statement may keep.
3542+
self.startToken = program.startToken;
3543+
self.endToken = program.endToken;
35403544
return self;
35413545
};
35423546

@@ -9291,7 +9295,7 @@ const createPropertyMangler = (modules) => {
92919295
* @property {boolean | DriverMangleOptions | undefined} rename whether names are renamed
92929296
* @property {boolean} safari10 whether the output works around that engine
92939297
* @property {false | DriverSourceMapOptions} sourceMap how a source map is written, false for none
9294-
* @property {JavascriptSourceType | undefined} sourceType which production the source is: a module, a script, or an event handler's function body
9298+
* @property {JavascriptSourceType | undefined} sourceType which production the source is: a module, a script, or a function's body, a CommonJS module's or an event handler's
92959299
* @property {boolean} spidermonkey whether the sources are ESTree trees
92969300
* @property {boolean} timings whether how long each step took is answered
92979301
* @property {boolean} toplevel whether top-level names may be renamed or dropped
@@ -9703,14 +9707,14 @@ const installMinify = (modules) => {
97039707
);
97049708
}
97059709
options.format = options.format || options.output || {};
9706-
// A handler's body is its function's: the names it declares are that
9707-
// function's own, those it reads undeclared resolve outside it.
9708-
const functionBody = options.sourceType === "event-handler";
9709-
if (options.sourceType !== undefined) {
9710-
options.module = options.sourceType === "module";
9711-
}
9710+
// A function's body: the names it declares are that function's own,
9711+
// those it reads undeclared resolve outside it.
9712+
const { sourceType } = options;
9713+
const functionBody =
9714+
sourceType === "event-handler" || sourceType === "commonjs";
9715+
if (sourceType !== undefined) options.module = sourceType === "module";
97129716
if (functionBody) {
9713-
options.parse = { ...options.parse, [PARSE_FUNCTION_BODY]: true };
9717+
options.parse = { ...options.parse, [PARSE_FUNCTION_BODY]: sourceType };
97149718
}
97159719
setShorthand("ecma", options, ["parse", "compress", "format"]);
97169720
setShorthand("ie8", options, ["compress", "mangle", "format"]);
@@ -9920,8 +9924,8 @@ const installMinify = (modules) => {
99209924
if (options.enclose) toplevel = ast.wrapEnclose(toplevel, options.enclose);
99219925
// Compressed inside a function, so every rule a function's body reads by
99229926
// holds; printed without it, so it wraps every other wrapper.
9923-
const wrapsBody = functionBody && Boolean(options.compress);
9924-
if (wrapsBody) toplevel = ast.wrapFunctionBody(toplevel);
9927+
const bodyProgram = functionBody && options.compress ? toplevel : undefined;
9928+
if (bodyProgram !== undefined) toplevel = ast.wrapFunctionBody(toplevel);
99259929
if (
99269930
!options.wrap &&
99279931
!options.enclose &&
@@ -9966,7 +9970,9 @@ const installMinify = (modules) => {
99669970
);
99679971
assignNativeLookups(compressor, modules.nativeObjects);
99689972
toplevel = compressor.compress(toplevel);
9969-
if (wrapsBody) toplevel = ast.unwrapFunctionBody(toplevel);
9973+
if (bodyProgram !== undefined) {
9974+
toplevel = ast.unwrapFunctionBody(toplevel, bodyProgram);
9975+
}
99709976
// Unset, the printer keeps what "some" does.
99719977
const comments = hasOwn(options.format, "comments")
99729978
? options.format.comments
@@ -10010,7 +10016,8 @@ const installMinify = (modules) => {
1001010016
if (printsCode) {
1001110017
formatOptions = { ...options.format };
1001210018
if (destroysTree) formatOptions._destroy_ast = true;
10013-
if (functionBody) formatOptions._function_body = true;
10019+
// A handler is never joined to other code, so its last `;` goes.
10020+
if (sourceType === "event-handler") formatOptions._function_body = true;
1001410021
if (options.sourceMap) {
1001510022
if (options.sourceMap.includeSources && isToplevelNode(files)) {
1001610023
throw new Error("original source content unavailable");
@@ -13329,7 +13336,7 @@ const throwParseError = (message, filename, line, col, pos) => {
1332913336
throw new JSParseError(message, filename, line, col, pos);
1333013337
};
1333113338

13332-
/** @typedef {{ bare_returns?: boolean, ecma?: number | null, filename?: string | null, html5_comments?: boolean, module?: boolean, shebang?: boolean, toplevel?: Node | null, [PARSE_ONTO_TABLES]?: TokenTables, [PARSE_FUNCTION_BODY]?: boolean }} ParseOptions terser's parse options, the tables a parse joining a tree numbers its tokens after, and whether the source is a function's body */
13339+
/** @typedef {{ bare_returns?: boolean, ecma?: number | null, filename?: string | null, html5_comments?: boolean, module?: boolean, shebang?: boolean, toplevel?: Node | null, [PARSE_ONTO_TABLES]?: TokenTables, [PARSE_FUNCTION_BODY]?: "commonjs" | "event-handler" }} ParseOptions terser's parse options, the tables a parse joining a tree numbers its tokens after, and whether the source is a function's body */
1333313340

1333413341
/**
1333513342
* What the parse phase shares with the rest: the operator precedences and
@@ -17858,8 +17865,9 @@ const NO_TOKEN_TABLES = new TokenTables();
1785817865
// join that tree: its tokens are numbered after the tree's own.
1785917866
const PARSE_ONTO_TABLES = Symbol("tables a parse numbers its tokens after");
1786017867

17861-
// The parse option reading a source as a function's body, as an event handler
17862-
// attribute's is: `return` and `new.target` at its top, no `#!` line.
17868+
// The parse option reading a source as a function's body, the `sourceType`
17869+
// naming whose: `return` and `new.target` at its top, and a `#!` line opening
17870+
// only a CommonJS module's.
1786317871
const PARSE_FUNCTION_BODY = Symbol("a source read as a function body");
1786417872

1786517873
/**
@@ -18503,11 +18511,12 @@ const installFusedParse = (modules) => {
1850318511

1850418512
/**
1850518513
* @param {string} source the source
18506-
* @param {typeof PARSE_DEFAULTS & { [PARSE_FUNCTION_BODY]?: boolean }} options terser's parse options
18514+
* @param {typeof PARSE_DEFAULTS & { [PARSE_FUNCTION_BODY]?: "commonjs" | "event-handler" }} options terser's parse options
1850718515
* @returns {import("./syntax-parser").Options & { importPhases: boolean, lazyNodes: boolean }} what to ask webpack's parser for
1850818516
*/
1850918517
const parserOptions = (source, options) => {
18510-
const functionBody = options[PARSE_FUNCTION_BODY] === true;
18518+
const body = options[PARSE_FUNCTION_BODY];
18519+
const functionBody = body !== undefined;
1851118520
return {
1851218521
ecmaVersion: "latest",
1851318522
// acorn's `commonjs` reads the program as a function's body.
@@ -18528,7 +18537,9 @@ const installFusedParse = (modules) => {
1852818537
allowAwaitOutsideFunction: Boolean(options.module),
1852918538
allowSuperOutsideMethod: true,
1853018539
allowHashBang:
18531-
!functionBody && Boolean(options.shebang) && source.startsWith("#!"),
18540+
body !== "event-handler" &&
18541+
Boolean(options.shebang) &&
18542+
source.startsWith("#!"),
1853218543
// terser reads module syntax in a script too, and `import defer`.
1853318544
allowImportExportEverywhere: !functionBody && !options.module,
1853418545
importPhases: true,

‎test/unitCases/javascript/parser.unittest.js‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5834,7 +5834,25 @@ describe("parseProgram", () => {
58345834
"export {}",
58355835
[["import-outside-module", "export", "error", "0"]]
58365836
],
5837-
// Only a script or a module opens with a `#!` line.
5837+
// A CommonJS module is a function's body too, which a `#!` line may open.
5838+
["commonjs", "#!x\nif (a) return new.target; await = 1", []],
5839+
[
5840+
"commonjs",
5841+
'import x from "y"',
5842+
[["import-outside-module", "import", "error", "0"]]
5843+
],
5844+
[
5845+
"commonjs",
5846+
"import.meta",
5847+
[["import-meta-outside-module", "import", "warning", "0"]]
5848+
],
5849+
// A handler is never a module: what only a module may hold stays out.
5850+
[
5851+
"event-handler",
5852+
"import.meta",
5853+
[["import-meta-outside-module", "import", "warning", "0"]]
5854+
],
5855+
// Only a script, a module or a CommonJS module opens with a `#!` line.
58385856
[
58395857
"event-handler",
58405858
"#!x\n1",
@@ -5843,9 +5861,10 @@ describe("parseProgram", () => {
58435861
["module", "#!x\n1", []]
58445862
])("reads a source as %s: %j", (sourceType, source, expected) => {
58455863
const { program, errors } = collect(source, {
5846-
sourceType: /** @type {"module" | "script" | "event-handler"} */ (
5847-
sourceType
5848-
)
5864+
sourceType:
5865+
/** @type {import("../../../lib/javascript/syntax-parser").JavascriptSourceType} */ (
5866+
sourceType
5867+
)
58495868
});
58505869
expect(errors).toEqual(expected);
58515870
expect(program === undefined).toBe(

‎test/unitCases/javascript/printer.unittest.js‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
// cspell:ignore binop, fnames, propmangle, fargs, domprops, argnames, nondeferred, loopcontrol, Defun, defun, NOINLINE, Funarg, unmangleable, Unmangleable, thedef, funs
44
// cspell:ignore fnames
55
// cspell:ignore argnames, bcatch, bfinally, argname
6+
// cspell:ignore AAAWA MAAH
67
// cspell:ignore endline, endcol, endpos, nlb, thedef
78

89
const vm = require("vm");
@@ -14991,6 +14992,41 @@ describe("JavaScript minifier reading an event handler's body", () => {
1499114992
}
1499214993
});
1499314994

14995+
it.each([
14996+
// Its declared names are the module's own, its `#!` line kept.
14997+
[
14998+
"#!/usr/bin/env node\nvar v = 5; exports.x = v",
14999+
"#!/usr/bin/env node\nexports.x=5;"
15000+
],
15001+
["value = 5; alert(value)", "value=5,alert(value);"],
15002+
["x(); return new.target", "return x(),new.target;"]
15003+
])("should compress %j as a CommonJS module's body", async (source, expected) => {
15004+
const { minify } = await load();
15005+
const { code } = await minify(source, { sourceType: "commonjs" });
15006+
expect(code).toBe(expected);
15007+
});
15008+
15009+
it("should map a body where it was read", async () => {
15010+
const { minify } = await load();
15011+
const { code, map } = await minify(
15012+
{ "a.js": "var v = 5; alert(v)" },
15013+
{ sourceType: "event-handler", sourceMap: { asObject: true } }
15014+
);
15015+
expect(code).toBe("alert(5)");
15016+
// `alert` at column 0 maps back to column 11, where the source has it.
15017+
expect(map).toEqual(
15018+
expect.objectContaining({ names: ["alert"], mappings: "AAAWA,MAAH" })
15019+
);
15020+
});
15021+
15022+
it("should keep the comments around a body whose statements are dropped", async () => {
15023+
const { minify } = await load();
15024+
const { code } = await minify("/*! a */ var v = 5; f(v) /*! b */", {
15025+
sourceType: "event-handler"
15026+
});
15027+
expect(code).toBe("/*! a */f(5)/*! b */");
15028+
});
15029+
1499415030
it("should wrap a body as `wrap` and `enclose` ask, inside its function", async () => {
1499515031
const { minify } = await load();
1499615032
for (const given of [{ enclose: true }, { wrap: "lib" }]) {

‎types.d.ts‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15320,9 +15320,9 @@ type JavascriptNormalModule = NormalModule & {
1532015320
};
1532115321
declare interface JavascriptParseOptions {
1532215322
/**
15323-
* what the source is: a module (the default), a script, or an event handler's function body
15323+
* what the source is: a module (the default), a script, or a function's body, a CommonJS module's or an event handler's
1532415324
*/
15325-
sourceType?: "module" | "script" | "event-handler";
15325+
sourceType?: "module" | "commonjs" | "script" | "event-handler";
1532615326

1532715327
/**
1532815328
* optional callback for each parse error, with the CSS and HTML parsers' signature and Babel's `reasonCode` kebab-cased as `code`; `severity` is `"error"` for one that stops the parse and `"warning"` for one it reads past; without it, the errors read past go unreported
@@ -17906,9 +17906,9 @@ type JavascriptParserState = ParserStateBase<BuiltJavascriptModule> &
1790617906
*/
1790717907
declare interface JavascriptProcessOptions {
1790817908
/**
17909-
* what the source is: a module (the default), a script, or an event handler's function body
17909+
* what the source is: a module (the default), a script, or a function's body, a CommonJS module's or an event handler's
1791017910
*/
17911-
sourceType?: "module" | "script" | "event-handler";
17911+
sourceType?: "module" | "commonjs" | "script" | "event-handler";
1791217912

1791317913
/**
1791417914
* terser's format options a print reads, as `minify` reads them under `format`; `comments`, `beautify` and `source_map` are not read yet
@@ -20837,7 +20837,7 @@ declare interface MinifyOptions {
2083720837
/**
2083820838
* which production the source is, `module` read from it where given
2083920839
*/
20840-
sourceType?: "module" | "script" | "event-handler";
20840+
sourceType?: "module" | "commonjs" | "script" | "event-handler";
2084120841

2084220842
/**
2084320843
* which edition the output may use

0 commit comments

Comments
 (0)