Skip to content

Commit 1e8bbd0

Browse files
refactor(javascript): generate the minifier's folded built-ins from browser-compat-data
`tooling/generate-js-data.js` now writes the built-ins `improve` evaluates on literals into `syntax-printer-data.js`. Every member @mdn/browser-compat-data lists of `Math`, `Number`, `String`, `Array` and the global functions is placed as folded, or left out with a reason, and a bump that lists a new one fails generation until it is placed. A folded built-in must be in every Node `engines.node` allows, so `at` and `replaceAll` are no longer folded and output no longer depends on the build's Node; `fromCodePoint` and `normalize` (on ASCII) are now folded. Claude-Session: https://claude.ai/code/session_01ViD41i2KdwwGxrDk7yrjeD
1 parent 6fc0499 commit 1e8bbd0

7 files changed

Lines changed: 538 additions & 120 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,7 @@ Re-run `yarn fix:special` **before the next commit** after touching:
240240

241241
- An option type in `lib/**/*.js` or `declarations/**/*.ts` — rewrites that schema, its validator and `types.d.ts`.
242242
- JSDoc in `lib/**/*.js` reachable from a public export — `types.d.ts`.
243-
- `tooling/generate-runtime-code.js`, `generate-wasm-code.js`, `generate-css-data.js`, `generate-html-data.js`, `generate-js-data.js`, or anything they consume (incl. the `acorn` / `mdn-data` / `color-name` / `@webref/*` versions in `package.json` and `tooling/html-entities.json`).
243+
- `tooling/generate-runtime-code.js`, `generate-wasm-code.js`, `generate-css-data.js`, `generate-html-data.js`, `generate-js-data.js`, or anything they consume (incl. the `acorn` / `mdn-data` / `@mdn/browser-compat-data` / `color-name` / `@webref/*` versions in `package.json` and `tooling/html-entities.json`).
244244

245245
CI's `lint` job verifies these are current; `yarn fix` (`fix:code` + `fix:special` + `fmt`) is the preferred final step.
246246

‎docs/syntax.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Read this before touching the CSS, HTML or JavaScript parser, printer or minifie
2424
- `lib/css/data.js` — every table the CSS minifier looks names up in, plus the arithmetic its math-function descriptors bind to: from `mdn-data` + `color-name` (box shorthands, color-argument and math functions, named colors) and the generator's `SUPPLEMENT` of spec-prose tables and math primitives, by `tooling/generate-css-data.js` — which also holds the value-definition-syntax parser those grammars are read with, and runs generation only as the entry point so its tests can require it.
2525
- `lib/html/data.js` — every table the HTML parser and minifier look names up in: reflected-attribute tables from webref's HTML IDL (`@webref/idl`, `@webref/elements`) plus the generator's `SUPPLEMENT` and `PARSER_TABLES` of §13.2 tree-construction vocabulary, by `tooling/generate-html-data.js` — which also emits the `// #region html entities` block in `lib/html/syntax-parser.js` from the vendored `tooling/html-entities.json` (WHATWG's named character references).
2626
- `lib/javascript/data.js` — the JS parser's and printer's Unicode tables: run-length identifier ranges the tokenizer decodes at its first non-ASCII code point, and per-edition `\p{...}` property names (reached only for a pattern the engine rejected), which `tooling/generate-js-data.js` reads from the pinned acorn devDependency — bumping it moves them; plus the Unicode 8.0 identifier ranges the printer writes unescaped (so older engines read its output), read from terser's own frozen tables.
27-
- `lib/javascript/syntax-printer-data.js` — the tables only the printer reads, in their own module so a build that parses without minifying through the printer never loads them: the native-object tables of pure globals, calls, methods and static properties the compressor reads (`NATIVE_*`, built only when `nativeObjectTables()` is called, so loading the module allocates none), each name with its `unsafe` / `builtins_ecma >= N` conditions in terser's order, parsed with acorn out of terser's `lib/compress/native-objects.js`. Plus terser's node classes (`nodeClasses()`, built on call too): each class's base, the fields its constructor copies, the values on its prototype, and its children in the order its `_walk` and `_children_backwards` reach them, parsed out of terser's `lib/ast.js` — the printer builds every node class and its walk from them. Plus the tokenizer's and parser's tables (`parserTables()`), parsed out of terser's `lib/parse.js`, and the DOM property names property mangling keeps (`domProperties()`), out of its `tools/domprops.js`.
27+
- `lib/javascript/syntax-printer-data.js` — the tables only the printer reads, in their own module so a build that parses without minifying through the printer never loads them: the native-object tables of pure globals, calls, methods and static properties the compressor reads (`NATIVE_*`, built only when `nativeObjectTables()` is called, so loading the module allocates none), each name with its `unsafe` / `builtins_ecma >= N` conditions in terser's order, parsed with acorn out of terser's `lib/compress/native-objects.js`. Plus terser's node classes (`nodeClasses()`, built on call too): each class's base, the fields its constructor copies, the values on its prototype, and its children in the order its `_walk` and `_children_backwards` reach them, parsed out of terser's `lib/ast.js` — the printer builds every node class and its walk from them. Plus the tokenizer's and parser's tables (`parserTables()`), parsed out of terser's `lib/parse.js`, and the DOM property names property mangling keeps (`domProperties()`), out of its `tools/domprops.js`. Plus the built-ins `improve` evaluates on literals (`foldedBuiltIns()`): `tooling/generate-js-data.js` places every member `@mdn/browser-compat-data` lists of `Math`, `Number`, `String`, `Array` and the global functions in `FOLDED_BUILT_INS` or, with the reason, `UNFOLDED_BUILT_INS`, and fails when a bump lists one in neither, or a folded one is missing, misplaced, or newer in Node than `engines.node` (folding it would tie the output to the build's Node).
2828

2929
## CSS
3030

‎lib/javascript/syntax-printer-data.js‎

Lines changed: 88 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
*/
55

66
// GENERATED by tooling/generate-js-data.js — do not edit.
7-
// Sources: acorn 8.18.0, terser 5.51.2. The tables read from terser are under its
7+
// Sources: terser 5.51.2, @mdn/browser-compat-data 8.1.4. The tables read from terser are under its
88
// license, quoted in full in syntax-printer.js.
99

1010
"use strict";
@@ -2471,7 +2471,94 @@ const domProperties = () =>
24712471
" "
24722472
);
24732473

2474+
/**
2475+
* @typedef {object} FoldedBuiltIns
2476+
* @property {string[]} globals the global functions
2477+
* @property {Record<string, string[]>} statics the functions of each global object
2478+
* @property {Record<string, string[]>} methods the prototype methods of each constructor
2479+
* @property {string[]} unicodeDependent the string methods folded on ASCII only
2480+
*/
2481+
2482+
// The built-ins the `improve` phase evaluates on literals, by owner, as
2483+
// tooling/generate-js-data.js checks them against @mdn/browser-compat-data.
2484+
/**
2485+
* @returns {FoldedBuiltIns} the tables, fresh on each call
2486+
*/
2487+
const foldedBuiltIns = () => ({
2488+
globals: [
2489+
"decodeURI",
2490+
"decodeURIComponent",
2491+
"encodeURI",
2492+
"encodeURIComponent",
2493+
"isFinite",
2494+
"isNaN",
2495+
"parseFloat",
2496+
"parseInt"
2497+
],
2498+
statics: {
2499+
Math: [
2500+
"abs",
2501+
"ceil",
2502+
"clz32",
2503+
"floor",
2504+
"fround",
2505+
"imul",
2506+
"max",
2507+
"min",
2508+
"round",
2509+
"sign",
2510+
"sqrt",
2511+
"trunc"
2512+
],
2513+
Number: [
2514+
"isFinite",
2515+
"isInteger",
2516+
"isNaN",
2517+
"isSafeInteger",
2518+
"parseFloat",
2519+
"parseInt"
2520+
],
2521+
String: ["fromCharCode", "fromCodePoint"]
2522+
},
2523+
methods: {
2524+
String: [
2525+
"charAt",
2526+
"charCodeAt",
2527+
"codePointAt",
2528+
"concat",
2529+
"endsWith",
2530+
"includes",
2531+
"indexOf",
2532+
"lastIndexOf",
2533+
"normalize",
2534+
"padEnd",
2535+
"padStart",
2536+
"repeat",
2537+
"replace",
2538+
"slice",
2539+
"startsWith",
2540+
"substr",
2541+
"substring",
2542+
"toLowerCase",
2543+
"toUpperCase",
2544+
"trim",
2545+
"trimEnd",
2546+
"trimStart"
2547+
],
2548+
Array: ["join"]
2549+
},
2550+
unicodeDependent: [
2551+
"normalize",
2552+
"toLowerCase",
2553+
"toUpperCase",
2554+
"trim",
2555+
"trimEnd",
2556+
"trimStart"
2557+
]
2558+
});
2559+
24742560
module.exports.domProperties = domProperties;
2561+
module.exports.foldedBuiltIns = foldedBuiltIns;
24752562
module.exports.nativeObjectTables = nativeObjectTables;
24762563
module.exports.nodeClasses = nodeClasses;
24772564
module.exports.parserTables = parserTables;

‎lib/javascript/syntax-printer.js‎

Lines changed: 37 additions & 105 deletions
Original file line numberDiff line numberDiff line change
@@ -19326,6 +19326,28 @@ const installImprove = (modules) => {
1932619326
return compressed;
1932719327
};
1932819328

19329+
const foldedTables = require("./syntax-printer-data").foldedBuiltIns();
19330+
19331+
/** @type {Map<string, Set<string>>} */
19332+
const foldedStatics = new Map();
19333+
for (const owner of Object.keys(foldedTables.statics)) {
19334+
foldedStatics.set(owner, new Set(foldedTables.statics[owner]));
19335+
}
19336+
const foldedGlobals = new Set(foldedTables.globals);
19337+
const foldedStringMethods = new Set(foldedTables.methods.String);
19338+
const foldedArrayMethods = new Set(foldedTables.methods.Array);
19339+
const unicodeDependentMethods = new Set(foldedTables.unicodeDependent);
19340+
// Every name a folded call can end in, for the check every call meets.
19341+
const foldedNames = new Set([
19342+
...foldedTables.globals,
19343+
...Object.keys(foldedTables.statics).reduce(
19344+
(names, owner) => [...names, ...foldedTables.statics[owner]],
19345+
/** @type {string[]} */ ([])
19346+
),
19347+
...foldedTables.methods.String,
19348+
...foldedTables.methods.Array
19349+
]);
19350+
1932919351
/** @type {(value: unknown, orig: Node) => Node} */
1933019352
const makeNodeFromConstant = modules.common.make_node_from_constant;
1933119353
/** @type {(node: Node) => boolean} */
@@ -19365,40 +19387,40 @@ const installImprove = (modules) => {
1936519387
const builtInOf = (call) => {
1936619388
const callee = call.expression;
1936719389
if (callee instanceof A.AST_SymbolRef) {
19368-
return isUndeclaredRef(callee) &&
19369-
Object.prototype.hasOwnProperty.call(FOLDED_GLOBALS, callee.name)
19370-
? { fn: FOLDED_GLOBALS[callee.name], receiver: undefined }
19390+
return isUndeclaredRef(callee) && foldedGlobals.has(callee.name)
19391+
? {
19392+
fn: /** @type {EXPECTED_ANY} */ (global)[callee.name],
19393+
receiver: undefined
19394+
}
1937119395
: undefined;
1937219396
}
1937319397
if (!(callee instanceof A.AST_Dot) || callee.optional) return undefined;
1937419398
const object = callee.expression;
1937519399
const name = callee.property;
1937619400
if (object instanceof A.AST_SymbolRef) {
1937719401
const methods = isUndeclaredRef(object)
19378-
? FOLDED_STATICS.get(object.name)
19402+
? foldedStatics.get(object.name)
1937919403
: undefined;
1938019404
return methods && methods.has(name)
1938119405
? {
19382-
fn: /** @type {EXPECTED_ANY} */ (FOLDED_STATIC_OWNERS)[object.name][
19383-
name
19384-
],
19406+
fn: /** @type {EXPECTED_ANY} */ (global)[object.name][name],
1938519407
receiver: undefined
1938619408
}
1938719409
: undefined;
1938819410
}
1938919411
if (object instanceof A.AST_String) {
1939019412
// Case and white space follow the engine's Unicode version.
19391-
if (UNICODE_DEPENDENT_METHODS.has(name) && !ASCII.test(object.value)) {
19413+
if (unicodeDependentMethods.has(name) && !ASCII.test(object.value)) {
1939219414
return undefined;
1939319415
}
19394-
return FOLDED_STRING_METHODS.has(name)
19416+
return foldedStringMethods.has(name)
1939519417
? {
1939619418
fn: /** @type {EXPECTED_ANY} */ (String.prototype)[name],
1939719419
receiver: object.value
1939819420
}
1939919421
: undefined;
1940019422
}
19401-
if (object instanceof A.AST_Array && name === "join") {
19423+
if (object instanceof A.AST_Array && foldedArrayMethods.has(name)) {
1940219424
/** @type {unknown[]} */
1940319425
const elements = [];
1940419426
for (const element of object.elements) {
@@ -19407,7 +19429,10 @@ const installImprove = (modules) => {
1940719429
if (value === NOT_CONSTANT) return undefined;
1940819430
elements.push(value);
1940919431
}
19410-
return { fn: Array.prototype.join, receiver: elements };
19432+
return {
19433+
fn: /** @type {EXPECTED_ANY} */ (Array.prototype)[name],
19434+
receiver: elements
19435+
};
1941119436
}
1941219437
return undefined;
1941319438
};
@@ -19477,7 +19502,7 @@ const installImprove = (modules) => {
1947719502
: callee instanceof AST_SymbolRef
1947819503
? callee.name
1947919504
: undefined;
19480-
return FOLDED_NAMES.has(name) &&
19505+
return foldedNames.has(name) &&
1948119506
!(optimized instanceof AST_New) &&
1948219507
compressor.option("evaluate")
1948319508
? foldBuiltInCall(optimized, compressor)
@@ -19489,99 +19514,6 @@ const installImprove = (modules) => {
1948919514
const NOT_CONSTANT = Symbol("not constant");
1949019515
const ASCII = /^[\0-\u007F]*$/;
1949119516

19492-
// The built-ins `improve` evaluates on literals: each exactly specified, so
19493-
// every engine answers alike, and none reading anything but its arguments;
19494-
// `split` is left out, an array literal gzipping worse than the string it splits.
19495-
const FOLDED_STRING_METHODS = new Set([
19496-
"at",
19497-
"charAt",
19498-
"charCodeAt",
19499-
"codePointAt",
19500-
"concat",
19501-
"endsWith",
19502-
"includes",
19503-
"indexOf",
19504-
"lastIndexOf",
19505-
"padEnd",
19506-
"padStart",
19507-
"repeat",
19508-
"replace",
19509-
"replaceAll",
19510-
"slice",
19511-
"startsWith",
19512-
"substr",
19513-
"substring",
19514-
"toLowerCase",
19515-
"toUpperCase",
19516-
"trim",
19517-
"trimEnd",
19518-
"trimStart"
19519-
]);
19520-
const UNICODE_DEPENDENT_METHODS = new Set([
19521-
"toLowerCase",
19522-
"toUpperCase",
19523-
"trim",
19524-
"trimEnd",
19525-
"trimStart"
19526-
]);
19527-
/** @type {Map<string, Set<string>>} */
19528-
const FOLDED_STATICS = new Map([
19529-
[
19530-
"Math",
19531-
new Set([
19532-
"abs",
19533-
"ceil",
19534-
"clz32",
19535-
"floor",
19536-
"fround",
19537-
"imul",
19538-
"max",
19539-
"min",
19540-
"round",
19541-
"sign",
19542-
"sqrt",
19543-
"trunc"
19544-
])
19545-
],
19546-
[
19547-
"Number",
19548-
new Set([
19549-
"isFinite",
19550-
"isInteger",
19551-
"isNaN",
19552-
"isSafeInteger",
19553-
"parseFloat",
19554-
"parseInt"
19555-
])
19556-
],
19557-
["String", new Set(["fromCharCode"])]
19558-
]);
19559-
const FOLDED_STATIC_OWNERS = { Math, Number, String };
19560-
/** @type {Record<string, (...args: EXPECTED_ANY[]) => unknown>} */
19561-
const FOLDED_GLOBALS = {
19562-
decodeURI,
19563-
decodeURIComponent,
19564-
encodeURI,
19565-
encodeURIComponent,
19566-
// The globals convert their argument first, where `Number`'s do not.
19567-
// eslint-disable-next-line unicorn/prefer-number-properties
19568-
isFinite,
19569-
// eslint-disable-next-line unicorn/prefer-number-properties
19570-
isNaN,
19571-
parseFloat: Number.parseFloat,
19572-
parseInt: Number.parseInt
19573-
};
19574-
// Every name a folded call can end in, for the check every call meets.
19575-
const FOLDED_NAMES = new Set([
19576-
...FOLDED_STRING_METHODS,
19577-
...[...FOLDED_STATICS.values()].reduce(
19578-
(names, methods) => [...names, ...methods],
19579-
/** @type {string[]} */ ([])
19580-
),
19581-
...Object.keys(FOLDED_GLOBALS),
19582-
"join"
19583-
]);
19584-
1958519517
// What a regular expression's source may hold for terser to build it: no
1958619518
// pattern that backtracks without bound.
1958719519
const SAFE_REGEXP_SOURCE = /^[\\/|\0\s\w^$.[\]()]*$/;

‎test/helpers/oxcSmaller.js‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,6 @@ const OXC_SMALLER_BY_REASON = [
128128
"oxc minifier: ecmascript/may_have_side_effects.rs:1250",
129129
"oxc minifier: ecmascript/may_have_side_effects.rs:1251",
130130
"oxc minifier: ecmascript/may_have_side_effects.rs:1252",
131-
"oxc minifier: ecmascript/may_have_side_effects.rs:1255",
132131
"oxc minifier: ecmascript/may_have_side_effects.rs:1259",
133132
"oxc minifier: ecmascript/may_have_side_effects.rs:1266",
134133
"oxc minifier: ecmascript/may_have_side_effects.rs:1267",
@@ -166,8 +165,6 @@ const OXC_SMALLER_BY_REASON = [
166165
"oxc minifier: ecmascript/may_have_side_effects.rs:1340",
167166
"oxc minifier: ecmascript/may_have_side_effects.rs:1341",
168167
"oxc minifier: ecmascript/may_have_side_effects.rs:1365",
169-
"oxc minifier: ecmascript/may_have_side_effects.rs:1374",
170-
"oxc minifier: ecmascript/may_have_side_effects.rs:1375",
171168
"oxc minifier: ecmascript/may_have_side_effects.rs:1396",
172169
"oxc minifier: ecmascript/may_have_side_effects.rs:1399",
173170
"oxc minifier: ecmascript/may_have_side_effects.rs:1400",
@@ -2382,6 +2379,16 @@ const OXC_SMALLER_BY_REASON = [
23822379
"oxc minifier: ecmascript/value_type.rs:62"
23832380
]
23842381
],
2382+
[
2383+
"oxc folds `at` and `replaceAll` on literals, which webpack leaves to the engine: the oldest Node it builds on lacks them, and folding them would tie the output to the build's Node",
2384+
[
2385+
"oxc minifier: peephole/replace_known_methods.rs:172",
2386+
"oxc minifier: peephole/replace_known_methods.rs:173",
2387+
"oxc minifier: peephole/replace_known_methods.rs:174",
2388+
"oxc minifier: peephole/replace_known_methods.rs:175",
2389+
"oxc minifier: peephole/replace_known_methods.rs:177"
2390+
]
2391+
],
23852392
[
23862393
"oxc resolves a `switch` on a known value, or writes a `switch` shorter",
23872394
[

‎test/unitCases/syntax-printer.unittest.js‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -225,7 +225,7 @@ const IMPROVED_CASES = [
225225
],
226226
[
227227
"a string method called on a literal",
228-
'console.log("abcde".charAt(1), "abc".at(-1), "abcde".replaceAll("c", "x"));',
228+
'console.log("abcde".charAt(1), "abc".normalize(), "abcde".replace("c", "x"));',
229229
{ compress: {}, mangle: false }
230230
],
231231
[
@@ -240,7 +240,7 @@ const IMPROVED_CASES = [
240240
],
241241
[
242242
"`Number` and `String` functions called on literals",
243-
"console.log(Number.isInteger(5), Number.isSafeInteger(2 ** 53), String.fromCharCode(65, 66));",
243+
"console.log(Number.isInteger(5), Number.isSafeInteger(2 ** 53), String.fromCharCode(65, 66), String.fromCodePoint(67));",
244244
{ compress: {}, mangle: false }
245245
],
246246
[
@@ -302,6 +302,7 @@ const KEPT_CASES = [
302302
["a string method left out", 'try { console.log("a".join()); } catch (e) { console.log(2); }'],
303303
["an array of something no literal", "console.log([Math.random() > 2].join());"],
304304
["a function left out", 'console.log(Math.sin(1), "a,b".split(","));'],
305+
["a method the oldest Node lacks", 'console.log("abc".at(-1), "abc".replaceAll("b", "x"));'],
305306
["an optional call", 'console.log("abc"?.charAt(1));']
306307
];
307308

0 commit comments

Comments
 (0)