Skip to content

Commit 11f73f2

Browse files
alexander-akaitclaude
alexander-akait
authored and
claude
committed
fix(javascript): name a parse error whose message holds a line terminator
A duplicate export or attribute name can hold a cooked line terminator, which the code table's `.*` did not match, so the search ran past the catch-all and threw. Templates now match across line terminators. Also: parseProgram takes the fast constructor at offset 0, the HTML tables it added move to the generator, and the no-callback behavior is documented. Claude-Session: https://claude.ai/code/session_015gdnghKbLJ7VvJ5nNCNHhH
1 parent 8b7870c commit 11f73f2

9 files changed

Lines changed: 88 additions & 38 deletions

File tree

‎docs/syntax.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ One visitor serves several node types through an array, resolved once in `use()`
8484
Conventions every language keeps, and a new member follows:
8585

8686
- **A node is an integer id**, valid until the next parse; `0` is no node, and an HTML attribute's id is negative. The path is one object rebound before each callback; read it during the callback, never keep it.
87-
- **A parse entry point returns ids too**, read through the exported path `A` until the next parse (`parseHtml`, CSS's `parseA*`, JavaScript's `parseProgram`), and takes a `parseError(input, code, start, end, severity)` option reporting each parse error of the tokenizer and of the parser or tree construction: `"error"` where end of input cuts a token or rule short, or where a JavaScript one stops the parse, and `"warning"` otherwise. No spec names them all, so HTML's tree-construction errors take html5lib-python's names, JavaScript's take Babel's `reasonCode` kebab-cased (read off probes by `tooling/generate-js-data.js`), and CSS's are webpack's own.
87+
- **A parse entry point returns ids too**, read through the exported path `A` until the next parse (`parseHtml`, CSS's `parseA*`, JavaScript's `parseProgram`), and takes a `parseError(input, code, start, end, severity)` option reporting each parse error of the tokenizer and of the parser or tree construction: `"error"` where end of input cuts a token or rule short, or where a JavaScript one stops the parse, and `"warning"` otherwise; without the option, a `"warning"` goes unreported and `parseProgram` returns `undefined` for an `"error"`. No spec names them all, so HTML's tree-construction errors take html5lib-python's names, JavaScript's take Babel's `reasonCode` kebab-cased (read off probes by `tooling/generate-js-data.js`), and CSS's are webpack's own.
8888
- **The node is the last argument and optional**, defaulting to the current one: `path.range()`, `path.range(other)`, `path.child(0)`.
8989
- **A node has at most one `name` and one `value`; every other part is a `Field`, a `Flag` or a `Part`** (its position, and its text when it has more than the name and value), so the generic members cover any grammar instead of one method per part.
9090
- **`name` and `value` are what the language's spec defines**, decoded as ESTree's `Identifier.name` and `Literal.value` are: escapes and character references resolved, a number as a number, a comment's text without its delimiters. **`source` is exactly the input**, so the spelling as written is `source(path.startOf(Part.name), path.endOf(Part.name))`. A node without a name or value answers `""`, a missing part `null`.

‎lib/css/syntax-parser.js‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1598,9 +1598,8 @@ const Part = {
15981598

15991599
/**
16001600
* A node reference into the struct-of-arrays AST: an integer id indexing the
1601-
* parallel columns. Read fields through the exported accessor `A`. Refs are only
1602-
* valid until the next parse — the columns are reused across parses — so
1603-
* consume a tree fully before parsing again.
1601+
* parallel columns, read through the exported accessor `A`. Refs are only valid
1602+
* until the next parse, which reuses the columns, so consume a tree before then.
16041603
* @typedef {number} CssNodeRef
16051604
*/
16061605

‎lib/html/data.js‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1997,6 +1997,43 @@ const JAVASCRIPT_SCRIPT_TYPES = new Set([
19971997
"text/x-javascript"
19981998
]);
19991999

2000+
/**
2001+
* Start tags besides the void elements whose self-closing flag tree construction acknowledges.
2002+
* @type {Set<string>}
2003+
*/
2004+
const ACKNOWLEDGED_NON_VOID = new Set([
2005+
"basefont",
2006+
"bgsound",
2007+
"frame",
2008+
"image"
2009+
]);
2010+
2011+
/**
2012+
* What the end of the body may leave open without a parse error: elements whose end tag is optional.
2013+
* @type {Set<string>}
2014+
*/
2015+
const END_TAG_OPTIONAL_AT_BODY_END = new Set([
2016+
"dd",
2017+
"dt",
2018+
"li",
2019+
"optgroup",
2020+
"option",
2021+
"p",
2022+
"rb",
2023+
"rp",
2024+
"rt",
2025+
"rtc",
2026+
"tbody",
2027+
"td",
2028+
"tfoot",
2029+
"th",
2030+
"thead",
2031+
"tr",
2032+
"body",
2033+
"html"
2034+
]);
2035+
2036+
module.exports.ACKNOWLEDGED_NON_VOID = ACKNOWLEDGED_NON_VOID;
20002037
module.exports.ADDRESS_DIV_P = ADDRESS_DIV_P;
20012038
module.exports.APPLET_MARQUEE_OBJECT = APPLET_MARQUEE_OBJECT;
20022039
module.exports.BLOCK_END = BLOCK_END;
@@ -2016,6 +2053,7 @@ module.exports.DOM_TOKEN_LIST_ATTRIBUTES = DOM_TOKEN_LIST_ATTRIBUTES;
20162053
module.exports.EMPTY_ELEMENT_DROPPED = EMPTY_ELEMENT_DROPPED;
20172054
module.exports.EMPTY_METADATA_ELEMENTS = EMPTY_METADATA_ELEMENTS;
20182055
module.exports.EMPTY_REMOVABLE_ATTRIBUTES = EMPTY_REMOVABLE_ATTRIBUTES;
2056+
module.exports.END_TAG_OPTIONAL_AT_BODY_END = END_TAG_OPTIONAL_AT_BODY_END;
20192057
module.exports.ENUMERATED_ATTRIBUTE_NAMES = ENUMERATED_ATTRIBUTE_NAMES;
20202058
module.exports.ENUMERATED_KEYWORDS = ENUMERATED_KEYWORDS;
20212059
module.exports.EVENT_HANDLER_ATTRIBUTES = EVENT_HANDLER_ATTRIBUTES;

‎lib/html/syntax-parser.js‎

Lines changed: 2 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ const { buildDataURI, readEmbeddedDataURI } = require("../util/dataURL");
1717
const { haltVisitors, resumeVisitors } = require("../util/haltVisitors");
1818

1919
const {
20+
ACKNOWLEDGED_NON_VOID,
2021
ADDRESS_DIV_P,
2122
APPLET_MARQUEE_OBJECT,
2223
BLOCK_END,
@@ -36,6 +37,7 @@ const {
3637
EMPTY_ELEMENT_DROPPED,
3738
EMPTY_METADATA_ELEMENTS,
3839
EMPTY_REMOVABLE_ATTRIBUTES,
40+
END_TAG_OPTIONAL_AT_BODY_END,
3941
ENUMERATED_ATTRIBUTE_NAMES,
4042
ENUMERATED_KEYWORDS,
4143
EVENT_HANDLER_ATTRIBUTES,
@@ -8362,28 +8364,6 @@ const hasUnclosedElement = () => {
83628364
const isHtmlNamed = (node, name) =>
83638365
node !== 0 && _namespaceOf(node) === NS_HTML && _tagNameOf(node) === name;
83648366

8365-
// What the end of the body may leave open: elements whose end tag is optional.
8366-
const END_TAG_OPTIONAL_AT_BODY_END = new Set([
8367-
"dd",
8368-
"dt",
8369-
"li",
8370-
"optgroup",
8371-
"option",
8372-
"p",
8373-
"rb",
8374-
"rp",
8375-
"rt",
8376-
"rtc",
8377-
"tbody",
8378-
"td",
8379-
"tfoot",
8380-
"th",
8381-
"thead",
8382-
"tr",
8383-
"body",
8384-
"html"
8385-
]);
8386-
83878367
const closeIfPInButtonScope = () => {
83888368
if (inButtonScope("p")) closePElement();
83898369
};
@@ -9617,13 +9597,6 @@ const vetoesContentMode = (name) => {
96179597
let parseErrorCallback;
96189598
// Whether the start tag being processed had its self-closing flag acknowledged.
96199599
let selfClosingAcknowledged = false;
9620-
// Start tags whose `/>` the tree builder acknowledges besides the void elements.
9621-
const ACKNOWLEDGED_NON_VOID = new Set([
9622-
"basefont",
9623-
"bgsound",
9624-
"frame",
9625-
"image"
9626-
]);
96279600
/** @type {NonNullable<HtmlParseOptions["parseError"]>} */
96289601
const handleParseError = (input, code, start, end, severity) => {
96299602
if (code === "eof-in-tag") eofInTag = true;

‎lib/javascript/syntax-parser.js‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21131,7 +21131,7 @@ const A = {
2113121131
/**
2113221132
* @typedef {object} JavascriptParseOptions
2113321133
* @property {"module" | "script"=} sourceType a module (the default) or a script
21134-
* @property {((input: string, code: string, start: number, end: number, severity: ParseErrorSeverity) => void)=} parseError optional callback for each parse error, with the CSS and HTML parsers' signature and Babel's `reasonCode` kebab-cased as `code`; `severity` is `"error"` for one that stops the parse and `"warning"` for one it reads past
21134+
* @property {((input: string, code: string, start: number, end: number, severity: ParseErrorSeverity) => void)=} parseError optional callback for each parse error, with the CSS and HTML parsers' signature and Babel's `reasonCode` kebab-cased as `code`; `severity` is `"error"` for one that stops the parse and `"warning"` for one it reads past; without it, the errors read past go unreported
2113521135
*/
2113621136

2113721137
// Whether `parseProgram` is running, which reads past the errors it can, and
@@ -21152,7 +21152,7 @@ const parseErrorCode = (message) => {
2115221152
if (parseErrorPatterns === undefined) {
2115321153
parseErrorPatterns = PARSE_ERROR_CODES.map(([template, code]) => [
2115421154
new RegExp(
21155-
`^${template.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*")}$`
21155+
`^${template.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, "[\\s\\S]*")}$`
2115621156
),
2115721157
code
2115821158
]);
@@ -21216,7 +21216,7 @@ const parseProgram = (input, pos = 0, options = {}) => {
2121621216
syntaxTree: tree
2121721217
},
2121821218
input,
21219-
pos
21219+
pos === 0 ? undefined : pos
2122021220
).parse()
2122121221
)
2122221222
);

‎test/unitCases/javascript/parser.unittest.js‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5166,6 +5166,15 @@ describe("parseProgram", () => {
51665166
]);
51675167
});
51685168

5169+
it("names an error whose message holds a line terminator", () => {
5170+
const { program, errors } = collect(
5171+
'export { a as "x\\ny", b as "x\\ny" }; var a, b',
5172+
{ sourceType: "module" }
5173+
);
5174+
expect(program).toBeDefined();
5175+
expect(errors.map(([code]) => code)).toEqual(["duplicate-export"]);
5176+
});
5177+
51695178
it("reads past an error without a callback", () => {
51705179
expect(parseProgram("let a; let a")).toBeDefined();
51715180
expect(parseProgram("a ~ ")).toBeUndefined();

‎tooling/generate-html-data.js‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1700,6 +1700,37 @@ const PARSER_TABLES = [
17001700
"text/x-ecmascript",
17011701
"text/x-javascript"
17021702
]
1703+
],
1704+
[
1705+
"ACKNOWLEDGED_NON_VOID",
1706+
"set",
1707+
"Start tags besides the void elements whose self-closing flag tree construction acknowledges.",
1708+
["basefont", "bgsound", "frame", "image"]
1709+
],
1710+
[
1711+
"END_TAG_OPTIONAL_AT_BODY_END",
1712+
"set",
1713+
"What the end of the body may leave open without a parse error: elements whose end tag is optional.",
1714+
[
1715+
"dd",
1716+
"dt",
1717+
"li",
1718+
"optgroup",
1719+
"option",
1720+
"p",
1721+
"rb",
1722+
"rp",
1723+
"rt",
1724+
"rtc",
1725+
"tbody",
1726+
"td",
1727+
"tfoot",
1728+
"th",
1729+
"thead",
1730+
"tr",
1731+
"body",
1732+
"html"
1733+
]
17031734
]
17041735
];
17051736

‎tooling/generate-js-data.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2150,7 +2150,7 @@ const PARSE_ERRORS = [
21502150
*/
21512151
const parseErrorTemplateRegExp = (template) =>
21522152
new RegExp(
2153-
`^${template.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*")}$`
2153+
`^${template.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, "[\\s\\S]*")}$`
21542154
);
21552155

21562156
/**

‎types.d.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15254,7 +15254,7 @@ declare interface JavascriptParseOptions {
1525415254
sourceType?: "module" | "script";
1525515255

1525615256
/**
15257-
* optional callback for each parse error, with the CSS and HTML parsers' signature and Babel's `reasonCode` kebab-cased as `code`; `severity` is `"error"` for one that stops the parse and `"warning"` for one it reads past
15257+
* optional callback for each parse error, with the CSS and HTML parsers' signature and Babel's `reasonCode` kebab-cased as `code`; `severity` is `"error"` for one that stops the parse and `"warning"` for one it reads past; without it, the errors read past go unreported
1525815258
*/
1525915259
parseError?: (
1526015260
input: string,

0 commit comments

Comments
 (0)