Skip to content

Commit 74f38b7

Browse files
build(tooling): align the compare sweeps across languages (#22850)
* build(tooling): share expectation and respelling filters across compare sweeps A narrowed `--invariants` run (SPELLING, FIXTURE, PRESET or RELATION="respelling <name>") reported expectations it never reached as stale in the CSS, HTML and XML sweeps, and RELATION="respelling <name>" ran no respelling there. Move the JS sweep's filter into the harness as `wantedExpectations` and `wantedRespelling` and use them in all four. Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV * build(tooling): align the compare sweeps across languages - `Expected.source` takes a list of fixtures; the CSS `*zoom` entry names its four, so a narrowed run no longer reports it stale. - XML: invariants run the embedded CSS renderer presets the webpack rows use, `meaning` reads stylesheets in their canonical CSS form under the preset's target, sources are swept cut short (`cutsAt`, shared with HTML), and `PROFILE=1` prints. - JS: the printing table shows `2nd` and dropped legal notices, and purity digests the printed output. - CSS, HTML and XML dispatch through `runMode`, with tests. - Fix a blank SVG `<style>` the input ran out in inside a CDATA section being printed with the section still open, found by the XML cuts. Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV * fix(html): escape a bare `&` in minified XML The XML minifier kept the source's spelling of a text or attribute value whenever it was shorter, so an `&` opening no reference (`a & b`, `&amp` the input ran out in) was written back bare and the output was not well-formed. Keep the source spelling only where every `&` opens a reference. Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV * build(tooling): report JS invariants before the install, as CSS, HTML and XML do Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV * fix(html): keep an XML reference as written only when its name is a Name - An `&` followed by a name XML's `Name` production refuses (`&foo?;`, `&1a;`) is escaped when minifying, as the decoder left it text. - The JS comparison matches legal notices one by one, so a changed or duplicated notice counts as dropped. - The XML idempotence checks read CSS under the preset's target, as `meaning` does. Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV * fix(html): read an XML reference's name with one class per part The alternation for surrogate pairs inside the repeated name part was flagged by CodeQL as able to backtrack exponentially; each part is now a single character class reading surrogates per code unit. Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV * docs(agents): forbid adding private exports to share an internal helper Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV * refactor(html): export the XML well-formed text check as a public helper `isXmlWellFormedText` replaces the private `_isXmlWellFormedText` export, and AGENTS.md now forbids private exports in every file of the project, testing being the only exception. Claude-Session: https://claude.ai/code/session_01Bqhj38umy4coKMvXQT28NV
1 parent 90b6205 commit 74f38b7

18 files changed

Lines changed: 871 additions & 199 deletions
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": patch
3+
---
4+
5+
Keep minified XML well-formed: close a cut-off CDATA, escape a bare `&`.

‎.github/workflows/test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,7 @@ jobs:
160160
run: yarn test:invariants:css --require-corpus
161161

162162
# One language's finding must not hide another's: each names a repro of
163-
# its own and all three are read in the same sitting. `!cancelled()`
163+
# its own and all four are read in the same sitting. `!cancelled()`
164164
# rather than `always()`, so a cancelled run stops here instead of
165165
# sweeping again.
166166
- name: HTML printer invariants

‎AGENTS.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,12 @@ Spell names out in full (`attributeCount`, not `attrCnt`), beyond the abbreviati
107107
108108
Path-shape regexps and their helpers live in `lib/util/identifier.js`: **import from there — never re-declare a local copy**, even a one-liner like `/\\/g`. Export one that is defined but not exported; define one locally only when nothing fits ([details](docs/code-style.md#path-regexps-and-helpers-live-in-one-file)).
109109

110+
### No private exports
111+
112+
> [!REQUIRED]
113+
114+
**Never add a `_`-prefixed export** (`module.exports._foo`) to any file in the project to share an internal helper with another file: every export is reachable, so other developers come to depend on it and it can't change freely. Keep the helper in the one file that uses it, or move the logic so one file owns it; a helper useful in its own right is exported under a public name with full JSDoc instead, as the HTML and CSS `escape*` / `decode*` helpers are. **The only exception is testing**: a private export a test needs to reach code no public entry point can.
115+
110116
### Don't enumerate module or source types
111117

112118
> [!REQUIRED]

‎lib/html/syntax-parser.js‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6392,6 +6392,11 @@ const isXmlChar = (code) =>
63926392
const XML_REFERENCE_REGEXP =
63936393
/&(?:#(?:x([\da-fA-F]+)|(\d+))|([^\s&;#<>"'=/%]+));/g;
63946394
const XML_CHARACTER_REFERENCE_REGEXP = /&#(?:x([\da-fA-F]+)|(\d+));/g;
6395+
// An `&` opening no well-formed reference (§4.1, a `Name` per §2.3), which a
6396+
// strict parser refuses. One class per part, surrogates read per code unit, so
6397+
// a long name never backtracks; the decoder kept it as text, so it is escaped.
6398+
const XML_BARE_AMPERSAND_REGEXP =
6399+
/&(?!(?:#(?:x[\da-fA-F]+|\d+)|[:A-Z_a-z\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u2070-\u218F\u2C00-\u2FEF\u3001-\uDB7F\uF900-\uFDCF\uFDF0-\uFFFD\u200C-\u200D][-.0-9:A-Z_a-z\u00B7\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u037D\u037F-\u1FFF\u203F\u2040\u2070-\u218F\u2C00-\u2FEF\u3001-\uDFFF\uF900-\uFDCF\uFDF0-\uFFFD\u200C-\u200D]*);)/;
63956400
// WHY: §3.3.3 normalizes each literal whitespace character of an attribute value
63966401
// to a space after §2.11 has folded CRLF, so a CRLF is one space; replacement
63976402
// text of an entity is already folded, so there each character counts alone.
@@ -6523,6 +6528,19 @@ const escapeXmlText = (data) =>
65236528
.replace(/>/g, "&gt;")
65246529
.replace(/\r/g, "&#13;");
65256530

6531+
/**
6532+
* Whether XML text or an attribute value, as written in its source, may be
6533+
* written back unchanged: no `<` and every `&` opening a reference (§4.1).
6534+
* @example
6535+
* isXmlWellFormedText("a &amp; b"); // true
6536+
* isXmlWellFormedText("&undeclared;"); // true: a reference, declared or not
6537+
* isXmlWellFormedText("a & b"); // false: a strict parser refuses the bare `&`
6538+
* @param {string} raw the text or attribute value as written
6539+
* @returns {boolean} true when it is well-formed as written
6540+
*/
6541+
const isXmlWellFormedText = (raw) =>
6542+
!raw.includes("<") && !XML_BARE_AMPERSAND_REGEXP.test(raw);
6543+
65266544
/**
65276545
* Escape an attribute value for XML (§3.1): `&`, `<` and the delimiter, plus the
65286546
* whitespace §3.3.3 would otherwise normalize to a space.
@@ -13377,7 +13395,7 @@ const _xmlOpenTag = (path, minify) => {
1337713395
_attrValues[i] === null &&
1337813396
valueStart > 0 &&
1337913397
raw.length < value.length &&
13380-
!raw.includes("<")
13398+
isXmlWellFormedText(raw)
1338113399
) {
1338213400
const written = _htmlSource.charAt(valueStart - 1);
1338313401
if (written === '"' || written === "'") {
@@ -15783,6 +15801,7 @@ module.exports.grammar = grammar;
1578315801
module.exports.isAllAsciiWhitespace = isAllAsciiWhitespace;
1578415802
module.exports.isAsciiAlphanumeric = isAsciiAlphanumeric;
1578515803
module.exports.isAsciiWhitespace = isAsciiWhitespace;
15804+
module.exports.isXmlWellFormedText = isXmlWellFormedText;
1578615805
module.exports.metaTag = metaTag;
1578715806
module.exports.parseCssUrls = parseCssUrls;
1578815807
module.exports.parseHtml = parseHtml;

‎lib/html/syntax-printer.js‎

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ const {
7979
escapeXmlText,
8080
isAllAsciiWhitespace,
8181
isAsciiAlphanumeric,
82+
isXmlWellFormedText,
8283
startsWithAsciiWhitespace,
8384
stripAsciiWhitespace
8485
} = require("./syntax-parser");
@@ -316,19 +317,17 @@ const _printXml = (path, writer, minify) => {
316317
case NodeType.Text: {
317318
const raw = path.source();
318319
const data = _dataOf(path.node);
319-
if (minify && _isXmlStylesheet(path)) {
320-
return data.trim() === ""
321-
? raw
322-
: _S._deferEmbeddedSource === undefined
323-
? _xmlCharacterData(_renderEmbedded(data, CSS_TYPE))
324-
: _deferEmbedded(data, CSS_TYPE, (rendered) =>
325-
_xmlCharacterData(
326-
typeof rendered === "string" ? rendered : data
327-
)
328-
);
320+
const stylesheet = minify && _isXmlStylesheet(path);
321+
if (stylesheet && data.trim() !== "") {
322+
return _S._deferEmbeddedSource === undefined
323+
? _xmlCharacterData(_renderEmbedded(data, CSS_TYPE))
324+
: _deferEmbedded(data, CSS_TYPE, (rendered) =>
325+
_xmlCharacterData(typeof rendered === "string" ? rendered : data)
326+
);
329327
}
330328
// A CDATA section the input ran out in would hold the end tags after it.
331-
if (!minify) {
329+
// A blank stylesheet is kept as written, as the renderer is never asked.
330+
if (!minify || stylesheet) {
332331
return raw.includes("<![CDATA[") && !raw.endsWith("]]>")
333332
? escapeXmlText(data)
334333
: raw;
@@ -342,8 +341,10 @@ const _printXml = (path, writer, minify) => {
342341
escaped = `<![CDATA[${data}]]>`;
343342
}
344343
// The source's own spelling when shorter: a declared entity reference, or
345-
// a `>` it left bare. A `<` in it is a CDATA section, never kept as is.
346-
return raw.length < escaped.length && !raw.includes("<") ? raw : escaped;
344+
// a `>` it left bare. A `<` is a CDATA section, a bare `&` no reference.
345+
return raw.length < escaped.length && isXmlWellFormedText(raw)
346+
? raw
347+
: escaped;
347348
}
348349
case NodeType.Comment: {
349350
const source = path.source();

‎test/unitCases/tooling/compare-css.unittest.js‎

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,42 @@
11
"use strict";
22

3+
const CACHE_NAME = "css-tool-comparison";
4+
5+
const installed = jest.fn(() => Promise.resolve(CACHE_NAME));
6+
7+
// A real install would `npm ci` the corpus into a contributor's cache, and every
8+
// packed stylesheet and built fixture is taken as there, so `--setup` fetches
9+
// and builds nothing: what is under test is that it installs and stops.
10+
jest.mock("../../../tooling/compare-tools-harness", () => ({
11+
...jest.requireActual("../../../tooling/compare-tools-harness"),
12+
exists: () => Promise.resolve(true),
13+
installPackages: installed
14+
}));
15+
316
const { SourceProcessor } = require("../../../lib/css/syntax");
4-
const { COLD_BRANCHES } = require("../../../tooling/compare-css-tools");
17+
const {
18+
COLD_BRANCHES,
19+
runMode
20+
} = require("../../../tooling/compare-css-tools");
21+
22+
// Read before anything clears the mock: an unguarded script installs while
23+
// being required, and a `beforeEach` would erase exactly that evidence.
24+
const callsWhileRequiring = installed.mock.calls.length;
25+
26+
describe("the CSS tool comparison", () => {
27+
it("should start nothing when the script is required rather than run", () => {
28+
expect(callsWhileRequiring).toBe(0);
29+
});
30+
31+
it("should install the corpus and stop on `--setup`", async () => {
32+
installed.mockClear();
33+
34+
await runMode("--setup");
35+
36+
expect(installed).toHaveBeenCalledTimes(1);
37+
expect(installed).toHaveBeenCalledWith(CACHE_NAME);
38+
});
39+
});
540

641
// What each rewrite the fixture is there for writes. Every one was counted at
742
// zero over every stylesheet on disk before the fixture was written, so an edit

‎test/unitCases/tooling/compare-harness.unittest.js‎

Lines changed: 110 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ const { Readable } = require("stream");
88
const { pathToFileURL } = require("url");
99
const {
1010
compress,
11+
cutsAt,
1112
exists,
1213
filterFrom,
1314
findingGroups,
@@ -26,7 +27,9 @@ const {
2627
profileShares,
2728
run,
2829
sweepExitCode,
29-
sweepMode
30+
sweepMode,
31+
wantedExpectations,
32+
wantedRespelling
3033
} = require("../../../tooling/compare-tools-harness");
3134

3235
/** @type {string} */
@@ -833,6 +836,112 @@ describe("compare-tools-harness", () => {
833836
expect(text).toContain("idempotence — bytes");
834837
expect(text).toContain("stale — 1 expectation(s) matched nothing");
835838
});
839+
840+
it("lets an entry naming several fixtures cover a group reaching them", () => {
841+
const groups = findingGroups([
842+
{
843+
relation: "idempotence",
844+
contains: "a ->",
845+
source: ["one", "two"],
846+
why: "those fixtures write it"
847+
}
848+
]);
849+
groups.add(finding("idempotence", " a -> b"), "default", "one.html");
850+
groups.add(finding("idempotence", " a -> b"), "default", "two.html");
851+
expect(written(groups).count).toBe(0);
852+
groups.add(finding("idempotence", " a -> b"), "default", "three.html");
853+
expect(written(groups).count).toBe(2);
854+
});
855+
});
856+
857+
describe("cutsAt", () => {
858+
it("cuts inside the first construct past the midpoint, else the first", () => {
859+
const source = "<a>1</a><b>2</b><c>3</c>";
860+
expect(cutsAt(source, [["in a start tag", /<[a-z]/g]])).toEqual([
861+
["in a start tag", "<a>1</a><b>2</b><c"]
862+
]);
863+
expect(cutsAt(source, [["in a comment", /<!--./g]])).toEqual([]);
864+
expect(cutsAt("<a>1</a>xyz", [["in a tag", /<[a-z]/g]])).toEqual([
865+
["in a tag", "<a"]
866+
]);
867+
});
868+
869+
it("leaves out a cut that would keep the whole source", () => {
870+
expect(cutsAt("ab<a", [["in a start tag", /<[a-z]/g]])).toEqual([]);
871+
});
872+
});
873+
874+
describe("narrowing a sweep", () => {
875+
afterEach(() => {
876+
delete process.env.RELATION;
877+
delete process.env.SPELLING;
878+
});
879+
880+
it("asks every respelling when nothing narrows the run", () => {
881+
expect(wantedRespelling("quote-double")).toBe(true);
882+
});
883+
884+
it("asks a respelling `RELATION` names alone or as `respelling`", () => {
885+
process.env.RELATION = "respelling quote-double";
886+
expect(wantedRespelling("quote-double")).toBe(true);
887+
expect(wantedRespelling("quote-single")).toBe(false);
888+
process.env.RELATION = "respelling";
889+
expect(wantedRespelling("quote-single")).toBe(true);
890+
process.env.RELATION = "spans";
891+
expect(wantedRespelling("quote-single")).toBe(false);
892+
});
893+
894+
it("asks only the respelling `SPELLING` names", () => {
895+
process.env.SPELLING = "unquote";
896+
expect(wantedRespelling("unquote")).toBe(true);
897+
expect(wantedRespelling("quote-double")).toBe(false);
898+
});
899+
900+
/** @type {import("../../../tooling/compare-tools-harness").Expected[]} */
901+
const EXPECTED = [
902+
{ relation: "idempotence", contains: "a", why: "…" },
903+
{ relation: "respelling quote-double", contains: "b", why: "…" },
904+
{ relation: "idempotence", contains: "c", source: "one", why: "…" },
905+
{ relation: "idempotence", contains: "d", presets: ["legacy"], why: "…" },
906+
{
907+
relation: "idempotence",
908+
contains: "e",
909+
source: ["three", "one"],
910+
why: "…"
911+
}
912+
];
913+
914+
/**
915+
* @param {string[]} labels the fixtures the run reads
916+
* @param {string[]} presets the option sets it prints under
917+
* @returns {string[]} what each kept entry contains
918+
*/
919+
const kept = (labels, presets) =>
920+
wantedExpectations(EXPECTED, labels, presets).map(
921+
(entry) => entry.contains
922+
);
923+
924+
it("keeps every entry a full run reaches", () => {
925+
expect(kept(["one.html"], ["default", "legacy"])).toEqual([
926+
"a",
927+
"b",
928+
"c",
929+
"d",
930+
"e"
931+
]);
932+
});
933+
934+
it("drops an entry for a fixture or preset the run does not read", () => {
935+
expect(kept(["two.html"], ["default"])).toEqual(["a", "b"]);
936+
});
937+
938+
it("drops an entry for a relation or respelling the run does not ask", () => {
939+
process.env.SPELLING = "unquote";
940+
expect(kept(["one.html"], ["legacy"])).toEqual(["a", "c", "d", "e"]);
941+
delete process.env.SPELLING;
942+
process.env.RELATION = "respelling quote-double";
943+
expect(kept(["one.html"], ["legacy"])).toEqual(["b"]);
944+
});
836945
});
837946
});
838947

‎test/unitCases/tooling/compare-html.unittest.js‎

Lines changed: 32 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,46 @@
11
"use strict";
22

3+
const CACHE_NAME = "html-tool-comparison";
4+
5+
const installed = jest.fn(() => Promise.resolve(CACHE_NAME));
6+
7+
// A real install would `npm ci` the corpus into a contributor's cache, and
8+
// nothing here reads it: the built documents are the ones under test.
9+
jest.mock("../../../tooling/compare-tools-harness", () => ({
10+
...jest.requireActual("../../../tooling/compare-tools-harness"),
11+
installPackages: installed
12+
}));
13+
314
const fs = require("fs");
415
const path = require("path");
516
const { SourceProcessor } = require("../../../lib/html/syntax");
617
const {
718
COLD_CONSTRUCTS,
819
EMBEDDED_LEGACY_OPTIONS,
9-
EMBEDDED_OPTIONS
20+
EMBEDDED_OPTIONS,
21+
runMode
1022
} = require("../../../tooling/compare-html-tools");
1123
const { collectFiles } = require("../../../tooling/compare-tools-harness");
1224

25+
// Read before anything clears the mock: an unguarded script installs while
26+
// being required, and a `beforeEach` would erase exactly that evidence.
27+
const callsWhileRequiring = installed.mock.calls.length;
28+
29+
describe("the HTML tool comparison", () => {
30+
it("should start nothing when the script is required rather than run", () => {
31+
expect(callsWhileRequiring).toBe(0);
32+
});
33+
34+
it("should install the corpus and stop on `--setup`", async () => {
35+
installed.mockClear();
36+
37+
await expect(runMode("--setup")).resolves.toBe(CACHE_NAME);
38+
39+
expect(installed).toHaveBeenCalledTimes(1);
40+
expect(installed).toHaveBeenCalledWith(CACHE_NAME);
41+
});
42+
});
43+
1344
const ROOT = path.join(__dirname, "..", "..", "..");
1445

1546
// cspell:ignore popovertarget popovertargetaction contenteditable exportparts

0 commit comments

Comments
 (0)