Skip to content

Commit 51420a1

Browse files
fix(html): keep a repeated token list token unless the list is sorted
`_normalizeTokenList` dropped a token already in a `DOMTokenList` attribute whenever `normalizeListAttributes` was on, which is the default. `classList` folds a repeat, but `getAttribute`, `className`, `classList.value` and an attribute selector matching on the whole value all read the string, so the rewrite was observable — the same reason `sortTokenLists` is off by default. The repeat now goes only under `sortTokenLists`, which already rewrote the value. Duplicate *attribute* names are untouched: dropping the second is §13.2.5.33, and Chrome parses the reported input the same way. Refs #22013
1 parent ab04b83 commit 51420a1

12 files changed

Lines changed: 66 additions & 75 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"webpack": patch
3+
---
4+
5+
Keep a repeated HTML token-list token unless `sortTokenLists` asks for the set.

‎declarations/WebpackOptions.d.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2613,7 +2613,7 @@ export interface OptimizationMinimizeHtml {
26132613
*/
26142614
normalizeEnumeratedAttributes?: boolean;
26152615
/**
2616-
* Normalize a list-shaped attribute value: a space-separated token list (`class`, `rel`, `part`, …), a comma-separated one (`accept`, `sizes`, …), a `srcset` and the viewport `<meta content>`. On by default. Reordering a token list is `sortTokenLists`, which is separate and off by default.
2616+
* Normalize a list-shaped attribute value: a space-separated token list (`class`, `rel`, `part`, …), a comma-separated one (`accept`, `sizes`, …), a `srcset` and the viewport `<meta content>`. On by default: only the separators between the tokens move, so the value reads back as the same list. Reordering a token list, or folding a repeat away, is `sortTokenLists`, which is separate and off by default.
26172617
* @since 5.110.0
26182618
*/
26192619
normalizeListAttributes?: boolean;
@@ -2658,7 +2658,7 @@ export interface OptimizationMinimizeHtml {
26582658
*/
26592659
sortAttributes?: boolean;
26602660
/**
2661-
* Print every space-separated token list the DOM reads as a set — `class`, `rel`, `part`, `sandbox`, `blocking`, `itemprop` / `itemref` / `itemtype`, `<output for>` and `<link sizes>` — in token order. Nothing matching those reads order, so this only makes the same markup compress better across pages. Off by default: a script reading `className` or `rel` back sees the new order. Asked for on its own: a list this reorders is rewritten whether or not `normalizeListAttributes` is on, since another order is another spelling. The lists the DOM does not read as a set are left alone whatever this says — `ping` is the order its requests go out in and `accesskey` the order its keys are tried.
2661+
* Print every space-separated token list the DOM reads as a set — `class`, `rel`, `part`, `sandbox`, `blocking`, `itemprop` / `itemref` / `itemtype`, `<output for>` and `<link sizes>` — in token order, with a repeated token dropped. Nothing matching those reads order or counts a repeat, so this only makes the same markup compress better across pages. Off by default: a script reading `className` or `rel` back, and an attribute selector matching on the whole value, both see the new string. Asked for on its own: a list this reorders is rewritten whether or not `normalizeListAttributes` is on, since another order is another spelling. The lists the DOM does not read as a set are left alone whatever this says — `ping` is the order its requests go out in and `accesskey` the order its keys are tried.
26622662
* @since 5.110.0
26632663
*/
26642664
sortTokenLists?: boolean;

‎lib/html/syntax.js‎

Lines changed: 16 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -8955,7 +8955,7 @@ const mirrorSelectedContent = (node, select) => {
89558955
* @property {boolean=} mergeStyles print a run of adjacent `<style>` elements as one sheet, which removes elements (default false)
89568956
* @property {boolean=} mergeScripts print a run of adjacent bare `<script>` elements whose bodies this print writes itself as one, which removes elements, hoists a later body's declarations into the ones before it, and lets a failing body end the whole run (default false)
89578957
* @property {boolean=} sortAttributes print an element's attributes commonest name first, ties by name, which nothing in HTML reads (default false)
8958-
* @property {boolean=} sortTokenLists print every token list the DOM reads as a set (`class`, `rel`, `part`, …) in token order (default false)
8958+
* @property {boolean=} sortTokenLists print every token list the DOM reads as a set (`class`, `rel`, `part`, …) in token order, a repeat dropped (default false)
89598959
* @property {(boolean | "smart" | "all")=} removeRedundantAttributes drop an attribute whose value is the element's own default; `true` is `"smart"`, and `"all"` also drops the spec defaults a selector can match (default false)
89608960
* @property {(boolean | "smart" | "all")=} removeImpliedTags how much of the `<html>` / `<head>` / `<body>` shell §13.1.2.4 lets the parser imply may be left out: `"smart"` leaves out only the `<html>` start tag, `true` (or `"all"`) all six, `false` none (default `"smart"`)
89618961
* @property {boolean=} deferSrcdoc whether an `<iframe srcdoc>` is among what `deferEmbeddedSource` collects (default true); false for a caller that minifies them itself, which keeps the attribute on the normal path and its shorter delimiter
@@ -10080,49 +10080,14 @@ const _isEventHandlerAttribute = (tagName, name) => {
1008010080
};
1008110081

1008210082
/**
10083-
* Whether a space-separated list names the same token twice, read span by span
10084-
* so nothing is allocated. The lists are short — `class` is a handful of tokens
10085-
* — so comparing each pair beats building a set of them.
10086-
* @param {string} list a collapsed, trimmed token list
10087-
* @returns {boolean} true when some token appears more than once
10088-
*/
10089-
const _repeatsAToken = (list) => {
10090-
const length = list.length;
10091-
for (let start = 0; start < length;) {
10092-
let end = list.indexOf(" ", start);
10093-
if (end === -1) end = length;
10094-
const width = end - start;
10095-
for (let other = end + 1; other < length;) {
10096-
let otherEnd = list.indexOf(" ", other);
10097-
if (otherEnd === -1) otherEnd = length;
10098-
if (otherEnd - other === width) {
10099-
let i = 0;
10100-
while (
10101-
i < width &&
10102-
list.charCodeAt(start + i) === list.charCodeAt(other + i)
10103-
) {
10104-
i++;
10105-
}
10106-
if (i === width) return true;
10107-
}
10108-
other = otherEnd + 1;
10109-
}
10110-
start = end + 1;
10111-
}
10112-
return false;
10113-
};
10114-
10115-
/**
10116-
* Collapse an ASCII-whitespace-separated token list (`class`) to single spaces,
10117-
* and — where the DOM reflects the list as a `DOMTokenList`, an ordered *set*,
10118-
* so a repeat was never a second token — drop a token already in it. Order is
10119-
* kept either way: the first spelling of each token stays put.
10083+
* Collapse an ASCII-whitespace-separated token list (`class`) to single spaces.
10084+
* Which tokens it names is left exactly as written unless `sortTokenLists`
10085+
* reads the list as the set the DOM reflects, which also folds a repeat away.
1012010086
* @param {string} raw raw (undecoded) value
10121-
* @param {boolean} unique whether a repeated token may go (`DOM_TOKEN_LIST_ATTRIBUTES`)
10122-
* @param {boolean} sort whether to print the tokens in order (`sortTokenLists`)
10087+
* @param {boolean} sort whether to print the tokens as a sorted set (`sortTokenLists`)
1012310088
* @returns {string} the collapsed list
1012410089
*/
10125-
const _normalizeTokenList = (raw, unique, sort) => {
10090+
const _normalizeTokenList = (raw, sort) => {
1012610091
// One walk answers "already collapsed?" — the common already-minified list
1012710092
// rewrites nothing, so it must not pay the trim + regex replace either.
1012810093
let dirty = false;
@@ -10148,12 +10113,9 @@ const _normalizeTokenList = (raw, unique, sort) => {
1014810113
const collapsed = dirty ? _asciiTrim(raw).replace(/[\t\n\f\r ]+/g, " ") : raw;
1014910114
if (dirty ? !collapsed.includes(" ") : !hasSpace) return collapsed;
1015010115
// Cutting the list into tokens is the allocation this runs often enough to
10151-
// care about, and almost no list repeats one — so the check reads the spans
10152-
// in place and only a list that does pay for the pieces. Sorting needs them
10153-
// either way.
10154-
if (!sort && !(unique && _repeatsAToken(collapsed))) return collapsed;
10155-
if (sort) return _sortedTokenList(collapsed, unique);
10156-
return [...new Set(collapsed.split(" "))].join(" ");
10116+
// care about, so only a run that reorders them pays for the pieces.
10117+
if (!sort) return collapsed;
10118+
return _sortedTokenList(collapsed);
1015710119
};
1015810120

1015910121
// One list's token spans, reused: ordering a list allocates only its result.
@@ -10183,13 +10145,12 @@ const _compareTokens = (list, aStart, aEnd, bStart, bEnd) => {
1018310145
};
1018410146

1018510147
/**
10186-
* `collapsed`'s tokens in sorted order, deduplicated when it is a set. They are
10187-
* ordered where they sit, so the pieces `split` would build are never built.
10148+
* `collapsed`'s tokens in sorted order, with a repeat dropped. They are ordered
10149+
* where they sit, so the pieces `split` would build are never built.
1018810150
* @param {string} collapsed a single-space-separated list
10189-
* @param {boolean} unique whether repeats collapse
10190-
* @returns {string} the ordered list
10151+
* @returns {string} the ordered set
1019110152
*/
10192-
const _sortedTokenList = (collapsed, unique) => {
10153+
const _sortedTokenList = (collapsed) => {
1019310154
const starts = _tokenSpanStarts;
1019410155
const ends = _tokenSpanEnds;
1019510156
let count = 0;
@@ -10221,7 +10182,6 @@ const _sortedTokenList = (collapsed, unique) => {
1022110182
let out = "";
1022210183
for (let i = 0; i < count; i++) {
1022310184
if (
10224-
unique &&
1022510185
i !== 0 &&
1022610186
_compareTokens(
1022710187
collapsed,
@@ -10325,10 +10285,10 @@ const _rewriteAttributeValue = (element, name, raw, viewport, enumeratedOn) => {
1032510285
(tokenListOn === null || tokenListOn.has(element))
1032610286
) {
1032710287
// Only the lists the DOM reflects as a `DOMTokenList` are sets, so only
10328-
// they may be reordered — `ping` is the order its requests go out in and
10329-
// `accesskey` the order its keys are tried.
10288+
// they may be reordered or have a repeat folded away — `ping` is the order
10289+
// its requests go out in and `accesskey` the order its keys are tried.
1033010290
const isSet = DOM_TOKEN_LIST_ATTRIBUTES.has(name);
10331-
return _normalizeTokenList(raw, isSet, _sortTokenLists && isSet);
10291+
return _normalizeTokenList(raw, _sortTokenLists && isSet);
1033210292
}
1033310293
if (_transforms.normalizeListAttributes) {
1033410294
if (viewport && name === "content") return _normalizeViewport(raw);

‎schemas/WebpackOptions.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4205,7 +4205,7 @@
42054205
"added": "5.110.0"
42064206
},
42074207
"normalizeListAttributes": {
4208-
"description": "Normalize a list-shaped attribute value: a space-separated token list (`class`, `rel`, `part`, …), a comma-separated one (`accept`, `sizes`, …), a `srcset` and the viewport `<meta content>`. On by default. Reordering a token list is `sortTokenLists`, which is separate and off by default.",
4208+
"description": "Normalize a list-shaped attribute value: a space-separated token list (`class`, `rel`, `part`, …), a comma-separated one (`accept`, `sizes`, …), a `srcset` and the viewport `<meta content>`. On by default: only the separators between the tokens move, so the value reads back as the same list. Reordering a token list, or folding a repeat away, is `sortTokenLists`, which is separate and off by default.",
42094209
"type": "boolean",
42104210
"added": "5.110.0"
42114211
},
@@ -4264,7 +4264,7 @@
42644264
"added": "5.110.0"
42654265
},
42664266
"sortTokenLists": {
4267-
"description": "Print every space-separated token list the DOM reads as a set — `class`, `rel`, `part`, `sandbox`, `blocking`, `itemprop` / `itemref` / `itemtype`, `<output for>` and `<link sizes>` — in token order. Nothing matching those reads order, so this only makes the same markup compress better across pages. Off by default: a script reading `className` or `rel` back sees the new order. Asked for on its own: a list this reorders is rewritten whether or not `normalizeListAttributes` is on, since another order is another spelling. The lists the DOM does not read as a set are left alone whatever this says — `ping` is the order its requests go out in and `accesskey` the order its keys are tried.",
4267+
"description": "Print every space-separated token list the DOM reads as a set — `class`, `rel`, `part`, `sandbox`, `blocking`, `itemprop` / `itemref` / `itemtype`, `<output for>` and `<link sizes>` — in token order, with a repeated token dropped. Nothing matching those reads order or counts a repeat, so this only makes the same markup compress better across pages. Off by default: a script reading `className` or `rel` back, and an attribute selector matching on the whole value, both see the new string. Asked for on its own: a list this reorders is rewritten whether or not `normalizeListAttributes` is on, since another order is another spelling. The lists the DOM does not read as a set are left alone whatever this says — `ping` is the order its requests go out in and `accesskey` the order its keys are tried.",
42684268
"type": "boolean",
42694269
"added": "5.110.0"
42704270
}

‎test/HtmlSyntax.unittest.js‎

Lines changed: 19 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5470,20 +5470,25 @@ describe("SourceProcessor — token list values", () => {
54705470
expect(minify('<div class="ab ">x</div>')).toContain("class=ab");
54715471
});
54725472

5473-
it("drops a repeated token only where the DOM folds it away", () => {
5474-
// `class` / `rel` reflect a `DOMTokenList`, which is an ordered set.
5475-
expect(minify('<div class="a b a">x</div>')).toContain('class="a b"');
5473+
it("keeps a repeated token, which `getAttribute` reads back", () => {
5474+
// Folding a repeat away is `sortTokenLists`: `classList` is a set, but
5475+
// `getAttribute`, `className` and `[class="a b a"]` read the string.
5476+
expect(minify('<div class="a b a">x</div>')).toContain('class="a b a"');
54765477
expect(minify('<link rel="preload preload" href=x>')).toContain(
5477-
"rel=preload"
5478+
'rel="preload preload"'
54785479
);
5479-
// `ping` is read back as written and sends one request per token; the
5480-
// other two reflect no token list either.
54815480
expect(minify('<a href=x ping="/p /p">l</a>')).toContain('ping="/p /p"');
54825481
expect(minify('<table><tr><td headers="h h">c</table>')).toContain(
54835482
'headers="h h"'
54845483
);
54855484
expect(minify('<div accesskey="k k">x</div>')).toContain('accesskey="k k"');
54865485
});
5486+
5487+
it("collapses the separators around a repeat without dropping it", () => {
5488+
expect(minify('<div class="a \n b \n a">x</div>')).toContain(
5489+
'class="a b a"'
5490+
);
5491+
});
54875492
});
54885493

54895494
describe("SourceProcessor — sortAttributes / sortTokenLists", () => {
@@ -5544,13 +5549,17 @@ describe("SourceProcessor — sortAttributes / sortTokenLists", () => {
55445549
).toBe('<div class="btn btn-lg btn-sm">x</div>');
55455550
});
55465551

5547-
it("drops a set's repeats, ordered or not", () => {
5552+
it("drops a set's repeats, which only ordering it does", () => {
55485553
expect(
55495554
minify('<div class="b a b a c">x</div>', { sortTokenLists: true })
55505555
).toBe('<div class="a b c">x</div>');
5551-
expect(minify('<div class="b a b a c">x</div>')).toBe(
5552-
'<div class="b a c">x</div>'
5553-
);
5556+
expect(
5557+
minify('<link rel="preload preload" href=x>', { sortTokenLists: true })
5558+
).toBe("<link rel=preload href=x>");
5559+
// A list the DOM does not reflect as a set keeps both spellings.
5560+
expect(
5561+
minify('<a href=x ping="/p /p">l</a>', { sortTokenLists: true })
5562+
).toBe('<a href=x ping="/p /p">l</a>');
55545563
});
55555564

55565565
it("leaves foreign content alone, where a name is not case-folded", () => {

‎test/configCases/html/minimize-attributes/__snapshots__/ConfigCacheTest.snap‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ b\\">newline in a value</p>
3030
<svg viewBox=\\"0 0 8 8\\"><rect x=1 y=2 /><circle R=1 /></svg>
3131
<div class=a>source casing</div>
3232
<p a=1>duplicate attribute</p>
33+
34+
<p class=\\"a b a\\">repeated class token</p>
35+
36+
<input type=text {% if required %} required{% endif>
3337
<p data-slash=/end/>value ending in a slash</p>
3438
<a href=x/>unquoted value ending in a slash</a>
3539
<pre> preformatted

‎test/configCases/html/minimize-attributes/__snapshots__/ConfigTest.snap‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ b\\">newline in a value</p>
3030
<svg viewBox=\\"0 0 8 8\\"><rect x=1 y=2 /><circle R=1 /></svg>
3131
<div class=a>source casing</div>
3232
<p a=1>duplicate attribute</p>
33+
34+
<p class=\\"a b a\\">repeated class token</p>
35+
36+
<input type=text {% if required %} required{% endif>
3337
<p data-slash=/end/>value ending in a slash</p>
3438
<a href=x/>unquoted value ending in a slash</a>
3539
<pre> preformatted

‎test/configCases/html/minimize-attributes/page.html‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,12 @@
4545
<svg viewBox="0 0 8 8"><rect x="1" y="2"/><circle R="1"/></svg>
4646
<DIV CLASS="a">source casing</DIV>
4747
<p a=1 a=2>duplicate attribute</p>
48+
<!-- `classList` folds the repeat, but `getAttribute`, `className` and
49+
`[class="a b a"]` all read the string, so it stays -->
50+
<p class="a b a">repeated class token</p>
51+
<!-- not HTML: the `%}` pairs tokenize as attribute names, so the second is a
52+
duplicate the spec drops — what a browser parses this to (#22013) -->
53+
<input type="text"{% if required %} required{% endif %}>
4854
<p data-slash="/end/">value ending in a slash</p>
4955
<a href=x/>unquoted value ending in a slash</a>
5056
<pre> preformatted

‎test/configCases/html/minimize-transforms/__snapshots__/ConfigCacheTest.snap‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,8 @@ exports[`ConfigCacheTestCases html minimize-transforms minimize-transforms shoul
4949
<label for=\\"a b\\">for keeps its spaces: an id may contain one</label>
5050
<output for=\\"a b\\">for is a token list here</output>
5151
<link rel=stylesheet sizes=\\"16x16 32x32\\" href=s.css>
52-
<link rel=preload as=script blocking=render href=p.js>
52+
53+
<link rel=preload as=script blocking=\\"render render\\" href=p.js>
5354
<div part=\\"a b\\" itemscope itemtype=https://schema.org/Thing>globals</div>
5455
<div headingreset headingoffset=2 tabindex=-1>heading globals</div>
5556
<table><tr><th id=h>h<td headers=\\"h h2\\" rowspan=2>headers is a token list</table>

‎test/configCases/html/minimize-transforms/__snapshots__/ConfigTest.snap‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,8 @@ exports[`ConfigTestCases html minimize-transforms minimize-transforms should com
4949
<label for=\\"a b\\">for keeps its spaces: an id may contain one</label>
5050
<output for=\\"a b\\">for is a token list here</output>
5151
<link rel=stylesheet sizes=\\"16x16 32x32\\" href=s.css>
52-
<link rel=preload as=script blocking=render href=p.js>
52+
53+
<link rel=preload as=script blocking=\\"render render\\" href=p.js>
5354
<div part=\\"a b\\" itemscope itemtype=https://schema.org/Thing>globals</div>
5455
<div headingreset headingoffset=2 tabindex=-1>heading globals</div>
5556
<table><tr><th id=h>h<td headers=\\"h h2\\" rowspan=2>headers is a token list</table>

0 commit comments

Comments
 (0)