Repository navigation
Commit 99dd2c7
authored
feat(minify): safe HTML minify transforms, driven by the spec grammars (#21586)
* fix(minify): keep HTML text intact across CR line endings
Two defects made CRLF-authored HTML lose characters when minified.
`leadingWs` split a character token by counting its decoded data against
raw source offsets, so a run whose decoding shortened it (CRLF, or a
character reference) got a text node whose span named the wrong source.
The `Text` printer took `raw.length === data.length` to mean nothing was
decoded, but preprocessing maps a lone CR to LF without changing length,
so a bare CR reached the output -- and a truncated span from the first
defect passed the same test, printing `<head>\r\n\t` as `<head>\r\n`.
Validated against the html5lib suite (13379 cases).
* feat(minify): drop attribute quotes an HTML value does not need
§13.1.2.3 admits an unquoted value that is non-empty and free of
whitespace, quotes, backtick, `=`, `<` and `>`. Values are serialized from
their raw source span, so the quotes come off without re-encoding, and a
character reference decodes the same either way.
The tag was previously copied byte-for-byte so the sentinels webpack's own
late asset passes leave behind stayed matchable. Those passes run after
minification, so the one that assumed quotes -- the `integrity` sentinel --
now matches the quote optionally and back-references it to keep the pair
symmetric; it already re-quoted its replacement, which an SRI list needs
for its spaces and `=` padding. A third-party post-processor that regexes
emitted HTML assuming quotes would still not match, which is why
`removeAttributeQuotes` is off by default in html-minifier.
Checked over every HTML fixture in `test/` (236 documents): the parsed DOM
is unchanged in all of them and 188 got smaller, 2.97% overall.
* feat(minify): drop HTML whitespace nothing renders
Whitespace directly under `<head>` or `<html>` sits outside any block
formatting context, so no styling makes it visible -- the same reasoning
that already drops inert comments. `<body>`'s whitespace does render and is
left alone.
The CRLF regression tests moved from `<head>` to `<colgroup>`: all three
insertion modes that split a leading whitespace run through `leadingWs`
insert into `<head>`, `<html>` or `<colgroup>`, and only the last one still
keeps the run where the split is observable. Both re-targeted tests were
checked to still fail against the pre-fix parser.
Over every HTML fixture in `test/` (237 documents): 533 inert nodes
dropped, no other DOM change, 0.96% smaller.
* feat(minify): omit an HTML end tag the next element implies
§13.1.2.4 lets `</li>`, `</p>`, `</td>`, `</tr>`, `</option>` and the rest
go when a qualifying element follows or nothing else is left in the parent.
The printer previously materialized every one of them.
`<html>`, `<head>` and `<body>` are deliberately left alone: their tags are
omissible too, but non-browser consumers routinely mis-parse a document
without them.
Two conditions are easy to miss and both are enforced. A trailing `</p>`
stays inside `a`/`audio`/`del`/`ins`/`map`/`noscript`/`video` and any
autonomous custom element, whose content model would otherwise absorb what
follows. And the qualifying sibling has to actually print a start tag -- a
parser-implied `<tbody>` / `<tr>` is emitted transparently, so dropping
`</thead>` in front of one would continue the section instead of closing
it. Without that second guard eight table shapes re-parsed differently.
The decision reads the tree, not the emitted text, so an intervening
comment keeps the tag even though minification drops it -- bigger than
necessary, never wrong.
Checked by re-parsing: 304 constructed cases (every element crossed with
every kind of following sibling, plus each `</p>` parent carve-out) and
every HTML fixture in `test/`, with no DOM change beyond one `<noscript>`
scripting-flag artifact that predates this change.
* feat(minify): fold HTML casing and tags the parser re-implies
Three transforms whose result re-parses to the identical tree.
HTML tag and attribute names are ASCII-lowercased, which is exactly what
parsing does to them. Only ASCII: `path.tagName()` folds with Unicode
`toLowerCase`, which rewrote `<aÀ-豈>` to `<aà-豈>` -- a name the parser
leaves alone. Foreign content keeps its source bytes entirely, because SVG
admits element names no adjustment table would case-correct on the way back
in, and because the interned foreign attribute name is html5lib's `xlink
href` spelling rather than the serialized `xlink:href`.
A `/` before `>` is dropped on HTML elements, where the parser ignores it.
It survives on foreign elements, where it self-closes, and where it is the
last character of an unquoted value.
A `<tbody>` start tag goes when the parser re-implies it: the first child
is a `<tr>`, it carries no attributes, and no preceding table section
dropped its own end tag -- otherwise these rows continue that section.
Re-parsed and compared over 304 constructed cases and every HTML fixture in
`test/`: no DOM change, 11 fixtures smaller.
* feat(minify): reduce srcset whitespace to what the grammar needs
The candidate list is re-serialized through webpack's own `parseSrcset`,
so a comma inside a URL (a `data:` URI, a `?w=1,2` query) is not mistaken
for a separator the way a regex would.
Two traps, both caught by checking the parsed candidate list before and
after. A descriptor-less candidate keeps a space after its comma: its URL
runs to the next whitespace, so `a.png, b.png 2x` collapsed to
`a.png,b.png 2x` parses as one candidate named `a.png,b.png`. And
`parseSrcset` throws on a value the grammar rejects, which now hands the
value back untouched instead of failing the build.
A value carrying a character reference is left alone -- it decodes to
something the grammar reads differently -- as is foreign content, where
`srcset` is not this attribute.
This changes the attribute's bytes, so `getAttribute("srcset")` differs
while the candidate list does not. Checked over every HTML fixture in
`test/`: no candidate list changed.
* feat(minify): minify an inline `<style>` body
An inline sheet now goes through webpack's own CSS minifier -- the one that
runs on CSS assets -- so it is held to exactly the same rules, shorthand
collapsing and all. `lib/html/syntax.js` already lazily links `../css/syntax`
for `parseCssUrls`, so this adds no dependency.
Minified only when the body is CSS: a `type` other than `text/css` means it
is a template, not a sheet. An empty body and anything the CSS parser
rejects are handed back untouched. Foreign content is included, since an SVG
`<style>` is CSS too.
An inline `<script>` is left alone. Minifying JS needs terser, which reaches
webpack only through `minimizer-webpack-plugin` and exposes an async API,
while this printer is synchronous -- so it needs a design, not a patch.
Checked over every HTML fixture in `test/`: nothing outside a `<style>` body
changed.
* chore(minify): note the inline-script minification that is still open
* feat(minify): rewrite style, viewport and doctype through their grammars
A `style` attribute is a declaration list, so it goes through webpack's CSS
minifier inside a throwaway rule; a value that does not come back as exactly
one rule escaped the wrapper (a stray `}`, a custom property holding braces)
and keeps its source spelling.
A `<meta name=viewport>` content list ignores the whitespace around its
`,` / `;` / `=` separators, so that whitespace goes. No other `content` is
touched -- a description is prose.
Parsing ASCII-lowercases the doctype keyword and name, so both are folded.
The public and system identifiers behind them are case-sensitive strings and
stay as written, and the space after the keyword stays too: omitting it is a
`missing-whitespace-before-doctype-name` parse error.
A rewritten value now picks its own quote, since a minified declaration list
can carry either kind (`grid-template-areas`).
Checked over every HTML fixture in `test/`: every value still parses to the
same declarations, candidates and pairs.
* feat(minify): collapse a class list and a boolean attribute's value
`class` is an ASCII-whitespace-separated token list, so its separators go to
one space and the ends are trimmed. The token set the DOM reads is unchanged.
A boolean attribute's presence is its value, so the canonical spellings
(`disabled`, `disabled=""`, `DISABLED="DISABLED"`) lose theirs. Collapsing is
gated on the elements each attribute is boolean *on*: `checked` means nothing
on a `<div>`, where it is an ordinary attribute whose value a script may read.
A value the spec does not canonicalize (`disabled="true"`) is left as written.
Both change the attribute string while keeping what the DOM parses, so they
are noted alongside the other value rewrites as wanting an option.
Checked over every HTML fixture in `test/`: same token sets, same boolean
presence, nothing else moved.
* test(tooling): compare an app shell, and a `<style>` body as CSS
Neither installed fixture carries an inline `<style>`, a `srcset` or a
boolean attribute, so the comparison could not see what any minifier does
with them. The added shell is the third real shape: weight in critical CSS
and form markup.
Its `<style>` body also exposed the fingerprint treating CSS as bytes, which
reported every minifier that rewrites a sheet as losing text -- four of the
six on that fixture. It is canonicalized through webpack's CSS minifier now,
so the column reports what a sheet means. A CSS-level mistake is left to
webpack's CSS suites; this tool checks the HTML around it, and it still
catches the inline JS that htmlnano and swc rewrite.
* refactor(minify): move the HTML minify tables into `lib/html/data.js`
The optional-end-tag conditions, the boolean-attribute inventory and the
attribute-kind sets now sit beside the algorithm rather than inside it, the
way `lib/css/data.js` does. That one is generated from `mdn-data`; these are
transcribed from the spec, and the file says so along with what a generator
would still need — no dataset states "is this attribute boolean on this
element" or the optional-tag conditions yet.
The tree-construction tables (`VOID`, `SPECIAL`, `FORMATTING`, the scope
sets) stay where they are: they are the parsing algorithm's own steps, not
data a source could supply.
Every minify transform moves to `configCases/html/minimize-transforms`, one
snapshot over a corpus carrying each transform and each case it declines, so
the assertion is the output a real build emits. The 452 lines of unit tests
they replace are gone. The CR cases stay a unit test and say why:
`.gitattributes` checks every `test/` file out with LF, so a CR only reaches
the parser from a string built in the test.
* fix(minify): keep a body's leading whitespace when its tag is implied
A parser-implied `<body>` prints transparently, which puts a leading
whitespace run where re-parsing drops it: before `<body>` starts, the
insertion modes ignore whitespace. `</html> foo` came back as `foo`, losing
a text node the source had. The tag materializes now when the run would
otherwise be eaten.
Found by running the html5lib serializer corpus (228 cases) through minify
and comparing the parsed trees -- it flagged four documents, all this one
bug. The behaviour predates this branch; `origin/main` loses the run too.
* test(html): assert minification keeps a document's tree over a corpus
A snapshot records what the minifier emits; it cannot say whether the output
still means the same thing. `configCases/html/minimize-round-trip` parses
each corpus document before and after minification and compares the trees,
so a transform that changes meaning fails whatever the bytes look like.
The comparison canonicalizes exactly what the transforms are licensed to
rewrite -- a token list, a declaration list, a srcset's candidates, a
viewport's pairs, a boolean attribute's presence, a `<style>` body, and the
whitespace nothing renders -- and nothing else.
Sixteen documents, one per construct family a minifier classically breaks:
table sections, nested lists, option groups, the adoption agency, paragraph
siblings, SVG and MathML integration points, `<template>` content, attribute
value edge cases, verbatim text, conditional comments, CSS, srcset, quoting,
an implied `<body>`, and a body whose text starts with whitespace.
The corpus is ours rather than ported: the other minifiers publish no tests
in their packages, and this environment cannot reach their repositories.
Checked to have teeth against the parent commit, where the leading-whitespace
document fails with the text node it lost.
* chore: split the minify changeset by bump level and subject
* feat(minify): strip the whitespace in a JSON `<script>` body
`application/json`, `application/ld+json` (any `+json` subtype), `importmap`
and `speculationrules` bodies are JSON, and JSON's whitespace is only between
tokens. Stripping it needs no async minifier, so this is the one inline
`<script>` body that does not wait on the terser question.
Every literal is copied byte for byte rather than re-serialized.
`JSON.parse` + `JSON.stringify` would round `12345678901234567890` to
`...567000`, rewrite `1.0` to `1`, drop the first of two duplicate keys and
turn `é` into `é` -- all meaning changes this has no business making. A
body the JSON parser rejects is left alone, as is any other `type`.
* feat(minify): collapse every whitespace-separated token list, not just class
`rel`, `ping`, `headers`, `sandbox`, `itemprop`, `itemref`, `itemtype`,
`accesskey`, `blocking` and `part` are the same kind of value as `class` --
an ASCII-whitespace-separated token list whose set is what the DOM reads --
so they take the collapse `class` already had.
`for` stays out: on `<output>` it is a token list, but on `<label>` it is one
id, and an id may contain a space.
* feat(minify): normalize URL, integer and comma-list attribute values
Three more grammars whose whitespace carries nothing.
A URL attribute's ends are trimmed: the URL parser strips leading and
trailing C0 controls and spaces before resolving, so the result is the same
URL.
An integer attribute loses its surrounding whitespace and leading zeros --
the parse rules skip the first and read the rest as base ten. Both are gated
on the elements the attribute is an integer on: `width` on `<img>` is one,
but on `<td>` the legacy dimension grammar also admits a percentage, so that
one is left alone.
A comma-separated list (`accept`, `coords`, `sizes`) loses the whitespace
around its commas. `sizes` keeps the whitespace inside an item, which its
media condition needs.
The fixtures show no gain from any of this -- generated markup already emits
tidy values -- but hand-written markup does not.
* fix(test): match an unquoted src, and minify in-process for coverage
Two things CI caught that local runs could not.
`process-assets/html-plugin`'s inline plugin matched `<script src="...">`
with the quotes required. It runs at `OPTIMIZE_INLINE`, after minification
at `OPTIMIZE_SIZE`, so it stopped matching once the quotes went and the
script was never inlined. The matcher takes them optionally now -- the same
fix `INTEGRITY_SENTINEL_REGEXP` needed, and the second quote-assuming
consumer this change has turned up.
The minify config cases now run the minimizer with `parallel: false`. Its
worker pool is another process, so the coverage instrument saw none of the
minify code the cases exercise -- patch coverage read 64% for lines the
tests do run. In-process the four cases cover 58.69% of `lib/html/syntax.js`
on their own.
* fix(html): fold and trim only ASCII in the attribute grammars
The attribute rewrites trimmed with String#trim, which also strips NBSP and
the other Unicode spaces the HTML integer, token-list, URL, comma-list and
viewport grammars read as ordinary characters, so a leading NBSP could turn
an unparsable integer into a valid one or drop a class token. Trim ASCII
whitespace instead, and reject a signed value on a non-negative integer
attribute, which its parse rules do not accept.
The same applies to case folding: the parser's unknown-name fallback and the
printer both used a Unicode toLowerCase, which maps U+212A onto k and U+0130
onto two characters. Fold ASCII only.
* feat(html): complete the optional-tag rules and stop escaping a bare >
Adds the three §13.1.2.4 omissions that were still missing: a <caption> and a
<colgroup> end tag when no whitespace or comment follows them, and a
<colgroup> start tag when a <col> opens it and no preceding colgroup dropped
its own end tag. <html>, <head> and <body> stay out on purpose.
A > in text is only ever a character, so escaping it costs bytes and buys
nothing. Minification emits it bare; the default serialization keeps §13.3.
* test(html): find inlined scripts with the parser, refresh the cache snapshot
The html-plugin case matched <script src=...></script> with a regex, which
CodeQL flagged as a tag filter that misses forms like </script >. It now walks
webpack's own HTML parser, which gives the element's source offsets and the
decoded src directly, so no tag pattern is needed.
ConfigCacheTest.snap was left behind when the transforms corpus grew.
* fix(html): keep the whitespace inside a comma-list item
"Split a string on commas" strips each token's edges and nothing else, but the
rewrite also deleted the whitespace inside an item of accept and coords. That
changes what the DOM keeps: `accept="image/ png"` is an invalid token the
browser holds verbatim and became the valid `image/png`, and coords parses
whitespace as a separator, so `1 2,3` went from three numbers to two.
Trim the edges only, which is what sizes already did — the three attributes
now share one rule, so the table is a Set.1 parent 572fd32 commit 99dd2c7
53 files changed
Lines changed: 1929 additions & 151 deletions
File tree
- .changeset
- lib/html
- test
- configCases
- css
- minimize-minimizer-detection
- minimize-minimizer-in-plugins
- minimize-minimizer-include-only
- html
- minimize-attributes/__snapshots__
- minimize-experiments-auto
- minimize-experiments-true
- minimize-integrity
- src
- minimize-round-trip
- cases
- minimize-transforms
- __snapshots__
- minimize
- process-assets/html-plugin
- tooling
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
25 | 28 | | |
26 | 29 | | |
27 | 30 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
| 121 | + | |
121 | 122 | | |
122 | 123 | | |
123 | 124 | | |
| |||
154 | 155 | | |
155 | 156 | | |
156 | 157 | | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
157 | 161 | | |
158 | 162 | | |
159 | 163 | | |
| |||
176 | 180 | | |
177 | 181 | | |
178 | 182 | | |
| 183 | + | |
179 | 184 | | |
180 | 185 | | |
181 | 186 | | |
| |||
185 | 190 | | |
186 | 191 | | |
187 | 192 | | |
| 193 | + | |
188 | 194 | | |
189 | 195 | | |
190 | 196 | | |
| |||
226 | 232 | | |
227 | 233 | | |
228 | 234 | | |
| 235 | + | |
229 | 236 | | |
230 | 237 | | |
231 | 238 | | |
| |||
245 | 252 | | |
246 | 253 | | |
247 | 254 | | |
| 255 | + | |
248 | 256 | | |
249 | 257 | | |
250 | 258 | | |
| |||
312 | 320 | | |
313 | 321 | | |
314 | 322 | | |
| 323 | + | |
315 | 324 | | |
316 | 325 | | |
317 | 326 | | |
| |||
406 | 415 | | |
407 | 416 | | |
408 | 417 | | |
| 418 | + | |
409 | 419 | | |
410 | 420 | | |
411 | 421 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
250 | 250 | | |
251 | 251 | | |
252 | 252 | | |
253 | | - | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
254 | 257 | | |
255 | | - | |
| 258 | + | |
256 | 259 | | |
257 | 260 | | |
258 | 261 | | |
259 | 262 | | |
260 | | - | |
| 263 | + | |
261 | 264 | | |
262 | 265 | | |
263 | 266 | | |
| |||
534 | 537 | | |
535 | 538 | | |
536 | 539 | | |
537 | | - | |
| 540 | + | |
538 | 541 | | |
539 | 542 | | |
540 | 543 | | |
| |||
0 commit comments