Skip to content

hyperframes check: flag text drawn in a fallback face (VA-3327) - #7330

Closed
jerrai-bot-heygen wants to merge 2 commits into
mainfrom
jerrai/hf-check_text_in_fallback_face-11062026
Closed

jerrai-bot-heygen wants to merge 2 commits into
mainfrom
jerrai/hf-check_text_in_fallback_face-11062026

Conversation

@jerrai-bot-heygen

@jerrai-bot-heygen jerrai-bot-heygen commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A composition set Korean text in IBM Plex Mono, which has no Hangul. Chrome quietly drew it in a system fallback face, and hyperframes check said nothing. VA-3327 asks for a check that flags any text drawn in a face other than the declared one, whatever the script or cause.

Design

hyperframes check gets a fonts section beside layout, contrast and craft.

  • For each visible element with its own non-whitespace text, ask Chrome which platform fonts drew it with CDP CSS.getPlatformFontsForNode, on the same Puppeteer page the other sections use (page.createCDPSession()).
  • Compare against the first family of the computed font-family. System faces are matched by family name. Chrome reports a webfont by the file's internal name, which can be unrelated to the CSS name, so a custom face counts as the declared one when the declared family is a loaded @font-face.
  • Any other face gives one text_in_fallback_face finding per element, graded by outcome: error when the declared face drew none of the run's characters (the whole run fell back, the VA-3327 case), warning for a partial fallback. A repeated element and cause across seeks merges, keeping the worse severity.
  • Generic keywords (sans-serif, monospace, system-ui, -apple-system, BlinkMacSystemFont, ...) as the first family name no face, so they are skipped.
  • Emoji and symbols are ignored when deciding fallback (Extended_Pictographic, Emoji_Presentation, \p{S}, plus ZWJ, variation selectors and keycap), so "Launch day 🚀" in Inter is clean.
  • A platform-reported face that is the OS's metric-compatible substitute counts as the declared face, from one small documented table (OS_SUBSTITUTES): Liberation Sans for Arial/Helvetica, Liberation Serif for Times New Roman/Times, Liberation Mono for Courier New/Courier, .SF NS for the SF names. Core OS substitutes only, no per-language entries.
  • By design a custom webfont later in the author's own stack counts as declared: Korean drawn by a second listed webfont such as "Noto Sans KR" passes (isDeclaredFace documents this).
  • No cmap parsing and no per-language branches. A missing glyph, an unloaded webfont and a unicode-range gap all show up the same way.
  • To grade and name the affected characters, each distinct character of an element that shows a foreign face is probed alone in a throwaway off-screen span styled like the element, then removed. Results are cached per computed font and character.
  • It runs after the seek grid, at up to 3 seeks spread across the film so text that enters late is covered. Same element and same cause across seeks merge into one finding with times.

Code: packages/cli/src/utils/checkFonts.ts (new), wired through checkBrowser.ts (driver collectFontFallback), checkPipeline.ts (auditFonts, report section, error count, finding crops), checkTypes.ts, checkFailureCodes.ts, and a Fonts block in the human report.

Report shape

report.fonts is a normal section (ok, errorCount, warningCount, infoCount, findings). A finding:

{
  "code": "text_in_fallback_face",
  "severity": "error",
  "selector": "#caption",
  "text": "한국어",
  "declaredFamily": "IBM Plex Mono",
  "usedFamily": "Noto Sans CJK KR",
  "glyphCount": 3,
  "message": "Text in #caption is declared in \"IBM Plex Mono\" but 3 glyph(s) were drawn in \"Noto Sans CJK KR\".",
  "fixHint": "Load a face for \"IBM Plex Mono\" that covers these characters, or declare a family that does.",
  "sourceFile": "index.html",
  "bbox": { "x": 0, "y": 32, "width": 800, "height": 46 },
  "time": 0.5,
  "times": [0.5, 1]
}

Cost

Measured with collectFontFallback on repo composition blocks in headless Chromium on this box, 3 consecutive seeks (ms each, text-bearing elements in brackets):

block elements ms per seek
ios26-liquid-glass 55 9, 4, 4
vpn-youtube-spot 51 31, 17, 20
message-thread-reveal 130 131, 110, 98
app-showcase (webfont unavailable offline) 40 516, 350, 320
ui-3d-reveal (webfont unavailable offline) 136 614, 489, 506

A clean film costs tens of milliseconds for all 3 seeks. The cost is the per-character probe, paid only where a foreign face shows up, so the worst case is a film whose declared webfont is missing. Before the changes below that case took 2.4 s per seek (ui-3d-reveal, 118 flagged elements). Now: element-level reads are pipelined, per-character probes are batched per element and cached per computed font, and flagged elements are capped at 30 per seek, so the worst case is about 0.5 s per seek (about 1.5 s for the 3 seeks). A film with more than 30 affected elements reports the first 30, which already says the font is wrong.

Known limit

If the box has no fallback face for a script at all, Chrome draws the missing glyph as a notdef box in the declared face and reports the declared face. That case is not a fallback, so this check does not see it. The fixture therefore uses a unicode-range gap, which gives a real fallback on any machine.

Tests

  • checkFonts.chromium.test.ts (real Chromium, 11 tests): a webfont declared for capitals only. ABC xyz is a partial fallback, a warning naming xyz and the fallback face. xyz xyz is a whole-run fallback, an error. An emoji-only run, and capitals with emoji and symbols, are not flagged. A -apple-system, BlinkMacSystemFont stack is not flagged. Fully covered text, a generic first family and hidden text are not flagged. isDeclaredFace takes the OS substitutes and a later webfont in the stack, and rejects a wrong substitute. runAuditGrid through createPageDriver fills fontFindings.
  • The existing check.test.ts, checkPipeline*, checkBrowser.test.ts suites pass with the new section (fixtures gain fonts / fontFamily / a stub CDP session).
  • tsc --noEmit shows no errors in the touched files, and oxlint / oxfmt are clean on them.

Follow-up outside this PR

The hyperframes-cli skill line listing the --json envelope sections (skills/hyperframes-cli/references/lint-validate-inspect.md) and its mirrors do not mention fonts yet; left out to keep the mirror check untouched. Say if you want it here.

On the experiment-framework side, hf_check/pipeline.py passes a fonts section through unchanged (its stages read any member with a findings list). One small EF follow-up is needed so the turn-end report path keeps it: add "fonts" to BROWSER_SECTIONS in cutroom/agent/plugins/dsh/hf_check/turn.py.

— Jerrai

Adds a fonts section to hyperframes check. For each element with text, Chrome's
CSS.getPlatformFontsForNode says which faces drew it; any face other than the
declared first family is a text_in_fallback_face error.

Jerrai
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Fallow audit report

Quality gate failed

Found 2 findings.

Dead code (2)
Severity Rule Location Description
major fallow/unused-export packages/cli/src/utils/checkFonts.ts:4 Export 'FONT_FALLBACK_CODE' is never imported by other modules
major fallow/unused-export packages/cli/src/utils/checkPipeline.ts:2297 Export 'STOP_CODES' is never imported by other modules

Generated by fallow.

Gate outcomes: failed audit-verdict.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 119 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

Unstable (1)

  • seqab-none-pct-r0-root-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 6, 2026
…, record duplicate rule

Review PR heygen-com#7304-heygen-com#7335: 10 merged already adopted or deferred; 10 open or draft PRs
pinned (heygen-com#7304, heygen-com#7308, heygen-com#7311, heygen-com#7315, heygen-com#7321, heygen-com#7330-heygen-com#7334); open duplicates of
pinned heygen-com#7281/heygen-com#7285/heygen-com#7286/heygen-com#7296 are tracked by those pins. strict exit 0;
monitored 110 -> 120.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@jerrai-bot-heygen

Copy link
Copy Markdown
Contributor Author

Closing for now; the branch stays. VA-3327 itself was fixed by the 32 MB install_font cap in EF #66451. This glyph-coverage check was the optional third item, its CI is red and Magi's review fixes are still open, and nobody will pick it up this week. It can come back as a new PR from this branch. — Jerrai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant