Skip to content

fix(engine): the drawElement self-check reads its truth on a page the render does not use - #6316

Merged
miguel-heygen merged 5 commits into
mainfrom
checklong/de-verify-truth-events
Oct 25, 2026
Merged

miguel-heygen merged 5 commits into
mainfrom
checklong/de-verify-truth-events

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

What

The drawElement self-check reads its ground-truth screenshots on a disposable page, with timeline callbacks running, instead of on the render's own page with callbacks suppressed. Compositions with videos keep the old in-page path (the video injector is tied to the render page); a follow-up covers them.

Why

The check compared each drawElement frame with a screenshot taken after __hf.seek(t, { suppressEvents: true }). That skips GSAP callbacks, so anything a callback draws (a count-up's text, a class swap that shows a panel, a timeline call) was missing from the "truth" while the render, whose per-frame seek runs callbacks, drew it. The check failed (11 to 30 dB against the 32 dB bar) and the whole render restarted on the slower screenshot path.

Making the seek run callbacks on the render's own page is not enough, and is unsafe: the check's forward seeks then run a callback that changes the page cumulatively (a call that sets text, say) and leave it changed, so the real render starts from a polluted page. In that case the check passed at 70 dB while the drawElement video showed the changed text before the callback's time. Taking the truth on a separate page (the pattern createStaticVerificationPage already documents for static-frame verification) removes both problems.

Related work

None open. Size: a lone engine fix under the usual PR size floor; no other open PR touches the engine's capture code.

How

  • seekDeVerifyTruth(page, t, suppressEvents) is the one seek used for the truth.
  • A composition without videos opens a disposable page with the existing createStaticVerificationPage, seeks eventfully there (frame 0, then ascending, like the render) and closes it; the render's page is never touched.
  • If that page cannot be opened, the check logs it and reads the truth on the render page with callbacks suppressed, exactly as before this change, so a failed open never fails the render. (The static-frame verify degrades the same way.)
  • A composition with videos reads its truth on the render page as before, with callbacks suppressed and the page returned to frame 0.
  • Cost: one extra page load per video-free drawElement init; on the plain-tween composition below the render took 6.7 s both ways.

Test plan

  • Unit: frameCapture-verifySeek.test.ts covers the seek with callbacks running and suppressed, and a page without the seek protocol; lint, format and typecheck clean.
  • Real renders on a laptop, 20 s 1080p30, released 0.8.198 bundle against the same bundle with only this change:
composition released with this change
plain tweens passes, 6.7 s passes, 6.7 s
count-up text in onUpdate fails at 30.3 dB, screenshot re-render 29.6 s passes (75 to 77 dB), 11.9 s
onStart swaps a full-screen panel fails at 11.6 dB, 30.8 s passes, 12.4 s
tl.call toggles full-screen panels fails at 11.6 dB, 29.4 s passes, 11.8 s; frame at 5 s is the right panel
tl.call sets text at 3 s fails at 20.3 dB, 29.7 s passes (70 dB), 11.8 s; frame at 1 s shows the text before the call

The last row is the one an eventful seek on the render page got wrong (it passed but showed the text after the call at 1 s). Output sizes match the screenshot render in every row.

  • Not covered: compositions with videos (unchanged path), and a render-level test of the open-failure fallback, which needs a real browser session.

  • Unit tests added/updated

  • Manual testing performed

  • Documentation updated (if applicable)

  • Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

…xt it should accept

The check's ground-truth screenshot was taken after a seek that suppressed timeline callbacks, so text a callback writes (a count-up) was stale in the truth while the real render, which runs callbacks, showed it correctly. The check failed and the whole render restarted on the slower screenshot path.
@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Parked as a draft: with only the eventful seek, a composition whose callback changes the page cumulatively (a timeline call that sets text) passes the check at 70 dB but the drawElement video shows the changed state at frames before the call, because the check's own forward seeks run the callback and leave the page changed for the real render. The check has to take its truth on a page the render does not use. Reworking that here.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head 4fe97cae

The fix is right. The render's own DE loop calls __hf.seek(t), which defaults to eventful, and the static-verify seek already passes suppressEvents: false, so the truth now matches the render. The only blocker is that the Comments check fails on this head. ↓ inline

Scope

Read the captureDeVerificationFrames seek and the new helper and test. I checked the render-path seeks (drawElementService.ts:1092, seekPageTimeline, static verify frameCapture.ts:3643) and the runtime default (init.ts:4665). CI was still running when I posted this review.

return [...Array.from({ length: k - 1 }, (_, i) => (i + 1) / k), 0.95];
}

/** Runs timeline callbacks like the render's seek; suppressed, a count-up's text is stale in the truth. */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Comments check fails here: the comment ratchet reports frameCapture.ts at 21.8% (1153/5284) against 21.8% (1152/5281) at the base, and this added docblock line is the one extra comment line. Drop it; the helper's name and frameCapture-verifySeek.test.ts already carry the why.

Suggested change
/** Runs timeline callbacks like the render's seek; suppressed, a count-up's text is stale in the truth. */

… render does not use

Callbacks now run in the truth seek (a count-up or class swap no longer looks stale), and they run on a disposable page so a callback that changes the page cumulatively cannot change what the render draws. Compositions with videos keep the in-page path, because the video injector is tied to the render page.
@miguel-heygen miguel-heygen changed the title fix(engine): the drawElement self-check no longer rejects count-up text it should accept fix(engine): the drawElement self-check reads its truth on a page the render does not use Oct 25, 2026
@miguel-heygen
miguel-heygen marked this pull request as ready for review October 25, 2026 15:16

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 1 nit · head 9e6cb7d8

The Comments blocker from 4fe97cae is fixed. Reading the truth on a disposable page makes the eventful seek safe for video-free comps. That page is createStaticVerificationPage, which already serves static verify. One new blocker: a failure to open that page now fails the render instead of degrading. ↓ inline

  • nit: the PR body still describes the 4fe97cae approach (always suppressEvents: false on the render page). It doesn't mention the disposable page, or that video comps still read their truth with callbacks suppressed.
Scope

I read the rewritten captureDeVerificationFrames, createStaticVerificationPage, the finalizeDrawElementInit call path, and the producer injector. createVideoFrameInjector returns null without a frame lookup, so isolated covers the video-free render sessions, not just probes. CI (main and Windows) was still running when I posted this review.

// A video-free comp reads its truth on a disposable page: callbacks that run there
// cannot leave the render's own page changed. The video injector is tied to the render page.
const isolated = !session.onBeforeCapture;
const truthPage = isolated ? await createStaticVerificationPage(session) : page;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a second full page load (goto, pollHfReady, sub-composition and font waits) to every video-free drawElement init, and nothing catches it here. finalizeDrawElementInit → initializeSession has no try/catch, so a navigation timeout or a killed tab under memory pressure now fails session init in captureStage's ensureSessionReady or the streaming stage. Before, verify only took screenshots on a page that had already loaded. The static verify call to the same helper (frameCapture.ts:3795) catches this and degrades with verification_failed. Do the same here: on a creation failure, fall back to page with suppressEvents: true, which is the pre-PR behavior.

Suggested change
const truthPage = isolated ? await createStaticVerificationPage(session) : page;
const truthPage = isolated
? await createStaticVerificationPage(session).catch(() => null)
: page;

(…then const onTruthPage = truthPage ?? page, and suppress events and skip the close whenever the fallback was taken.)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed in 3ec057b: when the truth page cannot open, the check falls back to the render page with callbacks suppressed and does not close it.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 140 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (2)

  • seqheldtwice-tlhold-px-r0-root-z100: tracking 120.29, pressJump 312.3, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport false / tracking 0.13, pressJump 0.11, drop 0.07, reload 0.07, render 0.02, renderKey -, undo true, teleport true / tracking 0.13, pressJump 0.11, drop 0.07, reload 0.07, render 0.02, renderKey -, undo true, teleport true
  • seqplainkeys-none-pct-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Head 3ec057b: a truth page that cannot be opened now logs it and reads the truth on the render page with callbacks suppressed (the pre-change behaviour), so a failed open never fails the render. The PR body is refreshed to describe the disposable-page approach and the video-composition scope. No unit test for the fallback itself: the function has no seam without a real browser session; the open-failure path is the same .catch degrade the static-frame verify already uses.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 2 resolved, 0 open, 0 new · 9e6cb7d8 → 3ec057bf. CI is green on this head.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 25, 2026
Merged via the queue into main with commit 21f315e Oct 25, 2026
117 of 118 checks passed
@miguel-heygen
miguel-heygen deleted the checklong/de-verify-truth-events branch October 25, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant