Skip to content

perf(engine): skip the screenshot on frames the page reports unchanged in beginframe capture - #7027

Merged
miguel-heygen merged 5 commits into
mainfrom
perf/beginframe-static-reuse
Nov 2, 2026
Merged

miguel-heygen merged 5 commits into
mainfrom
perf/beginframe-static-reuse

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

What

On Linux (BeginFrame capture), a frame the page reports as unchanged is no longer screenshotted. It reuses the previous frame's bytes.

Why

Renders with held or still stretches (title cards, pauses, cuts after a short tween) paid a full screenshot encode for every frame. The log said static-frame dedup: disabled (capture mode beginframe, not screenshot), and the existing reuse in beginFrameCapture never fired: a BeginFrame that carries a screenshot request always reports hasDamage: true, even for an identical second tick, so noDamageFrames was 0 on every render (360/360 with damage on a 12 s mostly-still fixture).

Related work

Refs the render-speed gap with frame-in-page capture. This is the "reuse paint when nothing changed" half and is independent of the capture-transport work.

How

beginFrameCapture (screenshotService.ts) now sends a screenshot-less BeginFrame first when it holds the previous frame. Its hasDamage is the page's own report for the seek just made. No damage returns the cached buffer; damage takes the screenshot as before. The screenshot after the probe is taken with noDisplayUpdates: a displayed screenshot frame leaves damage that the next probe reports (measured: S=true, p=true, p=false), while an undisplayed one does not. A settle frame to absorb that damage would draw page changes that nothing captures, so there is none. Any BeginFrame sent from elsewhere moves the display past the cached buffer, so the one such site that can run after captures start (frameCapture.ts, the first-capture seek presentation) clears it with forgetBeginFrameCapture. Warmup, commit and probe-liveness BeginFrames run at init with an empty cache; drawelement mode never reaches the cache.

Nothing is predicted from the timeline. The existing timeline-based dedup (armStaticDedup) is untouched and still skips in beginframe mode, because its verification screenshots are not taken through BeginFrame.

Test plan

  • Unit tests added/updated: beginFrameReuse.browser.test.ts drives real headless-shell with BeginFrame control. Unchanged page gives hasDamage: false and identical bytes; a changed pixel gives damage and different bytes. Fails on main at expect(held2.hasDamage).toBe(false), passes 5 runs in a row after the fix. A second test queues three chained rAF paints across frames and asserts the last returned frame equals a fresh capture; it fails at the previously reviewed head 26a3f1f and passes now.
  • Manual testing performed: three fixtures rendered with the main build and this build back to back on the same devbox (30 fps, 1 worker, 1280x720, same loaded box, two to three runs each, so read the ratios, not the seconds):
Fixture Capture, main Capture, this PR Frames reused Frames differing in the output mp4
Held still (12 s, tweens at 0-2 s and 10.5-11.5 s) 8.3 s, 10.5 s, 11.4 s (three main runs) 7.6 s, 9.4 s 269 of 360 0 of 360
Cut-heavy (12 scenes, hard cuts, short tween each) 11.1 s, 12.5 s 5.8 s, 7.6 s 240 of 360 0 of 360
Video (10 s clip under a caption) 16.9 s, 16.8 s, 20.3 s 16.1 s, 20.3 s, 18.9 s 0 of 300 0 of 300

Output identity is per-frame md5 of the decoded mp4s from both builds. The video fixture changes every frame, so it correctly reuses nothing and shows no slowdown from the extra settle frame.

Seek and the ffmpeg write still run on every frame; skipping the seek needs a signal from before the seek, which only the timeline-based path has.

  • Documentation updated (if applicable)
  • Comments follow CONTRIBUTING.md "Comments"

Size

93 changed lines: one self-contained engine change plus its browser test. It has no other open PR in its lane to fold into.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes at 26a3f1fc: a change that lands in the post-screenshot settle frame is never captured, and later frames reuse the stale buffer.

Blocker. The settle BeginFrame (screenshotService.ts, screenshotTicks + interval / 4) runs a full main frame and updates the display. Anything committed in that frame is drawn there and nowhere else: rAF callbacks queued during the screenshot frame, or a late video frame / image decode / font swap. The next probe then reports hasDamage: false relative to the settle frame and returns lastFrameCache, which predates the change. The stale frame then repeats for the whole still stretch, until something else damages. On main the next frame's screenshot picks the change up, so at worst one frame is late.

Repro (headless-shell 152, same flags as beginFrameReuse.browser.test.ts). After the page settles, queue rAF → green, rAF → blue, rAF → yellow, then call beginFrameCapture three times:

  • A: hasDamage: true. The buffer pixel is blue, but the DOM is already yellow.
  • B, C: hasDamage: false, and each returns A's blue buffer byte for byte.
  • After forgetBeginFrameCapture, a fresh capture is yellow.

So the test name's "never a stale one" doesn't hold. The three fixtures came out identical only because nothing in them changes after a screenshot frame.

I tried the obvious fix: a settle with noDisplayUpdates: true closes the hole, but then the probe always reports damage and nothing is reused (your test fails at held.hasDamage). The fix needs a different signal, e.g. a page-side change counter (MutationObserver plus media/rAF activity) checked between the screenshot and the probe, or gating reuse to pages that can't change outside the seek. Please also add a test like the repro above, one that fails at this head.

Non-blocking:

  • Measured: the PR's browser test passes here.
  • frameCapture.ts also sends BeginFrames for warmup and commit (init only, cache empty) and in drawelement mode. Neither reaches the cache today, because the first-capture forgetBeginFrameCapture covers page reuse. That's worth a line in the comment.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 106 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

Unstable (1)

  • seqrotatenudge-none-pct-r0-root-z100: tracking 0.03, pressJump 0, drop 0, reload 3.03, render 0.04, renderKey -, undo false, teleport true / tracking 0.03, pressJump 0, drop 0, reload 0, render 0.08, renderKey -, undo true, teleport true / tracking 0.03, pressJump 0, drop 0, reload 0, render 0.08, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at df8cc999. This resolves the blocker I raised on 26a3f1fc.

  • Removing the settle frame and taking the post-probe screenshot with noDisplayUpdates closes the gap. A change made in the screenshot frame isn't displayed yet, so the next probe draws it, reports damage and captures it. Nothing gets drawn without being captured.
  • Measured with my earlier repro, extended to a four-step rAF chain and then a synchronous change. Every change appears in the output within one frame, the held frames afterwards match a fresh capture, and nothing stays stale.
  • The new test fails with screenshotService.ts from 26a3f1fc (1 failed, 1 passed) and passes at this head (3 runs, 2/2).

Non-blocking:

  • Every capture after the first now runs two main frames (probe + screenshot) instead of one. Content driven by rAF frame count therefore advances twice per output frame, and intermediate states can be skipped (the blue step of the chain never appears). Seek-driven compositions are unaffected, and raw-rAF compositions are already sent to screenshot capture. Worth a line in the comment.
  • The body still says the video fixture "shows no slowdown from the extra settle frame". There's no settle frame any more.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes at 91fda7e4: the new install step fails in CI, so the Flake gate is red at this head.

Blocker. The setup-test-workspace step id: shell exits 127 in the Flake gate (job 125464920196):

npx --yes @puppeteer/browsers install chrome-headless-shell@stable --path "$dir"
sh: 1: browsers: not found

npx inside the workspace can't resolve the package's browsers bin, so the shell is never installed and the job fails before flake-gate.mjs runs. The PR therefore doesn't yet do what its title says. Something like npx --yes --package=@puppeteer/browsers@<version> browsers install … should fix it, but please confirm on a CI run before re-pinning.

Non-blocking:

  • Pin the shell version (or reuse the one the engine pins) instead of @stable. Otherwise the gate can change underneath you when Chrome ships.
  • If find comes back empty, the output path is "". ?? doesn't fall through on an empty string, so HF_BEGINFRAME_SHELL="" silently skips the suite in the one job meant to run it. Fail the step when the binary isn't found.
  • The engine source hasn't changed since df8cc999, which I approved. The test-side basename filter is fine.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes at 874e0471: the Flake gate is still red, with the same browsers: not found / exit 127 in the id: shell step (job 125476447776). The browser test never runs.

Cause, reproduced locally. The command fails only when run from the repo root:

# in the hyperframes checkout
$ npx --yes --package=@puppeteer/[email protected] browsers --version
sh: 1: browsers: not found
# in /tmp
$ npx --yes --package=@puppeteer/[email protected] browsers --version
3.2.2

Inside the bun workspace, npx resolves against the workspace's own dependencies and never puts the browsers bin on PATH. Run it from outside the checkout, for example:

dir="$RUNNER_TEMP/headless-shell"
(cd "$RUNNER_TEMP" && npx --yes --package=@puppeteer/[email protected] browsers install chrome-headless-shell@stable --path "$dir")
shell=$(find "$dir" -type f -name chrome-headless-shell | head -n 1)
test -n "$shell"
echo "path=$shell" >> "$GITHUB_OUTPUT"

Please show a green Flake gate before the next re-pin. The test -n also covers my earlier note that an empty path silently skips the suite. Pinning @puppeteer/browsers is good; the shell itself is still @stable, which is non-blocking.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at cbffe241. The CI blocker is resolved.

  • Switching to bunx works. In the Flake gate (job 125482640805) it installed [email protected] and passed that path as HF_BEGINFRAME_SHELL. The gate then ran beginFrameReuse.browser.test.ts 10 times under CPU contention: 10/10 passed, taking about 40 s, so the suite actually ran rather than being skipped. The Flake gate is green.
  • The engine source matches df8cc999, which I approved after checking the stale-frame fix. Since then only CI setup and the test's shell selection have changed.

Non-blocking (carried over): the shell is still @stable, and an empty find result would silently skip the suite. A test -n guard would make that fail loudly.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 3bb0a9a Nov 2, 2026
93 checks passed
@miguel-heygen
miguel-heygen deleted the perf/beginframe-static-reuse branch November 2, 2026 19:35
zohaibtariq pushed a commit to zohaibtariq/hyperframes that referenced this pull request Nov 8, 2026
zohaibtariq pushed a commit to zohaibtariq/hyperframes that referenced this pull request Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant