Skip to content

fix(producer): a BeginFrame capture failure re-renders on a screenshot session - #7070

Merged
miguel-heygen merged 1 commit into
mainfrom
cloudperf/beginframe-capture-fallback
Nov 3, 2026
Merged

miguel-heygen merged 1 commit into
mainfrom
cloudperf/beginframe-capture-fallback

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

When a render's streaming capture fails in BeginFrame mode, the render restarts once on a fresh screenshot session, whatever the failure was. Before, only stalls, encoder death, transient browser errors and drawElement failures restarted; any other BeginFrame capture failure failed the render.

Why

On Linux with chrome-headless-shell, BeginFrame capture is noticeably faster than screenshot capture, and hosts that opt into it (PRODUCER_FORCE_SCREENSHOT=false) want that speed without a new way to fail. Screenshot capture is the baseline every other fallback already returns to, so a BeginFrame-only failure should cost one re-render, not the render.

Related work

Refs #7066 (a render request can force screenshot capture). Together they let a caller retry in screenshot mode, and make the engine do the same inside one attempt.

How

  • shouldRetryViaPinnedFallback takes isBeginFrameCapture: true means the failed attempt captured through BeginFrame, and that alone allows the restart. Cancellation and encoder interruption still never restart.
  • The streaming catch sets it from the failed attempt's plan: not forced to screenshot, not drawElement, and resolveParallelCaptureMode resolved beginframe for this render.
  • The restart goes through the existing replanAfterFailure, which always sets forceScreenshot: true, so the restarted attempt is not BeginFrame and a second failure is final.

Scope: the streaming route. Distributed chunks already have their own whole-chunk screenshot retry (runCaptureWithScreenshotFallback).

Test plan

  • Unit tests added/updated: renderOrchestrator.beginFrameRetry.test.ts runs executeRenderJob with the streaming stage scripted per attempt.
    • A non-stall capture failure in a BeginFrame attempt restarts with forceScreenshot: true. Fails without the fix: the render rejects with the capture error after one attempt.
    • An attempt that already captured in screenshot mode does not restart.
    • A second failure after the screenshot restart is final.
  • Mutations: dropping the isBeginFrameCapture branch, or passing false at the call site, each fail the two restart tests.
  • renderOrchestrator.test.ts (295), renderOrchestrator.stall.test.ts (9), renderOrchestrator.segmentedDeRetry.test.ts (7) pass; new file passes three runs in a row; tsc --noEmit, oxlint, oxfmt, comment ratchet clean.
  • Manual testing performed: in a Linux container with PRODUCER_FORCE_SCREENSHOT=false, a composition that froze the page tripped the BeginFrame watchdog and the engine restarted it on a screenshot session within the same render.
  • Documentation updated (if applicable)
  • Comments follow CONTRIBUTING.md "Comments"

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at 6b2dcfed. No blocker found.

Verified

  • The body names the test that fails without the fix: renderOrchestrator.beginFrameRetry.test.ts. I checked it against main's renderOrchestrator.ts. On main, "re-renders a BeginFrame attempt…" and "does not retry a second failure…" both fail: one attempt, then the render rejects with the first capture error. All 3 pass at this head.
  • renderOrchestrator.test.ts, .stall, .segmentedDeRetry and .beginFrameRetry pass: 314/314.
  • The retry is bounded. The re-render runs inside the catch, so a second failure propagates. replanAfterFailure sets forceScreenshot: true on every sdr_streaming branch, so the restarted attempt evaluates isBeginFrameCapture as false.
  • isBeginFrameCapture matches the attempt's own label predicate (!plan.forceScreenshot && !useDrawElement && parallelCaptureMode === "beginframe"). It is checked after the cancellation and encoder-interrupted guards, so those still never restart.

Non-blocking notes

  1. The scope is the streaming route only, as the body says. A BeginFrame failure on the plain sdr_disk capture path still fails the render: the disk catch only retries drawElement errors. Routes that go there include webm, png sequences, and BeginFrame with the stream router off. Worth a follow-up if "any BeginFrame failure costs one re-render" is meant to hold everywhere.
  2. "Whatever the failure was" includes failures that will happen again no matter the capture mode, such as a composition that throws, or ENOSPC. Those now take one extra full screenshot render before failing. That is a reasonable trade for a single bounded retry; just noting the cost.
  3. resolveParallelCaptureMode cannot see the engine's software-GPU clamp (its own docstring says so). A render that the clamp moved to screenshot is still labelled BeginFrame and gets one extra screenshot retry. This is harmless.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 106 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

Unstable (1)

  • seqplainkeys-none-pct-r0-nested-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 9a2cd9e Nov 3, 2026
91 checks passed
@miguel-heygen
miguel-heygen deleted the cloudperf/beginframe-capture-fallback branch November 3, 2026 08:02
@miguel-heygen miguel-heygen mentioned this pull request Nov 3, 2026
1 task done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant