Skip to content

fix(producer): segmented and sequential capture check drawElement samples - #7255

Merged
miguel-heygen merged 2 commits into
mainfrom
cloudperf/segmented-drain-guard
Nov 5, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
cloudperf/segmented-drain-guard

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

What

Segmented capture, and the streaming stage's sequential loop (drawElement without worker encode), now run the drawElement self-check: at each sampled frame, the drawElement frame is compared against the screenshot taken before the canvas was injected, and a frame below the PSNR floor fails the render with DrawElementVerificationError before it reaches the encoder.

Why

The check existed only inside the streaming stage's drain guard. The segmented stage and the streaming sequential loop both capture with captureFrameToBuffer and never read session.deVerifyFrames, so a drawElement render on either path had no quality net: a damaged frame was encoded and shipped. This has to land before segmented capture becomes the default.

Related work

Refs the segmented capture work (Phase 2). Does not touch the GPU-less Linux BeginFrame hang or its stack.

How

  • The sample comparison moves out of createDrainFrameGuard into one exported function, verifyDrawElementSample. The drain guard calls it exactly as before, so the worker-encode drain does not change.
  • captureSegmentFrames and the streaming sequential loop call the same function on every frame before writing it. Sessions not in drawElement mode, and frames that are not samples, return immediately.
  • Only the sample check is shared. The drain guard's blank-frame part stays streaming-only, because captureFrameToBuffer already runs the engine's own blank guard on this path.
  • The orchestrator already re-renders a segmented attempt by screenshot on DrawElementVerificationError (renderOrchestrator.segmentedDeRetry.test.ts), so nothing changes there.
  • Each worker session arms its own ground truth at the same sample indices, recycled sessions included. Whichever worker captures a sampled frame checks it.

Test plan

  • Segmented unit tests added. They run 2 workers over 9 frames with samples armed at frames 2 and 7.
    • One test checks that both samples are compared against their own ground truth.
    • The other checks that a sample below the floor fails the render, wrapping DrawElementVerificationError, and is never written to the encoder.
  • Unit tests added for the streaming sequential loop: the armed sample at frame 3 is compared, and a sample below the floor fails the render with DrawElementVerificationError.
  • fails without the fix: "drawElement self-verify > compares every armed sample against its ground truth" and "drawElement self-verify > fails the render before a sample below the floor is encoded" (captureSegmentedStage.test.ts); "drawElement self-verify on the sequential loop > compares the armed sample against its ground truth" and "drawElement self-verify on the sequential loop > fails the render on a sample below the floor" (captureStreamingStage.test.ts). The segmented pair also goes red when the shared check returns without comparing. All pass again once restored.
  • captureSegmentedStage.test.ts and captureStreamingStage.test.ts: 73 pass, 3 runs in a row.
  • renderOrchestrator.segmentedDeRetry.test.ts (vitest): 7 pass, 3 runs in a row.
  • tsc --noEmit (producer), oxlint and oxfmt on the changed files: clean. Run on a Linux box.
  • Manual render: not done. The change routes frames through a check that is already exercised on the streaming path.
  • Comments follow CONTRIBUTING.md.

@miguel-heygen miguel-heygen changed the title fix(producer): segmented capture checks drawElement frames against their screenshots fix(producer): segmented and sequential capture check drawElement samples Nov 5, 2026

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head 24686388

  • blocker (PR body, process gate): a fix PR here needs a fails without the fix: line that names a test. The body has only "Fails without the fix. The segmented tests go red…", which doesn't name one. I checked the claim, and it holds: with the two new verifyDrawElementSample calls removed, exactly these 4 fail and 69 pass. Adding this line clears it:
    - fails without the fix: "drawElement self-verify > compares every armed sample against its ground truth" and "> fails the render before a sample below the floor is encoded" (captureSegmentedStage.test.ts); "drawElement self-verify on the sequential loop > compares the armed sample against its ground truth" and "> fails the render on a sample below the floor" (captureStreamingStage.test.ts).

I found no code blockers. Once the body is fixed, this is an approve.

Scope

Read the whole diff. The extraction matches the old drain-guard body, and the only change is that stats is now optional. The drain guard still passes stats, so its telemetry is unchanged. verifyDiskDrawElementSamples doesn't run on either new path: segmented doesn't go through parallelCoordinator, and streaming is skipped. Truth frames exclude deRiskFrames, so screenshot-substituted risk frames aren't compared. The error is thrown before writeFrame on both paths. At head, the 73 tests in captureSegmentedStage and captureStreamingStage pass under bun.

— Genesis V2 reviewer

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 1 resolved, 0 open, 0 new · 24686388 (head unchanged). The body now has a fails without the fix: line naming the 4 tests, the same 4 I saw fail with the two new call sites removed. Some CI runs on this head were cancelled and others were still running; none had failed.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 118 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

Unstable (1)

  • seqfastkeys-none-px-r0-root-z100: tracking 0.07, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 5, 2026
Merged via the queue into main with commit 8df4f21 Nov 5, 2026
159 of 160 checks passed
@miguel-heygen
miguel-heygen deleted the cloudperf/segmented-drain-guard branch November 5, 2026 19:10
@vanceingalls vanceingalls mentioned this pull request Nov 5, 2026
2 of 3 tasks
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 6, 2026
…view

Adopts heygen-com#7240 (deferred frame extraction leaves the distributed code),
heygen-com#7241 (a capture segment extracts and deletes its own frames), heygen-com#7255
(segmented and sequential capture check drawElement samples), heygen-com#7254
(normalize-audio --reference stops short of the peak ceiling), heygen-com#7256
(engine names a failed script's origin first), heygen-com#7258 (SVG gradient text
judged by its stops) and heygen-com#7242 (windowed source frames, off unless
HF_VIDEO_FRAME_WINDOW=true). Release v0.9.17 is not synced wholesale;
the producer packed budget takes upstream's +12000 from this range.
Watermark advances to 32eb4c3.

Co-authored-by: Vance Ingalls <[email protected]>
Co-authored-by: Miguel Ángel <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant