Repository navigation
fix(engine): a chunk's first frame shows its own time, not the composition's opening frame - #5807
Closed
miguel-heygen wants to merge 1 commit into
Closed
miguel-heygen wants to merge 1 commit into
miguel-heygen wants to merge 1 commit into
Conversation
…ition's opening frame
Contributor
Edit accuracy: accurate 271 (base branch 271), smooth 168 of thoseThe gate passes. Quarantined, measured but not gated (0) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the user sees
A distributed render (plan, then chunks, then assemble) could start any chunk after the first on the composition's opening frame: the frame at time 0, with the background video on its first frame and no overlays yet. It is one wrong frame per chunk boundary, so it shows as a flash at every seam. Whether a given chunk got it was down to timing, so two renders of the same plan gave different videos.
Cause
On a BeginFrame capture session, the first display-producing
HeadlessExperimental.beginFramecan return the frame Chrome composited during session init, not the seek the page has already applied. Logged just before that capture, the DOM was already correct (title opacity 1, native<video>hidden, injected frame visible and decoded), and Chrome answeredhasDamage: truewith a fresh screenshot. In some runs that screenshot was still the init frame. Every later frame of the session was on time (checked against the source video, frame by frame).What triggers it is the chunk's BeginFrame liveness probe (
renderChunk→beginFrameSessionNeedsScreenshotFallback→probeBeginFrameLiveness). It sends one more BeginFrame on the capture session after init and before the first seek. On main, on the host where it was stale 9 times in 10:noDisplayUpdates: trueSo any extra BeginFrame between init and the first seek is enough, whether or not it displays. The probe itself is needed: it is how a chunk detects a SwiftShader compositor stall. Local renders never probe their capture sessions this way, which is why they are not affected (next section).
Local renders (
hyperframes render): not affectedDo local multi-worker renders flash too? No. 0 of 240 worker-start frames were stale on main, and 0 of 240 with this PR. I rendered the same pinned fixture locally with
--workers 1, with--workers 4(the default interleaved split: worker i draws frames i, i+4, ..., so the worker starts are frames 0 to 3), and with--workers 4plusHF_CAPTURE_PARALLEL_STREAM=false(the contiguous disk split: worker starts 0, 75, 150, 225). That was 5 runs of each idle and 5 under 16 CPU hogs, on two Linux hosts, plus 15 extra loaded--workers 4rounds alternating main and this PR. Every log reportedcaptureMode: beginframeand the requested worker count. Worker-start frames were compared with the--workers 1render by per-frame PSNR (a stale one is about 21 to 25 dB).--workers 1frame 0--workers 4interleaved starts--workers 4contiguous starts--workers 1Local output is byte-identical between main and this PR on both hosts, so the extra BeginFrame changes nothing there. One loaded
--workers 4render with this PR differed at frame 5 only. That frame is worker 1's second frame, not a worker start, and showed the title one step behind the video. It did not recur in 15 alternating rounds (main 0 of 20 loaded--workers 4renders, this PR 1 of 20), so I cannot tell whether it is related.Fix
In
captureFrameSurface, the first BeginFrame capture of a session first displays the seek once. It sends one visual BeginFrame with no screenshot at the capture tick minus half an interval. Ticks stay monotonic: the producer's liveness probe sits at base − 5 intervals and the init commit at base − 6. This covers chunk workers, parallel workers and retry pages, because each one is a fresh session. It adds one BeginFrame per session.Measured
A 10 s 1080p30 fixture with an H.264 background video, a GSAP title and AAC audio, split into 5 chunks. It was generated once and pinned by sha256, and every chunk of the same plan was rendered 10 times (5 idle, 5 under CPU hogs).
Which way the race goes depends on the host: host A was usually stale when idle, host B only when loaded. On both hosts chunk 0 also flipped between Chrome's own paint of the native video and the injected extracted frame. With the fix it always shows the injected frame. A stale first frame measures 21 dB against the correct one; the rest of the chunk differs only by the encoder carrying that frame forward (about 50 dB).
Tests: the new
frameCapture-firstBeginFrame.test.tsfails on main (both cases) and passes with the fix, 3 runs in a row. The engine's frameCapture and screenshotService suites pass (31 files, 333 tests). The full engine unit suite passes except 4ffprobe.test.tscases that read an LFS fixture which was not fetched in this checkout; they fail identically on main.Not exercised
beginframebranch changed.The plan step was deterministic apart from the plan folder path baked into
planHash, which #5782 already fixes.