Skip to content

fix(core): the first cut after Play starts on time instead of running ahead - #6794

Merged
miguel-heygen merged 2 commits into
mainfrom
fix/learn-start-latency-from-cuts
Oct 30, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
fix/learn-start-latency-from-cuts

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

What

The first cut after Play no longer starts its incoming video too early. Before, that video ran about 50 ms ahead of the playhead for its whole clip: it caught up by holding a frame just after the cut, then reached its last frame early and held it until the next cut. On a short first clip that was one or two dropped frames on every Play from the start.

Why

#6494 starts an upcoming video early by the start delay learned from earlier starts. It also learns from the video that Play itself starts, and that start waits on the playhead's own start as well as the video's. Measured in Studio preview, a start at Play took 64-74 ms against 41-60 ms for a start at a cut. The learned value is an average, so that one sample set the lead for the first cut alone and pulled the next ones up.

Skipping that sample is not enough: with nothing learned, the first cut starts on its tick about 45 ms late, and a late clip catches up by skipping a frame, usually its last one before the next cut.

How

syncRuntimeMedia records whether the playhead was running on the previous call. A video started on a call where it was not (the first tick of playback) counts at half weight; starts at cuts, and starts after a seek during playback, count in full as before. Half is picked from the measurement above, not derived: a cut's delay was 0.55-0.94 of Play's, so half errs late by well under a frame rather than early, and cut samples then take over the average.

Size

Under 100 lines on purpose: one fix to how the first cut after Play learns its start delay, with its test. Every Play from the start drops frames at the first cuts until it lands, and no other open PR touches this code to carry it.

Related work

Refines #6494.

Test plan

  • Unit tests: media.test.ts learns half the start delay of a video started by Play. The other start-latency tests now start their first video with the playhead already running, which is the case they describe.
  • fails without the fix: counting a Play start in full, or as zero, fails learns half the start delay of a video started by Play and only that test.
  • vitest run src/runtime in packages/core: 109 files, 2655 tests passed. oxfmt, oxlint, core tsc and the comment ratchet clean.
  • Manual testing in Studio preview, Chrome 156 on an M4 Pro MacBook Pro (macOS 26.5, 120 Hz display), playback started with Studio's Play button, 9 runs per side. Media is one synthetic 44-minute 1080p H.264 file at 23.976 fps with long keyframe gaps (7 s apart on average, up to 10.4 s), cut into a 149-clip edit whose second clip is 0.74 s long. Both sides carry fix(core): long cut-list edits play every cut without dropped or late frames #6785; the measured builds predate its dropped-frame guard, which does not touch starts.

Play at 0 s, 20 s per run, picture timed from each video's requestVideoFrameCallback:

9 runs, before → this branch
dropped frames (a frame held 2+ frame periods), total 14 → 0
runs with any frame held over 1.5 frame periods 9 → 1 (one 67 ms hold)
longest picture hold, median run 108 ms → under 63 ms
latest first frame after the playhead reached a clip, median run 30 ms → 30 ms
runs where a cut lost its outgoing clip's last frame 0 → 0
frames shown past an out-point 0 → 0
Play press to the first moving frame, median 232 ms → 215 ms

Before, the drops sat at the first two cuts (6.9-7.0 s and 7.6-7.7 s).

An earlier head of this branch skipped the Play sample instead: 4 drops in 9 runs, the first cut's first frame 64 ms late (median), and a lost last frame in 6 of 9 runs. Half weight removes all three.

With Play at 64 s, where Play itself starts the video under the playhead, the 1-3 drops within 2 s of Play are not this change's: 9 runs interleaved run by run with the previous head, 19 → 16.

For scale: one plain <video> playing the same file on the same machine drops 1-2 frames a minute, and started mid-file it held one frame in its first second in 2 of 9 runs.

  • Documentation updated (not applicable)
  • Comments follow CONTRIBUTING.md "Comments"

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 106 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (3)

  • seqplainkeys-none-center-r0-nested-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqplainkeys-none-pct-r0-nested-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true
  • seqab-none-pct-r0-root-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 0 nits · head a142f3da

I ran the named test locally. does not learn from a video started by Play fails with main's media.ts and passes here (179/179). Pause and a non-continuing seek both sync with playing: false, so the next Play's first tick is never sampled. A seek during playback still learns, as the body says. The first-frame cost of about one frame on the first cut after Play is disclosed and is the owner's call. All workflows are green on this head.

Scope

I read the whole diff (media.ts, media.test.ts) and checked syncRuntimeMedia's only caller, syncMediaForCurrentState in init.ts, on the play, pause and seek paths.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 resolved, 0 open, 0 new · a142f3da → 8e804432. The named test passes here and fails under both mutations I tried: a Play start counted in full (main's behaviour) and counted as zero (the previous head). All workflows are green on this head.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 07c714c Oct 30, 2026
189 of 190 checks passed
@miguel-heygen
miguel-heygen deleted the fix/learn-start-latency-from-cuts branch October 30, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant