Skip to content

perf(core): Safari stops a long video's clips from loading the whole file at once - #7436

Merged
miguel-heygen merged 10 commits into
mainfrom
perf/safari-clip-in-point
Nov 10, 2026
Merged

miguel-heygen merged 10 commits into
mainfrom
perf/safari-clip-in-point

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

What

On Safari in the player, a film cut from one long file (32 MB or more) no longer stands still for about 25 s while it loads. Only the clips on screen and the next take read the file. Every other clip of that file holds no source until it is due, and a clip opens at its in-point instead of at the head of the file. Sound for such a file plays from its element instead of first fetching the whole file to decode it. A smaller file, Chrome, Studio's preview and renders behave as before.

Why

Every clip of one long file opened the file at page load, and WebKit reads tens of MB from the head of the file for each one, whatever its preload. On top of that, the tap's sound unlock (primeMediaForLaterStarts) calls play() on every later clip, which opens its file even at preload="none", and the decode path fetched the whole 68 MB file at the tap. All of this shares one link. On an 8 MB/s link the clip on screen starved and the playhead stood still for about 25 s.

Measured in Safari on an iOS 26.5 Simulator, with the page's media held to 8 MB/s across all connections, over the first 25 s of a film of 10 takes cut from one 68 MB, 321 s file. 10 interleaved plays per arm, median (min to max), page readings:

longest stand-still MB moved first sound frozen picture
main 26.5 s (5.0 to 27.3) 565 (466 to 569) 235 ms (217 to 249) 73 to 80 %
this PR 0.35 s (0.0 to 10.7) 284 (266 to 363) 319 ms (207 to 670) 2.5 to 38 %
  • Sound against picture at each take start is unchanged: median -42 ms on both (this PR -77 to +1 ms over 30 take starts, main -69 to -17 ms over 10).
  • Not fixed yet, and coming in a follow-up:
    • About one play in five (3 of 10 here) still stands still for 4.6 to 10.7 s. When a take's clips get their source back, WebKit re-reads the file from byte 0 for each one despite the fragment (two 23 MB reads in the analysed play), and the take's own reads share the link with them. The same head re-read is most of the bytes this PR still moves.
    • First sound comes about 85 ms later than main. Likely cause: the tap unlock now starts the later clips' reads at their in-points, and they compete with the first clip. Not proven yet.
  • 5.6 MB film, 5 interleaved plays: same as main on every column (123 MB both, stand-still 0.2 s both, frozen picture 3.3 to 5.7 % vs 4.1 to 5.4 %, 4 to 5 audio seeks vs 4, first sound 286 to 322 ms vs 292 to 340 ms).

Related work

Refs the Safari long-file loading work. Replaces this PR's earlier fetch-ahead with eight slots, which was bimodal (4.9 to 32 s in the same setup). #7447 (audio from the element through a decode length limit) is superseded by the long-file check here and can close.

How

  • One decider: preloadNearPlayhead (init.ts) already decides each tick which clips are near (on screen, armed next on its track, or next take). A held clip gets its source back there (giveBackWhenDue) before the media sync plays or seeks it. A clip that has passed is released by releaseMedia, which holds it again instead of re-setting its source.
  • One owner of the authored source: unproxiedSrc (proxySrc.ts) returns the source the author wrote while the runtime holds a clip or has written its fragment. The clip grouping by file, load diagnostics, the picker, the timeline asset url and proxy swaps read it. A script or binding that sets another source ends the hold, and that source is the one the clip plays. A proxy swap on a held clip gives the file back first, so the copy replaces the right file.
  • preloadMedia.ts asks once per file for its size with a one-byte range request. For a file of 32 MB or more, on WebKit and in the player frame only, a later clip that WebKit has not started reading is held with no source. Every other clip opens at #t=<in-point>. A clip WebKit already began reading at parse opens at its in-point instead of being held: holding those too cut requests from about 300 to 80 but made the stand-still 4.8 to 14.9 s, because each take then fetched its metadata under load when due.
  • decodesFirst (init.ts) leaves a long file on its element: its decode fetches the whole file first.
  • The decode cache key drops the fragment, so clips of one file still share one decode.
  • Rendering does not reach any of this: every step returns outside the player frame or off WebKit, and the readers fall back to the src attribute exactly as before when the runtime has not touched a clip. Search first: the hold reuses stopMediaDownload/releaseMedia. git grep "bytes=0-0" packages/core/src finds one other one-byte request, proxyWait.ts, which polls a proxy copy until it is served and reads no size, so the size check is new.
  • The two proxy triggers (handleMetadataForProxy, handleErrorForProxy) now share one path, because the file's duplication check flagged their 19 identical lines once the file changed.
  • Package size budget: @hyperframes/producer unpacked goes from a budget of 3,130,000 to 3,162,274 bytes. Base 3,160,731, head 3,162,274: the +1,543 bytes are exactly the growth of dist/hyperframe.runtime.iife.js (651,645 to 653,188), the only producer file this changes. The producer ships that one runtime file because renders inject it into the render page, so the WebKit-only code rides along there unused. Keeping it out would mean separate preview and render runtime builds, a separate change. Main already sat 0.98 % over the old budget.

Test plan

  • Unit tests added/updated:
    • init.longSourceHold.test.ts: held past the next take and still naming its file; given back at its in-point on a paused seek; a binding set while held is kept and played; let go once passed and given back on a seek back.
    • preloadMedia.test.ts: held later clip, held again after release, a bound clip not held, a clip WebKit already read opens at its in-point, plus the earlier in-point cases.
    • mediaProxy.test.ts: a held clip's swap copies its own file.
    • init.test.ts: a long file's clip stays on its element.
    • webAudioTransport.test.ts, mediaLoadFailure.test.ts.
    • 2,963 of the runtime tests pass on a Linux box; the 43 in compositionLoader.test.ts fail there the same way on unmodified main (box setup), and pass in CI.
  • fails without the fix: init.longSourceHold.test.ts fails 3 of 4 with the giveBackWhenDue call removed from preloadNearPlayhead; "swaps a clip held with no src after giving it back its file" in mediaProxy.test.ts fails with its line removed; "leaves a clip of a long file on its element: decoding it would fetch the whole file first" in init.test.ts fails with the long-file check removed from decodesFirst.
  • Manual testing performed: the Simulator runs above (10 plays per arm, long film; 5 per arm, short film), and one headless Chromium play each of main and this PR (stand-still 0.0 s, 25.0 s wall, both). Renders: the CI regression shards pass at this head. Not run: a real iPhone, and sound by ear (the Simulator played muted, so first sound is a page reading).
  • Documentation updated (not applicable)
  • Comments follow CONTRIBUTING.md "Comments"

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 117 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

@miguel-heygen
miguel-heygen force-pushed the perf/safari-clip-in-point branch from e5671b8 to 3614aca Compare November 9, 2026 08:43
@miguel-heygen
miguel-heygen marked this pull request as draft November 9, 2026 09:59
@miguel-heygen
miguel-heygen force-pushed the perf/safari-clip-in-point branch from c2c2d4c to 00dcbfa Compare November 9, 2026 11:16
@miguel-heygen miguel-heygen changed the title perf(core): Safari clips open at their in-point perf(core): Safari clips open at their in-point and are fetched ahead Nov 9, 2026
@miguel-heygen
miguel-heygen force-pushed the perf/safari-clip-in-point branch from 6a25337 to a97070e Compare November 10, 2026 10:10
@miguel-heygen miguel-heygen changed the title perf(core): Safari clips open at their in-point and are fetched ahead perf(core): Safari stops a long video's clips from loading the whole file at once Nov 10, 2026
@miguel-heygen
miguel-heygen marked this pull request as ready for review November 10, 2026 11:34

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 2 nits · head 2bfc550b

The three fails without the fix: claims check out. With the giveBackWhenDue call taken out of preloadNearPlayhead, init.longSourceHold.test.ts fails 3 of 4. With giveBackWhenDue taken out of swapToProxy, the held-clip swap test fails. With !isLongSource taken out of decodesFirst, the long-file test fails. All 530 tests in the six touched suites pass at this head, and CI is green.

Scope

I read every non-test file end to end. I also checked the other places that read src: previewVideoClipsAt only sees a held clip after preloadNearPlayhead has given it back, because the same deferral rules gate both. The proxy URL keeps #t=, which does no harm because the fragment never reaches the server. I did not repeat the Simulator measurements.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 10, 2026
Merged via the queue into main with commit fecde99 Nov 10, 2026
145 checks passed
@miguel-heygen
miguel-heygen deleted the perf/safari-clip-in-point branch November 10, 2026 13:18
@miguel-heygen miguel-heygen mentioned this pull request Nov 10, 2026
3 of 4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant