Skip to content

fix(player): gate play on real composition readiness, not just duration - #4054

Merged
miguel-heygen merged 7 commits into
mainfrom
feat/ad80-assets-ready
Sep 18, 2026
Merged

miguel-heygen merged 7 commits into
mainfrom
feat/ad80-assets-ready

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

Summary

Play now gates on the composition's real declared-media readiness (video/audio/image/font loads), not just on a known duration. Two consumers previously had gaps that could enable Play, or drop a Play click, before what it plays was actually buffered:

  • The <hyperframes-player> web component: an early play() call made before the runtime probe resolved used to be silently dropped. It now queues and retries once ready, and no longer double-fires the public play event while doing so.
  • HeyGen's Studio timeline: both of its readiness call sites (the direct-adapter path and the message-based fallback) now go through one shared store action instead of duplicating a generation-counter-and-settle pattern across two files, and that action is correctly invalidated on a project switch or a full timeline reset.

A new shared module, @hyperframes/core/composition-readiness, is the single place both consumers ask "is this composition's declared media actually ready," so a future readiness input (compute, paint-idle) has one place to plug into.

This ships as a correctness improvement on its own merits. It does not claim to be the fix for a specific reported live bug — that investigation is still open.

What changed

  • packages/core/src/compositionReadiness.ts (new): scanPendingCompositionAssets, mediaReadinessInput, settleCompositionReadiness — resolves synchronously when nothing is pending, asynchronously with a timeout otherwise.
  • packages/player/src/hyperframes-player.ts: play() queues instead of drops when called before the runtime is ready, and only dispatches the public play event once, on the call that actually starts playback. _waitForAssetsReady now uses the shared gate.
  • packages/studio/src/player/store/readinessSlice.ts (new): a store slice owning requestTimelineReady(doc), with a generation guard that's invalidated on every timeline reset and on any direct setTimelineReady call, so a stale in-flight wait can never resolve into a project it was superseded by.
  • packages/studio/src/player/hooks/useTimelineSyncCallbacks.ts and useTimelinePlayer.ts: both readiness call sites now call the shared store action instead of each keeping its own generation counter.

Pre-review pass

An independent reviewer (fresh context, no access to this PR's framing) audited the diff before this was posted, hunting for exactly the two failure classes this kind of change tends to hide: a stale async wait resolving into the wrong state, and a duplicated side effect. It found two real, confirmed defects, both fixed in a follow-up commit with a regression test proving each was reachable before the fix (verified by reverting the fix and re-running):

Area Status Note
Module-level generation counter semantics Fixed Was never invalidated by a project switch or a full reset; now bumped on both
Sync-callback contract at all readiness call sites Audited — clean No caller assumes async-only
play() queue-and-consume logic Fixed Was dispatching the public play event twice for one early call
Dead-code removal Audited — clean The old promise-only wrapper this replaces has no remaining callers
Realm-guard dedup (player's iframe-realm DOM checks, now shared with core) Audited — clean Both existing importers resolve correctly
Timeout handling Audited — clean Timer is cleared on the real-promise path too
Hostile ordering — composition swap mid-wait (player) Audited — clean Covered by existing tests
Hostile ordering — project switch mid-wait (Studio) Fixed Same root cause as the generation-counter finding
Hostile ordering — zero declared media / already loaded Audited — clean Resolves synchronously as intended
Hostile ordering — media that never settles Audited — clean Times out rather than hanging

Test plan

  • packages/core: 7/7 (compositionReadiness.test.ts)
  • packages/player: 376/376, tsc --noEmit clean
  • packages/studio: 1413/1413 (player-area suites), tsc --noEmit clean
  • Two new regression tests added, each confirmed to fail without its fix and pass with it: one for the double play-event dispatch, one for the stale-readiness-across-project-switch race

… playing

The player used to fire ready/autoplay the moment the timeline resolved,
even if the composition's video, images or fonts hadn't loaded yet -
visible as a flash of missing content or a frozen first frame. It now
waits for those assets (readyState >= HAVE_FUTURE_DATA for media, decoded
images, document.fonts.ready) before play() proceeds, showing its own
loading overlay (reusing the shader-transition loading panel) while it
waits, bounded by an 8s timeout that plays anyway and logs which assets
were still stuck.

Settles synchronously with no overlay when nothing needs waiting (already
loaded, or a cross-origin composition the player has no DOM access to
inspect), so today's immediate-autoplay behavior is unchanged for the
common case. A new assetsReady getter and assetsready event mirror the
gate for hosts that want to observe it.
An internal tracking id leaked into a code comment and a test name in the
previous commit. This repo is public; internal references don't belong here.
An independent review found that a composition swap mid-wait (a new src or
srcdoc arriving while the previous composition's media/images/fonts were
still loading) had no way to invalidate the old wait. If the old wait's
promise resolved after the swap, it could mark the NEW composition ready
and even play it, while the new composition's own assets were still
pending - defeating the point of waiting in the first place. The same gap
let a late-resolving wait flip a disconnected player's paused state back to
false and fire a spurious play event after removal.

Fixed both with one generation counter: every new wait captures the current
generation, disconnect and reload each bump it, and a wait's eventual
settle no-ops if its generation is no longer current.

Also fixed: the loading overlay's aria-label ("Loading assets") never got
restored once the asset wait was done, so a later shader-transition overlay
on the same player kept misannouncing itself to assistive tech.
…t two

The generation-counter fix only bumped at two of the six places that set
_ready = false ahead of a navigation, reload, or disconnect - src/srcdoc
attribute changes, a sandbox-origin policy reload, and a shader-options
reload all started a new navigation without invalidating the previous
wait, leaving the same race open through those paths.

One _invalidateAssetsWait() helper now owns the reset (assetsReady,
pendingPlay, generation, the loading attribute, hiding the overlay) and
every one of the six sites calls it before starting whatever comes next.
Add a shared composition-readiness gate in packages/core, exported as the
@hyperframes/core/composition-readiness subpath. It resolves once every
declared input settles (media for now: video/audio/image/font loads), or
after a timeout, and calls back synchronously when nothing is pending so a
caller that plays immediately after sees the decision already applied.

The player element now queues an early play() call instead of dropping it
when called before the runtime is ready, and both of Studio's readiness
call sites (the direct-adapter path and the message-based fallback) go
through one store action, requestTimelineReady, instead of duplicating a
generation-counter-and-settle pattern across two hook files.
…ble-firing readiness

An independent review of the readiness gate found two real races:

play() queued before the runtime was ready re-ran its full body once ready,
dispatching the public "play" event twice for one user action. It now
dispatches "play" only on the call that actually starts playback.

Studio's requestTimelineReady generation guard was never invalidated by a
project switch or a full timeline reset, so a slow-loading project's
in-flight wait could resolve after the switch and mark the new, unchecked
project ready. Resetting the timeline now bumps the generation, and setting
timelineReady directly does too.

@somanshreddy somanshreddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — core design is sound, but one real (nonblocking, should-fix) bug in the queue-play lifecycle, and it's untested. Holding my stamp on it; CI is all-pending so it can't merge yet regardless. Manual pass (Codex disabled per workspace cap).

Bug: a play queued while assets buffer is not cancelled by pause()/seek(), so it fires against the user's pause when assets settle.
Sequence (heavy comp — the exact scenario this PR targets):

  1. Assets still buffering. User presses play → if (this._ready && !this._assetsReady) { this._pendingPlay = true; return; } (hyperframes-player.ts:364). Nothing plays; overlay shows.
  2. User presses pause (or seek). Both set _paused = true but neither clears _pendingPlay nor calls _invalidateAssetsWait (that's wired only to _ready = false sites).
  3. Assets settle → _settleAssetsReady → if (this._pendingPlay) this.play() (:1062), and play() has no _paused guard — it sets _paused = false and starts playback + dispatches play.

Result: the composition plays even though the user paused/seeked while it buffered. The buffering window is precisely the heavy-comp case this PR exists for, so it's reachable, not theoretical.

The same shape applies to the pre-existing runtime-readiness queue (_onProbeReady/_onRuntimeTimelineReady also do if (… || _pendingPlay) this.play() with no _paused guard), so the cleanest fix is at the cancel sites: clear _pendingPlay = false in pause() and seek() — one place, covers both queue paths. (Guarding each retry with !this._paused also works but is the enumerate-every-site version.)

Untested: the asset-ready gate tests (:2765+) cover doc === null, defer-then-play, and the 8s timeout — none exercises pause/seek during the pending window. A test that queues a play, pauses, then settles the asset and asserts isPlaying() === false (and no play event) would fail today.

Verified sound:

  • Core compositionReadiness module is well-built — realm-aware asset scan (cross-realm iframe instanceof handled), settles on canplay-or-error (a failed media element can't block), 8s timeout fail-opens so play never hangs, onSettled fires exactly once, synchronous callback when nothing's pending (no regression for ready comps).
  • Parity holds — player and Studio both consume the one @hyperframes/core/composition-readiness module. Studio's readinessSlice guards the async settle with a module-scope generation counter so a superseded wait can't set timelineReady for stale content; the player's _waitForAssetsReady uses the same generation pattern (_assetsGeneration) and _invalidateAssetsWait on teardown.

Will stamp once the pending checks are green and the pause/seek-cancels-pending gap is closed (or we agree it's a fast-follow).

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting one change at 41704f94e89de3d531ebb43d5e7c1c38665c4298. The core module, the Studio slice and the export wiring are all sound — the single problem is in the player's queue-play lifecycle, and it fires in exactly the window this PR creates.

The change

pause() and seek() don't clear _pendingPlay, so a queued play survives an explicit user pause and then starts playback anyway.

_pendingPlay is set in two places (hyperframes-player.ts:365, :388) and cleared in only two: the top of play() (:368) and _invalidateAssetsWait() (:1069). _invalidateAssetsWait() is called from navigation, disconnect, shader-option change and iframe reload — never from user transport control. pause() (:405) and seek() (:421) both set this._paused = true and neither touches the flag.

So, on a heavy composition inside the up-to-8s asset wait:

  1. _ready is true, _assetsReady is false. play() early-returns at :364 and parks _pendingPlay = true. Nothing visibly starts — _paused stays true and controlsApi.updatePlaying is never called.
  2. The user pauses, or scrubs. pause() dispatches pause; seek() honours its own documented contract that a seek lands paused. Either way the host now believes playback is stopped, and _pendingPlay is still true.
  3. Assets settle. _settleAssetsReady (:1062) calls this.play() with no _paused guard, which falls straight through the :364 early-return (_assetsReady is now true), sets _paused = false, sends play to the runtime and dispatches the public play event.

The composition starts playing against the user's explicit pause, seconds after they stopped it. It's small and it's recoverable, but it makes pause() non-authoritative during the feature's own target scenario, and it is new behaviour — before this PR there was no queue for a pause to lose a race against.

The fix is one place, two lines — clear the flag wherever the user takes control:

  pause() {
    this._pendingPlay = false;
    if (!this._tryDirectTimelinePause()) this._sendControl("pause");

and the same first line at the top of seek(). seek() needs it as much as pause() does, since seek's contract is that it lands paused.

What I verified at source

  • The ready-ordering the whole gate depends on is correct. Both _onRuntimeTimelineReady (:983-987) and _onProbeReady (:1001-1005) set this._ready = true before calling _waitForAssetsReady, so by the time the trailing if (autoplay || _pendingPlay) this.play() runs, play()'s :364 guard can actually see the pending state. If those were ordered the other way this gate would be a no-op on the autoplay path.
  • The generation guards hold. In the player, every _ready = false site calls _invalidateAssetsWait() first, and _settleAssetsReady is idempotent (:1060). In Studio, timelineReadyGeneration is bumped on requestTimelineReady, on setTimelineReady, and in resetPlaybackReadinessState() — which createTimelineResetState() spreads, so beginTimelineSession invalidates too. I could not construct a stale settle that lands in a superseded project.
  • The synchronous-settle claim is real, and it matters: settleCompositionReadiness calls back before returning when nothing is pending, which is why _waitForAssetsReady can check if (!this._assetsReady) afterwards and skip the loading overlay for the common case rather than flashing it.
  • The export wiring is complete and the repo's own gates agree. ./composition-readiness is added in all three required places (dev exports, publishConfig.exports, package-subpaths.json), consistent with the neighbouring ./mediaTiming entry, and tsc emits dist/compositionReadiness.js with no build-entry registration needed. I ran the gates rather than eyeballing it: check:package-subpaths → "Package subpaths verified: 12 public workspaces"; check:package-cycles → no new cycle; check:workspace-contracts → clean; scripts/package-subpaths.test.mjs → 4/4.
  • packages/core compositionReadiness.test.ts → 7/7, matching the test plan.

What I could not run, and the false finding I nearly filed

I could not run packages/player's suite, so the "376/376" and "1,413/1,413" claims are unverified by me.

This one is worth spelling out because it looked like a finding. packages/player/vitest.config.ts carries a hand-maintained alias list, and it covers exactly three specifiers — @hyperframes/core/composition-contract, /slideshow, /runtime/protocol. Those are precisely the three @hyperframes/core subpaths the player imported before this PR. This PR adds a fourth import, @hyperframes/core/composition-readiness, and does not add a fourth alias — and the suite fails for me with Missing "./composition-readiness" specifier in "@hyperframes/core" package, 143 of 154 red. That reads like an exhaustive list with one entry missing.

It isn't. I ran the control before writing it up: in my environment all four of those specifiers fail to resolve, including the three that are fine on main. My borrowed node_modules has no workspace link for @hyperframes/core at all, so nothing resolves through its export map; the alias list was simply masking that for the other three and left the new one exposed. So this is my rig, not your branch, and I am not reporting it as a defect.

The residual, stated as a question rather than a finding: I have no evidence either way about whether that alias list is load-bearing in CI. If your local run of the player suite was against a full bun install, export-map resolution answered and there is nothing to do. Worth one glance, no more.

Non-blocking

  1. collectPendingCompositionAssets leaks its media listeners on the timeout path (compositionReadiness.ts:49-60). onSettled is the only thing that removes the canplay/error pair, so when settleCompositionReadiness resolves through the 8s timeout instead, those listeners stay attached and the per-element promise stays pending. Bounded and low-impact — but the elements it leaks on are exactly the stuck ones you're already warning about.
  2. _onProbeReady has no if (this._ready) return guard, while _onRuntimeTimelineReady (:984) does. That asymmetry predates this PR, but it now has a wider blast radius: a late probe callback arriving after the runtime path already won would re-enter _waitForAssetsReady, bump the generation and re-show the loading overlay on a composition that had already settled.
  3. Everything else in the pre-review table I audited independently and agree with; the readiness slice's module-scope generation counter is the right call for state no component renders from.

Happy to re-read as soon as _pendingPlay is cleared on the user-control paths — that alone clears my block.

— Rames

// fallow-ignore-next-line complexity
play() {
if (this._ready && !this._assetsReady) {
this._pendingPlay = true;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pause() and seek() never clear this, and _invalidateAssetsWait() — the only other place that does — is wired to navigation and disconnect, not to transport control.

That leaves the window this PR introduces: play() parks _pendingPlay = true here without starting anything, the user then pauses or scrubs, and when the assets settle _settleAssetsReady (line 1062) calls play() with no _paused guard. Playback starts against the pause, seconds later.

Clear it wherever the user takes control. In pause():

  pause() {
    this._pendingPlay = false;
    if (!this._tryDirectTimelinePause()) this._sendControl("pause");

and the same line at the top of seek(), whose own contract is that a seek lands paused.

@jrusso1020

Copy link
Copy Markdown
Collaborator

Closing out the vitest-alias question I left open in my review — it's an artifact of my environment, not a defect on this branch. Evidence, in order:

  1. packages/core/package.json at 41704f94 declares the subpath:
    "./composition-readiness": { "bun": "./src/compositionReadiness.ts", "node": "./dist/compositionReadiness.js", "import": "./src/compositionReadiness.ts", "types": ... }.
    The failure I hit was Missing "./composition-readiness" specifier in "@hyperframes/core" package — that error only comes from an exports map that lacks the key, so whatever package.json my run resolved was not this branch's.
  2. Node picks the node condition for these specifiers, not import — measured, not assumed: import.meta.resolve("@hyperframes/core/composition-readiness") from packages/player returns packages/core/dist/compositionReadiness.js. src/compositionReadiness.ts is inside core's tsconfig include: ["src/**/*"] and outside every exclude, so tsc emits that dist file.
  3. .github/workflows/ci.yml:297 runs bun run --cwd packages/core build before :303 runs bun run --filter '!@hyperframes/producer' test. The target exists by the time the player suite resolves it. If Vite's SSR resolver picks import instead, that's src/compositionReadiness.ts, which always exists. Both branches resolve, so there's no missing-4th-alias failure in CI.

For contrast, @hyperframes/core/runtime/protocol genuinely does need its alias: core's tsconfig excludes src/runtime, and none of the three pre-tsc build scripts emit dist/runtime/protocol.js, so that subpath has no built target for the condition Node picks. The discriminator isn't "is it exported" — it's "does the build emit a target for the condition that wins".

Non-blocking nit: a 4th alias would still be worth adding, since without it the player suite exercises the built composition-readiness while the other three run against source, and a local vitest run on a tree where core hasn't been built will fail on it.

The _pendingPlay blocker in my review stands unchanged.

— Rames

A play() queued while assets buffer (_pendingPlay) used to fire once they
settled even after an explicit pause or seek during the wait. pause() and
seek() now clear the flag, so a user pause/seek wins.
@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Fixed at 39e58ba: pause() and seek() now clear _pendingPlay, same as the
existing clear in _invalidateAssetsWait(). A play() queued while assets are
still buffering no longer survives an explicit pause or seek during the wait.

Added one test per ordering, per the request:

  • pause while waiting cancels the queued play
  • seek while waiting cancels the queued play
  • play() called twice while waiting still settles into exactly one start
    (renamed to say explicitly this is play()'s pre-existing double-call
    idempotency, not the pause/seek cancel — an independent review caught that
    the original name implied it covered the fix when it didn't)

Mutation-tested: reverting the two _pendingPlay = false lines fails the new
pause/seek tests; restoring them is byte-identical to this diff. Full player
suite green (379/379).

Simplify + independent adversarial review recorded against 39e58ba.

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — this clears my 5244505964.

Delta against the head I blocked (41704f94..39e58ba8): ahead 1, behind 0, two files, +90/−0, of which the product change is two lines.

The fix is the one I asked for, and it is in the better of the two possible places. this._pendingPlay = false at the top of pause() (:409) and seek() (:429) — at the flag rather than at a fire site. That matters more than it looks, because the queued play has three fire sites and my review named one. _settleAssetsReady:1070 was the one I traced; _onRuntimeTimelineReady:995 and _onProbeReady:1013 also do if (this.hasAttribute("autoplay") || this._pendingPlay) this.play(). Clearing at the source covers all three, including the two I never mentioned.

I checked those two for the same bug arriving through the other half of that condition — user pauses during the wait, a settle later plays anyway because autoplay is still on the element — and it is not reachable. Both are load-time readiness handlers (_onRuntimeTimelineReady returns early on this._ready, and both set it), so they run before there is anything for a user to pause, and a composition swap goes through _invalidateAssetsWait(), which clears the flag and then legitimately re-autoplays the new composition.

The new tests discriminate, and I proved it by mutation rather than by reading them. With the fix in place the asset-ready gate block is 9 passed. With only the two assignments removed and the test file untouched: exactly 2 failed, 7 passed — the two failures being the pause case and the seek case. The third new test ("settles a play() called twice while buffering into exactly one playback start (pre-existing idempotency, not the pause/seek cancel)") stays green under the mutation, so its name is honest: it documents behaviour that predates this commit instead of quietly claiming the new one.

One environment note for whoever runs this next: the block fails 9-for-9 with Failed to resolve import "@hyperframes/core/composition-readiness" until packages/core is built, because that export's node condition resolves to dist/compositionReadiness.js and a fresh bun install does not produce it. Every arm failing — including the pre-existing tests — is the only reason I read that as my own rig rather than as the branch.

Review by Rames

@miguel-heygen
miguel-heygen merged commit 72b223d into main Sep 18, 2026
59 checks passed
@miguel-heygen
miguel-heygen deleted the feat/ad80-assets-ready branch September 18, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants