Skip to content

Commit 4ed269a

Browse files
perf(core): playback housekeeping runs on a timer, and rebinds stop re-sampling silent media (#4595)
* perf(core): playback housekeeping runs on a timer, and rebinds stop re-sampling silent media While playing, the transport re-posted the clip manifest every 20 frames and re-bound media every 30, frame-counted, so a 120 Hz display did both twice as often for a film that was not changing. Playing, those jobs now run when the composition changes (rate-limited as before) and on a one-second poll for what nothing can push. A rebind re-probed every media element's volume by seeking the whole timeline 60 times per second of clip, about 1,500 seeks and 50-140 ms in one frame. The probe seeks with events suppressed, so only a tween on the element that names volume can move it; an element no tween touches is now answered from the tweens without seeking. * fix(core): a data-width or data-height change applies while playing and parked postTimeline applies the root's sizing, and it now runs when the composition revision moves. The revision only watched timing attributes, so a size edit waited for a pause while playing, and on main it was missed while parked too. The observer now also watches data-width and data-height. The tick comment is trimmed to three lines, the tick reuses its playing flag, and the cadence test is renamed to what it checks now. * test(core): share one 120 Hz frame step in the transport park tests The two playing tests each defined the same frame helper; one frame120Hz now sits beside settle and all four loops use it. No source change. * refactor(core): drop the playing poll doc comments The comment ratchet does not let a file's comment share rise; the reason lives in the PR body. * fix(core): a playing clip's track, label and z-index reach Studio again The playing path had dropped the periodic manifest post, and the composition observer does not see data-track-index, data-timeline-label or an inline z-index on an existing clip (watching style would fire on every GSAP frame). On the playing poll the runtime now builds the manifest and posts it only when it differs from the last one posted, so an edit lands within a second and a steady film still posts nothing. * fix(core): keep sampling volume when a tween drives a non-DOM object A tween on a plain-object proxy whose setter writes the element's volume names neither the element nor volume, so the fast skip dropped it and WebAudio-routed preview played the static data-volume (0 where the film asks for 0.375). The skip now applies only when every tween targets DOM nodes; any other target falls back to the full volume probe, as on main. * fix(core): skip the volume probe again for spacers, calls and counters The non-DOM fallback fired on any plain-object or function target, so a tl.to({}, {duration}) spacer, a tl.call or a tweened counter brought back the full probe on ordinary films. It now falls back only when a tweened property resolves to a setter on the target or its prototype chain, which is what a gain proxy or a class with a volume setter uses; data properties skip. * fix(core): post a clip edit made while playing when playback stops A label, track or inline z-index edited while playing and then paused before the next one-second poll was never posted: once parked, nothing watches it. The tick where playback stops now runs the same manifest diff once and posts only if the manifest changed.
1 parent 13bee74 commit 4ed269a

6 files changed

Lines changed: 304 additions & 21 deletions

File tree

‎packages/core/src/runtime/init.ts‎

Lines changed: 38 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@ import { swallow } from "./diagnostics";
103103
import {
104104
CHANGE_DRIVEN_SERVICE_MIN_INTERVAL_MS,
105105
MEDIA_BIND_INTERVAL_FRAMES,
106+
PLAYING_POLL_INTERVAL_MS,
106107
TIMELINE_POST_INTERVAL_FRAMES,
107108
shouldAttemptPeriodicTimelineBind,
108109
} from "./timelineRebindPolicy";
@@ -1217,7 +1218,8 @@ export function initSandboxRuntimeModular(): void {
12171218
childList: true,
12181219
subtree: true,
12191220
attributes: true,
1220-
attributeFilter: DURATION_FLOOR_INPUT_ATTRIBUTES,
1221+
// Sizing inputs too: postTimeline applies them, and it runs when this revision moves.
1222+
attributeFilter: [...DURATION_FLOOR_INPUT_ATTRIBUTES, "data-width", "data-height"],
12211223
});
12221224
// Media duration changes are published as events, not DOM mutations, and
12231225
// they do not bubble — so listen in the capture phase, which reaches every
@@ -3007,6 +3009,7 @@ export function initSandboxRuntimeModular(): void {
30073009
let assetsReadyStarted = false;
30083010
let assetsSettled = false;
30093011
let liveRootDurationOverrideSeconds = 0;
3012+
let lastPostedManifest = "";
30103013
const computeClipTreeSignature = (): string => {
30113014
let sig = "";
30123015
for (const el of document.querySelectorAll("[data-start]")) {
@@ -3032,6 +3035,7 @@ export function initSandboxRuntimeModular(): void {
30323035
canonicalFps: state.canonicalFps,
30333036
});
30343037
window.__clipManifest = payload;
3038+
lastPostedManifest = JSON.stringify(payload);
30353039

30363040
const currentSignature = computeClipTreeSignature();
30373041
if (clipTreeSignature !== currentSignature) {
@@ -3065,6 +3069,14 @@ export function initSandboxRuntimeModular(): void {
30653069
scheduleRootStageLayoutDiagnostics();
30663070
};
30673071

3072+
// What no observer sees (a label, a track or an inline z-index edited on a playing clip) still reaches Studio.
3073+
const postTimelineIfManifestChanged = () => {
3074+
const manifest = JSON.stringify(
3075+
collectRuntimeTimelinePayload({ canonicalFps: state.canonicalFps }),
3076+
);
3077+
if (manifest !== lastPostedManifest) postTimeline();
3078+
};
3079+
30683080
const finitePositiveDuration = (value: number | null | undefined): number =>
30693081
typeof value === "number" && Number.isFinite(value) && value > 0 ? value : 0;
30703082

@@ -3829,6 +3841,9 @@ export function initSandboxRuntimeModular(): void {
38293841
/** A composition change has been seen and not yet carried to consumers. */
38303842
let compositionChangePending = false;
38313843
let lastChangeDrivenServiceAtMs = Number.NEGATIVE_INFINITY;
3844+
let lastPlayingPollAtMs = Number.NEGATIVE_INFINITY;
3845+
let playingPollWitness = "";
3846+
let lastTickPlaying = false;
38323847

38333848
const seekRuntimeTimeline = (
38343849
timeline: RuntimeTimelineLike,
@@ -4195,30 +4210,32 @@ export function initSandboxRuntimeModular(): void {
41954210
try {
41964211
transportTickCount += 1;
41974212

4198-
// The three periodic jobs below fire on a frame counter, which is only a
4199-
// proxy for "the document may have changed since last time". The counter
4200-
// stops advancing while the loop is parked, so on THAT path the question
4201-
// is asked directly: the composition timing revision moves on exactly the
4202-
// inputs these three derive from.
4203-
//
4204-
// Only on that path. While the clock is playing the counter advances at
4205-
// frame rate and is already a good cadence, and raising it is a real
4206-
// regression: postTimeline walks the whole document, so a composition
4207-
// that mutates the DOM every frame would turn one post per 20 frames into
4208-
// one per frame. And the change-driven path is rate-limited to the same
4209-
// posts-per-second the counter yields at 60 Hz, so no consumer ever sees
4210-
// a faster cadence than the frame counter alone produced.
4213+
// The jobs below run when the composition revision moves (rate-limited). What no
4214+
// observer sees is polled: on the frame counter paused, on a timer playing.
42114215
const timingRevision = readCompositionTimingRevision();
42124216
if (timingRevision !== lastSeenTimingRevision) {
42134217
lastSeenTimingRevision = timingRevision;
42144218
compositionChangePending = true;
42154219
}
42164220
const nowMs = Date.now();
4221+
const playing = clock.isPlaying();
4222+
const playingPollDue = playing && nowMs - lastPlayingPollAtMs >= PLAYING_POLL_INTERVAL_MS;
4223+
const stoppedPlaying = lastTickPlaying && !playing;
4224+
lastTickPlaying = playing;
4225+
if (playingPollDue) {
4226+
lastPlayingPollAtMs = nowMs;
4227+
const witness = readParkedPollWitness();
4228+
if (witness !== playingPollWitness) {
4229+
playingPollWitness = witness;
4230+
compositionChangePending = true;
4231+
}
4232+
}
42174233
const changeDrivenService =
42184234
compositionChangePending &&
4219-
!clock.isPlaying() &&
42204235
nowMs - lastChangeDrivenServiceAtMs >= CHANGE_DRIVEN_SERVICE_MIN_INTERVAL_MS;
42214236
if (changeDrivenService) lastChangeDrivenServiceAtMs = nowMs;
4237+
const pausedCounterDue = (interval: number) =>
4238+
!playing && transportTickCount % interval === 0;
42224239

42234240
// Slower operations: timeline binding (~every 60 frames / ~1s at 60fps).
42244241
// `compositionChanged` goes IN to the policy, never around it: the policy
@@ -4227,10 +4244,11 @@ export function initSandboxRuntimeModular(): void {
42274244
if (
42284245
shouldAttemptPeriodicTimelineBind({
42294246
tick: transportTickCount,
4230-
isPlaying: clock.isPlaying(),
4247+
isPlaying: playing,
42314248
hasCapturedTimeline: state.capturedTimeline != null,
42324249
currentTimeSeconds: clock.now(),
42334250
compositionChanged: changeDrivenService,
4251+
playingPollDue,
42344252
})
42354253
) {
42364254
const prevTimeline = state.capturedTimeline;
@@ -4246,7 +4264,7 @@ export function initSandboxRuntimeModular(): void {
42464264
postTimeline();
42474265
}
42484266
}
4249-
if (changeDrivenService || transportTickCount % TIMELINE_POST_INTERVAL_FRAMES === 0) {
4267+
if (changeDrivenService || pausedCounterDue(TIMELINE_POST_INTERVAL_FRAMES)) {
42504268
// The manifest is what carries a composition change to its consumers,
42514269
// so posting it is what discharges the pending flag — whichever path
42524270
// got here. Clearing it when the change is merely SEEN would drop it
@@ -4259,8 +4277,10 @@ export function initSandboxRuntimeModular(): void {
42594277
// change happens to wake the loop again.
42604278
postTimeline();
42614279
compositionChangePending = false;
4280+
} else if (playingPollDue || stoppedPlaying) {
4281+
postTimelineIfManifestChanged();
42624282
}
4263-
if (changeDrivenService || transportTickCount % MEDIA_BIND_INTERVAL_FRAMES === 0) {
4283+
if (changeDrivenService || pausedCounterDue(MEDIA_BIND_INTERVAL_FRAMES)) {
42644284
bindMediaMetadataListeners();
42654285
}
42664286

‎packages/core/src/runtime/mediaVolumeEnvelope.test.ts‎

Lines changed: 116 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
/** @vitest-environment jsdom */
2-
import { describe, expect, it } from "vitest";
2+
import gsap from "gsap";
3+
import { describe, expect, it, vi } from "vitest";
34
import {
45
interpolateVolumeGain,
56
probeAndCacheElementVolume,
@@ -140,6 +141,120 @@ describe("probeAndCacheElementVolume", () => {
140141
expect(cache.has(audio)).toBe(false);
141142
});
142143

144+
it("does not seek to probe an element whose volume no tween names", () => {
145+
const audio = document.createElement("audio");
146+
audio.dataset.start = "0";
147+
audio.dataset.duration = "25";
148+
document.body.append(audio);
149+
const title = document.createElement("h1");
150+
151+
let seekCount = 0;
152+
const timeline = {
153+
totalTime(next?: number) {
154+
if (next !== undefined) seekCount += 1;
155+
return 0;
156+
},
157+
getChildren: () => [
158+
{ targets: () => [title], vars: { volume: 0, opacity: 1 } },
159+
{ targets: () => [audio], vars: { playbackRate: 2 } },
160+
],
161+
};
162+
const cache = new WeakMap<HTMLMediaElement, { time: number; volume: number }[]>();
163+
164+
// What a rebind does for every bound media element: 25 s at 60 samples/s on main.
165+
probeAndCacheElementVolume(audio, timeline, 25, cache);
166+
167+
expect(seekCount).toBe(0);
168+
expect(cache.has(audio)).toBe(false);
169+
});
170+
171+
it("samples a gain tweened through a plain-object proxy that writes the element's volume", () => {
172+
const audio = document.createElement("audio");
173+
audio.dataset.start = "0";
174+
audio.dataset.duration = "1";
175+
audio.dataset.volume = "0";
176+
document.body.append(audio);
177+
const proxy = {
178+
get gain() {
179+
return audio.volume;
180+
},
181+
set gain(value: number) {
182+
audio.volume = value;
183+
},
184+
};
185+
const timeline = gsap.timeline({ paused: true });
186+
timeline.fromTo(proxy, { gain: 0 }, { gain: 0.75, duration: 1, ease: "none" }, 0);
187+
const cache = new WeakMap<HTMLMediaElement, { time: number; volume: number }[]>();
188+
189+
probeAndCacheElementVolume(audio, timeline, 1, cache);
190+
191+
const envelope = cache.get(audio);
192+
expect(envelope).toBeDefined();
193+
expect(interpolateVolumeGain(envelope!, 0.5)).toBeCloseTo(0.375, 3);
194+
});
195+
196+
it.each([
197+
["a {} spacer", (tl: gsap.core.Timeline) => tl.to({}, { duration: 25 })],
198+
["a tl.call", (tl: gsap.core.Timeline) => tl.call(() => {}, [], 3)],
199+
["a plain counter", (tl: gsap.core.Timeline) => tl.to({ n: 0 }, { n: 100, duration: 5 })],
200+
])("does not seek for %s", (_name, add) => {
201+
const audio = document.createElement("audio");
202+
audio.dataset.start = "0";
203+
audio.dataset.duration = "25";
204+
document.body.append(audio);
205+
const timeline = gsap.timeline({ paused: true });
206+
add(timeline);
207+
const seek = vi.spyOn(timeline, "totalTime");
208+
209+
probeAndCacheElementVolume(audio, timeline, 25, new WeakMap());
210+
211+
expect(seek.mock.calls.filter((args) => args.length > 0)).toHaveLength(0);
212+
});
213+
214+
it("samples a class instance whose volume setter writes the element", () => {
215+
const audio = document.createElement("audio");
216+
audio.dataset.start = "0";
217+
audio.dataset.duration = "1";
218+
audio.dataset.volume = "0";
219+
document.body.append(audio);
220+
class Fader {
221+
get volume() {
222+
return audio.volume;
223+
}
224+
set volume(value: number) {
225+
audio.volume = value;
226+
}
227+
}
228+
const timeline = gsap.timeline({ paused: true });
229+
timeline.fromTo(new Fader(), { volume: 0 }, { volume: 0.5, duration: 1, ease: "none" }, 0);
230+
const cache = new WeakMap<HTMLMediaElement, { time: number; volume: number }[]>();
231+
232+
probeAndCacheElementVolume(audio, timeline, 1, cache);
233+
234+
expect(interpolateVolumeGain(cache.get(audio) ?? [], 0.5)).toBeCloseTo(0.25, 3);
235+
});
236+
237+
it("still samples a keyframed volume tween on the element", () => {
238+
const audio = document.createElement("audio");
239+
audio.dataset.volume = "1";
240+
document.body.append(audio);
241+
242+
const timeline = {
243+
totalTime(next?: number) {
244+
if (next !== undefined) audio.volume = next >= 1 ? 0 : 1;
245+
return 0;
246+
},
247+
getChildren: () => [{ targets: () => [audio], vars: { keyframes: [{ volume: 0 }] } }],
248+
};
249+
const cache = new WeakMap<HTMLMediaElement, { time: number; volume: number }[]>();
250+
251+
probeAndCacheElementVolume(audio, timeline, 2, cache);
252+
253+
expect(cache.get(audio)).toEqual(
254+
expect.arrayContaining([expect.objectContaining({ volume: 0 })]),
255+
);
256+
});
257+
143258
it("restores the timeline playhead after sampling volume automation", () => {
144259
const audio = document.createElement("audio");
145260
audio.dataset.volume = "1";

‎packages/core/src/runtime/mediaVolumeEnvelope.ts‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,9 @@ function probeKeyframesInWindow(
202202
return hasAutomation ? keyframes : null;
203203
}
204204

205-
export type RuntimeTimelineRef = Partial<Pick<RuntimeTimelineLike, "totalTime" | "seek">>;
205+
export type RuntimeTimelineRef = Partial<
206+
Pick<RuntimeTimelineLike, "totalTime" | "seek" | "getChildren">
207+
>;
206208

207209
export interface VolumeProbeOptions {
208210
/**
@@ -216,6 +218,46 @@ export interface VolumeProbeOptions {
216218
allowLiveTimelineSeek?: boolean;
217219
}
218220

221+
function namesKey(vars: unknown, matches: (key: string) => boolean, depth = 0): boolean {
222+
if (depth > 3 || vars === null || typeof vars !== "object") return false;
223+
if (Array.isArray(vars)) return vars.some((item) => namesKey(item, matches, depth + 1));
224+
if (Object.getPrototypeOf(vars) !== Object.prototype) return false;
225+
return Object.entries(vars).some(
226+
([key, value]) => matches(key) || namesKey(value, matches, depth + 1),
227+
);
228+
}
229+
230+
const isDomNode = (target: unknown): boolean =>
231+
typeof (target as { nodeType?: unknown } | null)?.nodeType === "number";
232+
233+
function hasSetter(target: unknown, key: string): boolean {
234+
for (let o = Object(target); o; o = Object.getPrototypeOf(o)) {
235+
const descriptor = Object.getOwnPropertyDescriptor(o, key);
236+
if (descriptor) return typeof descriptor.set === "function";
237+
}
238+
return false;
239+
}
240+
241+
/**
242+
* Whether seeking `timeline` can move `el.volume`: a tween on the element that names `volume`, or a tween
243+
* on a non-DOM object whose tweened property is a setter (a gain proxy). Spacers, calls and counters skip.
244+
*/
245+
function timelineCanMoveVolume(timeline: RuntimeTimelineRef, el: HTMLMediaElement): boolean {
246+
if (typeof timeline.getChildren !== "function") return true;
247+
try {
248+
return timeline.getChildren(true, true, false).some((tween) => {
249+
const targets: unknown[] = tween.targets?.() ?? [];
250+
return targets.some((target) =>
251+
isDomNode(target)
252+
? target === el && namesKey(tween.vars, (key) => key === "volume")
253+
: namesKey(tween.vars, (key) => hasSetter(target, key)),
254+
);
255+
});
256+
} catch {
257+
return true;
258+
}
259+
}
260+
219261
/**
220262
* Probe a media element and, if volume automation is detected, store a
221263
* NORMALISED envelope in `cache`. Safe to call with a null timeline — returns
@@ -238,6 +280,8 @@ export function probeAndCacheElementVolume(
238280
if (!timeline) return;
239281
if (!isMediaElement(mediaEl)) return;
240282
if (compositionDuration <= 0) return;
283+
// Sampling is ~60 whole-timeline seeks per second of clip, paid again on every rebind.
284+
if (!timelineCanMoveVolume(timeline, mediaEl)) return;
241285

242286
const seekFn = (t: number) => {
243287
try {

‎packages/core/src/runtime/timelineRebindPolicy.test.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ describe("shouldAttemptPeriodicTimelineBind", () => {
3232
isPlaying: true,
3333
hasCapturedTimeline: true,
3434
currentTimeSeconds: PLAY_REBIND_HOLD_SECONDS - 0.001,
35+
playingPollDue: true,
3536
}),
3637
).toBe(false);
3738
expect(
@@ -40,6 +41,7 @@ describe("shouldAttemptPeriodicTimelineBind", () => {
4041
isPlaying: true,
4142
hasCapturedTimeline: true,
4243
currentTimeSeconds: PLAY_REBIND_HOLD_SECONDS,
44+
playingPollDue: true,
4345
}),
4446
).toBe(true);
4547
});
@@ -51,7 +53,21 @@ describe("shouldAttemptPeriodicTimelineBind", () => {
5153
isPlaying: true,
5254
hasCapturedTimeline: false,
5355
currentTimeSeconds: 0,
56+
playingPollDue: true,
5457
}),
5558
).toBe(true);
5659
});
60+
61+
it("checks on the poll timer, not the frame counter, while playing", () => {
62+
const playing = {
63+
tick: TIMELINE_REBIND_INTERVAL_FRAMES,
64+
isPlaying: true,
65+
hasCapturedTimeline: true,
66+
currentTimeSeconds: PLAY_REBIND_HOLD_SECONDS,
67+
};
68+
expect(shouldAttemptPeriodicTimelineBind(playing)).toBe(false);
69+
expect(shouldAttemptPeriodicTimelineBind({ ...playing, tick: 7, playingPollDue: true })).toBe(
70+
true,
71+
);
72+
});
5773
});

‎packages/core/src/runtime/timelineRebindPolicy.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ export const MEDIA_BIND_INTERVAL_FRAMES = 30;
1010
* it: `TIMELINE_POST_INTERVAL_FRAMES` frames at a 60 Hz display.
1111
*/
1212
export const CHANGE_DRIVEN_SERVICE_MIN_INTERVAL_MS = (1000 * TIMELINE_POST_INTERVAL_FRAMES) / 60;
13+
export const PLAYING_POLL_INTERVAL_MS = (1000 * TIMELINE_REBIND_INTERVAL_FRAMES) / 60;
1314

1415
export function shouldAttemptPeriodicTimelineBind(input: {
1516
tick: number;
@@ -24,6 +25,7 @@ export function shouldAttemptPeriodicTimelineBind(input: {
2425
* async rebind cannot race the first two seconds of playback.
2526
*/
2627
compositionChanged?: boolean;
28+
playingPollDue?: boolean;
2729
}): boolean {
2830
// The hold is the outer rule and applies to every trigger.
2931
if (
@@ -34,6 +36,7 @@ export function shouldAttemptPeriodicTimelineBind(input: {
3436
return false;
3537
}
3638
if (input.compositionChanged === true) return true;
39+
if (input.isPlaying) return input.playingPollDue === true;
3740
return (
3841
Number.isInteger(input.tick) &&
3942
input.tick > 0 &&

0 commit comments

Comments
 (0)