Skip to content

Commit 9140c0e

Browse files
fix(core): keep authored gain above unity off el.volume in the sandbox bridge (heygen-com#3349)
Authoring a clip above unity gain throws at runtime today. ## What breaks `MAX_AUDIO_GAIN_DB = 12` makes `data-volume` legal up to ~3.98. The sandbox runtime's volume bridge assigns the product straight to the element: ```ts el.volume = clipVolume * volume; // init.ts, onSetVolume ``` `HTMLMediaElement.volume` is spec-pinned to [0,1] and **throws `IndexSizeError`** outside it — verified in Chrome, and the test DOM agrees: ``` el.volume = 2 → IndexSizeError: Failed to set the 'volume' property... ``` The throw lands inside a `for` loop over every media element, so it takes the rest of the loop with it: every clip after the boosted one keeps whatever volume it already had, while `state.bridgeVolume` says the change was applied. A composition with one boosted clip stops responding to the volume control for every clip authored after it. ## The fix Clamp what the element receives. That is not lossy, because the element was never where the boost lived — the transport gets the authored gain unclamped, and this PR pins that half too: - `syncRuntimeMedia` hands `onElementVolume` both the element's clamped volume **and** the authored gain, so the transport can have the boost the element cannot hold. - `setElementVolume` keeps that gain on the per-element node, clamped only to `MAX_AUDIO_GAIN`. Those two paths already worked; they were untested, and they are the reason clamping the element is the right half to clamp. ## Tests - `init.test.ts` — a boosted clip followed by a quieter one, both seeded with sentinels, then the real `set-volume` control message. Asserts the boosted element lands at 1 **and** that the clip after it still gets its own volume, which is what a throw mid-loop strands. - `media.test.ts` — the transport receives the authored gain while the element stays legal. - `webAudioTransport.test.ts` — the per-element gain node keeps a boost above unity. All three mutation-checked: removing the clamp reds the first, and clamping the gain at either transport seam reds the others. ## Provenance This is the last unlanded piece of heygen-com#3280. That PR was rebased onto current `main` and collapsed from +3050 to +944, of which everything except these lines is either already merged (heygen-com#3308, heygen-com#3309, heygen-com#3333, heygen-com#3339) or duplicated by the open heygen-com#3306 and heygen-com#3310. Cutting it out separately because the throw is live on `main` now and shouldn't wait behind a PR that is otherwise redundant.
1 parent d09145f commit 9140c0e

4 files changed

Lines changed: 67 additions & 1 deletion

File tree

‎packages/core/src/runtime/init.test.ts‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,37 @@ describe("initSandboxRuntimeModular", () => {
140140
},
141141
);
142142

143+
it("keeps a boosted clip legal on the element when the bridge sets volume", () => {
144+
// `data-volume` may hold up to 12 dB of authored gain. `el.volume` is
145+
// spec-pinned to [0,1] and THROWS outside it, so assigning the product raw
146+
// aborted the loop — every element after the boosted one kept its old
147+
// volume, and the bridge's own state said otherwise.
148+
document.body.innerHTML =
149+
`<div data-composition-id="main" data-root="true">` +
150+
`<audio data-start="0" data-volume="3.98"></audio>` +
151+
`<audio data-start="0" data-volume="0.5"></audio>` +
152+
`</div>`;
153+
window.__timelines = {};
154+
initSandboxRuntimeModular();
155+
156+
const [boosted, quiet] = Array.from(document.querySelectorAll("audio"));
157+
if (!boosted || !quiet) throw new Error("expected both clips");
158+
// Sentinels, so the assertions cannot be satisfied by what the runtime
159+
// already applied while starting up.
160+
boosted.volume = 0.2;
161+
quiet.volume = 0.1;
162+
163+
window.dispatchEvent(
164+
new MessageEvent("message", {
165+
data: { source: "hf-parent", type: "control", action: "set-volume", volume: 1 },
166+
}),
167+
);
168+
169+
expect(boosted.volume).toBe(1);
170+
// The clip after the boosted one is what a throw mid-loop strands.
171+
expect(quiet.volume).toBeCloseTo(0.5, 5);
172+
});
173+
143174
afterEach(() => {
144175
window.__hfRuntimeTeardown?.();
145176
document.body.innerHTML = "";

‎packages/core/src/runtime/init.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3159,7 +3159,12 @@ export function initSandboxRuntimeModular(): void {
31593159
if (!(el instanceof HTMLMediaElement)) continue;
31603160
const parsed = parseFloat(el.dataset.volume ?? "");
31613161
const clipVolume = Number.isFinite(parsed) ? parsed : 1;
3162-
el.volume = clipVolume * volume;
3162+
// `data-volume` carries authored gain, which goes above unity now that
3163+
// the ceiling is 12 dB — and `el.volume` is spec-pinned to [0,1], so
3164+
// assigning the product raw THROWS IndexSizeError and takes the rest of
3165+
// the loop with it. The element carries the legal part; the boost above
3166+
// unity belongs to Web Audio, which already has it from `setVolume`.
3167+
el.volume = Math.max(0, Math.min(1, clipVolume * volume));
31633168
}
31643169
},
31653170
onSetMediaOutputMuted: (muted) => {

‎packages/core/src/runtime/media.test.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -436,6 +436,27 @@ describe("syncRuntimeMedia", () => {
436436
expect(only).toBeCloseTo(0.55, 5);
437437
});
438438

439+
it("sends boosted author gain to Web Audio while keeping the native element legal", () => {
440+
const clip = createMockClip({ start: 0, end: 10, volume: 3.98 });
441+
Object.defineProperty(clip.el, "readyState", { value: 4, writable: true });
442+
let transportGain = -1;
443+
444+
syncRuntimeMedia({
445+
clips: [clip],
446+
timeSeconds: 1,
447+
playing: true,
448+
playbackRate: 1,
449+
// Third arg is the authored gain, which is the one the transport wants;
450+
// the second is the element's, which the spec pins to [0,1].
451+
onElementVolume: (_el, _effectiveVolume, authorVolume) => {
452+
transportGain = authorVolume;
453+
},
454+
});
455+
456+
expect(transportGain).toBeCloseTo(3.98, 5);
457+
expect(clip.el.volume).toBe(1);
458+
});
459+
439460
/**
440461
* The render bakes the lane at CLIP-LOCAL time: prepareAudioTrack already
441462
* cut the wav with `-ss mediaStart`, so its t=0 is the clip's start, and

‎packages/core/src/runtime/webAudioTransport.test.ts‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -306,6 +306,15 @@ describe("WebAudioTransport", () => {
306306
});
307307

308308
describe("schedulePlayback timing", () => {
309+
it("keeps author boost above unity on the per-element gain node", async () => {
310+
const { transport, mock, gen } = setupTransport(100);
311+
312+
await transport.schedulePlayback(mockEl, mockBuffer, 0, 0, 0, 1, gen);
313+
transport.setElementVolume(mockEl, 3.98);
314+
315+
expect(mock.gainNode.gain.value).toBeCloseTo(3.98, 5);
316+
});
317+
309318
it("starts in-progress clips immediately with correct buffer offset", async () => {
310319
const { transport, mock, gen } = setupTransport(100);
311320

0 commit comments

Comments
 (0)