Skip to content

Commit 99f42be

Browse files
vanceingallsclaude
andauthored
feat(engine): render grouped audio through a summed, FX-processed bus (#3289)
* feat(core): route grouped audio through a group bus in preview An audio element carrying `data-audio-group` no longer lands its gain on the master bus directly — it feeds a per-group `GainNode` (built lazily on first use, one per group id) which itself feeds master, so members of the same group sum before the ear, ready for a group-level FX chain and volume/mute in later steps. An id with no matching `<hf-audio-group>` element still gets a plain, unprocessed bus rather than losing the track. The group's own chain and volume lane are wired through the same `attachElementFxChain`/`scheduleVolumeLane` every element already uses, against the group's clock — composition time (design doc §1.3), since a group has no `data-start` and a missing one parses as 0. The bus persists across `stopAll()` (mirroring `_masterGain`'s own lifecycle) so replaying a group does not rebuild its chain; only `destroy()` disposes it. Render is untouched — stays flat until B4; `audio-groups` is still a 0% canary so nothing ships this to a real composition without hand-authoring `data-audio-group`. Also: `audioGroupOf` (B1) crashed on any element lacking a real `tagName`/ `getAttribute` — exactly the shape of most `HTMLMediaElement` test doubles in this suite, including this file's own `mockEl`. Made it tolerant, same style as `readChain`'s existing guard in `runtime/audioFx.ts`. `schedulePlayback` was already 110 lines pre-existing before this diff; extracted `resolveDestination` and `handleSourceEnded` to shrink it to 92, then suppressed the remainder (inherently sequential graph wiring, not a decision tree) per the same precedent B2 used on `TimelineLogicalRow`. Co-Authored-By: Claude Sonnet 5 <[email protected]> * feat(studio,lint): carve targets voiceover groups — always, when plural Plural voiceover carve now targets a group instead of naming each clip: `resolveCarveSourceIds` (core `audioGroups.ts`) expands a group id to its current members at analysis time, so a clip added to the group later is covered without touching `sources`. The picker (`useFxCarve.ts`) offers a grouped voice as one option instead of one row per member, tests overlap as a union of member spans (a group overlaps the bed if ANY member does), and prefers a qualifying group over its individual members in `autoSourceIds`. Picking two or more ungrouped voice clips in the carve flow now mints a group behind them (`mintGroupId`, de-duped against every id in the document) and writes `data-audio-group` on each picked clip atomically, one undo entry — `createAudioGroupAndAssignMembers` in `timelineTrackVisibility.ts` copies `setElementsHidden`'s multi-target write shape. The DSP is untouched: `mixCarveSources` already sums multiple sources correctly (verified in the design doc's own investigation) — this only fixes the picker. New lint rule `audio_carve_ungrouped_sources` (`packages/lint/src/rules/ media.ts`, alongside `audio_volume_double_automation`) warns when a `data-fx-carve`'s `sources` names two or more plain clip ids instead of a group — the shape that silently rots when a clip is added. `/hyperframes- audio` states the same rule as an invariant, not a tip, with the grouped- narration HTML example from the design doc. The group-matching and auto-group logic (`withAutoGroupedSources`, `collectCarveCandidates`) is split into `useFxCarveGrouping.ts` — `useFxCarve.ts` was pushing past the 600-line cap. `resolveNextCarveSettings` is deliberately NOT an `async function`: wrapping it in one would force a microtask on every call, including the synchronous branch — the exact bug `withAutoGroupedSources`'s own sync-when-possible contract exists to avoid, and one caught via `propertyPanelAudioFxGroup.test.tsx` (10 failures) before fixing it back to a plain function the caller conditionally awaits. Also extracted `useEffectiveTimelineDuration` out of `App.tsx` and `useRemoveBackground` out of `StudioRightPanel.tsx` (both pushed past 600 lines from an added prop wire), and decomposed `useFxCarve.ts`'s picker IIFE to clear fallow's complexity gate. Co-Authored-By: Claude Sonnet 5 <[email protected]> * feat(engine): render grouped audio through a summed, FX-processed bus Renders what B3 already routes in preview: a group's members sub-mix into one PCM WAV at full composition length (adelay already places each member at its composition position, so the group WAV's t=0 IS composition time), run through the group's own FX chain and automation via the same applyAudioFxChain/envelope-bake path a member uses, then fold into the flat track list as one processed AudioTrack — the final mixAudioTracks call never has to know groups exist. Gain law verified against plans/spikes/amix-nesting-spike.sh (brought over from the plans branch, along with audioMixer.grouping.test.ts, since both were committed there and never merged to origin/main — every step branch in this stack descends from origin/main): the sub-mix's own amix prefers normalize=0 (nulls exactly against a flat mix), falling back to per-node compensation by the group's OWN member count only when this ffmpeg build's amix rejects the option. Carrying any other count into a nested amix node is the exact +2.499 dB silent failure the spike measured — confirmed by a manual mutation check (wrong-count compensation landed 3.5 dB hot, exactly 20*log10(3/2) for a 2-member group compensated as 3; reverted after confirming the level test catches it). A group element carrying data-hidden drops every member before the sub-mix ever runs (RULES: mute-by-drop, never mute-by-volume-0) — parseAudioElements now resolves groups once per parse and skips hidden-group members the same way it already skips data-hidden ancestors. HfAudioGroup (packages/core/src/audioGroups.ts, from B1) gains fxChain, automation, volume and hidden, read off the group element the same way resolveAudioGroups already reads data-label — audioGroups.test.ts updated for the wider shape plus new coverage for the added reads. it.todo("mixes a grouped composition at the same level as the ungrouped one") is now a real, passing test; two more added per the step doc (FX routing isolation, member-level envelope survives grouping). Co-Authored-By: Claude Sonnet 5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5 <[email protected]>
1 parent 485c037 commit 99f42be

7 files changed

Lines changed: 712 additions & 10 deletions

File tree

‎packages/core/src/audioGroups.test.ts‎

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,15 +23,25 @@ describe("resolveAudioGroups", () => {
2323
<audio id="sfx-1"></audio>
2424
`;
2525
const groups = resolveAudioGroups(document);
26-
expect(groups).toEqual([{ id: "voiceover", label: "Voiceover", memberIds: ["vo-1", "vo-2"] }]);
26+
expect(groups).toEqual([
27+
{
28+
id: "voiceover",
29+
label: "Voiceover",
30+
memberIds: ["vo-1", "vo-2"],
31+
volume: 1,
32+
hidden: false,
33+
},
34+
]);
2735
});
2836

2937
it("resolves from member tags alone when the group element is absent, label = id", () => {
3038
document.body.innerHTML = `
3139
<audio id="vo-1" data-audio-group="narration"></audio>
3240
`;
3341
const groups = resolveAudioGroups(document);
34-
expect(groups).toEqual([{ id: "narration", label: "narration", memberIds: ["vo-1"] }]);
42+
expect(groups).toEqual([
43+
{ id: "narration", label: "narration", memberIds: ["vo-1"], volume: 1, hidden: false },
44+
]);
3545
});
3646

3747
it("ignores data-audio-group on the group element itself (groups do not nest)", () => {
@@ -40,7 +50,9 @@ describe("resolveAudioGroups", () => {
4050
<audio id="vo-1" data-audio-group="outer"></audio>
4151
`;
4252
const groups = resolveAudioGroups(document);
43-
expect(groups).toEqual([{ id: "outer", label: "outer", memberIds: ["vo-1"] }]);
53+
expect(groups).toEqual([
54+
{ id: "outer", label: "outer", memberIds: ["vo-1"], volume: 1, hidden: false },
55+
]);
4456
expect(audioGroupOf(document.getElementById("outer") as Element)).toBeNull();
4557
});
4658

@@ -59,6 +71,37 @@ describe("resolveAudioGroups", () => {
5971
document.body.innerHTML = `<video id="v-1" data-audio-group="voiceover"></video>`;
6072
expect(resolveAudioGroups(document)).toEqual([]);
6173
});
74+
75+
it("reads the group element's fx chain, automation, volume and hidden", () => {
76+
document.body.innerHTML = `
77+
<hf-audio-group id="voiceover" data-fx-chain='{"version":1,"nodes":[]}' data-automation='{"lanes":[]}' data-volume="0.5" data-hidden></hf-audio-group>
78+
<audio id="vo-1" data-audio-group="voiceover"></audio>
79+
`;
80+
const groups = resolveAudioGroups(document);
81+
expect(groups).toEqual([
82+
{
83+
id: "voiceover",
84+
label: "voiceover",
85+
memberIds: ["vo-1"],
86+
fxChain: '{"version":1,"nodes":[]}',
87+
automation: '{"lanes":[]}',
88+
volume: 0.5,
89+
hidden: true,
90+
},
91+
]);
92+
});
93+
94+
it("defaults volume to 1 and hidden to false when a group element exists but carries neither", () => {
95+
document.body.innerHTML = `
96+
<hf-audio-group id="voiceover"></hf-audio-group>
97+
<audio id="vo-1" data-audio-group="voiceover"></audio>
98+
`;
99+
const [group] = resolveAudioGroups(document);
100+
expect(group?.volume).toBe(1);
101+
expect(group?.hidden).toBe(false);
102+
expect(group?.fxChain).toBeUndefined();
103+
expect(group?.automation).toBeUndefined();
104+
});
62105
});
63106

64107
describe("audioGroupOf", () => {

‎packages/core/src/audioGroups.ts‎

Lines changed: 36 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@
99
* nothing here routes or sums audio yet.
1010
*/
1111

12+
import { HF_AUDIO_FX_ATTR } from "./audioFx.js";
13+
import { HF_AUDIO_AUTOMATION_ATTR } from "./audioAutomation.js";
14+
1215
export const HF_AUDIO_GROUP_TAG = "hf-audio-group";
1316
export const HF_AUDIO_GROUP_ATTR = "data-audio-group";
1417

@@ -32,6 +35,38 @@ export interface HfAudioGroup {
3235
label: string;
3336
/** Member element ids, in document order. */
3437
memberIds: string[];
38+
/** Serialised FX chain JSON from the group element's `data-fx-chain`, when set. */
39+
fxChain?: string;
40+
/** Serialised automation JSON from the group element's `data-automation`, when set. */
41+
automation?: string;
42+
/** The group element's `data-volume`, defaulting to 1 when absent or there is no group element. */
43+
volume: number;
44+
/**
45+
* The group element's `data-hidden`. Render drops every member rather than
46+
* zeroing them (RULES: mute-by-drop, never mute-by-volume-0) — B5 defines
47+
* the UI for this; this field just makes the read available now.
48+
*/
49+
hidden: boolean;
50+
}
51+
52+
function parseGroupVolume(el: Element | undefined): number {
53+
const raw = el?.getAttribute("data-volume");
54+
const parsed = raw ? parseFloat(raw) : 1;
55+
return Number.isFinite(parsed) ? parsed : 1;
56+
}
57+
58+
function buildGroup(id: string, memberIds: string[], el: Element | undefined): HfAudioGroup {
59+
const fxChain = el?.getAttribute(HF_AUDIO_FX_ATTR);
60+
const automation = el?.getAttribute(HF_AUDIO_AUTOMATION_ATTR);
61+
return {
62+
id,
63+
label: el?.getAttribute("data-label") || id,
64+
memberIds,
65+
...(fxChain ? { fxChain } : {}),
66+
...(automation ? { automation } : {}),
67+
volume: parseGroupVolume(el),
68+
hidden: el?.hasAttribute("data-hidden") ?? false,
69+
};
3570
}
3671

3772
/**
@@ -58,9 +93,7 @@ export function resolveAudioGroups(root: ParentNode): HfAudioGroup[] {
5893

5994
const groups: HfAudioGroup[] = [];
6095
for (const [id, memberIds] of membersByGroup) {
61-
const el = groupElements.get(id);
62-
const label = el?.getAttribute("data-label") || id;
63-
groups.push({ id, label, memberIds });
96+
groups.push(buildGroup(id, memberIds, groupElements.get(id)));
6497
}
6598
return groups;
6699
}

0 commit comments

Comments
 (0)