Repository navigation
Commit 99f42be
feat(engine): render grouped audio through a summed, FX-processed bus (#3289)
* feat(core): route grouped audio through a group bus in preview
An audio element carrying `data-audio-group` no longer lands its gain on
the master bus directly — it feeds a per-group `GainNode` (built lazily on
first use, one per group id) which itself feeds master, so members of the
same group sum before the ear, ready for a group-level FX chain and
volume/mute in later steps. An id with no matching `<hf-audio-group>`
element still gets a plain, unprocessed bus rather than losing the track.
The group's own chain and volume lane are wired through the same
`attachElementFxChain`/`scheduleVolumeLane` every element already uses,
against the group's clock — composition time (design doc §1.3), since a
group has no `data-start` and a missing one parses as 0. The bus persists
across `stopAll()` (mirroring `_masterGain`'s own lifecycle) so replaying a
group does not rebuild its chain; only `destroy()` disposes it.
Render is untouched — stays flat until B4; `audio-groups` is still a 0%
canary so nothing ships this to a real composition without hand-authoring
`data-audio-group`.
Also: `audioGroupOf` (B1) crashed on any element lacking a real `tagName`/
`getAttribute` — exactly the shape of most `HTMLMediaElement` test doubles
in this suite, including this file's own `mockEl`. Made it tolerant, same
style as `readChain`'s existing guard in `runtime/audioFx.ts`.
`schedulePlayback` was already 110 lines pre-existing before this diff;
extracted `resolveDestination` and `handleSourceEnded` to shrink it to 92,
then suppressed the remainder (inherently sequential graph wiring, not a
decision tree) per the same precedent B2 used on `TimelineLogicalRow`.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
* feat(studio,lint): carve targets voiceover groups — always, when plural
Plural voiceover carve now targets a group instead of naming each clip:
`resolveCarveSourceIds` (core `audioGroups.ts`) expands a group id to its
current members at analysis time, so a clip added to the group later is
covered without touching `sources`. The picker (`useFxCarve.ts`) offers a
grouped voice as one option instead of one row per member, tests overlap
as a union of member spans (a group overlaps the bed if ANY member does),
and prefers a qualifying group over its individual members in
`autoSourceIds`.
Picking two or more ungrouped voice clips in the carve flow now mints a
group behind them (`mintGroupId`, de-duped against every id in the
document) and writes `data-audio-group` on each picked clip atomically,
one undo entry — `createAudioGroupAndAssignMembers` in
`timelineTrackVisibility.ts` copies `setElementsHidden`'s multi-target
write shape. The DSP is untouched: `mixCarveSources` already sums
multiple sources correctly (verified in the design doc's own
investigation) — this only fixes the picker.
New lint rule `audio_carve_ungrouped_sources` (`packages/lint/src/rules/
media.ts`, alongside `audio_volume_double_automation`) warns when a
`data-fx-carve`'s `sources` names two or more plain clip ids instead of a
group — the shape that silently rots when a clip is added. `/hyperframes-
audio` states the same rule as an invariant, not a tip, with the grouped-
narration HTML example from the design doc.
The group-matching and auto-group logic (`withAutoGroupedSources`,
`collectCarveCandidates`) is split into `useFxCarveGrouping.ts` —
`useFxCarve.ts` was pushing past the 600-line cap. `resolveNextCarveSettings`
is deliberately NOT an `async function`: wrapping it in one would force a
microtask on every call, including the synchronous branch — the exact bug
`withAutoGroupedSources`'s own sync-when-possible contract exists to avoid,
and one caught via `propertyPanelAudioFxGroup.test.tsx` (10 failures)
before fixing it back to a plain function the caller conditionally awaits.
Also extracted `useEffectiveTimelineDuration` out of `App.tsx` and
`useRemoveBackground` out of `StudioRightPanel.tsx` (both pushed past 600
lines from an added prop wire), and decomposed `useFxCarve.ts`'s picker
IIFE to clear fallow's complexity gate.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
* feat(engine): render grouped audio through a summed, FX-processed bus
Renders what B3 already routes in preview: a group's members sub-mix into
one PCM WAV at full composition length (adelay already places each member
at its composition position, so the group WAV's t=0 IS composition time),
run through the group's own FX chain and automation via the same
applyAudioFxChain/envelope-bake path a member uses, then fold into the flat
track list as one processed AudioTrack — the final mixAudioTracks call
never has to know groups exist.
Gain law verified against plans/spikes/amix-nesting-spike.sh (brought over
from the plans branch, along with audioMixer.grouping.test.ts, since both
were committed there and never merged to origin/main — every step branch in
this stack descends from origin/main): the sub-mix's own amix prefers
normalize=0 (nulls exactly against a flat mix), falling back to per-node
compensation by the group's OWN member count only when this ffmpeg build's
amix rejects the option. Carrying any other count into a nested amix node
is the exact +2.499 dB silent failure the spike measured — confirmed by a
manual mutation check (wrong-count compensation landed 3.5 dB hot, exactly
20*log10(3/2) for a 2-member group compensated as 3; reverted after
confirming the level test catches it).
A group element carrying data-hidden drops every member before the sub-mix
ever runs (RULES: mute-by-drop, never mute-by-volume-0) — parseAudioElements
now resolves groups once per parse and skips hidden-group members the same
way it already skips data-hidden ancestors.
HfAudioGroup (packages/core/src/audioGroups.ts, from B1) gains fxChain,
automation, volume and hidden, read off the group element the same way
resolveAudioGroups already reads data-label — audioGroups.test.ts updated
for the wider shape plus new coverage for the added reads.
it.todo("mixes a grouped composition at the same level as the ungrouped
one") is now a real, passing test; two more added per the step doc (FX
routing isolation, member-level envelope survives grouping).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
---------
Co-authored-by: Claude Sonnet 5 <[email protected]>1 parent 485c037 commit 99f42be
7 files changed
Lines changed: 712 additions & 10 deletions
File tree
- packages
- core/src
- engine/src/services
- plans/spikes
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
27 | 35 | | |
28 | 36 | | |
29 | 37 | | |
30 | 38 | | |
31 | 39 | | |
32 | 40 | | |
33 | 41 | | |
34 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
35 | 45 | | |
36 | 46 | | |
37 | 47 | | |
| |||
40 | 50 | | |
41 | 51 | | |
42 | 52 | | |
43 | | - | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
44 | 56 | | |
45 | 57 | | |
46 | 58 | | |
| |||
59 | 71 | | |
60 | 72 | | |
61 | 73 | | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
62 | 105 | | |
63 | 106 | | |
64 | 107 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
12 | 15 | | |
13 | 16 | | |
14 | 17 | | |
| |||
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
35 | 70 | | |
36 | 71 | | |
37 | 72 | | |
| |||
58 | 93 | | |
59 | 94 | | |
60 | 95 | | |
61 | | - | |
62 | | - | |
63 | | - | |
| 96 | + | |
64 | 97 | | |
65 | 98 | | |
66 | 99 | | |
| |||
0 commit comments