Skip to content

Commit 5fd84c3

Browse files
vanceingallsclaude
andauthored
feat(studio,core): a volume and a living meter on the group row (#3290)
* feat(studio,core): a volume and a living meter on the group row B7: the group bus strip — droppable, and deliberately minimal per the casual-user design constraints (groups doc §5): a volume slider, a level bar that moves with the sound, and the words "Too loud" when it clips. No dB numbers, no peak-hold readout, no routing row. Transport (core): groupInput() now routes each group through input -> [FX chain or dry passthrough] -> output -> master, with one AnalyserNode per group tapped off `output` (post-FX, so the meter reads what the bus actually outputs) — fftSize 256, level not spectrum. groupLevel(groupId) returns RMS-ish level 0..1 + a clipped flag off a reused per-group buffer (no per-frame allocation), or null when the group is idle/unknown. The runtime posts group-levels messages only while playing, piggybacking the existing message channel rather than adding a new poll loop. Studio: groupLevels.ts is a plain pub-sub store (mirrors liveTime.ts's shape) fed by useTimelinePlayer's message handler via parseGroupLevelsMessage; useGroupLevel throttles re-renders to ~33ms. TimelineGroupBusStrip renders in the group row's own `∿` lane area (STRIP_H, already sized in B2's row-height pipeline) — drag writes live via onSetAudioGroupAttributeLive, release commits one undo entry via onSetAudioGroupAttributeQuiet (packages/studio/src/hooks/ timelineAudioGroupVolume.ts, extracted from timelineTrackVisibility.ts to stay under the 600-line cap; mirrors FxParamRow's live/commit split). "Too loud" holds for ~2s after the last clipped block, tracked in the component, not the transport. volumeByGroup mirrors labelByGroup in useTimelineTrackDerivations.ts so the strip's slider round-trips the group's own data-volume. Fixed two pre-existing group-routing tests in webAudioTransport.test.ts that hardcoded gain-node creation order/count — B7 inserts an extra `output` gain node between the group's input and master (for the meter to tap), which shifted node indices the tests asserted on directly. Co-Authored-By: Claude Sonnet 5 <[email protected]> * fix(studio,core): keep useTimelinePlayer under the size cap and the level buffer non-shared Two CI gates, both from this branch's own additions. `File size check`: `useTimelinePlayer.ts` sat at 599 lines on main and the group-levels branch pushed it to 605 (cap 600). Extracted the `window.message` router — which already carried a `fallow-ignore-next-line complexity` admitting it had outgrown its home — into `previewMessageRouter.ts`, with the fixture lease, sender check and protocol accept-gate collapsed into one `acceptedPreviewMessage` so the listener is a flat dispatch and the suppression is retired rather than moved. Same branches, same refs, no behaviour change; the file lands at 561. `Test: runtime contract`: `levelBuf: Float32Array` resolves to `Float32Array<ArrayBufferLike>` under `tsconfig.runtime.json`, and `getFloatTimeDomainData` will not take a possibly-shared buffer (TS2345). Pinned the field to `Float32Array<ArrayBuffer>`, which is what `new Float32Array(analyser.fftSize)` already produces. Also drops `EditorShell.selectionSync.test.tsx`'s `vi.mock("./StudioFeedbackBar")` — main deleted that component in favour of `feedback/StudioFeedbackCard`, and touching this file for the group prop put the dangling path in fallow's scope. --------- Co-authored-by: Claude Sonnet 5 <[email protected]>
1 parent 99f42be commit 5fd84c3

26 files changed

Lines changed: 977 additions & 93 deletions

‎packages/core/src/runtime/init.ts‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2145,6 +2145,25 @@ export function initSandboxRuntimeModular(): void {
21452145
scheduleRootStageLayoutDiagnostics();
21462146
};
21472147

2148+
/** One meter reading per group with an active member — polled from the
2149+
* transport's analyser, not the DOM, so an idle group (never played, no
2150+
* matching `<hf-audio-group>`) is simply absent rather than reported as
2151+
* zero. Cheap when nothing is grouped: `groupIds()` is empty. */
2152+
const postGroupLevels = () => {
2153+
const groupIds = webAudio.groupIds();
2154+
if (groupIds.length === 0) return;
2155+
const levels = groupIds
2156+
.map((groupId) => {
2157+
const reading = webAudio.groupLevel(groupId);
2158+
return reading ? { groupId, ...reading } : null;
2159+
})
2160+
.filter(
2161+
(entry): entry is { groupId: string; level: number; clipped: boolean } => entry !== null,
2162+
);
2163+
if (levels.length === 0) return;
2164+
postRuntimeMessage({ source: "hf-preview", type: "group-levels", levels });
2165+
};
2166+
21482167
const finitePositiveDuration = (value: number | null | undefined): number =>
21492168
typeof value === "number" && Number.isFinite(value) && value > 0 ? value : 0;
21502169

@@ -2902,6 +2921,9 @@ export function initSandboxRuntimeModular(): void {
29022921
if (transportTickCount % 30 === 0) {
29032922
bindMediaMetadataListeners();
29042923
}
2924+
if (clock.isPlaying()) {
2925+
postGroupLevels();
2926+
}
29052927

29062928
// Sync clock duration with the resolved timeline each tick (catches async
29072929
// rebinds, live data-duration edits). Never shrink while playing — transient

‎packages/core/src/runtime/types.ts‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,15 @@ export type RuntimePerformanceMessage = {
197197
tags: Record<string, string | number | boolean | null>;
198198
};
199199

200+
/** One audio group's live meter reading, polled from the transport each tick
201+
* while playing. A group id absent from `levels` is idle/unknown (no active
202+
* member) — the studio side treats that as "no reading", not zero. */
203+
export type RuntimeGroupLevelsMessage = {
204+
source: "hf-preview";
205+
type: "group-levels";
206+
levels: Array<{ groupId: string; level: number; clipped: boolean }>;
207+
};
208+
200209
export type RuntimeOutboundMessage =
201210
| RuntimeStateMessage
202211
| RuntimeTimelineMessage
@@ -210,7 +219,8 @@ export type RuntimeOutboundMessage =
210219
| RuntimeMediaAutoplayBlockedMessage
211220
| RuntimeReadyMessage
212221
| RuntimeAnalyticsMessage
213-
| RuntimePerformanceMessage;
222+
| RuntimePerformanceMessage
223+
| RuntimeGroupLevelsMessage;
214224

215225
export type RuntimePlayer = {
216226
_timeline: RuntimeTimelineLike | null;

‎packages/core/src/runtime/webAudioTransport.test.ts‎

Lines changed: 92 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -665,6 +665,12 @@ describe("WebAudioTransport", () => {
665665
connect: ReturnType<typeof vi.fn>;
666666
disconnect: ReturnType<typeof vi.fn>;
667667
}[] = [];
668+
const analysers: {
669+
fftSize: number;
670+
connect: ReturnType<typeof vi.fn>;
671+
disconnect: ReturnType<typeof vi.fn>;
672+
getFloatTimeDomainData: ReturnType<typeof vi.fn>;
673+
}[] = [];
668674
const masterGain = { gain: { value: 1 }, connect: vi.fn(), disconnect: vi.fn() };
669675
const ctx = {
670676
currentTime,
@@ -684,10 +690,20 @@ describe("WebAudioTransport", () => {
684690
gainNodes.push(node);
685691
return node;
686692
}),
693+
createAnalyser: vi.fn(() => {
694+
const node = {
695+
fftSize: 2048,
696+
connect: vi.fn(),
697+
disconnect: vi.fn(),
698+
getFloatTimeDomainData: vi.fn(),
699+
};
700+
analysers.push(node);
701+
return node;
702+
}),
687703
destination: {},
688704
close: vi.fn(),
689705
};
690-
return { ctx, gainNodes, masterGain };
706+
return { ctx, gainNodes, analysers, masterGain };
691707
}
692708

693709
function setupGroupTransport(currentTime = 100) {
@@ -745,22 +761,27 @@ describe("WebAudioTransport", () => {
745761
await scheduleGrouped(transport, gen, "a", "vo");
746762
await scheduleGrouped(transport, gen, "b", "vo");
747763

748-
// Member gain nodes: index 0 (a) and index 2 (b) — index 1 is the
749-
// group's own input gain, built inside a's schedule call.
750-
expect(mock.gainNodes.length).toBeGreaterThanOrEqual(3);
764+
// Member gain nodes: index 0 (a) and index 3 (b) — index 1/2 are the
765+
// group's own input/output gain pair (B7's meter taps `output`),
766+
// built inside a's schedule call.
767+
expect(mock.gainNodes.length).toBeGreaterThanOrEqual(4);
751768
const groupInput = firstGroupInput(mock);
769+
const groupOutput = mock.gainNodes[2]!;
752770
const aGain = mock.gainNodes[0]!;
753-
const bGain = mock.gainNodes[2]!;
771+
const bGain = mock.gainNodes[3]!;
754772

755773
// Neither member connects straight to master — both feed the shared bus.
756774
expect(aGain.connect).toHaveBeenCalledWith(groupInput);
757775
expect(bGain.connect).toHaveBeenCalledWith(groupInput);
758776
expect(aGain.connect).not.toHaveBeenCalledWith(mock.masterGain);
759777
expect(bGain.connect).not.toHaveBeenCalledWith(mock.masterGain);
760778

761-
// The bus itself is what reaches master — a plain sum, no processing,
762-
// since neither member's group has a chain-bearing `<hf-audio-group>`.
763-
expect(groupInput.connect).toHaveBeenCalledWith(mock.masterGain);
779+
// The bus's input never reaches master directly — it lands on the
780+
// output gain (the dry passthrough, since neither member's group has a
781+
// chain-bearing `<hf-audio-group>`), and THAT reaches master.
782+
expect(groupInput.connect).not.toHaveBeenCalledWith(mock.masterGain);
783+
expect(groupInput.connect).toHaveBeenCalledWith(groupOutput);
784+
expect(groupOutput.connect).toHaveBeenCalledWith(mock.masterGain);
764785
});
765786

766787
it("a second member of an already-open group does not rebuild the group bus", async () => {
@@ -779,7 +800,9 @@ describe("WebAudioTransport", () => {
779800

780801
await scheduleGrouped(transport, gen, "a", "orphan-group"); // no matching element
781802

782-
expect(firstGroupInput(mock).connect).toHaveBeenCalledWith(mock.masterGain);
803+
const groupOutput = mock.gainNodes[2]!;
804+
expect(firstGroupInput(mock).connect).toHaveBeenCalledWith(groupOutput);
805+
expect(groupOutput.connect).toHaveBeenCalledWith(mock.masterGain);
783806
});
784807

785808
it("group volume rides the group's own data-volume via its automation lane, not the member's", async () => {
@@ -814,6 +837,66 @@ describe("WebAudioTransport", () => {
814837
// Still only one group-input gain ever created for "vo".
815838
expect(mock.gainNodes.filter((n) => n === groupInput)).toHaveLength(1);
816839
});
840+
841+
describe("groupLevel meter (B7)", () => {
842+
it("groupLevel returns null for an unknown/idle group id", () => {
843+
const { transport } = setupGroupTransport();
844+
expect(transport.groupLevel("never-played")).toBeNull();
845+
});
846+
847+
it("creates exactly one analyser per group, lazily, on first member", async () => {
848+
const { transport, mock, gen } = setupGroupTransport();
849+
expect(mock.analysers).toHaveLength(0);
850+
851+
await scheduleGrouped(transport, gen, "a", "vo");
852+
expect(mock.analysers).toHaveLength(1);
853+
expect(mock.analysers[0]!.fftSize).toBe(256); // level, not spectrum
854+
855+
await scheduleGrouped(transport, gen, "b", "vo");
856+
expect(mock.analysers).toHaveLength(1); // second member reuses the bus
857+
858+
expect(transport.groupIds()).toEqual(["vo"]);
859+
});
860+
861+
it("groupLevel reads RMS off the group's own analyser once a member is scheduled", async () => {
862+
const { transport, mock, gen } = setupGroupTransport();
863+
await scheduleGrouped(transport, gen, "a", "vo");
864+
865+
const analyser = mock.analysers[0]!;
866+
analyser.getFloatTimeDomainData.mockImplementation((buf: Float32Array) => {
867+
buf.fill(0.5);
868+
});
869+
870+
const reading = transport.groupLevel("vo");
871+
expect(reading).not.toBeNull();
872+
expect(reading!.level).toBeCloseTo(0.5, 5);
873+
expect(reading!.clipped).toBe(false);
874+
});
875+
876+
it("flags clipped when any sample hits the ceiling", async () => {
877+
const { transport, mock, gen } = setupGroupTransport();
878+
await scheduleGrouped(transport, gen, "a", "vo");
879+
880+
const analyser = mock.analysers[0]!;
881+
analyser.getFloatTimeDomainData.mockImplementation((buf: Float32Array) => {
882+
buf.fill(0.1);
883+
buf[0] = 0.995;
884+
});
885+
886+
expect(transport.groupLevel("vo")!.clipped).toBe(true);
887+
});
888+
889+
it("disposes the analyser along with the rest of the group bus", async () => {
890+
const { transport, mock, gen } = setupGroupTransport();
891+
await scheduleGrouped(transport, gen, "a", "vo");
892+
const analyser = mock.analysers[0]!;
893+
894+
transport.destroy();
895+
896+
expect(analyser.disconnect).toHaveBeenCalled();
897+
expect(transport.groupLevel("vo")).toBeNull();
898+
});
899+
});
817900
});
818901

819902
describe("decodeAudioElement retry policy (late-asset self-heal)", () => {

‎packages/core/src/runtime/webAudioTransport.ts‎

Lines changed: 54 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,18 @@ export class WebAudioTransport {
127127
// group is scheduled. Lives for the session (mirrors `_masterGain`'s own
128128
// lifecycle) rather than being torn down on every `stopAll()`, so replaying
129129
// a group does not rebuild its chain; only `destroy()` disposes these.
130-
private _groups = new Map<string, { input: GainNode; dispose(): void }>();
130+
private _groups = new Map<
131+
string,
132+
{
133+
input: GainNode;
134+
analyser: AnalyserNode;
135+
// `Float32Array<ArrayBuffer>`, not bare `Float32Array`: the runtime
136+
// typecheck resolves the latter to `Float32Array<ArrayBufferLike>`, which
137+
// `getFloatTimeDomainData` rejects (it wants a non-shared buffer).
138+
levelBuf: Float32Array<ArrayBuffer>;
139+
dispose(): void;
140+
}
141+
>();
131142
// Composition-time reference frame: at AudioContext time `_rateAnchorCtx`,
132143
// composition time was `_rateAnchorComp`, and time has been advancing at
133144
// `_rate` composition-seconds per wallclock-second since.
@@ -294,22 +305,39 @@ export class WebAudioTransport {
294305
if (!this._ctx || !this._masterGain) return null;
295306

296307
const input = this._ctx.createGain();
308+
// Stable point the FX chain (or, when there's none, the dry passthrough —
309+
// see `attachElementFxChain`'s `detach()`) always lands on before master,
310+
// regardless of whether a chain is attached/detached/rebuilt later. B7's
311+
// meter taps here. B5's group mute gain MUST splice in before `output`
312+
// (between the FX chain and here), never after — the meter is defined to
313+
// read the group's true, honestly-muted level (design doc §5), and this
314+
// node is that contract's anchor.
315+
const output = this._ctx.createGain();
316+
output.connect(this._masterGain);
317+
const analyser = this._ctx.createAnalyser();
318+
analyser.fftSize = 256; // level, not spectrum
319+
output.connect(analyser);
320+
297321
const groupEl = doc.getElementById(groupId);
298322
const fx = attachElementFxChain(
299323
this._ctx,
300324
groupEl ?? { getAttribute: () => null },
301325
input,
302-
this._masterGain,
326+
output,
303327
timing,
304328
);
305329
if (groupEl) scheduleVolumeLane(groupEl, input, timing);
306330

307331
this._groups.set(groupId, {
308332
input,
333+
analyser,
334+
levelBuf: new Float32Array(analyser.fftSize),
309335
dispose: () => {
310336
try {
311337
fx?.dispose();
312338
input.disconnect();
339+
output.disconnect();
340+
analyser.disconnect();
313341
} catch {
314342
// Already torn down.
315343
}
@@ -318,6 +346,30 @@ export class WebAudioTransport {
318346
return input;
319347
}
320348

349+
/** Every group id currently routing audio (built lazily by `groupInput` —
350+
* a group with no active member yet has no entry here). */
351+
groupIds(): string[] {
352+
return [...this._groups.keys()];
353+
}
354+
355+
/**
356+
* RMS-ish level 0..1 and whether the last block clipped, for the group's
357+
* meter — or null when the group has no active member (idle/unknown).
358+
* Reuses a per-group buffer; no per-frame allocation.
359+
*/
360+
groupLevel(groupId: string): { level: number; clipped: boolean } | null {
361+
const g = this._groups.get(groupId);
362+
if (!g) return null;
363+
g.analyser.getFloatTimeDomainData(g.levelBuf);
364+
let sumSquares = 0;
365+
let clipped = false;
366+
for (const sample of g.levelBuf) {
367+
sumSquares += sample * sample;
368+
if (Math.abs(sample) >= 0.99) clipped = true;
369+
}
370+
return { level: Math.sqrt(sumSquares / g.levelBuf.length), clipped };
371+
}
372+
321373
/** Master, unless `el` belongs to a group — then that group's bus (built on
322374
* first use, per `groupInput`). */
323375
private resolveDestination(

‎packages/studio/src/App.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -548,6 +548,7 @@ export function StudioApp() {
548548
handleTimelineElementResize={timelineEditing.handleTimelineElementResize}
549549
handleTimelineGroupResize={timelineEditing.handleTimelineGroupResize}
550550
handleToggleTrackHidden={timelineEditing.handleToggleTrackHidden}
551+
setAudioGroupAttribute={timelineEditing.setAudioGroupAttribute}
551552
handleBlockedTimelineEdit={timelineEditing.handleBlockedTimelineEdit}
552553
handleTimelineElementSplit={timelineEditing.handleTimelineElementSplit}
553554
handleRazorSplit={timelineEditing.handleRazorSplit}

‎packages/studio/src/components/EditorShell.selectionSync.test.tsx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,6 @@ vi.mock("./nle/PreviewPane", () => ({ PreviewPane: () => null }));
5252
vi.mock("./nle/PreviewOverlays", () => ({ PreviewOverlays: () => null }));
5353
vi.mock("./nle/TimelinePane", () => ({ TimelinePane: () => null }));
5454
vi.mock("../captions/components/CaptionTimeline", () => ({ CaptionTimeline: () => null }));
55-
vi.mock("./StudioFeedbackBar", () => ({ StudioFeedbackBar: () => null }));
5655

5756
Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true });
5857

@@ -82,6 +81,7 @@ describe("EditorShell timeline selection sync", () => {
8281
handleTimelineElementResize={vi.fn()}
8382
handleTimelineGroupResize={vi.fn()}
8483
handleToggleTrackHidden={vi.fn()}
84+
setAudioGroupAttribute={{ setLive: vi.fn(), setQuiet: vi.fn() }}
8585
handleBlockedTimelineEdit={vi.fn()}
8686
handleTimelineElementSplit={vi.fn()}
8787
handleRazorSplit={vi.fn()}

‎packages/studio/src/components/EditorShell.tsx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ export function EditorShell({
8585
handleTimelineElementResize,
8686
handleTimelineGroupResize,
8787
handleToggleTrackHidden,
88+
setAudioGroupAttribute,
8889
handleBlockedTimelineEdit,
8990
handleTimelineElementSplit,
9091
handleRazorSplit,
@@ -135,6 +136,7 @@ export function EditorShell({
135136
handleTimelineElementResize,
136137
handleTimelineGroupResize,
137138
handleToggleTrackHidden,
139+
setAudioGroupAttribute,
138140
handleBlockedTimelineEdit,
139141
handleTimelineElementSplit,
140142
handleRazorSplit,

‎packages/studio/src/components/nle/useTimelineEditCallbacks.test.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,7 @@ function renderCallbacks(): { callbacks: TimelineEditCallbacks; unmount: () => v
104104
handleTimelineElementResize: vi.fn(),
105105
handleTimelineGroupResize: vi.fn(),
106106
handleToggleTrackHidden: vi.fn(),
107+
setAudioGroupAttribute: { setLive: vi.fn(), setQuiet: vi.fn() },
107108
handleBlockedTimelineEdit: vi.fn(),
108109
handleTimelineElementSplit: vi.fn(),
109110
handleRazorSplit: vi.fn(),

‎packages/studio/src/components/nle/useTimelineEditCallbacks.ts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,10 @@ export interface TimelineEditCallbackDeps {
3838
) => Promise<void> | void;
3939
handleTimelineGroupResize: NonNullable<TimelineEditCallbacks["onResizeElements"]>;
4040
handleToggleTrackHidden: (track: number, hidden: boolean) => Promise<void> | void;
41+
setAudioGroupAttribute: {
42+
setLive: (groupId: string, attr: string, value: string | null) => void;
43+
setQuiet: (groupId: string, attr: string, value: string | null, label: string) => Promise<void>;
44+
};
4145
handleBlockedTimelineEdit: (element: TimelineElement, intent: BlockedTimelineEditIntent) => void;
4246
handleTimelineElementSplit: (element: TimelineElement, splitTime: number) => Promise<void> | void;
4347
handleRazorSplit: (element: TimelineElement, splitTime: number) => Promise<void> | void;
@@ -99,6 +103,7 @@ export function useTimelineEditCallbacks({
99103
handleTimelineElementResize,
100104
handleTimelineGroupResize,
101105
handleToggleTrackHidden,
106+
setAudioGroupAttribute,
102107
handleBlockedTimelineEdit,
103108
handleTimelineElementSplit,
104109
handleRazorSplit,
@@ -185,6 +190,8 @@ export function useTimelineEditCallbacks({
185190
onResizeElement: handleTimelineElementResize,
186191
onResizeElements: handleTimelineGroupResize,
187192
onToggleTrackHidden: handleToggleTrackHidden,
193+
onSetAudioGroupAttributeLive: setAudioGroupAttribute.setLive,
194+
onSetAudioGroupAttributeQuiet: setAudioGroupAttribute.setQuiet,
188195
onBlockedEditAttempt: handleBlockedTimelineEdit,
189196
onSplitElement: handleTimelineElementSplit,
190197
onRazorSplit: handleRazorSplit,
@@ -378,6 +385,7 @@ export function useTimelineEditCallbacks({
378385
handleTimelineElementResize,
379386
handleTimelineGroupResize,
380387
handleToggleTrackHidden,
388+
setAudioGroupAttribute,
381389
handleBlockedTimelineEdit,
382390
handleTimelineElementSplit,
383391
handleRazorSplit,

‎packages/studio/src/contexts/TimelineEditContext.tsx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ export function TimelineEditProvider({
3434
value.onMoveElements,
3535
value.onResizeElement,
3636
value.onToggleTrackHidden,
37+
value.onSetAudioGroupAttributeLive,
38+
value.onSetAudioGroupAttributeQuiet,
3739
value.onBlockedEditAttempt,
3840
value.onSplitElement,
3941
value.onRazorSplit,

0 commit comments

Comments
 (0)