Skip to content

Commit 5240367

Browse files
vanceingallsclaude
andauthored
feat(core): the audio group model — element, membership, helpers (#3278)
* feat(core,studio): the character presets pitch shift unlocks Chipmunk, Giant, and Monster ship as presets on the pitchshift worklet P1 added: Chipmunk pitches up and adds sparkle, Giant pitches down with weight and a compressor to hold the extra low end together, Monster pitches down further with saturation growl and a close, tight reverb. Every param verified against the live effect registry rather than sketched — the compressor/reverb/saturate/shelf keys all match exactly. Each gets its own title treatment (font, size, tracking, hue) so the FX rack's per-preset styling coverage and hue-distance/background-uniqueness tests extend cleanly to the three new entries, and complaint-line copy in the non-voice vocabulary the audit test enforces (no speech words — "Giant" over CapCut's "Deep Voice", as the design doc records). Updates plans/audio-fx-presets.md's two limits paragraphs to record that pitch shift landed and this half of the character list now ships; Robot and Alien stay out of scope (ring modulation, still unbuilt). Co-Authored-By: Claude Sonnet 5 <[email protected]> * feat(core): the audio group model — element, membership, helpers Introduces <hf-audio-group> and data-audio-group as the group model B2–B7 and C1 build on: a non-rendering group element carries a label and (later) an FX chain, membership lives on the member's own data-audio-group attribute rather than DOM nesting, so a track removed from the document simply drops out of the group on the next resolve — nothing dangles. Groups do not nest: data-audio-group on the group element itself is ignored. A group with members but no <hf-audio-group> element still resolves, label falling back to the id, so hand-authored HTML degrades gracefully. Audio only in v1 — video members are ignored. Parse-only: nothing routes or sums audio yet (B3/B4). Adds the audio-groups canary at percentage: 0 gating the future Studio UI; the element and attribute parse and play regardless of enrollment. Verified rather than assumed per this plan's standing rule: the timeline's clip-collection selector ([data-start], [data-track-index], [data-composition-id], video, audio, img) already excludes the group element with zero changes, and no lint rule flags unknown elements or data-* attributes, so neither needed touching — confirmed by grep and by running `hyperframes lint` against a fixture containing the element (0 findings referencing it). The step doc's suggested display:none injection point (an existing base stylesheet in the runtime) does not exist in this codebase; skipped rather than inventing new infrastructure, since an empty, childless custom element already renders as a zero-size inline box with no visible output — the same reasoning the lint check above confirms empirically. Co-Authored-By: Claude Sonnet 5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5 <[email protected]>
1 parent 5e0cc75 commit 5240367

7 files changed

Lines changed: 297 additions & 0 deletions

File tree

‎packages/core/package-subpaths.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,12 @@
152152
"types": "./dist/audioCarve.d.ts",
153153
"environments": ["browser", "bun", "node"]
154154
},
155+
"./audio-groups": {
156+
"source": "./src/audioGroups.ts",
157+
"runtime": "./dist/audioGroups.js",
158+
"types": "./dist/audioGroups.d.ts",
159+
"environments": ["browser", "bun", "node"]
160+
},
155161
"./audio-automation": {
156162
"source": "./src/audioAutomation.ts",
157163
"runtime": "./dist/audioAutomation.js",

‎packages/core/package.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -166,6 +166,12 @@
166166
"import": "./src/audioCarve.ts",
167167
"types": "./src/audioCarve.ts"
168168
},
169+
"./audio-groups": {
170+
"bun": "./src/audioGroups.ts",
171+
"node": "./dist/audioGroups.js",
172+
"import": "./src/audioGroups.ts",
173+
"types": "./src/audioGroups.ts"
174+
},
169175
"./audio-automation": {
170176
"bun": "./src/audioAutomation.ts",
171177
"node": "./dist/audioAutomation.js",
@@ -480,6 +486,10 @@
480486
"import": "./dist/audioCarve.js",
481487
"types": "./dist/audioCarve.d.ts"
482488
},
489+
"./audio-groups": {
490+
"import": "./dist/audioGroups.js",
491+
"types": "./dist/audioGroups.d.ts"
492+
},
483493
"./audio-automation": {
484494
"import": "./dist/audioAutomation.js",
485495
"types": "./dist/audioAutomation.d.ts"
Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
import { beforeEach, describe, expect, it } from "vitest";
2+
import {
3+
audioGroupOf,
4+
ensureAudioGroupInertStyle,
5+
HF_AUDIO_GROUP_ATTR,
6+
resolveAudioGroups,
7+
} from "./audioGroups.js";
8+
9+
beforeEach(() => {
10+
document.body.innerHTML = "";
11+
// The inert stylesheet is injected once per document, so a leftover from an
12+
// earlier test would carry the assertion for the one after it.
13+
document.getElementById("__hf-audio-group-inert")?.remove();
14+
});
15+
16+
describe("resolveAudioGroups", () => {
17+
it("returns one group of two members plus ignores an ungrouped track", () => {
18+
document.body.innerHTML = `
19+
<hf-audio-group id="voiceover" data-label="Voiceover"></hf-audio-group>
20+
<audio id="vo-1" data-audio-group="voiceover"></audio>
21+
<audio id="vo-2" data-audio-group="voiceover"></audio>
22+
<audio id="sfx-1"></audio>
23+
`;
24+
const groups = resolveAudioGroups(document);
25+
expect(groups).toEqual([{ id: "voiceover", label: "Voiceover", memberIds: ["vo-1", "vo-2"] }]);
26+
});
27+
28+
it("resolves from member tags alone when the group element is absent, label = id", () => {
29+
document.body.innerHTML = `
30+
<audio id="vo-1" data-audio-group="narration"></audio>
31+
`;
32+
const groups = resolveAudioGroups(document);
33+
expect(groups).toEqual([{ id: "narration", label: "narration", memberIds: ["vo-1"] }]);
34+
});
35+
36+
it("ignores data-audio-group on the group element itself (groups do not nest)", () => {
37+
document.body.innerHTML = `
38+
<hf-audio-group id="outer" data-audio-group="outer"></hf-audio-group>
39+
<audio id="vo-1" data-audio-group="outer"></audio>
40+
`;
41+
const groups = resolveAudioGroups(document);
42+
expect(groups).toEqual([{ id: "outer", label: "outer", memberIds: ["vo-1"] }]);
43+
expect(audioGroupOf(document.getElementById("outer") as Element)).toBeNull();
44+
});
45+
46+
it("drops a member removed from the DOM on re-resolve — nothing dangles", () => {
47+
document.body.innerHTML = `
48+
<audio id="vo-1" data-audio-group="voiceover"></audio>
49+
<audio id="vo-2" data-audio-group="voiceover"></audio>
50+
`;
51+
expect(resolveAudioGroups(document)[0].memberIds).toEqual(["vo-1", "vo-2"]);
52+
53+
document.getElementById("vo-2")?.remove();
54+
expect(resolveAudioGroups(document)[0].memberIds).toEqual(["vo-1"]);
55+
});
56+
57+
it("ignores a data-audio-group on a video element (audio only in v1)", () => {
58+
document.body.innerHTML = `<video id="v-1" data-audio-group="voiceover"></video>`;
59+
expect(resolveAudioGroups(document)).toEqual([]);
60+
});
61+
});
62+
63+
describe("audioGroupOf", () => {
64+
it("reads the member's group id", () => {
65+
document.body.innerHTML = `<audio id="vo-1" data-audio-group="voiceover"></audio>`;
66+
expect(audioGroupOf(document.getElementById("vo-1") as Element)).toBe("voiceover");
67+
});
68+
69+
it("returns null when the attribute is absent", () => {
70+
document.body.innerHTML = `<audio id="vo-1"></audio>`;
71+
expect(audioGroupOf(document.getElementById("vo-1") as Element)).toBeNull();
72+
});
73+
74+
// The mirror of resolveAudioGroups' own video case. These two readers used to
75+
// disagree here: the resolver saw no group, this one answered "voiceover", so
76+
// preview routed a track through a bus the export would never build (the
77+
// render enforces audio-only in audioMixer).
78+
it("returns null for a video, matching resolveAudioGroups", () => {
79+
document.body.innerHTML = `<video id="v-1" data-audio-group="voiceover"></video>`;
80+
const el = document.getElementById("v-1") as Element;
81+
expect(audioGroupOf(el)).toBeNull();
82+
expect(resolveAudioGroups(document)).toEqual([]);
83+
});
84+
85+
it("returns null for an empty attribute, not an empty string", () => {
86+
document.body.innerHTML = `<audio id="vo-1" data-audio-group=""></audio>`;
87+
expect(audioGroupOf(document.getElementById("vo-1") as Element)).toBeNull();
88+
expect(resolveAudioGroups(document)).toEqual([]);
89+
});
90+
91+
// Groups do not nest, and the group element is not a member of itself.
92+
it("returns null for the group element even when it carries the attribute", () => {
93+
document.body.innerHTML = `<hf-audio-group id="bus" data-audio-group="other"></hf-audio-group>`;
94+
expect(audioGroupOf(document.getElementById("bus") as Element)).toBeNull();
95+
});
96+
});
97+
98+
describe("ensureAudioGroupInertStyle", () => {
99+
it("takes the group element out of layout", () => {
100+
document.body.innerHTML = `<hf-audio-group id="voiceover"></hf-audio-group>`;
101+
const el = document.getElementById("voiceover") as HTMLElement;
102+
ensureAudioGroupInertStyle(document);
103+
expect(getComputedStyle(el).display).toBe("none");
104+
});
105+
106+
// An unknown custom element is an ordinary inline box, so in a flex or grid
107+
// root it takes a slot: a gap, a justify-content share, and every
108+
// :nth-child after it shifts. An author rule must not be able to put it
109+
// back — and an id selector outranks this rule's type selector no matter
110+
// which stylesheet came last, so `!important` is the only thing holding the
111+
// contract. Dropping it makes this case fail.
112+
it("beats an author rule that outranks it on specificity", () => {
113+
document.head.insertAdjacentHTML(
114+
"beforeend",
115+
`<style id="author">#voiceover{display:flex}</style>`,
116+
);
117+
document.body.innerHTML = `<hf-audio-group id="voiceover"></hf-audio-group>`;
118+
ensureAudioGroupInertStyle(document);
119+
expect(getComputedStyle(document.getElementById("voiceover") as HTMLElement).display).toBe(
120+
"none",
121+
);
122+
document.getElementById("author")?.remove();
123+
});
124+
125+
it("injects once, however many times it is called", () => {
126+
ensureAudioGroupInertStyle(document);
127+
ensureAudioGroupInertStyle(document);
128+
expect(document.querySelectorAll("#__hf-audio-group-inert")).toHaveLength(1);
129+
});
130+
});
131+
132+
describe(HF_AUDIO_GROUP_ATTR, () => {
133+
it("is the attribute name membership is keyed on", () => {
134+
expect(HF_AUDIO_GROUP_ATTR).toBe("data-audio-group");
135+
});
136+
});

‎packages/core/src/audioGroups.ts‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
/**
2+
* The audio group model: a named bucket of audio tracks that shares a label,
3+
* an FX chain, and automation. Membership is held by the member (`data-audio-group`
4+
* pointing at a group id), not by the group nesting its members, so a track
5+
* dropped from the DOM simply disappears from the group on the next resolve —
6+
* nothing dangles.
7+
*
8+
* Parse-only: this module answers "what groups exist and who is in them," and
9+
* nothing here routes or sums audio yet.
10+
*/
11+
12+
export const HF_AUDIO_GROUP_TAG = "hf-audio-group";
13+
export const HF_AUDIO_GROUP_ATTR = "data-audio-group";
14+
15+
/**
16+
* v1 membership, in one place: an `<audio>` carrying a NON-EMPTY
17+
* `data-audio-group`.
18+
*
19+
* Both readers below derive from this string, because they disagreed when they
20+
* did not. `resolveAudioGroups` scanned `audio[...]` while `audioGroupOf`
21+
* returned the attribute off any element, so the same DOM answered "no group"
22+
* for a `<video data-audio-group="voiceover">` in one and `"voiceover"` in the
23+
* other. The render already enforces audio-only (see audioMixer's
24+
* `type === "audio"` guard), so the disagreement was preview routing a track
25+
* the export would never group.
26+
*/
27+
const MEMBER_SELECTOR = `audio[${HF_AUDIO_GROUP_ATTR}]`;
28+
29+
export interface HfAudioGroup {
30+
id: string;
31+
/** `data-label`, falling back to the id when absent. */
32+
label: string;
33+
/** Member element ids, in document order. */
34+
memberIds: string[];
35+
}
36+
37+
/**
38+
* Every group with at least one member, resolved from the live document.
39+
*
40+
* A group with members but no `<hf-audio-group>` element still resolves
41+
* (label = id) so a hand-authored composition degrades gracefully. Audio
42+
* only in v1 — a `data-audio-group` on a `<video>` is ignored.
43+
*/
44+
export function resolveAudioGroups(root: ParentNode): HfAudioGroup[] {
45+
const membersByGroup = new Map<string, string[]>();
46+
for (const member of root.querySelectorAll(MEMBER_SELECTOR)) {
47+
const groupId = member.getAttribute(HF_AUDIO_GROUP_ATTR);
48+
if (!groupId || !member.id) continue;
49+
const members = membersByGroup.get(groupId);
50+
if (members) members.push(member.id);
51+
else membersByGroup.set(groupId, [member.id]);
52+
}
53+
54+
const groupElements = new Map<string, Element>();
55+
for (const el of root.querySelectorAll(HF_AUDIO_GROUP_TAG)) {
56+
if (el.id) groupElements.set(el.id, el);
57+
}
58+
59+
const groups: HfAudioGroup[] = [];
60+
for (const [id, memberIds] of membersByGroup) {
61+
const el = groupElements.get(id);
62+
const label = el?.getAttribute("data-label") || id;
63+
groups.push({ id, label, memberIds });
64+
}
65+
return groups;
66+
}
67+
68+
/**
69+
* The group a member belongs to, or null — the same predicate
70+
* `resolveAudioGroups` scans with, so the two can never disagree about a given
71+
* element.
72+
*
73+
* Non-`<audio>` returns null: video is out of scope in v1, and so is
74+
* `data-audio-group` on an `<hf-audio-group>` itself (groups do not nest).
75+
* `data-audio-group=""` returns null rather than `""` — the resolver skips a
76+
* falsy id, and the "or null" in this contract has to mean it.
77+
*/
78+
export function audioGroupOf(el: Element): string | null {
79+
if (el.tagName?.toLowerCase() !== "audio") return null;
80+
return el.getAttribute(HF_AUDIO_GROUP_ATTR) || null;
81+
}
82+
83+
/**
84+
* Make `<hf-audio-group>` inert, once per document.
85+
*
86+
* The element is metadata — an id, a label, a chain, an automation lane — and
87+
* carries no content, but "no content" is not "no box": it is still an unknown
88+
* custom element, so in a flex or grid composition root it counts as an item
89+
* (taking a `gap`, shifting `justify-content`, moving every `:nth-child` after
90+
* it), and in inline formatting it can still open a line box. Authored layout
91+
* would shift by adding a group, which is not something a mixing decision is
92+
* allowed to do.
93+
*
94+
* `!important` because the rule has to beat an author rule that sets `display`
95+
* on the tag — inertness here is a contract, not a default. Emitted from the
96+
* runtime rather than the compiler so preview and render share one source.
97+
*/
98+
export function ensureAudioGroupInertStyle(doc: Document): void {
99+
const STYLE_ID = "__hf-audio-group-inert";
100+
if (!doc?.head || doc.getElementById(STYLE_ID)) return;
101+
const style = doc.createElement("style");
102+
style.id = STYLE_ID;
103+
style.textContent = `${HF_AUDIO_GROUP_TAG}{display:none!important}`;
104+
doc.head.appendChild(style);
105+
}

‎packages/core/src/canaryRegistry.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,17 @@ export const CANARIES: readonly CanaryDefinition[] = [
9999
owner: "vance",
100100
sunsetAfter: "2026-12-15",
101101
},
102+
{
103+
name: "audio-groups",
104+
percentage: 0,
105+
description:
106+
"Group audio tracks under a shared label, FX chain, and automation " +
107+
"clock. Gates the Studio UI for creating and managing groups; the " +
108+
"underlying <hf-audio-group> element and data-audio-group membership " +
109+
"parse and play regardless of enrollment.",
110+
owner: "vance",
111+
sunsetAfter: "2027-01-15",
112+
},
102113
] as const;
103114

104115
export function findCanary(name: string): CanaryDefinition | undefined {

‎packages/core/src/runtime/init.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ import { loadExternalCompositions, loadInlineTemplateCompositions } from "./comp
3939
import { applyCaptionOverrides } from "./captionOverrides";
4040
import { applyPositionEdits, installPositionEditsSeekReapply } from "./positionEdits";
4141
import { applyVariableBindings } from "./applyVariableBindings";
42+
import { ensureAudioGroupInertStyle } from "../audioGroups.js";
4243
import { createColorGradingRuntime, type RuntimeColorGradingApi } from "./colorGrading";
4344
import { TransportClock } from "./clock";
4445
import { WebAudioTransport } from "./webAudioTransport";
@@ -131,6 +132,10 @@ export function initSandboxRuntimeModular(): void {
131132
// custom props) — values are fixed for the page's lifetime, so applying
132133
// once at init keeps renders deterministic and seeks safe.
133134
applyVariableBindings(document);
135+
// `<hf-audio-group>` is metadata, so it must not occupy a box — see
136+
// ensureAudioGroupInertStyle. Injected here, before timelines bind, so no
137+
// captured frame ever sees the group as a layout item.
138+
ensureAudioGroupInertStyle(document);
134139
const exportRenderFps = resolveExportRenderFps();
135140
state.canonicalFps = exportRenderFps.fps ?? state.canonicalFps;
136141
setRuntimeProtocolFps(state.canonicalFps);

‎packages/studio/src/hooks/useDomEditCommits.test.tsx‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1376,6 +1376,30 @@ describe("useDomEditCommits attribute persist handling", () => {
13761376
}
13771377
});
13781378

1379+
it("sets and removes data-audio-group like any other data attribute", async () => {
1380+
stubPatchFetch({ ok: true, changed: true, matched: true });
1381+
const { iframe, element } = createPreviewElement();
1382+
const rendered = renderDomEditCommits(createSelection(element), iframe);
1383+
1384+
try {
1385+
await act(async () => {
1386+
await rendered.hook.handleDomAttributeLiveCommit("audio-group", "voiceover", undefined, {
1387+
previewOnly: true,
1388+
});
1389+
});
1390+
expect(element.getAttribute("data-audio-group")).toBe("voiceover");
1391+
1392+
await act(async () => {
1393+
await rendered.hook.handleDomAttributeLiveCommit("audio-group", "", undefined, {
1394+
previewOnly: true,
1395+
});
1396+
});
1397+
expect(element.getAttribute("data-audio-group")).toBeNull();
1398+
} finally {
1399+
rendered.cleanup();
1400+
}
1401+
});
1402+
13791403
it("keeps a data-attribute commit on success", async () => {
13801404
stubPatchFetch({
13811405
ok: true,

0 commit comments

Comments
 (0)