Skip to content

Commit b3c43e2

Browse files
feat(cli): add normalize-audio to match one clip's loudness to another (#3306)
* feat(cli): add normalize-audio to match one clip's loudness to another Measures two authored `<audio>` clips with FFmpeg's integrated EBU R128 loudness and writes the target's matching `data-volume`, leaving the reference untouched. The measurement is bounded to the window the composition actually plays. `data-end` bounds a clip's timeline window just as `data-duration` does, and `-ss`/`-t` belong before `-i`: after it they bound the OUTPUT, and with `-f null` there is none, so ebur128 keeps integrating past the clip. On a fixture whose played window is -61.8 LUFS inside a file that measures -27.9 whole, either mistake reports a loudness the composition never plays and "corrects" an already-matched clip by tens of dB. Two EBU R128 passes run between reading the composition and writing it, each bounded only by a two-minute timeout, and the skill docs tell agents to keep Studio open meanwhile — so the attribute patch is re-applied to a fresh read and written through a temp file and a rename. Under `--json` the failures are documents too: an agent doing `JSON.parse(stdout)` on a bare error line throws. A pair needing more than the +12 dB ceiling has a source-file problem rather than a mixer one — mixer gain raises the noise floor with the signal — so the refusal names the remedy. * fix(cli): validate --tolerance before paying for the measurement Each EBU R128 pass is bounded at 120s and normalize-audio runs two, so parsing the argument afterwards made a typo'd --tolerance cost both of them before failing on something that was wrong from the start. Not pinned by a test: the ordering is internal to the command and neither it nor the parser is exported, so covering it would mean restructuring for a spy rather than asserting the behaviour. * docs(cli): restore the blank line between the preview and normalize-audio sections Lost when I resolved the rebase conflict against the background-preview docs by hand instead of letting the formatter near it. oxfmt --check failed on the one file, which fails Preflight — and because preview-parity needs Preflight it skipped, and the preview-regression gate fails closed on a skip, so a missing newline read as a preview defect. The quieter half: the same needs chain meant the required Test context was never created at that head. Not failing — absent, so there was no test signal at all on the PR.
1 parent 9da422f commit b3c43e2

8 files changed

Lines changed: 859 additions & 2 deletions

File tree

‎docs/packages/cli.mdx‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ npm install -g hyperframes
2323
| You want to | Use |
2424
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
2525
| Start a project | [`init`](#init), [`add`](#add), [`catalog`](#catalog) |
26-
| Bring in source material | [`capture`](#capture), [`transcribe`](#transcribe), [`tts`](#tts), [`remove-background`](#remove-background), [`media-treatment`](#media-treatment), [`beats`](#beats) |
26+
| Bring in source material | [`capture`](#capture), [`transcribe`](#transcribe), [`tts`](#tts), [`remove-background`](#remove-background), [`media-treatment`](#media-treatment), [`beats`](#beats), [`normalize-audio`](#normalize-audio) |
2727
| Look at it, or share it | [`preview`](#preview), [`present`](#present-and-play), [`play`](#present-and-play), [`publish`](#publish) |
2828
| Find problems | [`lint`](#lint), [`check`](#check), [`snapshot`](#snapshot), [`keyframes`](#keyframes), [`compare`](#compare-and-grade-compare), [`grade-compare`](#compare-and-grade-compare) |
2929
| Make a file | [`render`](#render), [`benchmark`](#benchmark) |
@@ -428,6 +428,32 @@ Needs a local Chrome, the same one `render` uses. Run
428428
within a frame or two — a different headless-Chrome audio sample rate can shift
429429
a beat slightly.
430430

431+
## Look at it
432+
433+
### `normalize-audio`
434+
435+
Match one authored audio clip to a reference using the exact local media bytes
436+
and FFmpeg's integrated EBU R128 loudness measurement.
437+
438+
```bash
439+
# Measure only (the default)
440+
npx hyperframes normalize-audio --reference target-audio --target user-audio
441+
442+
# Persist the computed gain on #user-audio
443+
npx hyperframes normalize-audio --reference target-audio --target user-audio --write
444+
445+
# Agent-readable result
446+
npx hyperframes normalize-audio --reference target-audio --target user-audio --json
447+
```
448+
449+
`--reference` and `--target` are `<audio>` element ids, with or without `#`.
450+
The reference stays unchanged; the command accounts for both clips' current
451+
`data-volume` and writes the target's absolute matched gain. It measures
452+
`data-media-start` and `data-duration`, rejects remote or out-of-project sources,
453+
and refuses a result that exceeds Studio's +12 dB ceiling or would clip. Dry-run
454+
is deliberate: inspect the measurement before passing `--write`. Use
455+
`--tolerance <LU>` to change the default 0.5 LU no-op threshold.
456+
431457
### `preview`
432458

433459
Start a live preview server with hot reload.

‎packages/cli/README.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,19 @@ previews with `--status`, `--stop`, `--list`, and `--kill-all`. Add `--json` to
4747
managed lifecycle commands for machine-readable output. `--foreground --json`
4848
prints the ready-session envelope once, then remains attached until stopped.
4949

50+
### `normalize-audio`
51+
52+
Measure two local authored audio clips with integrated LUFS and match the target
53+
to the unchanged reference. The command is a dry run unless `--write` is passed:
54+
55+
```bash
56+
npx hyperframes normalize-audio --reference target-audio --target user-audio
57+
npx hyperframes normalize-audio --reference target-audio --target user-audio --write
58+
```
59+
60+
It updates only the target element's `data-volume` and refuses unsafe boosts
61+
that exceed Studio's +12 dB ceiling or would clip.
62+
5063
### `render`
5164

5265
Render a composition to MP4. Run from the project directory; the positional

‎packages/cli/src/cli.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,7 @@ const commandLoaders = {
134134
lint: () => import("./commands/lint.js").then((m) => m.default),
135135
check: () => import("./commands/check.js").then((m) => m.default),
136136
beats: () => import("./commands/beats.js").then((m) => m.default),
137+
"normalize-audio": () => import("./commands/normalize-audio.js").then((m) => m.default),
137138
inspect: () => import("./commands/inspect.js").then((m) => m.default),
138139
keyframes: () => import("./commands/keyframes.js").then((m) => m.default),
139140
layout: () => import("./commands/layout.js").then((m) => m.default),
Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
import { resolve } from "node:path";
2+
import { describe, expect, it } from "vitest";
3+
import {
4+
audioNormalizationPlan,
5+
audioTags,
6+
loudnessMeasureArgs,
7+
parseEbur128Summary,
8+
resolveLocalAudioPath,
9+
updateAudioVolume,
10+
} from "./normalize-audio.js";
11+
12+
const PROJECT = "/tmp/example-project";
13+
14+
describe("audioTags", () => {
15+
it("reads quoted attributes without treating a quoted > as the end of the tag", () => {
16+
const html = `<audio id='reference' title="a > b" src="assets/ref.mp4" data-volume="1"></audio>`;
17+
expect(audioTags(html)).toEqual([
18+
expect.objectContaining({ id: "reference", src: "assets/ref.mp4", volume: 1 }),
19+
]);
20+
});
21+
22+
it("rejects duplicate ids instead of normalizing an arbitrary element", () => {
23+
const html = `<audio id="voice" src="a.wav"></audio><audio id="voice" src="b.wav"></audio>`;
24+
expect(() => audioTags(html)).toThrow(/duplicate audio id "voice"/i);
25+
});
26+
27+
it("ignores audio-like text in comments, scripts, and styles", () => {
28+
const html = `
29+
<!-- <audio id="comment" src="comment.wav"></audio> -->
30+
<script>const example = '<audio id="script" src="script.wav"></audio>';</script>
31+
<style>.demo::after { content: '<audio id="style" src="style.wav">'; }</style>
32+
<audio id="real" src="real.wav"></audio>
33+
`;
34+
expect(audioTags(html).map((tag) => tag.id)).toEqual(["real"]);
35+
});
36+
37+
it.each([
38+
[`<audio id="bad" src="a.wav" data-volume="loud"></audio>`, /data-volume/i],
39+
[`<audio id="bad" src="a.wav" data-media-start="-1"></audio>`, /data-media-start/i],
40+
[`<audio id="bad" src="a.wav" data-duration="0"></audio>`, /data-duration/i],
41+
])("rejects an invalid authored number", (html, expected) => {
42+
expect(() => audioTags(html)).toThrow(expected);
43+
});
44+
45+
it("measures only the window a data-end trim actually plays", () => {
46+
// `data-end` is a first-class trim the parser, the runtime and the render
47+
// mixer all honour. Measuring the whole source for one of these clips reads
48+
// a loudness the composition never plays, and `--write` then "corrects" an
49+
// already-matched clip by tens of dB.
50+
const html = `<audio id="vo" src="vo.wav" data-start="4" data-end="8"></audio>`;
51+
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: 4 }));
52+
});
53+
54+
it("prefers an explicit data-duration over data-end", () => {
55+
const html = `<audio id="vo" src="vo.wav" data-start="4" data-end="8" data-duration="3"></audio>`;
56+
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: 3 }));
57+
});
58+
59+
it("leaves an untrimmed clip unbounded", () => {
60+
const html = `<audio id="vo" src="vo.wav" data-start="4"></audio>`;
61+
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: null }));
62+
});
63+
64+
it("ignores a data-end that does not outlast its start", () => {
65+
const html = `<audio id="vo" src="vo.wav" data-start="8" data-end="4"></audio>`;
66+
expect(audioTags(html)[0]).toEqual(expect.objectContaining({ duration: null }));
67+
});
68+
});
69+
70+
describe("loudnessMeasureArgs", () => {
71+
it("bounds the INPUT, so the loudness filter never sees past the clip", () => {
72+
// `-t` after `-i` bounds the output, and `-f null` has no output worth
73+
// bounding: ffmpeg keeps feeding the graph and ebur128 integrates audio the
74+
// clip never plays. Measured on a fixture whose played window is -61.8 LUFS,
75+
// output-side `-t` reported -33.8.
76+
const args = loudnessMeasureArgs("clip.wav", { mediaStart: 2, duration: 4 });
77+
expect(args.indexOf("-ss")).toBeLessThan(args.indexOf("-i"));
78+
expect(args.indexOf("-t")).toBeLessThan(args.indexOf("-i"));
79+
expect(args.slice(args.indexOf("-ss"), args.indexOf("-i"))).toEqual(["-ss", "2", "-t", "4"]);
80+
});
81+
82+
it("leaves an untrimmed clip unbounded", () => {
83+
const args = loudnessMeasureArgs("clip.wav", { mediaStart: 0, duration: null });
84+
expect(args).not.toContain("-ss");
85+
expect(args).not.toContain("-t");
86+
});
87+
});
88+
89+
describe("resolveLocalAudioPath", () => {
90+
it("resolves a local source and strips query and fragment suffixes", () => {
91+
expect(resolveLocalAudioPath(PROJECT, "assets/voice%20one.wav?v=2#clip")).toBe(
92+
resolve(PROJECT, "assets/voice one.wav"),
93+
);
94+
});
95+
96+
it.each(["https://cdn.example.com/a.wav", "/etc/passwd", "../outside.wav", "data:x"])(
97+
"rejects a non-project source: %s",
98+
(src) => expect(() => resolveLocalAudioPath(PROJECT, src)).toThrow(/local project file/i),
99+
);
100+
});
101+
102+
describe("parseEbur128Summary", () => {
103+
it("takes the final integrated loudness and true-peak summary", () => {
104+
const stderr = `
105+
[Parsed_ebur128_0] t: 1.0 I: -18.2 LUFS
106+
[Parsed_ebur128_0] Summary:
107+
Integrated loudness:
108+
I: -15.5 LUFS
109+
Threshold: -25.5 LUFS
110+
True peak:
111+
Peak: -3.2 dBFS
112+
`;
113+
expect(parseEbur128Summary(stderr)).toEqual({ integratedLufs: -15.5, truePeakDbfs: -3.2 });
114+
});
115+
116+
it("fails when FFmpeg did not produce a usable summary", () => {
117+
expect(() => parseEbur128Summary("no audio stream")).toThrow(/integrated loudness/i);
118+
});
119+
});
120+
121+
describe("audioNormalizationPlan", () => {
122+
it("preserves the reference and attenuates the louder target", () => {
123+
const plan = audioNormalizationPlan(
124+
{ id: "target-audio", volume: 1, integratedLufs: -15.5, truePeakDbfs: -3.2 },
125+
{ id: "user-audio", volume: 1, integratedLufs: -11.7, truePeakDbfs: -0.2 },
126+
);
127+
128+
expect(plan.gainDb).toBeCloseTo(-3.8, 6);
129+
expect(plan.volume).toBeCloseTo(0.645654, 5);
130+
expect(plan.projectedLufs).toBeCloseTo(-15.5, 6);
131+
expect(plan.projectedTruePeakDbfs).toBeCloseTo(-4, 6);
132+
});
133+
134+
it("includes the reference's authored gain in the target", () => {
135+
const plan = audioNormalizationPlan(
136+
{ id: "reference", volume: 2, integratedLufs: -20, truePeakDbfs: -8 },
137+
{ id: "target", volume: 0.5, integratedLufs: -18, truePeakDbfs: -6 },
138+
);
139+
140+
expect(plan.referenceLufs).toBeCloseTo(-13.9794, 4);
141+
expect(plan.volume).toBeCloseTo(1.588656, 6);
142+
expect(plan.projectedLufs).toBeCloseTo(plan.referenceLufs, 6);
143+
});
144+
145+
it("refuses a gain beyond Studio's +12 dB ceiling", () => {
146+
expect(() =>
147+
audioNormalizationPlan(
148+
{ id: "reference", volume: 1, integratedLufs: -5, truePeakDbfs: -1 },
149+
{ id: "target", volume: 1, integratedLufs: -30, truePeakDbfs: -30 },
150+
),
151+
).toThrow(/\+12 dB/i);
152+
});
153+
154+
it("refuses a boost that would clip", () => {
155+
expect(() =>
156+
audioNormalizationPlan(
157+
{ id: "reference", volume: 1, integratedLufs: -10, truePeakDbfs: -1 },
158+
{ id: "target", volume: 1, integratedLufs: -15, truePeakDbfs: -2 },
159+
),
160+
).toThrow(/clip/i);
161+
});
162+
});
163+
164+
describe("updateAudioVolume", () => {
165+
it("updates only the selected audio element and preserves surrounding source", () => {
166+
const html = `<!doctype html>\n<audio id="ref" src="a.wav" data-volume="1"></audio>\n<audio data-volume='1' id='target' src='b.wav'></audio>\n`;
167+
expect(updateAudioVolume(html, "target", 0.645654)).toBe(
168+
`<!doctype html>\n<audio id="ref" src="a.wav" data-volume="1"></audio>\n<audio data-volume='0.645654' id='target' src='b.wav'></audio>\n`,
169+
);
170+
});
171+
172+
it("adds data-volume when it is absent", () => {
173+
expect(updateAudioVolume(`<audio id="target" src="b.wav" />`, "target", 2)).toBe(
174+
`<audio id="target" src="b.wav" data-volume="2" />`,
175+
);
176+
});
177+
});

0 commit comments

Comments
 (0)