Skip to content

Commit 1544b51

Browse files
authored
feat(cli): beats tracks the music with the TypeScript beat tracker, without Chrome (#5634)
* feat(cli): audio answers beats, downbeats, bpm, sections and describe through a provider ladder Beats and downbeats try api, apple-mu, beat-this, then signal; sections try api, apple-mu, then signal; describe tries api, then apple-sound. bpm is read off whichever provider placed the beats. Each provider says whether it can run here and why not, and the ladder warns when it passes one over. The signal rung is a TypeScript tracker: log-band spectral flux, autocorrelation tempo with a log-normal prior, Ellis dynamic-programming beats with a windowed local tempo, a low-band bar phase, and Foote novelty sections snapped to beats. It reports no steady beat when the onsets do not repeat. The api rung posts the audio to HYPERFRAMES_AUDIO_API_URL with HYPERFRAMES_AUDIO_API_KEY and moves each answered time onto the nearest signal beat within a quarter beat, reporting what moved and what did not. beat-this runs Beat This! from a configured Python and local weights and never downloads. apple-sound runs macOS's sound classifier through a Swift helper. apple-mu reports that Music Understanding needs macOS 27. words are no longer accepted: they come from hyperframes transcribe. * docs(cli): audio's beats, downbeats, bpm, sections, describe and their provider ladder * feat(cli): beats tracks the music with the TypeScript beat tracker, without Chrome hyperframes beats and Studio's beat detection now share analyzeMusicFromSignal: band energies, then the spectral-flux tracker in audioRhythm.ts, then each beat's loudness for its strength. The CLI decodes with the same ffmpeg path as hyperframes audio, so beats no longer launches headless Chrome. Studio still decodes in the browser and passes every channel. The old analyzer averaged bpm-detective's tempo with an energy-peak tempo it had only compared after folding both into one octave, and phased a rigid grid from noisy peaks. On 22 real tracks with exact script grids it found the tempo 18% of the time with beat F 0.437; this finds it 91% of the time with F 0.887, and refuses the other two rather than guess. Below a tracker confidence of 0.2 no beats are reported, so ambient or rubato music gets no invented grid and the CLI still writes nothing. bpm-detective, the energy-peak grid, the headless analyzer and its prebuilt bundle are gone. * fix(core): tempo reads 96 BPM off beats on 10 ms frames, not 95.2 The tracker places beats on 10 ms frames, so a 625 ms beat lands 620 or 630 ms apart and the median interval reported 95.2 or 96.8 BPM. Tempo now averages the intervals within 10% of the median: as robust to a skipped or doubled beat, without the frame-sized step. Both hyperframes audio and hyperframes beats read it. * fix(cli): remove redundant audio context comment * fix(core): remove redundant tempo interval comment * fix(ci): record beat analyzer removals and browser evidence * fix(core): keep flat audio novelty in one section * chore: keep PR capture evidence out of the source tree * fix(cli): honor BPM provider pins before analysis
1 parent f55b366 commit 1544b51

21 files changed

Lines changed: 324 additions & 545 deletions

‎bun.lock‎

Lines changed: 0 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/packages/cli.mdx‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -483,8 +483,8 @@ npx hyperframes beats [dir] --json # { ok, file, count, bpm }
483483

484484
`beats` finds the music track — an `<audio>` element with
485485
`data-timeline-role="music"`, or an id like `music`, `bgm`, or `soundtrack` —
486-
runs the same decode and BPM analysis Studio uses inside headless Chrome, and
487-
writes `beats/<audio-path>.json`:
486+
decodes it with ffmpeg, runs the same beat tracker Studio uses, and writes
487+
`beats/<audio-path>.json`:
488488

489489
```json
490490
{
@@ -499,10 +499,10 @@ loads this file as-is and only generates one when none exists. `time` is seconds
499499
into the audio; `strength` (0–1) is relative loudness. Beats you add, move, or
500500
delete in Studio save back to the same file.
501501

502-
Needs a local Chrome, the same one `render` uses. Run
503-
`npx hyperframes browser ensure` if it is missing. Results match Studio's to
504-
within a frame or two — a different headless-Chrome audio sample rate can shift
505-
a beat slightly.
502+
A track without a steady beat (ambient, rubato, too short) gets no beat file:
503+
the command fails and writes nothing rather than inventing a grid. Results
504+
match Studio's to within a frame or two — Studio decodes in the browser, often
505+
at a different sample rate, which can shift a beat slightly.
506506

507507
### `audio`
508508

‎packages/cli/package.json‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,10 +27,9 @@
2727
"scripts": {
2828
"test": "vitest run",
2929
"dev": "tsx src/cli.ts",
30-
"build": "bun run build:fonts && tsup && bun run build:runtime && bun run build:beat-analyzer && bun run build:copy",
30+
"build": "bun run build:fonts && tsup && bun run build:runtime && bun run build:copy",
3131
"build:fonts": "node scripts/build-fonts.mjs",
3232
"build:runtime": "tsx scripts/build-runtime.ts",
33-
"build:beat-analyzer": "node scripts/build-beat-analyzer.mjs",
3433
"build:copy": "node scripts/build-copy.mjs",
3534
"typecheck": "tsc --noEmit",
3635
"gen:skills-manifest": "tsx scripts/gen-skills-manifest.ts",

‎packages/cli/scripts/build-beat-analyzer.mjs‎

Lines changed: 0 additions & 28 deletions
This file was deleted.

‎packages/cli/src/beats/headlessAnalyzer.ts‎

Lines changed: 0 additions & 153 deletions
This file was deleted.
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
2+
import { tmpdir } from "node:os";
3+
import { join } from "node:path";
4+
import { afterAll, describe, expect, it, vi } from "vitest";
5+
import { findFfBinary } from "@hyperframes/parsers/ff-binaries";
6+
import { encodeWav } from "../audio/analysis/providers/shared.js";
7+
8+
const RATE = 22_050;
9+
const root = mkdtempSync(join(tmpdir(), "hf-beats-"));
10+
afterAll(() => rmSync(root, { recursive: true, force: true }));
11+
12+
/** A project whose music track is `seconds` of noise clicks at `bpm`, or silence when `bpm` is null. */
13+
function project(name: string, bpm: number | null, seconds = 12): string {
14+
const dir = join(root, name);
15+
const samples = new Float32Array(seconds * RATE);
16+
let seed = 7;
17+
const noise = () => (seed = (seed * 1_103_515_245 + 12_345) % 2_147_483_648) / 2 ** 30 - 1;
18+
for (let t = 0.2; bpm && t < seconds - 0.1; t += 60 / bpm) {
19+
const start = Math.round(t * RATE);
20+
for (let i = 0; i < 0.05 * RATE; i++)
21+
samples[start + i]! += 0.6 * noise() * Math.exp(-i / RATE / 0.015);
22+
}
23+
mkdirSync(join(dir, "assets"), { recursive: true });
24+
writeFileSync(
25+
join(dir, "assets", "song.wav"),
26+
encodeWav({ sampleRate: RATE, channels: [samples] }),
27+
);
28+
writeFileSync(
29+
join(dir, "index.html"),
30+
`<div data-composition-id="main"><audio id="music" src="assets/song.wav"></audio></div>`,
31+
);
32+
return dir;
33+
}
34+
35+
async function runBeats(dir: string): Promise<{ exitCode: number; out: string }> {
36+
const command = (await import("./beats.js")).default as unknown as {
37+
run: (context: { args: Record<string, unknown> }) => Promise<void>;
38+
};
39+
const lines: string[] = [];
40+
const capture = (...parts: unknown[]) => void lines.push(parts.map(String).join(" "));
41+
const log = vi.spyOn(console, "log").mockImplementation(capture);
42+
const error = vi.spyOn(console, "error").mockImplementation(capture);
43+
let exitCode = 0;
44+
try {
45+
await command.run({ args: { dir, json: true } });
46+
} catch (thrown) {
47+
exitCode = (thrown as { result?: { exitCode?: number } }).result?.exitCode ?? -1;
48+
} finally {
49+
log.mockRestore();
50+
error.mockRestore();
51+
}
52+
return { exitCode, out: lines.join("\n") };
53+
}
54+
55+
const ffmpeg = findFfBinary("ffmpeg") && findFfBinary("ffprobe");
56+
57+
describe.skipIf(!ffmpeg)("hyperframes beats", () => {
58+
it("writes beats/<audio>.json in the beat-file format the Studio reads", async () => {
59+
const dir = project("steady", 120);
60+
const { exitCode, out } = await runBeats(dir);
61+
expect(exitCode).toBe(0);
62+
const file = JSON.parse(readFileSync(join(dir, "beats", "assets", "song.wav.json"), "utf8"));
63+
expect(Object.keys(file)).toEqual(["version", "audio", "beats"]);
64+
expect(file).toMatchObject({ version: 1, audio: "assets/song.wav" });
65+
for (const beat of file.beats) {
66+
expect(Object.keys(beat)).toEqual(["time", "strength"]);
67+
expect(beat.strength).toBeGreaterThanOrEqual(0);
68+
expect(beat.strength).toBeLessThanOrEqual(1);
69+
}
70+
const times: number[] = file.beats.map((b: { time: number }) => b.time);
71+
expect(times.length).toBeGreaterThan(18);
72+
expect(times.every((t) => Math.abs(((t - 0.2 + 0.25) % 0.5) - 0.25) <= 0.07)).toBe(true);
73+
const report = JSON.parse(out) as { ok: boolean; count: number; bpm: number };
74+
expect(report).toMatchObject({ ok: true, file: "beats/assets/song.wav.json" });
75+
expect(report.count).toBe(times.length);
76+
expect(Math.abs(report.bpm - 120)).toBeLessThanOrEqual(1);
77+
});
78+
79+
it("writes nothing for a track with no steady beat", async () => {
80+
const dir = project("silent", null);
81+
const { exitCode, out } = await runBeats(dir);
82+
expect(exitCode).toBe(1);
83+
expect(out).toMatch(/No beats detected in assets\/song\.wav/);
84+
expect(existsSync(join(dir, "beats"))).toBe(false);
85+
});
86+
});

‎packages/cli/src/commands/beats.ts‎

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,17 @@ import { failCommand } from "../utils/commandResult.js";
22
import { defineCommand } from "citty";
33
import { existsSync, readFileSync, mkdirSync, writeFileSync } from "node:fs";
44
import { resolve, join, dirname } from "node:path";
5-
import { findMusicAudioSrc, audioRelPathForSrc, serializeBeats } from "@hyperframes/core/beats";
5+
import {
6+
analyzeMusicFromSignal,
7+
audioRelPathForSrc,
8+
findMusicAudioSrc,
9+
serializeBeats,
10+
type MusicBeatAnalysis,
11+
} from "@hyperframes/core/beats";
612
import type { Example } from "./_examples.js";
713
import { resolveProject, type ProjectDir } from "../utils/project.js";
8-
import { analyzeBeatsHeadless, type HeadlessBeatResult } from "../beats/headlessAnalyzer.js";
14+
import type { AudioSignal } from "@hyperframes/core/audio-analysis";
15+
import { decodeAudio } from "../audio/analysis/decode.js";
916
import { c } from "../ui/colors.js";
1017

1118
export const examples: Example[] = [
@@ -35,19 +42,17 @@ function resolveMusicTarget(project: ProjectDir): { rel: string; audioPath: stri
3542
return { rel, audioPath };
3643
}
3744

38-
async function detect(audioPath: string): Promise<HeadlessBeatResult> {
45+
async function decode(audioPath: string): Promise<AudioSignal> {
3946
try {
40-
return await analyzeBeatsHeadless(readFileSync(audioPath));
47+
const decoded = await decodeAudio(audioPath);
48+
if (decoded) return decoded.signal;
4149
} catch (err) {
42-
const msg = err instanceof Error ? err.message : String(err);
43-
const hint = /chrome|executable|browser|ENOENT/i.test(msg)
44-
? "\nRun: npx hyperframes browser ensure"
45-
: "";
46-
fail(`Beat detection failed: ${msg}${hint}`);
50+
fail(`Beat detection failed: ${err instanceof Error ? err.message : String(err)}`);
4751
}
52+
fail(`Beat detection failed: ${audioPath} has no audio stream`);
4853
}
4954

50-
function report(file: string, result: HeadlessBeatResult, json: boolean): void {
55+
function report(file: string, result: MusicBeatAnalysis, json: boolean): void {
5156
if (json) {
5257
console.log(
5358
JSON.stringify({ ok: true, file, count: result.beatTimes.length, bpm: result.bpm }, null, 2),
@@ -64,7 +69,7 @@ function report(file: string, result: HeadlessBeatResult, json: boolean): void {
6469
export default defineCommand({
6570
meta: {
6671
name: "beats",
67-
description: "Detect beats in the music track (headless) and write beats/<audio>.json",
72+
description: "Detect beats in the music track and write beats/<audio>.json",
6873
},
6974
args: {
7075
dir: { type: "positional", description: "Project directory", required: false },
@@ -73,9 +78,9 @@ export default defineCommand({
7378
async run({ args }) {
7479
const project = resolveProject(args.dir);
7580
const { rel, audioPath } = resolveMusicTarget(project);
76-
if (!args.json) console.log(c.dim(`Analyzing ${rel} in headless Chrome…`));
81+
if (!args.json) console.log(c.dim(`Analyzing ${rel}…`));
7782

78-
const result = await detect(audioPath);
83+
const result = await analyzeMusicFromSignal(await decode(audioPath));
7984
// The Studio ignores a 0-beat file (treats it as a stale seed), so don't write one.
8085
if (result.beatTimes.length === 0) {
8186
fail(`No beats detected in ${rel} — nothing written. (Track may be silent/ambient.)`);

0 commit comments

Comments
 (0)