Skip to content

Commit 2497402

Browse files
feat(media-use): clone, list and delete HeyGen voices, and accept a host OAuth token (#4992)
* feat(media-use): clone, list and delete HeyGen voices, and accept a host OAuth token heygen-voice.mjs clones a voice from an mp3 or wav file, lists cloned voices by name prefix, and deletes one, printing HeyGen's own error message on any refusal. HEYGEN_ACCESS_TOKEN is now the first credential source and is sent as an OAuth Bearer token. * fix(media-use): heygen-tts prints HeyGen's own refusal instead of a generic line * chore(media-use): regenerate the skills manifest for the voice scripts * fix(media-use): voice list stops on a repeated page, delete accepts an empty reply, plain refusals * test(media-use): a voice list that pages forever fails at once instead of hanging * docs(media-use): keep heygen-voice's header to three lines, usage lives in tts.md
1 parent ed5c100 commit 2497402

9 files changed

Lines changed: 436 additions & 18 deletions

File tree

‎skills-manifest.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,8 +54,8 @@
5454
"files": 1
5555
},
5656
"media-use": {
57-
"hash": "92e7e0e170665b11",
58-
"files": 104
57+
"hash": "9b98931fc057f125",
58+
"files": 107
5959
},
6060
"motion-graphics": {
6161
"hash": "b01efe567a52b5eb",

‎skills/media-use/audio/references/tts.md‎

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,11 @@ Gemini is an explicit alternative to the automatic provider order below. A
4040
request to use Gemini already chooses the provider; do not redirect that user
4141
to HeyGen sign-in. Read the Gemini section for its credential requirement.
4242

43-
| Order | Provider | Env trigger | Voice IDs | Word timestamps | Audio format |
44-
| ----- | ----------------- | ------------------------------------------- | ------------------------------------------- | ----------------------------------------- | -------------------- |
45-
| 1 | HeyGen (Starfish) | `$HEYGEN_API_KEY` / `~/.heygen/credentials` | UUIDs from `GET /v3/voices?engine=starfish` | **Yes** (`word_timestamps[]` in response) | mp3 → wav via ffmpeg |
46-
| 2 | ElevenLabs | `$ELEVENLABS_API_KEY` | UUIDs from elevenlabs.io dashboard | No | mp3 → wav via ffmpeg |
47-
| 3 | Kokoro-82M | always (local fallback) | `am_michael`, `af_heart`, … (54 voices) | No | wav direct |
43+
| Order | Provider | Env trigger | Voice IDs | Word timestamps | Audio format |
44+
| ----- | ----------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------------- | -------------------- |
45+
| 1 | HeyGen (Starfish) | `$HEYGEN_ACCESS_TOKEN` / `$HEYGEN_API_KEY` / `~/.heygen/credentials` | UUIDs from `GET /v3/voices?engine=starfish`, or your cloned voice ids (`heygen-voice.mjs`) | **Yes** (`word_timestamps[]` in response) | mp3 → wav via ffmpeg |
46+
| 2 | ElevenLabs | `$ELEVENLABS_API_KEY` | UUIDs from elevenlabs.io dashboard | No | mp3 → wav via ffmpeg |
47+
| 3 | Kokoro-82M | always (local fallback) | `am_michael`, `af_heart`, … (54 voices) | No | wav direct |
4848

4949
```bash
5050
# Local Kokoro CLI
@@ -59,7 +59,8 @@ the skill's bundled script, which calls the HeyGen v3 REST API directly and need
5959
no CLI provider plumbing:
6060

6161
The script resolves a HeyGen credential the same way the CLI does — first source
62-
wins: `$HEYGEN_API_KEY` → `$HYPERFRAMES_API_KEY` → a project `.env` (auto-loaded,
62+
wins: a host-injected OAuth `$HEYGEN_ACCESS_TOKEN` → `$HEYGEN_API_KEY` →
63+
`$HYPERFRAMES_API_KEY` → a project `.env` (auto-loaded,
6364
walks up ≤5 dirs) → `~/.heygen/credentials` (shared with heygen-cli;
6465
`$HEYGEN_CONFIG_DIR` overrides the dir). An OAuth login is sent as
6566
`Authorization: Bearer`; an API key as `X-Api-Key`; both include
@@ -80,11 +81,27 @@ node skills/media-use/audio/scripts/heygen-tts.mjs ./script.txt -o narration.wav
8081
node skills/media-use/audio/scripts/heygen-tts.mjs --list # public starfish voices
8182
```
8283

83-
- **Voice:** `--voice <id>` must be a **starfish** voice_id (`--list`, or `GET /v3/voices?engine=starfish`). v2-catalog ids are rejected with HTTP 400. Omit `--voice` (English) and it defaults to **Marcia** (`05f19352e8f74b0392a8f411eba40de1`, a fixed default so the choice is deterministic). Non-English with no `--voice` falls back to the first matching catalog voice.
84+
- **Voice:** `--voice <id>` must be a **starfish** voice_id (`--list`, or `GET /v3/voices?engine=starfish`) or one of your cloned voice ids (below). v2-catalog ids are rejected with HTTP 400. Omit `--voice` (English) and it defaults to **Marcia** (`05f19352e8f74b0392a8f411eba40de1`, a fixed default so the choice is deterministic). Non-English with no `--voice` falls back to the first matching catalog voice.
8485
- **Output:** `.wav` → transcoded to 44.1k mono via ffmpeg; `.mp3` → raw bytes (no ffmpeg needed).
8586
- **Words:** `--words <path>` writes the flat `[{id,text,start,end}]` shape below, drop-in for the captions pipeline. HeyGen's `<start>`/`<end>` boundary sentinels are filtered out and ids are re-contiguous.
8687
- **Non-English:** `--lang <code>` (anything but `en`) is sent as the request `language`.
8788

89+
## Cloned voices — `scripts/heygen-voice.mjs`
90+
91+
Clone a voice from an mp3 or wav recording, then pass the printed id to
92+
`heygen-tts.mjs --voice`. Same credential resolution as above. Any API refusal
93+
(clone limit reached, plan upgrade required, voice not found) prints HeyGen's own
94+
message to stderr and exits 1.
95+
96+
```bash
97+
node skills/media-use/audio/scripts/heygen-voice.mjs clone take.mp3 --name "My voice" # → {"voice_id":"..."}
98+
node skills/media-use/audio/scripts/heygen-voice.mjs list --prefix "My" # → [{"voice_id","name","created_at"}]
99+
node skills/media-use/audio/scripts/heygen-voice.mjs delete <voice_id>
100+
```
101+
102+
`clone` waits up to 120 s for the clone to finish. `list` reads
103+
`GET /v3/voices?type=private`.
104+
88105
## When to use which provider
89106

90107
| Goal | Use |

‎skills/media-use/audio/scripts/heygen-tts.mjs‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ const voiceId = await resolveVoiceId({ provider: "heygen", userVoice, lang });
9494
if (!userVoice) console.error(`· using voice ${voiceId}`);
9595

9696
// ---------- synthesize (shared engine code) ----------
97-
const { ok, words } = await synthesizeOne({
97+
const { ok, words, error } = await synthesizeOne({
9898
provider: "heygen",
9999
text,
100100
voiceId,
@@ -103,7 +103,8 @@ const { ok, words } = await synthesizeOne({
103103
wavAbs: output,
104104
hyperframesDir: process.cwd(),
105105
});
106-
if (!ok) die("synthesis failed (HeyGen request/transcode error)");
106+
// HeyGen's own refusal, verbatim: a caller shows it or acts on it (a deleted voice, a limit).
107+
if (!ok) die(error ?? "synthesis failed (HeyGen request/transcode error)");
107108

108109
let wordCount = 0;
109110
if (wordsPath) {
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
import { test } from "node:test";
2+
import assert from "node:assert/strict";
3+
import { spawnSync } from "node:child_process";
4+
import { mkdtempSync } from "node:fs";
5+
import { tmpdir } from "node:os";
6+
import { join } from "node:path";
7+
import { fileURLToPath } from "node:url";
8+
9+
const script = fileURLToPath(new URL("./heygen-tts.mjs", import.meta.url));
10+
// Every fetch answers as HeyGen does for a voice that no longer exists.
11+
const refusing = `data:text/javascript,globalThis.fetch = async () => new Response(${JSON.stringify(
12+
JSON.stringify({ error: { code: "voice_not_found", message: "Voice not found: vc_gone" } }),
13+
)}, { status: 404, headers: { "content-type": "application/json" } });`;
14+
15+
test("a refused speech prints HeyGen's own message, so a caller can act on it", () => {
16+
const dir = mkdtempSync(join(tmpdir(), "heygen-tts-"));
17+
const run = spawnSync(
18+
process.execPath,
19+
["--import", refusing, script, "Hello", "--voice", "vc_gone", "-o", join(dir, "out.mp3")],
20+
{
21+
cwd: dir,
22+
encoding: "utf8",
23+
env: {
24+
...process.env,
25+
HEYGEN_API_KEY: "",
26+
HYPERFRAMES_API_KEY: "",
27+
HEYGEN_ACCESS_TOKEN: "t",
28+
},
29+
},
30+
);
31+
assert.equal(run.status, 1);
32+
assert.match(run.stderr, /✗ heygen-tts: Voice not found: vc_gone\n/);
33+
assert.doesNotMatch(
34+
run.stderr,
35+
/HTTP 404|"error"/,
36+
"HeyGen's message alone, not the raw response",
37+
);
38+
});
Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
#!/usr/bin/env node
2+
// HeyGen voice clones: `clone <audio> --name <n>`, `delete <voice_id>`, `list --prefix <p>` (references/tts.md).
3+
// The clone id works as `heygen-tts.mjs --voice <id>`. A refusal prints HeyGen's own message and exits 1.
4+
// Auth: same resolver as heygen-tts.mjs (lib/heygen.mjs).
5+
6+
import { readFileSync } from "node:fs";
7+
import { extname } from "node:path";
8+
import { fileURLToPath } from "node:url";
9+
import { parseArgs } from "node:util";
10+
import { heygenAuthHeaders, heygenJSON, heygenMessage, loadEnvFromDir } from "./lib/heygen.mjs";
11+
12+
const MEDIA_TYPES = { ".mp3": "audio/mpeg", ".wav": "audio/wav" };
13+
const POLL_MS = 2_000;
14+
const DEADLINE_MS = 120_000;
15+
16+
async function clone(file, name, headers, { sleep, now }) {
17+
const mediaType = MEDIA_TYPES[extname(file).toLowerCase()];
18+
if (!mediaType) throw new Error(`unsupported audio file ${file}: use .mp3 or .wav`);
19+
const data = readFileSync(file).toString("base64");
20+
const created = await heygenJSON("/voices/clone", {
21+
method: "POST",
22+
headers,
23+
body: { voice_name: name, audio: { type: "base64", media_type: mediaType, data } },
24+
});
25+
const id = created.data.voice_clone_id;
26+
const deadline = now() + DEADLINE_MS;
27+
for (;;) {
28+
const { data: voice } = await heygenJSON(`/voices/${id}`, { headers });
29+
if (voice.status === "complete") return { voice_id: voice.voice_id ?? id };
30+
if (voice.status === "failed") throw new Error(voice.failure_message || "voice clone failed");
31+
if (now() >= deadline) throw new Error(`voice clone ${id} not ready after 120 s`);
32+
await sleep(POLL_MS);
33+
}
34+
}
35+
36+
async function list(prefix, headers) {
37+
const mine = [];
38+
const seen = new Set();
39+
let token;
40+
do {
41+
if (token && seen.has(token)) throw new Error(`HeyGen returned page token ${token} twice`);
42+
if (token) seen.add(token);
43+
const query = new URLSearchParams({ type: "private", limit: "100" });
44+
if (token) query.set("token", token);
45+
const page = await heygenJSON(`/voices?${query}`, { headers });
46+
mine.push(...page.data.filter((v) => (v.name ?? "").startsWith(prefix)));
47+
token = page.has_more ? page.next_token : null;
48+
} while (token);
49+
// ponytail: the list response has no created_at, so one GET per match; bounded by the clone limit.
50+
return Promise.all(
51+
mine.map(async ({ voice_id, name }) => {
52+
const { data } = await heygenJSON(`/voices/${voice_id}`, { headers });
53+
return { voice_id, name, created_at: data.created_at ?? null };
54+
}),
55+
);
56+
}
57+
58+
export async function main(
59+
argv,
60+
{
61+
sleep = (ms) => new Promise((r) => setTimeout(r, ms)),
62+
now = Date.now,
63+
out = (s) => process.stdout.write(`${s}\n`),
64+
err = (s) => process.stderr.write(`${s}\n`),
65+
} = {},
66+
) {
67+
try {
68+
const { positionals, values } = parseArgs({
69+
args: argv,
70+
allowPositionals: true,
71+
options: { name: { type: "string" }, prefix: { type: "string", default: "" } },
72+
});
73+
const [command, arg] = positionals;
74+
if (command === "clone") {
75+
if (!arg || !values.name) throw new Error("usage: clone <audio-file> --name <name>");
76+
out(JSON.stringify(await clone(arg, values.name, heygenAuthHeaders(), { sleep, now })));
77+
} else if (command === "delete") {
78+
if (!arg) throw new Error("usage: delete <voice_id>");
79+
await heygenJSON(`/voices/${encodeURIComponent(arg)}`, {
80+
method: "DELETE",
81+
headers: heygenAuthHeaders(),
82+
});
83+
} else if (command === "list") {
84+
out(JSON.stringify(await list(values.prefix, heygenAuthHeaders())));
85+
} else {
86+
throw new Error("usage: heygen-voice.mjs clone|delete|list (see the header of this file)");
87+
}
88+
return 0;
89+
} catch (e) {
90+
err(heygenMessage(e));
91+
return 1;
92+
}
93+
}
94+
95+
if (process.argv[1] === fileURLToPath(import.meta.url)) {
96+
loadEnvFromDir(process.cwd());
97+
process.exitCode = await main(process.argv.slice(2));
98+
}

0 commit comments

Comments
 (0)