Skip to content

Commit 4a55bc8

Browse files
authored
feat(cli): add --lang and auto-infer phonemizer locale from voice prefix (#351)
* feat(cli): add --lang and auto-infer phonemizer locale from voice prefix `hyperframes tts` was calling Kokoro's `model.create(text, voice=, speed=)` with no language argument, so Kokoro's default phonemizer (en-us) was applied regardless of the voice selected. Picking `ef_dora` or `jf_alpha` and feeding it Spanish or Japanese text produced English-phonemized output. Closes #349. - `manager.ts`: add `SUPPORTED_LANGS`, `inferLangFromVoiceId`, and `isSupportedLang`. Attach a `defaultLang` field to every bundled voice and expand the bundled list with `ef_dora`, `ff_siwis`, `jf_alpha`, `zf_xiaobei` so `--list` surfaces multilingual options. - `synthesize.ts`: accept optional `lang: SupportedLang` in `SynthesizeOptions`, forward it to the Python worker as `argv[7]`. The worker introspects `Kokoro.create`'s signature and only passes `lang=` when the installed kokoro-onnx version supports it. Returned metadata now includes `lang` and `langApplied` so callers can detect silent no-ops. Bump the cached script filename to `synth-v2.py` so existing installs pick up the new script automatically. - `commands/tts.ts`: add `--lang, -l` with validation against `SUPPORTED_LANGS`. Resolution order is explicit `--lang` > inferred from voice prefix > `en-us`. When explicit lang disagrees with the voice-implied lang (legitimate for stylized accents), emit a dim-level hint; suppress under `--json`. When kokoro-onnx silently ignores the kwarg, log that too. Update `--list` with a new "Lang code" column and add multilingual examples. - Tests: new `manager.test.ts` covering every supported prefix, the unknown-prefix fallback, case-insensitivity, `isSupportedLang` validation, and a regression guard that every bundled voice has a valid `defaultLang` matching its ID. - Docs: `docs/packages/cli.mdx` and `skills/hyperframes/references/tts.md` updated with the flag, examples, the espeak-ng dependency note for non-English phonemization, and the voice-prefix → lang table. Backward compatibility: - English voices (a*/b* prefixes) continue to phonemize as en-us / en-gb — no change. - Non-English voices now phonemize correctly by default (bug fix, not a regression). - Older kokoro-onnx versions that don't know the `lang` kwarg keep working via signature introspection; the CLI logs a dim note if `--lang` was requested but ignored. Verification: - `bun --cwd packages/cli test` — 128 tests pass (incl. 17 new). - `bunx oxlint` and `bunx oxfmt --check` clean on changed files. - `bun run build` succeeds. - `npx tsx packages/cli/src/cli.ts tts --help` / `--list` render cleanly; invalid `--lang` produces a clean error with the valid-codes list. * refactor(cli): simplify tts --lang implementation Post-review cleanup on #351. Net -21 lines. - Drop `defaultLang` field + `makeVoice()` helper from VoiceInfo — compute via `inferLangFromVoiceId(v.id)` at read time in listVoices. The only reader was the --list table; caching the derived value on every voice added a self-consistency invariant we had to test. - Drop redundant `lang` field from SynthesizeResult — caller already knows the requested lang since it passed it in; only `langApplied` carries information the caller can't derive. - Use `errorBox` for --lang validation to match the house style in render.ts (other validation errors already use errorBox). - Reuse existing `langList` module constant in the validation error instead of re-joining SUPPORTED_LANGS. - Inline `DEFAULT_LANG` — used once in inferLangFromVoiceId. - Trim WHAT-restating comments and the duplicate prefix-enumeration JSDoc on inferLangFromVoiceId (VOICE_PREFIX_LANG already carries per-row comments). - Clean up orphaned `synth*.py` files in ~/.cache/hyperframes/tts when writing the current versioned script, so repeated upgrades don't leak files. - Drop the `EN-US` case-sensitive-rejection test assertion — the CLI lowercases input before validation, so accepting mixed case is a feature, not a bug. Tests: 16/16 in `manager.test.ts`, 127/127 full CLI suite pass. Lint + format + typecheck clean.
1 parent 577e822 commit 4a55bc8

6 files changed

Lines changed: 270 additions & 22 deletions

File tree

‎docs/packages/cli.mdx‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -305,6 +305,12 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
305305
# Adjust speech speed
306306
npx hyperframes tts "Slow and clear" --speed 0.8
307307

308+
# Generate Spanish speech (lang auto-detected from the `e` voice prefix)
309+
npx hyperframes tts "La reunión empieza a las nueve" --voice ef_dora --output es.wav
310+
311+
# Override the phonemizer (read English text with a French voice)
312+
npx hyperframes tts "Bonjour le monde" --voice af_heart --lang fr-fr
313+
308314
# Read text from a file
309315
npx hyperframes tts script.txt
310316

@@ -317,9 +323,14 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
317323
| `--output, -o` | Output file path (default: `speech.wav` in current directory) |
318324
| `--voice, -v` | Voice ID (run `--list` to see options) |
319325
| `--speed, -s` | Speech speed multiplier (default: 1.0) |
326+
| `--lang, -l` | Phonemizer locale (`en-us`, `en-gb`, `es`, `fr-fr`, `hi`, `it`, `pt-br`, `ja`, `zh`). When omitted, inferred from the voice ID prefix. |
320327
| `--list` | List available voices and exit |
321328
| `--json` | Output result as JSON |
322329

330+
<Tip>
331+
Voice IDs encode the phonemizer language in their first letter (`a`=American, `b`=British, `e`=Spanish, `f`=French, `h`=Hindi, `i`=Italian, `j`=Japanese, `p`=Brazilian Portuguese, `z`=Mandarin). `--lang` is only needed when you want to override that — for example, giving English text a French phonemizer for a stylized accent.
332+
</Tip>
333+
323334
<Tip>
324335
Combine `tts` with `transcribe` to generate narration and word-level timestamps for captions in a single workflow: generate the audio with `tts`, then transcribe the output with `transcribe` to get word-level timing.
325336
</Tip>

‎packages/cli/src/commands/tts.ts‎

Lines changed: 70 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,32 @@ export const examples: Example[] = [
77
["Choose a voice", 'hyperframes tts "Hello world" --voice am_adam'],
88
["Save to a specific file", 'hyperframes tts "Intro" --voice bf_emma --output narration.wav'],
99
["Adjust speech speed", 'hyperframes tts "Slow and clear" --speed 0.8'],
10+
[
11+
"Generate Spanish speech",
12+
'hyperframes tts "La reunión empieza a las nueve" --voice ef_dora --output es.wav',
13+
],
14+
[
15+
"Override phonemizer language",
16+
'hyperframes tts "Ciao a tutti" --voice af_heart --lang it --output accented.wav',
17+
],
1018
["Read text from a file", "hyperframes tts script.txt"],
1119
["List available voices", "hyperframes tts --list"],
1220
];
1321
import { resolve, extname } from "node:path";
1422
import * as clack from "@clack/prompts";
1523
import { c } from "../ui/colors.js";
16-
import { DEFAULT_VOICE, BUNDLED_VOICES } from "../tts/manager.js";
24+
import { errorBox } from "../ui/format.js";
25+
import {
26+
DEFAULT_VOICE,
27+
BUNDLED_VOICES,
28+
SUPPORTED_LANGS,
29+
inferLangFromVoiceId,
30+
isSupportedLang,
31+
type SupportedLang,
32+
} from "../tts/manager.js";
1733

1834
const voiceList = BUNDLED_VOICES.map((v) => `${v.id} (${v.label})`).join(", ");
35+
const langList = SUPPORTED_LANGS.join(", ");
1936

2037
export default defineCommand({
2138
meta: {
@@ -43,6 +60,11 @@ export default defineCommand({
4360
description: "Speech speed multiplier (default: 1.0)",
4461
alias: "s",
4562
},
63+
lang: {
64+
type: "string",
65+
description: `Phonemizer language (auto-detected from voice prefix when omitted). Options: ${langList}`,
66+
alias: "l",
67+
},
4668
list: {
4769
type: "boolean",
4870
description: "List available voices and exit",
@@ -94,15 +116,37 @@ export default defineCommand({
94116
process.exit(1);
95117
}
96118

119+
const inferredLang = inferLangFromVoiceId(voice);
120+
let lang: SupportedLang = inferredLang;
121+
if (args.lang != null) {
122+
const requested = String(args.lang).toLowerCase();
123+
if (!isSupportedLang(requested)) {
124+
errorBox("Invalid --lang", `Got "${args.lang}". Must be one of: ${langList}.`);
125+
process.exit(1);
126+
}
127+
lang = requested;
128+
}
129+
130+
// Mismatched voice/lang is a valid stylization (English text, French
131+
// phonemization for accent), so this is a hint, not an error.
132+
if (!args.json && args.lang != null && lang !== inferredLang) {
133+
console.log(
134+
c.dim(
135+
` Note: voice "${voice}" is ${inferredLang}, rendering with --lang ${lang} instead.`,
136+
),
137+
);
138+
}
139+
97140
// ── Synthesize ────────────────────────────────────────────────────
98141
const { synthesize } = await import("../tts/synthesize.js");
99142
const spin = args.json ? null : clack.spinner();
100-
spin?.start(`Generating speech with ${c.accent(voice)}...`);
143+
spin?.start(`Generating speech with ${c.accent(voice)} (${lang})...`);
101144

102145
try {
103146
const result = await synthesize(text, output, {
104147
voice,
105148
speed,
149+
lang,
106150
onProgress: spin ? (msg) => spin.message(msg) : undefined,
107151
});
108152

@@ -112,6 +156,8 @@ export default defineCommand({
112156
ok: true,
113157
voice,
114158
speed,
159+
lang,
160+
langApplied: result.langApplied,
115161
durationSeconds: result.durationSeconds,
116162
outputPath: result.outputPath,
117163
}),
@@ -122,6 +168,13 @@ export default defineCommand({
122168
`Generated ${c.accent(result.durationSeconds.toFixed(1) + "s")} of speech → ${c.accent(result.outputPath)}`,
123169
),
124170
);
171+
if (args.lang != null && !result.langApplied) {
172+
console.log(
173+
c.dim(
174+
" Note: installed kokoro-onnx version does not support the --lang kwarg; phonemization used Kokoro's default.",
175+
),
176+
);
177+
}
125178
}
126179
} catch (err) {
127180
const message = err instanceof Error ? err.message : String(err);
@@ -140,23 +193,29 @@ export default defineCommand({
140193
// ---------------------------------------------------------------------------
141194

142195
function listVoices(json: boolean): void {
196+
const rows = BUNDLED_VOICES.map((v) => ({ ...v, defaultLang: inferLangFromVoiceId(v.id) }));
197+
143198
if (json) {
144-
console.log(JSON.stringify(BUNDLED_VOICES));
199+
console.log(JSON.stringify(rows));
145200
return;
146201
}
147202

148203
console.log(`\n${c.bold("Available voices")} (Kokoro-82M)\n`);
149204
console.log(
150-
` ${c.dim("ID")} ${c.dim("Name")} ${c.dim("Language")} ${c.dim("Gender")}`,
205+
` ${c.dim("ID")} ${c.dim("Name")} ${c.dim("Language")} ${c.dim("Lang code")} ${c.dim("Gender")}`,
151206
);
152-
console.log(` ${c.dim("─".repeat(60))}`);
153-
for (const v of BUNDLED_VOICES) {
154-
const id = v.id.padEnd(18);
155-
const label = v.label.padEnd(13);
156-
const lang = v.language.padEnd(10);
157-
console.log(` ${c.accent(id)} ${label} ${lang} ${v.gender}`);
207+
console.log(` ${c.dim("─".repeat(72))}`);
208+
for (const row of rows) {
209+
const id = row.id.padEnd(18);
210+
const label = row.label.padEnd(13);
211+
const lang = row.language.padEnd(10);
212+
const code = row.defaultLang.padEnd(10);
213+
console.log(` ${c.accent(id)} ${label} ${lang} ${code} ${row.gender}`);
158214
}
159215
console.log(
160-
`\n ${c.dim("Use any Kokoro voice ID — see https://github.com/thewh1teagle/kokoro-onnx for all 54 voices")}\n`,
216+
`\n ${c.dim("Use any Kokoro voice ID — see https://github.com/thewh1teagle/kokoro-onnx for all 54 voices")}`,
217+
);
218+
console.log(
219+
` ${c.dim("Override phonemizer with --lang <" + SUPPORTED_LANGS.join("|") + ">")}\n`,
161220
);
162221
}
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
import { describe, expect, it } from "vitest";
2+
import {
3+
BUNDLED_VOICES,
4+
SUPPORTED_LANGS,
5+
inferLangFromVoiceId,
6+
isSupportedLang,
7+
} from "./manager.js";
8+
9+
describe("inferLangFromVoiceId", () => {
10+
it.each([
11+
["af_heart", "en-us"],
12+
["am_adam", "en-us"],
13+
["bf_emma", "en-gb"],
14+
["bm_george", "en-gb"],
15+
["ef_dora", "es"],
16+
["ff_siwis", "fr-fr"],
17+
["hf_alpha", "hi"],
18+
["if_sara", "it"],
19+
["jf_alpha", "ja"],
20+
["pf_dora", "pt-br"],
21+
["zf_xiaobei", "zh"],
22+
])("maps voice %s to lang %s", (voiceId, expected) => {
23+
expect(inferLangFromVoiceId(voiceId)).toBe(expected);
24+
});
25+
26+
it("falls back to en-us for unknown prefixes", () => {
27+
expect(inferLangFromVoiceId("xf_test")).toBe("en-us");
28+
expect(inferLangFromVoiceId("")).toBe("en-us");
29+
});
30+
31+
it("is case-insensitive on the prefix letter", () => {
32+
expect(inferLangFromVoiceId("EF_dora")).toBe("es");
33+
expect(inferLangFromVoiceId("ZF_xiaobei")).toBe("zh");
34+
});
35+
});
36+
37+
describe("isSupportedLang", () => {
38+
it("accepts every value in SUPPORTED_LANGS", () => {
39+
for (const lang of SUPPORTED_LANGS) {
40+
expect(isSupportedLang(lang)).toBe(true);
41+
}
42+
});
43+
44+
it("rejects invalid or misspelled lang codes", () => {
45+
expect(isSupportedLang("english")).toBe(false);
46+
expect(isSupportedLang("de")).toBe(false);
47+
expect(isSupportedLang("")).toBe(false);
48+
});
49+
});
50+
51+
describe("BUNDLED_VOICES", () => {
52+
// --lang is user-facing, so the voice list must give users a working
53+
// example in at least the most common non-English locales.
54+
it("exposes at least one voice per non-English language", () => {
55+
const langs = new Set(BUNDLED_VOICES.map((v) => inferLangFromVoiceId(v.id)));
56+
expect(langs.has("es")).toBe(true);
57+
expect(langs.has("fr-fr")).toBe(true);
58+
expect(langs.has("ja")).toBe(true);
59+
expect(langs.has("zh")).toBe(true);
60+
});
61+
});

‎packages/cli/src/tts/manager.ts‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,51 @@ const MODEL_URLS: Record<string, string> = {
1717
const VOICES_URL =
1818
"https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin";
1919

20+
// Locale codes accepted by Kokoro's phonemizer (misaki for English,
21+
// espeak-ng for everything else). Kept as a readonly tuple so the union
22+
// type below stays driven by this single source.
23+
export const SUPPORTED_LANGS = [
24+
"en-us",
25+
"en-gb",
26+
"es",
27+
"fr-fr",
28+
"hi",
29+
"it",
30+
"pt-br",
31+
"ja",
32+
"zh",
33+
] as const;
34+
35+
export type SupportedLang = (typeof SUPPORTED_LANGS)[number];
36+
37+
// Kokoro voice IDs are `<lang><gender>_<name>` — the first letter is
38+
// language, the second is gender. See https://github.com/hexgrad/kokoro.
39+
const VOICE_PREFIX_LANG: Record<string, SupportedLang> = {
40+
a: "en-us", // American English
41+
b: "en-gb", // British English
42+
e: "es", // Spanish
43+
f: "fr-fr", // French
44+
h: "hi", // Hindi
45+
i: "it", // Italian
46+
j: "ja", // Japanese
47+
p: "pt-br", // Brazilian Portuguese
48+
z: "zh", // Mandarin
49+
};
50+
51+
/**
52+
* Infer the phonemizer language from a Kokoro voice ID prefix.
53+
* Unknown prefixes fall back to `en-us` — Kokoro's text frontend is
54+
* English-trained, so that's the safe default.
55+
*/
56+
export function inferLangFromVoiceId(voiceId: string): SupportedLang {
57+
const first = voiceId.charAt(0).toLowerCase();
58+
return VOICE_PREFIX_LANG[first] ?? "en-us";
59+
}
60+
61+
export function isSupportedLang(value: string): value is SupportedLang {
62+
return (SUPPORTED_LANGS as readonly string[]).includes(value);
63+
}
64+
2065
// ---------------------------------------------------------------------------
2166
// Voices — Kokoro ships 54 voices across 8 languages. We expose a curated
2267
// default set and allow users to specify any valid Kokoro voice ID.
@@ -38,6 +83,10 @@ export const BUNDLED_VOICES: VoiceInfo[] = [
3883
{ id: "bf_emma", label: "Emma", language: "en-GB", gender: "female" },
3984
{ id: "bf_isabella", label: "Isabella", language: "en-GB", gender: "female" },
4085
{ id: "bm_george", label: "George", language: "en-GB", gender: "male" },
86+
{ id: "ef_dora", label: "Dora", language: "es", gender: "female" },
87+
{ id: "ff_siwis", label: "Siwis", language: "fr-FR", gender: "female" },
88+
{ id: "jf_alpha", label: "Alpha", language: "ja", gender: "female" },
89+
{ id: "zf_xiaobei", label: "Xiaobei", language: "zh", gender: "female" },
4190
];
4291

4392
export const DEFAULT_VOICE = "af_heart";

0 commit comments

Comments
 (0)