@@ -36,6 +36,10 @@ Use another voice only for a documented reason, and write the reason down.
3636
3737## Available routes
3838
39+ Gemini is an explicit alternative to the automatic provider order below. A
40+ request to use Gemini already chooses the provider; do not redirect that user
41+ to HeyGen sign-in. Read the Gemini section for its credential requirement.
42+
3943| Order | Provider | Env trigger | Voice IDs | Word timestamps | Audio format |
4044| ----- | ----------------- | ------------------------------------------- | ------------------------------------------- | ----------------------------------------- | -------------------- |
4145| 1 | HeyGen (Starfish) | ` $HEYGEN_API_KEY ` / ` ~/.heygen/credentials ` | UUIDs from ` GET /v3/voices?engine=starfish ` | ** Yes** (` word_timestamps[] ` in response) | mp3 → wav via ffmpeg |
@@ -83,12 +87,86 @@ node skills/media-use/audio/scripts/heygen-tts.mjs --list # public starfish vo
8387
8488## When to use which provider
8589
86- | Goal | Use |
87- | --------------------------------------------------------- | --------------------------------------------------- |
88- | Best voice quality + word timestamps in one call | ** HeyGen** |
89- | Drop-in cloud TTS, big voice catalog | ** ElevenLabs** |
90- | Offline, no API key, fast iteration | ** Kokoro** |
91- | Non-English multilingual with deterministic phonemization | ** Kokoro** (` ef_dora ` , ` jf_alpha ` , ` zf_xiaobei ` , …) |
90+ | Goal | Use |
91+ | --------------------------------------------------------- | --------------------------------------------------------- |
92+ | Best voice quality + word timestamps in one call | ** HeyGen** |
93+ | Drop-in cloud TTS, big voice catalog | ** ElevenLabs** |
94+ | Offline, no API key, fast iteration | ** Kokoro** |
95+ | Directed delivery with Gemini prebuilt or custom voices | ** Gemini** (explicit selection; transcription for timing) |
96+ | Non-English multilingual with deterministic phonemization | ** Kokoro** (` ef_dora ` , ` jf_alpha ` , ` zf_xiaobei ` , …) |
97+
98+ ## Gemini narration
99+
100+ Use the shared audio engine, not ` hyperframes tts ` . Authenticate with either:
101+
102+ - ` GEMINI_API_KEY ` or ` GOOGLE_API_KEY ` (first set key wins).
103+ - A service-account JSON file at ` GOOGLE_APPLICATION_CREDENTIALS ` , or injected
104+ JSON in ` GCS_CREDS ` . Install ` google-auth requests ` in the Python 3 environment
105+ used by the helper. A configured file takes precedence over injected JSON.
106+
107+ API keys take precedence over service accounts: unset both key variables to use
108+ OAuth. Never put credentials in a request file or composition. The helper
109+ obtains a fresh OAuth token for each generation with the
110+ ` generative-language.retriever ` scope. The quota project resolves from
111+ ` GOOGLE_CLOUD_PROJECT ` , then ` GCLOUD_PROJECT_ID ` , then the service-account JSON.
112+ User ADC files and metadata-server authentication are not supported.
113+
114+ Both routes call the Gemini Developer Interactions API, not Cloud TTS or Vertex
115+ AI. Those APIs have separate model catalogs and access requirements.
116+
117+ Save this as ` audio_request.json ` in the project:
118+
119+ ``` json
120+ {
121+ "provider" : " gemini" ,
122+ "tts_model" : " gemini-3.8-flash-tts" ,
123+ "voice" : " Kore" ,
124+ "lang" : " en" ,
125+ "style" : " Warm, clear, conversational. Leave a short pause between sentences." ,
126+ "lines" : [
127+ { "id" : " intro" , "text" : " Every word has a moment. Let the picture follow the voice." }
128+ ],
129+ "bgm" : { "mode" : " none" }
130+ }
131+ ```
132+
133+ ``` bash
134+ node < SKILL_DIR> /audio/scripts/audio.mjs \
135+ --request ./audio_request.json --hyperframes . --out ./audio_meta.json --only tts
136+ ```
137+
138+ The engine saves ` assets/voice/intro.wav ` , measures its duration, and transcribes
139+ it into ` voices[].words ` in ` audio_meta.json ` . Check that every requested line
140+ has audio and nonempty word timings before building a captioned video. Review
141+ the timings against the actual audio; transcription is estimated alignment,
142+ not native TTS timestamps. Do not distribute words evenly across a clip.
143+
144+ - ** Models:** ` gemini-3.8-flash-tts ` (default), ` gemini-3.8-flash-lite-tts ` ,
145+ ` gemini-3.1-flash-tts-preview ` , ` gemini-2.5-pro-preview-tts ` , and
146+ ` gemini-2.5-flash-preview-tts ` . Use these exact Developer API IDs; Cloud TTS
147+ aliases such as ` gemini-2.5-flash-tts ` are not accepted.
148+ - ** Voice:** ` Kore ` by default; pass another prebuilt voice or an existing custom
149+ voice ID for 3.8. Older models require prebuilt voices. Creating or replicating
150+ voices is outside this helper.
151+ - ** Delivery:** Put directions in ` style ` , not in spoken ` text ` . Each line can
152+ override ` style ` . 3.8 uses structured annotations; older models receive a
153+ delivery prompt before the transcript. Check that directions were not spoken.
154+ Use style for pacing; numeric ` speed ` must be omitted or 1.
155+ - ** Audio:** 3.8 returns a complete WAV. Older models return mono 16-bit PCM,
156+ which the helper wraps as WAV at the returned sample rate without resampling.
157+ No generation calls run during playback or rendering.
158+ - ** Timing:** This adapter requests no native word timestamps. It uses the
159+ existing transcription pass. ` lang ` selects transcription language; Gemini
160+ infers speech language from the text.
161+ - ** Workflow adapters:** Product-launch, faceless-explainer, and PR-video audio
162+ scripts accept `--provider gemini --voice Kore --tts-model gemini-3.8-flash-tts
163+ --style "Warm and clear"`.
164+
165+ An API error is reported as a failed line; an explicitly chosen Gemini voice
166+ never silently falls back to another provider. Check the engine's anomalies
167+ and output metadata, not only its exit code.
168+
169+ API contract: [ Google's speech generation guide] ( https://ai.google.dev/gemini-api/docs/speech-generation ) .
92170
93171## ffmpeg requirement
94172
0 commit comments