You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 773a2a1
Browse filesBrowse the repository at this point in the historyBrowse files
huddle(tts): prep short Pocket TTS prompts to prevent runaway generation
Pocket TTS' autoregressive LM has a stochastic sampler (temp=0.7, random
seed) and a hard 500-frame ceiling (~40s of audio at 12.5Hz Mimi frame
rate). On very short, unpunctuated, or lowercase inputs, the EOS logit
(threshold > -4) sometimes never fires within those 500 frames, so the
model produces tens of seconds of nonsensical "breathing" output.
Tyler hit this on the first 'yep' utterance after the Mary swap landed:
~30s of monster noise, then subsequent utterances were fine. Non-
deterministic by design — the next sampled trajectory escaped the trap.
Sherpa-onnx's C++ Pocket TTS impl does not run the prompt preparation
that upstream kyutai-labs/pocket-tts applies in Python. This commit
mirrors that recipe locally:
prepare_pocket_prompt():
1. Collapse interior whitespace.
2. Capitalize the first letter.
3. Append '.' if no terminal punctuation.
4. If <=4 words: prepend 8 spaces; bump frames_after_eos 1 -> 3.
5. Compute adaptive max_frames from word count (saturates at 500).
synth_chunk():
- Routes input through prepare_pocket_prompt before calling
generate_with_config.
- Plumbs frames_after_eos + max_frames via GenerationConfig.extra
(HashMap<String, serde_json::Value>) — exposed by the sherpa-onnx
1.12 Rust binding as a per-call escape hatch.
Source: pocket_tts.models.tts_model.prepare_text_prompt and
_estimate_max_gen_len in https://github.com/kyutai-labs/pocket-tts.
12 new unit tests cover: empty input, the literal 'yep' case (must
become ' Yep.'), threshold inclusivity at 4 vs 5 words, preserved
existing punctuation, whitespace collapsing, no-double-capitalize,
non-ASCII first letter (Cyrillic д -> Д), max_frames tightness/clamping/
monotonicity.
Test results:
cargo test --lib huddle::pocket -> 12/12 pass
cargo test --lib -> 317/317 pass (was 305 + 12 new)
cargo fmt --check -> clean
cargo clippy huddle::pocket -> clean (preexisting doc-list
overindent warning at line
41 not introduced here)
node check-file-sizes.mjs -> clean (pocket.rs override added)
Signed-off-by: Tyler Longwell <[email protected]>
["src-tauri/src/huddle/models.rs",950],// model download manager for Parakeet TDT-CTC STT + Pocket TTS with streaming downloads + SHA-256 verification + Rust-native tar extraction + version manifest + atomic swap + hot-start signaling + MODEL_LICENSE.txt sidecar (fail-closed readiness) + idempotent legacy Moonshine dir cleanup + tts_readiness_requires_license_sidecar test + Mary (VCTK p333) reference voice attribution block
71
71
["src-tauri/src/huddle/stt.rs",580],// STT pipeline + PTT edge-detection flush + PTT gating (is_speech AND ptt_active) + barge-in for VAD mode + rubato resampler + earshot VAD + sherpa-onnx transcription
72
+
["src-tauri/src/huddle/pocket.rs",560],// Pocket TTS engine wrapper + prepare_pocket_prompt (capitalize/punctuate/pad short inputs, mirror upstream pocket-tts prepare_text_prompt) + estimate_max_frames (adaptive max_frames cap to prevent runaway "monster breathing" generation when EOS logit fails to fire) + 12 unit tests
72
73
["src-tauri/src/huddle/preprocessing.rs",670],// TTS text preprocessing pipeline + unified split_sentences + int_to_words 0-999999 + URL trailing punctuation preservation + 23 unit tests
73
74
["src-tauri/src/huddle/relay_api.rs",520],// audio relay recv task + per-peer frame counting for remote human TTS interrupt + NIP-98 channel member query
0 commit comments