Skip to content

About

AI-Powered Text-to-Speech with Voice Cloning using Chatterbox TTS and Gradio interface

Topics

Resources

Stars

9 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

75 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸŽ™οΈ Chatterbox TTS App

AI-Powered Text-to-Speech with Voice Cloning using Chatterbox TTS and Gradio interface.

⚑ Model Zoo

Chatterbox is a family of three state-of-the-art, open-source text-to-speech models by Resemble AI:

Model Size Languages Key Features Best For
Chatterbox-Turbo 350M English Paralinguistic Tags ([laugh]), Lower Compute and VRAM Zero-shot voice agents, Production
Chatterbox-Multilingual 500M 23+ Zero-shot cloning, Multiple Languages Global applications, Localization
Chatterbox 500M English CFG & Exaggeration tuning General zero-shot TTS with creative controls

✨ Features

  • 🎭 Voice Cloning: Clone any voice with just 10 seconds of audio
  • ⚑ Turbo Mode: Ultra-fast generation with lower VRAM requirements
  • 🎭 Paralinguistic Tags: Add [laugh], [cough], [chuckle] for realism
  • 🌍 23+ Languages: Multilingual support (Arabic, Chinese, French, Spanish, etc.)
  • 🎨 Emotion Control: Adjust expressiveness and pacing
  • πŸ†“ Free & Open Source: MIT license, completely free to use
  • πŸ”’ Privacy: Runs completely locally on your machine
  • 🌐 Cross-Platform: Works on Windows, Mac, and Linux
  • πŸ–₯️ Web Interface: Easy-to-use Gradio interface

πŸš€ Quick Start

Prerequisites

  • Python 3.10–3.13 (Pinokio creates a Python 3.11 environment)
  • CUDA-compatible GPU (recommended) or CPU
  • Windows, Linux, or Apple Silicon macOS. Intel Macs are unsupported by the required PyTorch version. AMD on Windows uses CPU; supported AMD GPUs on Linux use ROCm. NVIDIA RTX 50-series (Blackwell) GPUs are not supported by the PyTorch 2.6 build Chatterbox requires, so the app falls back to CPU on them.

Installation

  1. Clone this repository:
git clone https://github.com/PierrunoYT/chatterbox-tts-pinokio.git
cd chatterbox-tts-pinokio
  1. Install dependencies:
cd app
python -m venv env
# Windows: env\Scripts\activate
# macOS/Linux: source env/bin/activate
python -m pip install -r requirements.txt

Note: This installs a default torch build only. For GPU acceleration (NVIDIA/AMD), install the matching torch build for your platform afterward β€” see the when blocks in torch.js at the repo root for the exact commands per platform. The Pinokio install flow (install.js) does this automatically.

  1. (Optional) For Chatterbox-Turbo model access, login to Hugging Face:
hf auth login

Or set HF_TOKEN in the env block of start.js (there's a commented-out example line to uncomment).

  1. Run the application:
python app.py
  1. Open the local URL printed in the terminal. Gradio chooses an available port starting at 7860.

In Pinokio, use Install, then Start. Update pulls launcher changes and reruns installation. If installation fails, Reset incomplete installation removes the environment so you can retry; generated audio is retained.

🎯 Usage

Basic Text-to-Speech

  1. Select your preferred model (Turbo, Multilingual, or Original)
  2. Enter your text in the input field
  3. Adjust emotion and CFG settings as desired
  4. Click "🎡 Generate"
  5. Download your generated audio

Turbo Model with Paralinguistic Tags

  1. Select the "⚑ Turbo (Fastest, English)" model
  2. Use tags in your text for added realism:
    • [laugh], [chuckle], [cough], [sigh]
    • Example: "Hi there! [chuckle] Let me tell you something funny."
  3. Generate ultra-fast, realistic speech

Voice Cloning

  1. Upload a reference audio file (10+ seconds recommended; Turbo requires more than 5 seconds)
  2. Enter your text
  3. Adjust settings
  4. Generate speech with the cloned voice

Multilingual Support

  1. Select the "🌍 Multilingual (23+ Languages)" model
  2. Enter text in any supported language (up to 300 characters)
  3. Select the language matching your text

🌍 Supported Languages (Multilingual Model)

Arabic (ar) β€’ Danish (da) β€’ German (de) β€’ Greek (el) β€’ English (en) β€’ Spanish (es) β€’ Finnish (fi) β€’ French (fr) β€’ Hebrew (he) β€’ Hindi (hi) β€’ Italian (it) β€’ Japanese (ja) β€’ Korean (ko) β€’ Malay (ms) β€’ Dutch (nl) β€’ Norwegian (no) β€’ Polish (pl) β€’ Portuguese (pt) β€’ Russian (ru) β€’ Swedish (sv) β€’ Swahili (sw) β€’ Turkish (tr) β€’ Chinese (zh)

🎨 Settings

  • Model Selection: Choose between Turbo (fastest), Multilingual (23+ languages), or Original (best quality)
  • Exaggeration (Original / Multilingual): Controls how expressive the speech is (0.0 = calm, 0.5 = default, up to 2.0 = very dramatic)
  • CFG (Original / Multilingual): Controls speech pacing (0.0 = slower/deliberate, 1.0 = faster/natural)
  • Paralinguistic Tags (Turbo only): [laugh], [chuckle], [cough], [sigh] for added realism

πŸ“ Project Structure

chatterbox-tts-pinokio/
β”œβ”€β”€ app/                 # Application code (Pinokio convention)
β”‚   β”œβ”€β”€ app.py           # Main Gradio application
β”‚   β”œβ”€β”€ requirements.txt # Python dependencies
β”‚   β”œβ”€β”€ overrides.txt    # uv overrides pinning upstream's unpinned dependencies
β”‚   └── outputs/         # Generated audio (created at runtime)
β”œβ”€β”€ install.js, start.js, …  # Pinokio launcher scripts (repo root)
└── icon.png             # Application icon (optional)

πŸ”§ Technical Details

  • Model: Chatterbox TTS by Resemble AI
  • Interface: Gradio web interface
  • Audio Format: WAV files
  • Device Support: CUDA GPU / CPU automatic detection

πŸ“ Tips for Best Results

Voice Cloning

  • Use at least 10 seconds of clear reference audio
  • Ensure single speaker with no background noise
  • WAV format preferred, 24kHz+ sample rate
  • Professional microphone recommended

Text Input

  • Use natural punctuation for better prosody
  • Longer texts generally produce better results
  • Avoid special characters or formatting

🀝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Run the regression checks with python -m unittest discover -s tests -v and node --test tests/launchers.test.js. Python tests require Gradio 6.8.0 and stub model inference, so no model download or GPU is needed.

With the full dependencies installed, python tests/smoke_runtime.py checks model imports, WAV round-trip, localhost startup, and a queued API validation request without downloading weights. GPU inference and speech quality require separate testing with the models.

πŸ“„ License

This project is open source and available under the MIT License.

πŸ™ Credits

πŸ› Issues

If you encounter any issues, please report them on the GitHub Issues page.

πŸ”Œ API Reference

The Gradio app exposes a programmatic API once running. Replace 7860 with the actual port shown in the Pinokio terminal.

Python (gradio_client)

from gradio_client import Client, handle_file

client = Client("http://127.0.0.1:7860")

result = client.predict(
    model_choice="⚑ Turbo (Fastest, English)",
    text="Hello, this is a test.",
    reference_audio=None,
    exaggeration=0.5,
    cfg_value=0.5,
    temperature=0.8,
    min_p=0.05,
    top_p=0.95,
    repetition_penalty=1.2,
    top_k=1000,
    norm_loudness=True,
    language_code="en",
    output_filename="output.wav",
    api_name="/generate_speech",
)
# result is (audio_filepath, status_message)
print(result)

JavaScript (fetch)

const response = await fetch("http://127.0.0.1:7860/gradio_api/call/generate_speech", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    data: [
      "⚑ Turbo (Fastest, English)", // model_choice
      "Hello, this is a test.",      // text
      null,                          // reference_audio
      0.5,                           // exaggeration
      0.5,                           // cfg_value
      0.8,                           // temperature
      0.05,                          // min_p
      0.95,                          // top_p
      1.2,                           // repetition_penalty
      1000,                          // top_k
      true,                          // norm_loudness
      "en",                        // language_code
      "output.wav"                   // output_filename
    ]
  })
});
const { event_id } = await response.json();

// Poll for result
const resultResponse = await fetch(`http://127.0.0.1:7860/gradio_api/call/generate_speech/${event_id}`);
const text = await resultResponse.text();
console.log(text);

curl

# Submit the request
EVENT_ID=$(curl -s -X POST http://127.0.0.1:7860/gradio_api/call/generate_speech \
  -H "Content-Type: application/json" \
  -d '{
    "data": [
      "⚑ Turbo (Fastest, English)",
      "Hello, this is a test.",
      null, 0.5, 0.5, 0.8, 0.05, 0.95, 1.2, 1000, true, "en", "output.wav"
    ]
  }' | python3 -c "import sys,json; print(json.load(sys.stdin)['event_id'])")

# Retrieve the result
curl -s http://127.0.0.1:7860/gradio_api/call/generate_speech/$EVENT_ID

About

AI-Powered Text-to-Speech with Voice Cloning using Chatterbox TTS and Gradio interface

Topics

Resources

Stars

9 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages