Blackwell not supported yet?

Launching the docker:

WARNING: Defaulting repo_id to hexgrad/Kokoro-82M. Pass repo_id='hexgrad/Kokoro-82M' to suppress this warning.
/app/.venv/lib/python3.10/site-packages/torch/nn/modules/rnn.py:123: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.2 and num_layers=1
warnings.warn(
/app/.venv/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py:143: FutureWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
WeightNorm.apply(module, name, dim)
/app/.venv/lib/python3.10/site-packages/torch/cuda/init.py:235: UserWarning:
NVIDIA RTX PRO 6000 Blackwell Workstation Edition with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

I've seen this error plenty of times and it's usually some package, torch or otherwise, that isn't updated for Blackwell compatibility (RTX 5060-5090, 6000).

FYI I was able to get kokoro to launch and work by updating torch inside the container:

python -m ensurepip --default-pip
python -m pip uninstall torch
python -m pip install --upgrade torch --index-url https://download.pytorch.org/whl/cu128

However, this messed up something with being able to actually launch the container. The container just immediately stops, no log, no nothing. I have to launch the container in interactive mode and manually run the entry script command:

In host:
docker run -it --name kokoro --entrypoint=/bin/bash -p 8880:8880 --gpus all kokoro-fastapi-gpu-updated

In the launched container interactive prompt:
./entrypoint.sh

which does work.. but it's kind of annoying to not just be able to click a button in the docker UI to start the container.

Anyway, it proved that updating torch does allow kokoro in the docker to load on my Blackwell card, not sure why the container doesn't launch normally anymore, though.

Fixed the issue with restarting it. I wasn't creating the new container with replacing the bash entrypoint I had used to be able to update the container, so the new container was just relaunching the bash instead. Launching the updated container with --entrypoint=/app/entrypoint.sh fixed it.

So yeah, if you want to update the official docker to fully support Blackwell all you should need to do on the existing container is have the latest torch per my command above.

Until a new release is cut you can use ghcr.io/rushyrush/kokoro-fastapi-gpu:v0.3.0.
I forked from #390 to get 5000 series cards working.

Any updates on this?

My temporary solution is to make this my Dockerfile it works on my Blackwell (5090)

Dockerfile

# Extend official kokoro-fastapi-gpu image and upgrade PyTorch for Blackwell (sm_120) support
FROM ghcr.io/remsky/kokoro-fastapi-gpu:latest

# Upgrade PyTorch to 2.9+ with CUDA 12.8 for Blackwell (RTX 5090, sm_120) support
RUN uv pip install --python /app/.venv/bin/python \
    torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

Until a new release is cut you can use ghcr.io/rushyrush/kokoro-fastapi-gpu:v0.3.0. I forked from #390 to get 5000 series cards working.

thanks for this - doesn't seem like its been merged and released yet so Im using your image and it works great on my 5060ti

@garciadias

weird that latest release shows "Use torch 2.8 for compatibility with 50 series by @garciadias in #390" but when I tried the docker image, my 5060ti did not work

weird that latest release shows "Use torch 2.8 for compatibility with 50 series by @garciadias in #390" but when I tried the docker image, my 5060ti did not work

I found the same when using the latest tag, I switched to v0.2.4-master and that works on a 5060Ti

升级 PyTorch 至 2.10.0+cu128 以支持 RTX 50 系列 GPU

came here with a new rtx6000 pro blackwell.. container flaps...
would love if you could support blackwelll from the compose, but that's the fun with cuda - upgrade and loose older support. :S

Its been awhile but I created a compatible dockerfile and dockercompose for blackwell, so not sure if it still works, but you can try. It uses Cuda 13 as base

Dockerfile:

# Use CUDA 13 base with cuDNN, Ubuntu 24.04
FROM --platform=$BUILDPLATFORM nvcr.io/nvidia/cuda:13.0.1-cudnn-devel-ubuntu24.04

# Enable universe repository and install dependencies
RUN apt-get update -y && \
    apt-get install -y software-properties-common && \
    add-apt-repository universe && \
    apt-get update -y && \
    apt-get install -y \
    python3 python3-venv python3-pip \
    espeak-ng espeak-ng-data \
    git libsndfile1 curl ffmpeg g++ cmake && \
    apt-get clean && rm -rf /var/lib/apt/lists/* && \
    mkdir -p /usr/share/espeak-ng-data && \
    ln -s /usr/lib/*/espeak-ng-data/* /usr/share/espeak-ng-data/ && \
    curl -LsSf https://astral.sh/uv/install.sh | sh && \
    mv /root/.local/bin/uv /usr/local/bin/ && \
    mv /root/.local/bin/uvx /usr/local/bin/ && \
    useradd -m -u 1001 appuser && \
    mkdir -p /app/api/src/models/v1_0 && \
    chown -R appuser:appuser /app

USER appuser
WORKDIR /app

# Copy dependency files
COPY --chown=appuser:appuser pyproject.toml ./pyproject.toml

# Install dependencies using system Python 3, then explicitly force the cu130 wheels
RUN uv venv --python python3 && \
    uv sync --extra gpu --no-cache && \
    uv pip install "torch>=2.4.0" "torchvision>=0.19.0" "torchaudio>=2.4.0" --index-url https://download.pytorch.org/whl/cu130
    
# Copy project files including models
COPY --chown=appuser:appuser api ./api
COPY --chown=appuser:appuser web ./web
COPY --chown=appuser:appuser docker/scripts/ ./
RUN chmod +x ./entrypoint.sh

# Set environment variables
ENV PATH="/app/.venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    PYTHONPATH=/app:/app/api \
    UV_LINK_MODE=copy \
    USE_GPU=true \
    PHONEMIZER_ESPEAK_PATH=/usr/bin \
    PHONEMIZER_ESPEAK_DATA=/usr/share/espeak-ng-data \
    ESPEAK_DATA_PATH=/usr/share/espeak-ng-data \
    DEVICE="gpu"

ENV DOWNLOAD_MODEL=true
# Download model if enabled
RUN if [ "$DOWNLOAD_MODEL" = "true" ]; then \
    python download_model.py --output api/src/models/v1_0; \
    fi

# Run FastAPI server through entrypoint.sh
CMD ["./entrypoint.sh"]

Docker-compose is pretty much the same:

name: kokoro-tts-gpu
services:
  kokoro-tts:
    # image: ghcr.io/remsky/kokoro-fastapi-gpu:v${VERSION}
    build:
      context: ../..
      dockerfile: docker/gpu/Dockerfile
    volumes:
      - ../../api:/app/api
    user: "1001:1001"  # Ensure container runs as UID 1001 (appuser)
    ports:
      - "8887:8880"
    environment:
      - PYTHONPATH=/app:/app/api
      - USE_GPU=true
      - PYTHONUNBUFFERED=1
      - API_LOG_LEVEL=DEBUG
      - NVIDIA_VISIBLE_DEVICES=0
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ['1']
              capabilities: [gpu]

Put both in the same directory and docker build it

fix(docker): aligns cu128 base image with wheel (12.9.1 → 12.8.1), m…

Should be resolved in latest v0.4.0, feel free to reopen if issues pop up

chore(release): 0.6.0 [skip ci]

Navigation Menu

Repository navigation

Blackwell not supported yet? #365

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Blackwell not supported yet? #365

Description

Activity

elkay commented on Jul 18, 2025

elkay commented on Jul 18, 2025

rushyrush commented on Oct 6, 2025

fighter3005 commented on Dec 13, 2025

jamie-beck commented on Jan 7, 2026

ampersandru commented on Jan 11, 2026

ampersandru commented on Jan 11, 2026

paulglavin commented on Jan 30, 2026

haydonryan commented on May 3, 2026

ampersandru commented on May 4, 2026

remsky commented on May 25, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions