sm_120 not compatible with current PyTorch #306

Describe the bug
I am installing using docker compose and encountered this issue right at the end. It states:
"NVIDIA GeForce RTX 5060 Ti with CUDA capability sm_120 is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90"

I have done some research from the PyTorch forums and found that there are recent builds that should support sm_120, is it possible to configure the installation to get torch versions that support sm_120?

Screenshots or console output
=> [kokoro-tts] exporting to image 244.3s
=> => exporting layers 200.5s
=> => exporting manifest sha256:56a0f89ce31bf37439762b6fddb1fcc1e5d3f61f66c6854ac1c763fd64d0d0 0.0s
=> => exporting config sha256:a00ded8a62309d6bdb77bd7e7c1cb0d800419d9a95fb1b6725a38fb9e73777b5 0.0s
=> => exporting attestation manifest sha256:4bb3ea6bb50f70797b9b2ea66db5839b59ef58ce764e2f9e42 0.0s
=> => exporting manifest list sha256:2a68fd93bec928a73a24ce8c8b961665794d3d568319bdb31b7d6e3e7 0.0s
=> => naming to docker.io/library/kokoro-tts-gpu-kokoro-tts:latest 0.0s
=> => unpacking to docker.io/library/kokoro-tts-gpu-kokoro-tts:latest 43.7s
=> [kokoro-tts] resolving provenance for metadata file 0.0s
[+] Running 3/3
✔ kokoro-tts Built 0.0s
✔ Network kokoro-tts-gpu_default Created 0.0s
✔ Container kokoro-tts-gpu-kokoro-tts-1 Created 0.6s
Attaching to kokoro-tts-1
kokoro-tts-1 |
kokoro-tts-1 | ==========
kokoro-tts-1 | == CUDA ==
kokoro-tts-1 | ==========
kokoro-tts-1 |
kokoro-tts-1 | CUDA Version 12.8.0
kokoro-tts-1 |
kokoro-tts-1 | Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
kokoro-tts-1 |
kokoro-tts-1 | This container image and its contents are governed by the NVIDIA Deep Learning Container License.
kokoro-tts-1 | By pulling and using the container, you accept the terms and conditions of this license:
kokoro-tts-1 | https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
kokoro-tts-1 |
kokoro-tts-1 | A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.
kokoro-tts-1 |
kokoro-tts-1 | 2025-05-05 15:08:56.289 | INFO | main:download_model:60 - Model files already exist and are valid
kokoro-tts-1 | INFO: Started server process [31]
kokoro-tts-1 | INFO: Waiting for application startup.
kokoro-tts-1 | 03:09:06 PM | INFO | main:57 | Loading TTS model and voice packs...
kokoro-tts-1 | 03:09:06 PM | INFO | model_manager:38 | Initializing Kokoro V1 on cuda
kokoro-tts-1 | 03:09:06 PM | DEBUG | paths:101 | Searching for model in path: /app/api/src/models
kokoro-tts-1 | 03:09:06 PM | INFO | kokoro_v1:46 | Loading Kokoro model on cuda
kokoro-tts-1 | 03:09:06 PM | INFO | kokoro_v1:47 | Config path: /app/api/src/models/v1_0/config.json
kokoro-tts-1 | 03:09:06 PM | INFO | kokoro_v1:48 | Model path: /app/api/src/models/v1_0/kokoro-v1_0.pth
kokoro-tts-1 | WARNING: Defaulting repo_id to hexgrad/Kokoro-82M. Pass repo_id='hexgrad/Kokoro-82M' to suppress this warning.
kokoro-tts-1 | /app/.venv/lib/python3.10/site-packages/torch/nn/modules/rnn.py:123: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.2 and num_layers=1
kokoro-tts-1 | warnings.warn(
kokoro-tts-1 | /app/.venv/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py:143: FutureWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
kokoro-tts-1 | WeightNorm.apply(module, name, dim)
kokoro-tts-1 | /app/.venv/lib/python3.10/site-packages/torch/cuda/init.py:235: UserWarning:
kokoro-tts-1 | NVIDIA GeForce RTX 5060 Ti with CUDA capability sm_120 is not compatible with the current PyTorch installation.
kokoro-tts-1 | The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
kokoro-tts-1 | If you want to use the NVIDIA GeForce RTX 5060 Ti GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
kokoro-tts-1 |
kokoro-tts-1 | warnings.warn(
kokoro-tts-1 | 03:09:08 PM | ERROR | main:70 | Failed to initialize model: Warmup failed: Failed to load model: Failed to load Kokoro model: CUDA error: no kernel image is available for execution on the device
kokoro-tts-1 | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
kokoro-tts-1 | For debugging consider passing CUDA_LAUNCH_BLOCKING=1
kokoro-tts-1 | Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
kokoro-tts-1 |
kokoro-tts-1 | ERROR: Traceback (most recent call last):
kokoro-tts-1 | File "/app/api/src/inference/kokoro_v1.py", line 59, in load_model
kokoro-tts-1 | self._model = self._model.cuda()
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1053, in cuda
kokoro-tts-1 | return self._apply(lambda t: t.cuda(device))
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 903, in _apply
kokoro-tts-1 | module._apply(fn)
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 903, in _apply
kokoro-tts-1 | module._apply(fn)
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 903, in _apply
kokoro-tts-1 | module._apply(fn)
kokoro-tts-1 | [Previous line repeated 1 more time]
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/rnn.py", line 290, in _apply
kokoro-tts-1 | self._init_flat_weights()
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/rnn.py", line 215, in _init_flat_weights
kokoro-tts-1 | self.flatten_parameters()
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/rnn.py", line 271, in flatten_parameters
kokoro-tts-1 | torch._cudnn_rnn_flatten_weight(
kokoro-tts-1 | RuntimeError: CUDA error: no kernel image is available for execution on the device
kokoro-tts-1 | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
kokoro-tts-1 | For debugging consider passing CUDA_LAUNCH_BLOCKING=1
kokoro-tts-1 | Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
kokoro-tts-1 |
kokoro-tts-1 |
kokoro-tts-1 | During handling of the above exception, another exception occurred:
kokoro-tts-1 |
kokoro-tts-1 | Traceback (most recent call last):
kokoro-tts-1 | File "/app/api/src/inference/model_manager.py", line 127, in load_model
kokoro-tts-1 | await self._backend.load_model(path)
kokoro-tts-1 | File "/app/api/src/inference/kokoro_v1.py", line 66, in load_model
kokoro-tts-1 | raise RuntimeError(f"Failed to load Kokoro model: {e}")
kokoro-tts-1 | RuntimeError: Failed to load Kokoro model: CUDA error: no kernel image is available for execution on the device
kokoro-tts-1 | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
kokoro-tts-1 | For debugging consider passing CUDA_LAUNCH_BLOCKING=1
kokoro-tts-1 | Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
kokoro-tts-1 |
kokoro-tts-1 |
kokoro-tts-1 | During handling of the above exception, another exception occurred:
kokoro-tts-1 |
kokoro-tts-1 | Traceback (most recent call last):
kokoro-tts-1 | File "/app/api/src/inference/model_manager.py", line 66, in initialize_with_warmup
kokoro-tts-1 | await self.load_model(model_path)
kokoro-tts-1 | File "/app/api/src/inference/model_manager.py", line 131, in load_model
kokoro-tts-1 | raise RuntimeError(f"Failed to load model: {e}")
kokoro-tts-1 | RuntimeError: Failed to load model: Failed to load Kokoro model: CUDA error: no kernel image is available for execution on the device
kokoro-tts-1 | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
kokoro-tts-1 | For debugging consider passing CUDA_LAUNCH_BLOCKING=1
kokoro-tts-1 | Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
kokoro-tts-1 |
kokoro-tts-1 |
kokoro-tts-1 | During handling of the above exception, another exception occurred:
kokoro-tts-1 |
kokoro-tts-1 | Traceback (most recent call last):
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 693, in lifespan
kokoro-tts-1 | async with self.lifespan_context(app) as maybe_state:
kokoro-tts-1 | File "/home/appuser/.local/share/uv/python/cpython-3.10.17-linux-x86_64-gnu/lib/python3.10/contextlib.py", line 199, in aenter
kokoro-tts-1 | return await anext(self.gen)
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 133, in merged_lifespan
kokoro-tts-1 | async with original_context(app) as maybe_original_state:
kokoro-tts-1 | File "/home/appuser/.local/share/uv/python/cpython-3.10.17-linux-x86_64-gnu/lib/python3.10/contextlib.py", line 199, in aenter
kokoro-tts-1 | return await anext(self.gen)
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 133, in merged_lifespan
kokoro-tts-1 | async with original_context(app) as maybe_original_state:
kokoro-tts-1 | File "/home/appuser/.local/share/uv/python/cpython-3.10.17-linux-x86_64-gnu/lib/python3.10/contextlib.py", line 199, in aenter
kokoro-tts-1 | return await anext(self.gen)
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 133, in merged_lifespan
kokoro-tts-1 | async with original_context(app) as maybe_original_state:
kokoro-tts-1 | File "/home/appuser/.local/share/uv/python/cpython-3.10.17-linux-x86_64-gnu/lib/python3.10/contextlib.py", line 199, in aenter
kokoro-tts-1 | return await anext(self.gen)
kokoro-tts-1 | File "/app/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 133, in merged_lifespan
kokoro-tts-1 | async with original_context(app) as maybe_original_state:
kokoro-tts-1 | File "/home/appuser/.local/share/uv/python/cpython-3.10.17-linux-x86_64-gnu/lib/python3.10/contextlib.py", line 199, in aenter
kokoro-tts-1 | return await anext(self.gen)
kokoro-tts-1 | File "/app/api/src/main.py", line 65, in lifespan
kokoro-tts-1 | device, model, voicepack_count = await model_manager.initialize_with_warmup(
kokoro-tts-1 | File "/app/api/src/inference/model_manager.py", line 99, in initialize_with_warmup
kokoro-tts-1 | raise RuntimeError(f"Warmup failed: {e}")
kokoro-tts-1 | RuntimeError: Warmup failed: Failed to load model: Failed to load Kokoro model: CUDA error: no kernel image is available for execution on the device
kokoro-tts-1 | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
kokoro-tts-1 | For debugging consider passing CUDA_LAUNCH_BLOCKING=1
kokoro-tts-1 | Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
kokoro-tts-1 |
kokoro-tts-1 |
kokoro-tts-1 | ERROR: Application startup failed. Exiting.
kokoro-tts-1 exited with code 3

Branch / Deployment used
Default master branch, running locally, via docker compose (gpu) exactly as instructed in the README

Operating System
Windows, Docker version 28.0.4, NVIDIA GPU

Additional context
I had no issue running this when i was using my old 40xx gpu. Not sure if I'm missing something.