Repository navigation
Multiple simultaneous requests or cancelling requests mid-stream causes server to crash #337
Description
Describe the bug
The Docker containers (both ghcr.io/remsky/kokoro-fastapi-gpu:latest and ghcr.io/remsky/kokoro-fastapi-cpu:latest) crash without any error message when there are repeated requests canceled mid-stream.
Branch / Deployment used
Using the Docker images specified in the documentation.
- GPU: ghcr.io/remsky/kokoro-fastapi-gpu:latest
- CPU: ghcr.io/remsky/kokoro-fastapi-cpu:latest
Operating System
Crash occures in Docker when running both on Windows 10 and Ubuntu 20.04.6 LTS
Additional context
Here's a minimal Python script that can fairly consistently crash the Docker container (may require several runs). You may need to adjust the timeout value depending on if you are running the GPU or CPU container and how fast your machine is.
import requests
# may need to run this script multiple times before the docker container will crash
# crash happens more consistently on /dev/captioned_speech, but I've made it happen on /v1/audio/speech as well with more requests
url = 'http://localhost:8880/dev/captioned_speech'
url = 'http://localhost:8880/v1/audio/speech'
data = {
'input': 'Hello, this is a longer message to give the client time to cancel before the server finishes streaming the response. More text, more text, 12345456698732565213214, 12341458731278643782'
}
# if using GPU, timeout should be lower
timeout = 0.3 # GPU
timeout = 5.0 # CPU
for i in range(20):
try:
resp = requests.post(url, json=data, timeout=timeout)
print(resp)
except (requests.exceptions.ReadTimeout, requests.exceptions.ConnectionError):
print(f'timed out request {i+1}')
print('Done')Reacted by Morgan Marshall, Victor Sonck, xxnuo, Mike Iversen, lionel-rowe, carpdance and Wikijito7
Activity
[-]Canceling requests mid-stream causes Docker container to crash[/-][+]Cancelling requests mid-stream causes Docker container to crash[/+]MorganMarshall commentedon Jun 18, 2025
Experiencing same behavior
Mobious commentedon Jun 18, 2025
I found a similar crash also can occur anytime your are running multiple simultaneous requests. In fact this may be the actual source of the problem as I realize that request processing may be overlapping in my original example if the server does not immediately stop processing when the client closes the connection.
I also tested running the server with uv outside of a Docker container and found the crash happens there as well, so the issue does not appear to be related to Docker.
[-]Cancelling requests mid-stream causes Docker container to crash[/-][+]Multiple simultaneous requests or cancelling requests mid-stream causes Docker container to crash[/+][-]Multiple simultaneous requests or cancelling requests mid-stream causes Docker container to crash[/-][+]Multiple simultaneous requests or cancelling requests mid-stream causes server to crash[/+]fireblade2534 commentedon Jun 20, 2025
@Mobious can you test it on the latest version and tell me if that stil lhappens for you?
Mobious commentedon Jun 21, 2025
@fireblade2534
I just pulled the most recent GPU image ghcr.io/remsky/kokoro-fastapi-gpu:latest, sha256:ce12c3d6f0718d09188f3355b71973cf14c12791b7e17a358fdb9dd3a6faca33
Server still crashed right after running my script, but now prints "free(): invalid pointer" at the end. Here's the output:
EDIT: Ran the test a few more times. Sometimes it still crashes with no error message, and some times it crashes with "double free or corruption (!prev)" instead of "free(): invalid pointer".
fireblade2534 commentedon Jun 21, 2025
Do you get a different error when you run it on Windows vs Linux. Also which is was the test above run on
Mobious commentedon Jun 23, 2025
That test was on Windows. Tested the new image on Ubuntu. Crash still occurs, but was never able to get any error messages from the server.
EDIT: Tested the newest CPU container as well (sha256:916e50b8ef753a547dd2e4d52dac8c971b84f7bdefa6ecf6cdb1abf13f4ba1b7). Crash still occurs on both Windows and Ubuntu. Was able to get the "double free or corruption (!prev)" error on Ubuntu.
xxnuo commentedon Jul 25, 2025
Same issue
FreedomLiX commentedon Jul 25, 2025
SAME !!!!!!!!!!!!!!!!!!!!!!!!!!!!!
remsky commentedon Aug 24, 2026
Not able to reproduce this on current master; ran the script from the issue against both endpoints plus a heavier pass (3x120 requests, 8 concurrent, mixed formats, most cut mid-stream), with 1 and 2 workers, saw no crashes or errors in the container log.
Closing as not reproducible on current, but feel free to reopen with a trace from a current image if it recurs
fix(streaming): close audio writer and generator on client disconnect (…
remsky commentedon Sep 9, 2026
Was able to reproduce during other changes, should be resolved now with the above PR.
chore(release): 0.6.0 [skip ci]