Repository navigation
Auto unload on timeout - #506
Merged
Merged
Conversation
added 2 commits
August 17, 2026 07:14
brycehenson
marked this pull request as ready for review
August 17, 2026 08:05
Owner
|
Nice work, careful timer handling. Two things:
|
Centralize request timeout accounting into ModelManager.hold(). Use it for generate_from_phonemes(). Remove MODEL_AUTO_UNLOAD_ENABLED. Replace it with MODEL_AUTO_UNLOAD_TIMEOUT_SECONDS=0 is the default/off state.
Contributor
Author
|
Thanks, good points. Implemented 1 and 2. validation with unit tests:
Validation with integrations tests:
ps: Thank you for building and maintaining such a great tool ! |
added 5 commits
August 20, 2026 06:57
… config.py inline comment on allow_dev_unload
Contributor
Author
|
@remsky merged in master and fixed a small bug in the |
Owner
|
@brycehenson Nope, looks good thanks! Merged in now |
github-actions Bot
pushed a commit
to autoeditor-video-or-audio/kokoro-tts-server
that referenced
this pull request
Oct 1, 2026
# [0.6.0](v0.5.0...v0.6.0) (2026-10-01) ### Bug Fixes * **api:** turn off CORS credentials for starlette 1.x ([abcb9cf](abcb9cf)) * **audio:** drop WAV close()-time junk trailer that caused chunk-end click ([remsky#467](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/467)) ([7649d86](7649d86)), closes [remsky#463](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/463) * **audio:** implement audio file source swapping for stream playback and seeking ([remsky#475](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/475)) ([2153529](2153529)) * **audio:** improve error handling for aborted audio streams ([106cbd4](106cbd4)) * **audio:** read finalize buffer before close except for ogg, fixes flac tail loss ([remsky#497](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/497)) ([ef13810](ef13810)) * **changelog:** update model validation details for consistency and error handling ([3eb9606](3eb9606)) * **chunking:** replace regex sentence splitter with UAX [remsky#29](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/29) segmentation ([remsky#308](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/308), adapts [remsky#415](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/415)) ([remsky#520](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/520)) ([d2aae10](d2aae10)) * **ci:** run the test job with the cpu extra ([c6398b9](c6398b9)) * **debug-endpoints:** gate /debug/* introspection routes behind ENABLE_DEBUG_ENDPOINTS, documentation ([316aa67](316aa67)) * **dependencies:** update fastapi to 0.116.1 and pin starlette dependency to avoid CVE ([c2687ec](c2687ec)) * **dev-unload:** Prevent POST /dev/unload behind an opt-in setting ([remsky#483](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/483)) ([24ac58e](24ac58e)), closes [remsky#478](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/478) * **docker:** aligns cu128 base image with wheel (12.9.1 → 12.8.1), minimum toolkit with Blackwell (sm_120) support (CUDA 12.8) ([0d6b4d8](0d6b4d8)), closes [remsky#306](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/306) [remsky#320](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/320) [remsky#389](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/389) [remsky#443](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/443) [remsky#365](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/365) * **docker:** compose defaults to idempotent model download; harden download_model scripts, static release artifact link ([0cf239e](0cf239e)) * **docker:** honour HOST/PORT in entrypoint for IPv6 binds ([remsky#408](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/408)), download_model defaults; drop api dev mount cruft ([remsky#522](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/522)) ([21a78e0](21a78e0)) * **download_model:** checksum verification for downloaded model files, avoid clobbering pre-existing model files ([aab59af](aab59af)) * honour default_voice_code in _split_multi_voice (issue [remsky#514](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/514)) ([remsky#515](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/515)) ([5ba470b](5ba470b)) * int16 overflow in silence-trim, invalid-escape in URL regex ([64cb068](64cb068)) * **mappings:** /v1/audio/voices returns id/name objects ([remsky#462](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/462)) to align with OpenWebUI ([7bbd3a8](7bbd3a8)) * **mappings:** update voice mappings for consistency, add missing audio file for 'bf_isabella' ([remsky#480](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/480)) ([c5ccfa1](c5ccfa1)) * **normalizer:** 'F' is normalised to "farad" instead of "fahrenheit" ([remsky#526](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/526)) ([f901ed8](f901ed8)) * **normalizer:** anchor decimal regex to prevent backtracking on digit floods ([af6e444](af6e444)) * **normalizer:** expand apostrophe-less "'re" contractions ([7746fbe](7746fbe)), closes [remsky#488](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/488) [remsky#488](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/488) * **normalizer:** expand wh-word 're contractions espeak voices as -ray ([remsky#488](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/488)) ([f3a88ab](f3a88ab)) * **normalizer:** handle double dashes in text normalization and improve token processing ([4f826d6](4f826d6)) * **normalizer:** handle version-like numbers before NUMBER_PATTERN splits ([a736569](a736569)) * **normalizer:** move range sub before NUMBER_PATTERN to fix minus handling ([6c5773d](6c5773d)) * **normalizer:** treat underscore as a word boundary in caps detection ([123580c](123580c)) * **paths:** allow operator symlinks in search roots, keep traversal containment ([9dac665](9dac665)) * **paths:** harden _find_file to stay contained to its search roots ([0cb3a0d](0cb3a0d)) * phonemes default voice code ([remsky#516](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/516)) ([9138d89](9138d89)) * preserve paragraph breaks as sentence boundaries in normalizer ([remsky#525](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/525)) ([fbbaa75](fbbaa75)), closes [remsky#519](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/519) * prevent ReDoS in text normalizer (URL/email/acronym patterns) ([remsky#489](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/489)) ([5fb4753](5fb4753)) * **rocm:** persist MIOpen cache and add FIND_MODE=2 default +warmup script ([remsky#469](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/469)) ([8ce0821](8ce0821)) * **schema, pause overflow:** caps total pause duration, input length, volume_multiplier, validates lang_code ([remsky#511](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/511)) ([316409d](316409d)) * **server_addons:** run KOKORO_SYNC_ON_START sync inside the upstream lifespan ([39f1f6d](39f1f6d)), closes [#1](#1) * **ssml, web:** SSML root validation, rate validation in VoiceSelector ([5d54647](5d54647)) * **ssml:** keep block boundaries, drop <desc>, add provider corpus, translate documents with an xml prolog, refuse DTDs, 400 on deep nesting, cap nesting depth ([ffb4040](ffb4040)) * **streaming:** close audio writer and generator on client disconnect ([remsky#288](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/288), [remsky#337](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/337)) ([remsky#533](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/533)) ([cf8d71e](cf8d71e)) * **tests:** lazy import integration reqs, exclude on CI ([94f729f](94f729f)) * **tuning:** decode reference clips in a child process, serialized tune requests ([e498565](e498565)) * **uv direct run deps:** pyopenjtalk-plus on native Windows, avoids MSVC ([remsky#508](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/508)); pending swapover after multi-platform image testing ([8b0a51c](8b0a51c)) * **voices:** combine endpoint accepts weighted syntax, matching speech endpoints ([remsky#285](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/285)) ([remsky#512](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/512)) ([fdbcf96](fdbcf96)) * **web download:** voice-timestamp as download names, associated url handling ([5ba5190](5ba5190)) * **web:** add Firefox-safe audio playback fallback when MSE MP3 is unsupported ([5e55b5c](5e55b5c)) * **web:** anchor the cast menu to its Options button, float it in the top layer ([86662e3](86662e3)) * **web:** assorted UI tweaks, responsiveness, animation ([91cd44d](91cd44d)) * **web:** drop auto-insert voice tag, enter key to save name, update related tests ([1b66dde](1b66dde)) * **web:** filtering, edit mode opens name as well ([db46865](db46865)) * **web:** firefox/supported playback handling and msging, bmac link, waveform bugfix ([a10fc75](a10fc75)) * **web:** pin playbar to bottom on slim view, responsive fixes, redesign tweaks ([328b816](328b816)) * **web:** playback slider step size; smoothness ([89850a9](89850a9)) * **webui:** bounded buffer to resolve 10 minute crashout, player state short ciruit on no-up updates to preserve responsiveness on long sessions. ([remsky#471](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/471)) ([3f64dc2](3f64dc2)) * **web:** voice search dropdown keys, clearable alias name, focus restore guard ([4aa5f1a](4aa5f1a)) * word timestamps for non-English (espeak) voices in captioned_speech ([remsky#484](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/484)) ([c0acb91](c0acb91)), closes [remsky#331](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/331) ### Features * **api, web ui:** include voice grades on the voice listing endpoint ([remsky#517](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/517)) ([5fb71ea](5fb71ea)) * **api:** add POST /dev/unload to release model from GPU VRAM ([remsky#474](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/474)) ([ff6efaf](ff6efaf)), closes [remsky#473](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/473) * **api:** optional model auto-unload after idle timeout ([remsky#506](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/506)) ([8307b0e](8307b0e)) * **benchmarks:** add model unload benchmarking script and update README with VRAM reclaim details ([6b58b53](6b58b53)) * **gpu:** adds clear cu128 opt-in via compose, setup for Blackwell ([f08279f](f08279f)) * **gpu:** adds clear cu128 opt-in via compose, setup for Blackwell / RTX 50-series release image ([65ef3b8](65ef3b8)) * improve voice selector and tag handling ([e65f1ba](e65f1ba)) * **multi-speaker:** add inline multi-speaker support, voice alias mappings, read-along scrobbing in web ui ([remsky#502](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/502)) ([5775958](5775958)), closes [remsky#272](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/272) * **normalizer:** caps_normalization option, spoken phone numbers, hertz plurals, drop the 'S pass, emoji tag flags and joiners, web toggles ([remsky#353](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/353)) ([794a27b](794a27b)) * **normalizer:** opt-in remove_emoji, 400 before streaming on unspeakable input ([remsky#353](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/353)) ([c3cb020](c3cb020)) * **player:** duration handling and update UI controls for audio playback ([a559254](a559254)) * **ssml, cleanup:** ENABLE_SSML kill switch, drop legacy settings, documentation of config variables) ([60d6862](60d6862)) * **ssml, speed/rate tag:** initial crack at integrating speed param per voice, ssml compliance ([8e4810a](8e4810a)) * **ssml:** adjust rate tag semantics to scale speaking pace and introduce baserate tag ([536bc76](536bc76)) * **ssml:** dedicated ssml router, shared rate clamp contract, alias rate visible + synced in web editor ([e282b40](e282b40)) * **ssml:** ssml flag on the speech endpoints, one call to translate and synthesize ([719f47a](719f47a)) * **time-to-first-audio:** lazy phonemization in get_sentence_info ([remsky#513](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/513)) ([61be98d](61be98d)) * **voice-clone-tune:** integrate Inno tuner, improve web ui ([e3049b2](e3049b2)) * **web,accessibility:** enhance keyboard navigation and ARIA attributes for improved accessibility ([8322a9b](8322a9b)) * **web:** add cast pins, resize insert button and ordering ([ffa52e1](ffa52e1)) * **web:** add duplicate to options, simplify suggested cast name rate to 3 digits ([fe9c7ae](fe9c7ae)) * **web:** add text normalization toggle to web UI settings ([remsky#523](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/523)) ([e57216c](e57216c)) * **web:** improve styling, responsiveness ([c53efd5](c53efd5)) * **web:** normalization options menu in player ([de18072](de18072)) * **web:** seamless auto swap-to-file ([c82946a](c82946a)) * **web:** show server version badge in the UI footer ([8e1bafe](8e1bafe)) * **web:** swap to the finished file as soon as it exists, allow minor skip on swap/ ([8cb69ac](8cb69ac)) * **web:** voice list sorted by grade then name, DEFAULT_VOICE honoured ([5fec3ad](5fec3ad)) ### Performance Improvements * **rocm:** disable MIOpen by default, add ENABLE_MIOPEN opt-in ([remsky#518](https://github.com/autoeditor-video-or-audio/kokoro-tts-server/issues/518)) ([5fc3ba3](5fc3ba3))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
adds unload after a period of inactivity.
request for feeback
Would you prefer a different flag name ?
TODO