Repository navigation
Studio: pick the quant of a GGUF dictation model in Voice settings - #119
Open
mahiatlinux wants to merge 13 commits into
Open
mahiatlinux wants to merge 13 commits into
mahiatlinux wants to merge 13 commits into
Conversation
Settings > Voice offered 6 of the 18 audio.cpp ASR folders the Transcribe page runs, and a saved folder id was reset to the default on reload. The picker now derives its GGUF rows from the same catalog Transcribe uses, keeps the existing keys, adds a Moonshine medium key, accepts folder ids, reads folder sizes from the cached variant listing, marks downloaded models, finds listed models by name in search, and adds Danish for Hviske.
A package-folder dictation model such as GigaAM ships several quants, but Settings > Voice could only run the folder's default one. The voice store now remembers the picked quant beside the model, like Transcribe does, and a Select under the model picker offers the folder's quants with sizes and an on-device dot. The quant travels with every Settings dictation path: load, download, raw transcription, chat audio upload, the Audio page's reference transcription and the download prompt, which now shows the quant's size. Offline, a picked quant that is not on disk is reported as not downloaded instead of an unknown variant, so the download prompt appears rather than an error.
…Select A stopped or failed download of a second quant read as complete, because a row counts as downloaded once any quant is cached; the mirror now checks cancel and error first. The upload dialog said Ready for a pinned quant that was not on disk, and the download prompt took the quant from live settings instead of the request; both now use the pinned quant. Unpinned, the quant Select shows what a bare load runs: the resident quant (mapping the cache's loose key), else the first cached one.
…dio-audio-stt-settings-quant # Conflicts: # studio/backend/core/inference/audio_cpp_models.py # studio/backend/tests/test_audio_cpp_umbrella_catalog.py # studio/frontend/src/features/settings/stores/stt-model-catalog.ts # studio/frontend/src/features/settings/tabs/voice-tab.tsx # studio/frontend/tests/audio-cpp-catalog.test.ts
Without a browser speech service, Studio switched dictation to local as soon as any quant of the model was on disk, so the next recording failed on a pinned quant that was not. The switch now checks that quant first, through the same helper upload readiness uses. A download's warm-up also remembers which quant it fetched, so picking another quant meanwhile no longer loads one that is not on disk.
…d quants With no quant pinned, the quant Select showed the resident quant as its value, so picking that same quant changed nothing and an idle unload later fell back to another one. The Select now holds the pin and shows the running quant as its placeholder, so any pick is saved. A quant the listing leaves out (an offline, cache-only listing) now counts as not downloaded, and a download adopted without a known quant only warms an unpinned row.
An offline listing keys cached files by what tells them apart, such as ctc/F16 for v3-ctc/F16, so treating a key it leaves out as not downloaded showed a cached GigaAM quant as missing and blocked offline dictation. A listing now marks the pinned quant missing only through its own row; anything else defers to the row status, and the backend's not-downloaded reply still raises the download prompt.
With another quant of the row cached, a failed download of the pinned quant read as not downloaded, with a fresh Download button, hiding the error. It now shows the same failed state and Retry a row gets.
Owner
Author
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…ll picked The Voice tab loads the model when a download it watched finishes, but it loaded the quant picked now, so switching quant during a download tried to load one that was not on disk. It now remembers the quant it started and loads only that one; a download it adopted loads only when no quant is pinned.
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Owner
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A package-folder dictation model such as GigaAM ships several quants, but Settings > Voice could only run the folder's default one. Voice settings now has a Quantization select under the model picker when the selected folder has more than one quant. It lists each quant with its size and an on-device dot, and picking one unloads the current model and loads the new quant.
Changes
sttGgufVariantnext tosttModel. It is cleared when the model changes (including the fallback to the default model on a language switch) and only rehydrated when the saved model is still the selected one.sttModelVariant()andwithSttVariant()instt-model-catalog.tsapply the quant only to package-folder rows. Saved keys and the other engines ignore it./transcribe/rawgetsrow:variant, so a cold sidecar does not fall back to the default quant), chat audio upload (pinned in the upload snapshot), the Audio page's reference transcription, and the download prompt. The download prompt now shows the quant's size, read from the cached GGUF listing.resolve_audio_cpp_stt_modelraisesSttModelNotDownloadedErrorinstead ofSttModelIdErrorwhen, offline, a picked quant is not on disk but its row is a runnable ASR model. Callers then show the download prompt instead of an unknown-variant error. A folder for another task (for example Kokoro with a quant) is still rejected as a wrong model.sttQuantLabelandsttQuantDescriptionin all 14 locales.Testing
npm test: 10756 passed. New cases intests/audio-cpp-catalog.test.tscover which rows take a saved quant, clearing and rehydrating the quant with its model, the quant reaching every Settings dictation path, and when the Select appears.npm run typecheck, stricti18n:check,catalog:checkandnpm run buildpass.test_audio_cpp_models,test_stt_audiocpp_details,test_audio_cpp_umbrella_catalog,test_openai_audio_transcriptions_route,test_stt_sidecarandtest_stt_transcription_cancellation.test_audio_cpp_modelsadds the offline not-downloaded quant case and the wrong-task case.POST /api/inference/audio/transcribe/rawwith GigaAM, on CPU and on an idle GPU::v3-ctc/F16loads that quant (loaded_variantmatches), the bare row keeps the resident quant,:multilingual-ctc/F16switches and returns the LibriSpeech transcript, and a quant that is not on disk returns 409 not downloaded (422 before the backend change).Mirror of unslothai/unsloth#12900 by @Etherll, opened for review only.
Do not merge here. Fixes agreed on this mirror must be applied to the source PR.