Repository navigation
fix(chat): use an idle timeout for streaming replies - #2651
Conversation
Reset the 60-second idle timer whenever response bytes arrive, including keepalives and hidden reasoning. Allow local and self-hosted models up to 10 minutes to send their first bytes. Add regression coverage for long replies, slow prefill, stalled streams, and cancellation. Fixes OpenWhispr#2648
The long-reply and keepalive tests ran 225 s, inside the 600 s local/LAN allowance before the first bytes, so they passed with the idle reset removed.
…cond use The cancel test's name promised a cleared timer it could not observe, and the long-task constant's comment only named note formatting.
The built-in server loads its model before the request's timer starts.
boseq
left a comment
There was a problem hiding this comment.
Thanks @atty57, this fixes #2648. I tested it on macOS against a mock local server: a local or LAN reply that streams past 60 s and a 90 s wait for the first token both finish now, and a stalled stream still errors 60 s after its last data.
I pushed a merge of main, made the long-stream tests run past the 600 s allowance so they fail without the fix, and fixed the wording in one test name and one comment.
Happy to help 🤗 |
🟡 Take it, then run the checks below. 70 files conflict, and upstream changed code behind 21 of our features. 129 upstream changes · 77 matter on Windows · 2 improvements · 51 fixes Conflicts: CLAUDE.md: OpenWhispr#1916 OpenWhispr#2520 OpenWhispr#2683 OpenWhispr#2516 OpenWhispr#2685 OpenWhispr#2643 OpenWhispr#2717 OpenWhispr#2694 OpenWhispr#2792 OpenWhispr#2622 OpenWhispr#2624 OpenWhispr#2630 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2098 OpenWhispr#2815 OpenWhispr#2812 OpenWhispr#2822 OpenWhispr#2817 OpenWhispr#2886 OpenWhispr#2880 OpenWhispr#2585 OpenWhispr#2958 OpenWhispr#2824 OpenWhispr#2251 OpenWhispr#2842 OpenWhispr#2934 main.js: OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2565 OpenWhispr#1982 OpenWhispr#2886 OpenWhispr#2989 OpenWhispr#2842 preload.js: OpenWhispr#2604 OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2517 OpenWhispr#2565 OpenWhispr#2654 OpenWhispr#2822 OpenWhispr#2879 OpenWhispr#2880 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010 src/App.jsx: OpenWhispr#2693 OpenWhispr#2988 OpenWhispr#2842 src/components/CleanupFailureToastListener.tsx: OpenWhispr#2634 src/components/CommandSearch.tsx: OpenWhispr#2880 OpenWhispr#3010 src/components/ControlPanel.tsx: OpenWhispr#2718 OpenWhispr#2694 OpenWhispr#2876 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#2990 OpenWhispr#3012 src/components/ControlPanelSidebar.tsx: OpenWhispr#2876 src/components/DictionaryView.tsx: OpenWhispr#2855 src/components/MeetingRecordingMount.tsx: OpenWhispr#2988 OpenWhispr#2989 src/components/OpenAICompatiblePanel.tsx: OpenWhispr#2652 src/components/ReasoningModelSelector.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2652 OpenWhispr#2958 src/components/SettingsModal.tsx: OpenWhispr#2654 OpenWhispr#2670 src/components/SettingsPage.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2643 OpenWhispr#2559 OpenWhispr#2654 OpenWhispr#2652 OpenWhispr#2098 OpenWhispr#2876 OpenWhispr#2989 src/components/TranscriptionModelPicker.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2652 OpenWhispr#1982 OpenWhispr#2958 src/components/chat/useChatStreaming.ts: OpenWhispr#2517 OpenWhispr#2694 OpenWhispr#2630 OpenWhispr#2655 OpenWhispr#2842 OpenWhispr#2855 src/components/dictionary/DictionaryWordList.tsx: OpenWhispr#2855 src/components/icons/primitives.tsx: OpenWhispr#2654 OpenWhispr#2876 OpenWhispr#2824 OpenWhispr#2855 src/components/notes/UploadAudioView.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824 src/components/settings/InferenceConfigEditor.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2694 OpenWhispr#2790 OpenWhispr#2670 OpenWhispr#2439 src/components/settings/MeetingSettings.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2989 src/components/settings/UploadSettings.tsx: OpenWhispr#2989 src/components/ui/ApiKeyInput.tsx: OpenWhispr#2652 src/components/ui/PromptStudio.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2694 OpenWhispr#2855 src/components/ui/SettingsSection.tsx: OpenWhispr#2824 src/components/ui/TranscriptionItem.tsx: OpenWhispr#2678 OpenWhispr#2824 src/config/prompts.ts: OpenWhispr#2517 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2842 src/config/prompts/index.ts: OpenWhispr#2855 src/helpers/audioManager.js: OpenWhispr#2683 OpenWhispr#2434 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2436 OpenWhispr#2586 OpenWhispr#2713 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2694 OpenWhispr#2559 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2989 OpenWhispr#2855 src/helpers/database.js: OpenWhispr#2604 OpenWhispr#2822 OpenWhispr#2817 OpenWhispr#2879 OpenWhispr#2827 OpenWhispr#2880 OpenWhispr#2926 OpenWhispr#2929 OpenWhispr#2921 OpenWhispr#2824 OpenWhispr#2933 OpenWhispr#2855 OpenWhispr#3010 OpenWhispr#2990 src/helpers/debugLogger.js: OpenWhispr#2565 OpenWhispr#2818 src/helpers/environment.js: OpenWhispr#2585 src/helpers/hotkeyManager.js: OpenWhispr#2622 OpenWhispr#2886 OpenWhispr#2742 src/helpers/ipcHandlers.js: OpenWhispr#2604 OpenWhispr#2100 OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2436 OpenWhispr#2517 OpenWhispr#2643 OpenWhispr#2644 OpenWhispr#2565 OpenWhispr#2718 OpenWhispr#2778 OpenWhispr#2559 OpenWhispr#2630 OpenWhispr#2791 OpenWhispr#2803 OpenWhispr#2654 OpenWhispr#2652 OpenWhispr#2098 OpenWhispr#2818 OpenWhispr#2822 OpenWhispr#2879 OpenWhispr#1982 OpenWhispr#2886 OpenWhispr#2880 OpenWhispr#2585 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010 src/helpers/openaiRealtimeStreaming.js: OpenWhispr#2100 OpenWhispr#2767 OpenWhispr#2989 src/helpers/orukeetStreaming.js: OpenWhispr#2436 src/helpers/providerConnectionTest.js: OpenWhispr#2652 src/helpers/selectionManager.js: OpenWhispr#2643 OpenWhispr#2717 OpenWhispr#2654 OpenWhispr#2842 src/helpers/transcriptionRoute.ts: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2958 OpenWhispr#2989 src/helpers/whisperServer.js: OpenWhispr#1982 OpenWhispr#2855 src/helpers/windowManager.js: OpenWhispr#2632 OpenWhispr#2644 OpenWhispr#2565 OpenWhispr#2624 OpenWhispr#2900 OpenWhispr#2886 OpenWhispr#2988 OpenWhispr#2842 src/helpers/windowsKeyManager.js: OpenWhispr#2886 src/hooks/useAudioRecording.js: OpenWhispr#2643 OpenWhispr#2644 OpenWhispr#2693 OpenWhispr#2634 OpenWhispr#2654 OpenWhispr#2655 src/services/BaseReasoningService.ts: OpenWhispr#2694 OpenWhispr#2654 OpenWhispr#2670 OpenWhispr#2855 src/services/ReasoningService.ts: OpenWhispr#2651 OpenWhispr#2694 OpenWhispr#2630 OpenWhispr#2654 OpenWhispr#2670 OpenWhispr#2845 OpenWhispr#2839 OpenWhispr#2958 OpenWhispr#2842 src/services/ai/inferenceProviders/anthropic.ts: OpenWhispr#2654 src/services/ai/inferenceProviders/enterprise.ts: OpenWhispr#2654 src/services/ai/inferenceProviders/gemini.ts: OpenWhispr#2694 OpenWhispr#2654 OpenWhispr#2958 src/services/ai/inferenceProviders/local.ts: OpenWhispr#2654 src/services/ai/inferenceProviders/openai.ts: OpenWhispr#2019 OpenWhispr#2694 OpenWhispr#2654 OpenWhispr#2840 OpenWhispr#2670 src/services/ai/inferenceProviders/tinfoil.ts: OpenWhispr#2654 OpenWhispr#2670 src/services/ai/inferenceProviders/types.ts: OpenWhispr#2654 src/stores/cleanupFailureStore.ts: OpenWhispr#2634 src/stores/settingsStore.ts: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2713 OpenWhispr#2694 OpenWhispr#2790 OpenWhispr#2425 OpenWhispr#2654 OpenWhispr#2989 OpenWhispr#2855 src/types/electron.ts: OpenWhispr#2604 OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2436 OpenWhispr#2517 OpenWhispr#2643 OpenWhispr#2565 OpenWhispr#2718 OpenWhispr#2778 OpenWhispr#2630 OpenWhispr#2791 OpenWhispr#2654 OpenWhispr#2652 OpenWhispr#2098 OpenWhispr#2822 OpenWhispr#2879 OpenWhispr#1982 OpenWhispr#2880 OpenWhispr#2921 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010 src/utils/dictionaryWordKey.js: OpenWhispr#2855 src/utils/snippets.ts: OpenWhispr#2855 test/components/directionalContentPolicy.test.js: OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010 OpenWhispr#2990 test/components/fieldDirectionPolicy.test.js: OpenWhispr#2652 OpenWhispr#2822 OpenWhispr#2809 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 test/helpers/cleanupFailureToast.test.js: OpenWhispr#2634 test/helpers/harness/audioManager.js: OpenWhispr#2810 test/helpers/policyRules.test.js: OpenWhispr#2988 test/helpers/retryTranscriptionHandler.test.js: OpenWhispr#2718 OpenWhispr#2988 OpenWhispr#2989 test/helpers/transcriptionFailureOutcome.test.js: OpenWhispr#2678 test/helpers/trayQuickActions.test.js: OpenWhispr#2988 test/helpers/useAudioRecordingCleanupFallback.test.js: OpenWhispr#2634 test/helpers/useChatStreamingTools.test.js: OpenWhispr#2517 OpenWhispr#2630 OpenWhispr#2810 OpenWhispr#2655 test/helpers/voicePillPresentation.test.js: OpenWhispr#2693 test/helpers/whisperGpuUpgradeReset.test.js: OpenWhispr#2585 test/stores/policyEffectiveVisionOverride.test.js: OpenWhispr#2958 Our features upstream touched: Self-hosting and build info (our 54d560e, e83ad4e): OpenWhispr#2565 OpenWhispr#2818 check: node scripts/selfhost.js check, then ship: the installed build-info.json names the commit and Settings > System > This build shows it Logs you control (our 2d60c94, 4200336, 13fbdfe): OpenWhispr#2565 OpenWhispr#2818 OpenWhispr#2585 check: Storage > Logs: switch Problems only / Everything / Off and dictate; the level follows and Clear logs works. node --import tsx --test test/helpers/logTextCanary.test.js: no dictated text in any log line, upstream's new ones included Bluetooth start cue (our 2582c67): OpenWhispr#2693 check: With Bluetooth headphones idle, start a dictation: the start cue is heard in full Floating icon placement (our 79c292d): OpenWhispr#2624 OpenWhispr#2855 check: Settings > Floating Icon: lock to a monitor and a grid spot; dictate with the mouse on another monitor, the pill stays put; sleep a monitor, it comes back Recording look, edge strip and tray badge (our bf28e6f, c1189f1, 5b6f156, bef2f90): OpenWhispr#2644 OpenWhispr#2693 OpenWhispr#2624 OpenWhispr#2654 OpenWhispr#2988 OpenWhispr#2842 OpenWhispr#2855 check: Beacon red, Full frame and tray badge on: dictate; the pill, frame and tray dot show red while recording and go at stop and cancel; REC shows the time OpenWhispr account & Cloud switch (our 89be0f8): OpenWhispr#2517 OpenWhispr#2824 OpenWhispr#2842 check: Privacy & Data: turn Cloud off; no request reaches openwhispr.com (log shows [cloud-off] blocked), no account surfaces show, local dictation works GitHub Copilot provider (our 5a317eb, 655b9a1, 5a45572): OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2634 OpenWhispr#2790 OpenWhispr#2845 OpenWhispr#2988 OpenWhispr#2842 check: Cleanup on GitHub Copilot: dictate twice; the log shows 'Copilot runtime prewarmed' and 'Copilot request finished' under 6 s, the cleaned text pastes Glossary corrections (our b6c9ce0): OpenWhispr#2436 OpenWhispr#2586 OpenWhispr#2634 OpenWhispr#2718 OpenWhispr#2678 OpenWhispr#2694 OpenWhispr#2559 OpenWhispr#2654 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 check: Add a glossary rule (e.g. 'little body' -> 'Little Buddy'), dictate the spelling; the pasted text and View steps show the rule fired Prompt library and test set (our 4e0739a, 8479b17): OpenWhispr#2694 OpenWhispr#2790 OpenWhispr#2654 OpenWhispr#2842 OpenWhispr#2933 OpenWhispr#2855 check: Prompt Studio: switch prompts, edit and save a draft, run the test set A vs B; the prompt in use is what a dictation sends Step history and the History view (our 97f52cc, 4999528, 042d889, 616ead3, 3184e26, 368e6ac): OpenWhispr#2634 OpenWhispr#2718 OpenWhispr#2622 OpenWhispr#2654 OpenWhispr#2742 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2933 OpenWhispr#3010 check: Dictate, open History: the row shows its steps, 👎 opens the review side by side with the player, Re-run cleanup replaces the text, the review hotkey opens the last dictation Dictation styles and style hotkeys (our b7314cc, 195aaec): OpenWhispr#2656 OpenWhispr#2586 OpenWhispr#2693 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2622 OpenWhispr#2886 OpenWhispr#2855 check: Turn a style on with a hotkey; dictate with it: the badge shows, the text is restyled, View steps shows the style step Settings layer, pages and search (our 5bdceae, 8f19d0d, e265357, e44c006, 91a2e0e, cf152fb): OpenWhispr#2652 check: Open Settings: every page in the menu renders, search ('hotkey', 'logs') lands on the row, Esc leaves, the view underneath keeps running AI Connections, model pickers and connection checks (our b792f1c, 6501a60, b1b4773, a44dac3, 6f43147, 33a15ca): OpenWhispr#2694 OpenWhispr#2652 OpenWhispr#2585 check: AI Connections: every saved connection checks green, the pickers on Cleanup and Recognition list each connection's models, a pick routes the next dictation there Storage page and recording quality (our 53f63d6): OpenWhispr#2656 check: Settings > Storage: sizes measure, Recordings 'Smaller' records at 32 kbps (file size), Clear app cache works, Privacy & Data's summaries link here Transcription request deadline and retry (our 323f0d6): OpenWhispr#2656 OpenWhispr#2693 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2988 check: Point the custom endpoint at a server that never answers: 'Taking longer than usual · Esc to cancel' after 8 s, Esc cancels, Retry re-sends the recording Azure Speech MAI-Transcribe-2 (our 85a04c3): OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2634 OpenWhispr#2845 OpenWhispr#2988 OpenWhispr#2855 check: Recognition on Azure Speech with a Foundry endpoint: dictate, it transcribes; Re-transcribe from History works Dictionary v2 (our a16ea57, 6ca5f92, d41bdf7, 9d098d6, 0fd61bf, 36d7f41, be53cf7): OpenWhispr#2436 OpenWhispr#2586 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2654 OpenWhispr#2879 OpenWhispr#2880 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2933 OpenWhispr#2855 OpenWhispr#3010 check: Dictionary: usage counts, coverage dots and pin order show; Tidy up applies and Undo restores; a learned word appears as a suggestion; Re-transcribe sends the hint list Graphics mode and tray warning (our 50c8790, b7c580b): OpenWhispr#2988 OpenWhispr#2842 check: Start with --use-gl=desktop: 'Graphics mode' warns fallback, the tray shows the amber dot and Restart; a normal start logs hardware Quiet empty dictations (our bc86fe7): OpenWhispr#2656 OpenWhispr#2693 OpenWhispr#2678 OpenWhispr#2842 check: Start a dictation and say nothing for 2 s: the pill spins down red, no error card, no History row Upstream strings we reworded (all locales): OpenWhispr#2643 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2822 OpenWhispr#2958 OpenWhispr#2824 OpenWhispr#2855 check: npm run i18n:check passes; the Settings pages and pill read the same in English after the merge CLAUDE.md sections we rewrote: OpenWhispr#2824 check: CLAUDE.md keeps upstream's new sections and ours; nothing describes code that no longer exists Tracked topics upstream touched: Workspaces, teams and guests in Settings: OpenWhispr#2876 OpenWhispr#2921 OpenWhispr#3010 Full review: C:\Git\Notes\obsidian\Work\Repo Specific Knowledge 🤔\OpenWhispr\Upstream reviews\2026-11-10 upstream b9e9ebe.md Conflicts resolved to keep upstream v1.11.0 behaviour and every fork feature; per-file notes and the decisions taken are in the session brief files/upstream-merge-v1.11.0-brief.md and the resolver's report. Co-authored-by: Copilot App <[email protected]> Copilot-Session: cb886a50-814f-49fb-938d-e5716305752a
Reset the 60-second idle timer whenever response bytes arrive, including keepalives and hidden reasoning. Allow local and self-hosted models up to 10 minutes to send their first bytes.
Add regression coverage for long replies, slow prefill, stalled streams, and cancellation.
Fixes #2648