Skip to content

fix(chat): use an idle timeout for streaming replies - #2651

Merged
boseq merged 5 commits into
OpenWhispr:mainfrom
atty57:fix/chat-streaming-idle-timeout
Oct 15, 2026
Merged

boseq merged 5 commits into
OpenWhispr:mainfrom
atty57:fix/chat-streaming-idle-timeout

Conversation

@atty57

@atty57 atty57 commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Reset the 60-second idle timer whenever response bytes arrive, including keepalives and hidden reasoning. Allow local and self-hosted models up to 10 minutes to send their first bytes.

Add regression coverage for long replies, slow prefill, stalled streams, and cancellation.

Fixes #2648

atty57 and others added 5 commits October 13, 2026 17:34
Reset the 60-second idle timer whenever response bytes arrive, including keepalives and hidden reasoning. Allow local and self-hosted models up to 10 minutes to send their first bytes.

Add regression coverage for long replies, slow prefill, stalled streams, and cancellation.

Fixes OpenWhispr#2648
The long-reply and keepalive tests ran 225 s, inside the 600 s local/LAN
allowance before the first bytes, so they passed with the idle reset removed.
…cond use

The cancel test's name promised a cleared timer it could not observe, and the
long-task constant's comment only named note formatting.
The built-in server loads its model before the request's timer starts.

@boseq boseq left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @atty57, this fixes #2648. I tested it on macOS against a mock local server: a local or LAN reply that streams past 60 s and a 90 s wait for the first token both finish now, and a stalled stream still errors 60 s after its last data.

I pushed a merge of main, made the long-stream tests run past the 600 s allowance so they fail without the fix, and fixed the wording in one test name and one comment.

@boseq
boseq merged commit 4e15c38 into OpenWhispr:main Oct 15, 2026
19 checks passed
@atty57

atty57 commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @atty57, this fixes #2648. I tested it on macOS against a mock local server: a local or LAN reply that streams past 60 s and a 90 s wait for the first token both finish now, and a stalled stream still errors 60 s after its last data.

I pushed a merge of main, made the long-stream tests run past the 600 s allowance so they fail without the fix, and fixed the wording in one test name and one comment.

Happy to help 🤗

spectre009 pushed a commit to spectre009/openwhispr that referenced this pull request Nov 11, 2026
🟡 Take it, then run the checks below. 70 files conflict, and upstream changed code behind 21 of our features.
129 upstream changes · 77 matter on Windows · 2 improvements · 51 fixes

Conflicts:
  CLAUDE.md: OpenWhispr#1916 OpenWhispr#2520 OpenWhispr#2683 OpenWhispr#2516 OpenWhispr#2685 OpenWhispr#2643 OpenWhispr#2717 OpenWhispr#2694 OpenWhispr#2792 OpenWhispr#2622 OpenWhispr#2624 OpenWhispr#2630 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2098 OpenWhispr#2815 OpenWhispr#2812 OpenWhispr#2822 OpenWhispr#2817 OpenWhispr#2886 OpenWhispr#2880 OpenWhispr#2585 OpenWhispr#2958 OpenWhispr#2824 OpenWhispr#2251 OpenWhispr#2842 OpenWhispr#2934
  main.js: OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2565 OpenWhispr#1982 OpenWhispr#2886 OpenWhispr#2989 OpenWhispr#2842
  preload.js: OpenWhispr#2604 OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2517 OpenWhispr#2565 OpenWhispr#2654 OpenWhispr#2822 OpenWhispr#2879 OpenWhispr#2880 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010
  src/App.jsx: OpenWhispr#2693 OpenWhispr#2988 OpenWhispr#2842
  src/components/CleanupFailureToastListener.tsx: OpenWhispr#2634
  src/components/CommandSearch.tsx: OpenWhispr#2880 OpenWhispr#3010
  src/components/ControlPanel.tsx: OpenWhispr#2718 OpenWhispr#2694 OpenWhispr#2876 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#2990 OpenWhispr#3012
  src/components/ControlPanelSidebar.tsx: OpenWhispr#2876
  src/components/DictionaryView.tsx: OpenWhispr#2855
  src/components/MeetingRecordingMount.tsx: OpenWhispr#2988 OpenWhispr#2989
  src/components/OpenAICompatiblePanel.tsx: OpenWhispr#2652
  src/components/ReasoningModelSelector.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2652 OpenWhispr#2958
  src/components/SettingsModal.tsx: OpenWhispr#2654 OpenWhispr#2670
  src/components/SettingsPage.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2643 OpenWhispr#2559 OpenWhispr#2654 OpenWhispr#2652 OpenWhispr#2098 OpenWhispr#2876 OpenWhispr#2989
  src/components/TranscriptionModelPicker.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2652 OpenWhispr#1982 OpenWhispr#2958
  src/components/chat/useChatStreaming.ts: OpenWhispr#2517 OpenWhispr#2694 OpenWhispr#2630 OpenWhispr#2655 OpenWhispr#2842 OpenWhispr#2855
  src/components/dictionary/DictionaryWordList.tsx: OpenWhispr#2855
  src/components/icons/primitives.tsx: OpenWhispr#2654 OpenWhispr#2876 OpenWhispr#2824 OpenWhispr#2855
  src/components/notes/UploadAudioView.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824
  src/components/settings/InferenceConfigEditor.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2694 OpenWhispr#2790 OpenWhispr#2670 OpenWhispr#2439
  src/components/settings/MeetingSettings.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2989
  src/components/settings/UploadSettings.tsx: OpenWhispr#2989
  src/components/ui/ApiKeyInput.tsx: OpenWhispr#2652
  src/components/ui/PromptStudio.tsx: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2694 OpenWhispr#2855
  src/components/ui/SettingsSection.tsx: OpenWhispr#2824
  src/components/ui/TranscriptionItem.tsx: OpenWhispr#2678 OpenWhispr#2824
  src/config/prompts.ts: OpenWhispr#2517 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2842
  src/config/prompts/index.ts: OpenWhispr#2855
  src/helpers/audioManager.js: OpenWhispr#2683 OpenWhispr#2434 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2436 OpenWhispr#2586 OpenWhispr#2713 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2694 OpenWhispr#2559 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2989 OpenWhispr#2855
  src/helpers/database.js: OpenWhispr#2604 OpenWhispr#2822 OpenWhispr#2817 OpenWhispr#2879 OpenWhispr#2827 OpenWhispr#2880 OpenWhispr#2926 OpenWhispr#2929 OpenWhispr#2921 OpenWhispr#2824 OpenWhispr#2933 OpenWhispr#2855 OpenWhispr#3010 OpenWhispr#2990
  src/helpers/debugLogger.js: OpenWhispr#2565 OpenWhispr#2818
  src/helpers/environment.js: OpenWhispr#2585
  src/helpers/hotkeyManager.js: OpenWhispr#2622 OpenWhispr#2886 OpenWhispr#2742
  src/helpers/ipcHandlers.js: OpenWhispr#2604 OpenWhispr#2100 OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2436 OpenWhispr#2517 OpenWhispr#2643 OpenWhispr#2644 OpenWhispr#2565 OpenWhispr#2718 OpenWhispr#2778 OpenWhispr#2559 OpenWhispr#2630 OpenWhispr#2791 OpenWhispr#2803 OpenWhispr#2654 OpenWhispr#2652 OpenWhispr#2098 OpenWhispr#2818 OpenWhispr#2822 OpenWhispr#2879 OpenWhispr#1982 OpenWhispr#2886 OpenWhispr#2880 OpenWhispr#2585 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010
  src/helpers/openaiRealtimeStreaming.js: OpenWhispr#2100 OpenWhispr#2767 OpenWhispr#2989
  src/helpers/orukeetStreaming.js: OpenWhispr#2436
  src/helpers/providerConnectionTest.js: OpenWhispr#2652
  src/helpers/selectionManager.js: OpenWhispr#2643 OpenWhispr#2717 OpenWhispr#2654 OpenWhispr#2842
  src/helpers/transcriptionRoute.ts: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2958 OpenWhispr#2989
  src/helpers/whisperServer.js: OpenWhispr#1982 OpenWhispr#2855
  src/helpers/windowManager.js: OpenWhispr#2632 OpenWhispr#2644 OpenWhispr#2565 OpenWhispr#2624 OpenWhispr#2900 OpenWhispr#2886 OpenWhispr#2988 OpenWhispr#2842
  src/helpers/windowsKeyManager.js: OpenWhispr#2886
  src/hooks/useAudioRecording.js: OpenWhispr#2643 OpenWhispr#2644 OpenWhispr#2693 OpenWhispr#2634 OpenWhispr#2654 OpenWhispr#2655
  src/services/BaseReasoningService.ts: OpenWhispr#2694 OpenWhispr#2654 OpenWhispr#2670 OpenWhispr#2855
  src/services/ReasoningService.ts: OpenWhispr#2651 OpenWhispr#2694 OpenWhispr#2630 OpenWhispr#2654 OpenWhispr#2670 OpenWhispr#2845 OpenWhispr#2839 OpenWhispr#2958 OpenWhispr#2842
  src/services/ai/inferenceProviders/anthropic.ts: OpenWhispr#2654
  src/services/ai/inferenceProviders/enterprise.ts: OpenWhispr#2654
  src/services/ai/inferenceProviders/gemini.ts: OpenWhispr#2694 OpenWhispr#2654 OpenWhispr#2958
  src/services/ai/inferenceProviders/local.ts: OpenWhispr#2654
  src/services/ai/inferenceProviders/openai.ts: OpenWhispr#2019 OpenWhispr#2694 OpenWhispr#2654 OpenWhispr#2840 OpenWhispr#2670
  src/services/ai/inferenceProviders/tinfoil.ts: OpenWhispr#2654 OpenWhispr#2670
  src/services/ai/inferenceProviders/types.ts: OpenWhispr#2654
  src/stores/cleanupFailureStore.ts: OpenWhispr#2634
  src/stores/settingsStore.ts: OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2713 OpenWhispr#2694 OpenWhispr#2790 OpenWhispr#2425 OpenWhispr#2654 OpenWhispr#2989 OpenWhispr#2855
  src/types/electron.ts: OpenWhispr#2604 OpenWhispr#2683 OpenWhispr#2656 OpenWhispr#2685 OpenWhispr#2436 OpenWhispr#2517 OpenWhispr#2643 OpenWhispr#2565 OpenWhispr#2718 OpenWhispr#2778 OpenWhispr#2630 OpenWhispr#2791 OpenWhispr#2654 OpenWhispr#2652 OpenWhispr#2098 OpenWhispr#2822 OpenWhispr#2879 OpenWhispr#1982 OpenWhispr#2880 OpenWhispr#2921 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010
  src/utils/dictionaryWordKey.js: OpenWhispr#2855
  src/utils/snippets.ts: OpenWhispr#2855
  test/components/directionalContentPolicy.test.js: OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855 OpenWhispr#3010 OpenWhispr#2990
  test/components/fieldDirectionPolicy.test.js: OpenWhispr#2652 OpenWhispr#2822 OpenWhispr#2809 OpenWhispr#2989 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855
  test/helpers/cleanupFailureToast.test.js: OpenWhispr#2634
  test/helpers/harness/audioManager.js: OpenWhispr#2810
  test/helpers/policyRules.test.js: OpenWhispr#2988
  test/helpers/retryTranscriptionHandler.test.js: OpenWhispr#2718 OpenWhispr#2988 OpenWhispr#2989
  test/helpers/transcriptionFailureOutcome.test.js: OpenWhispr#2678
  test/helpers/trayQuickActions.test.js: OpenWhispr#2988
  test/helpers/useAudioRecordingCleanupFallback.test.js: OpenWhispr#2634
  test/helpers/useChatStreamingTools.test.js: OpenWhispr#2517 OpenWhispr#2630 OpenWhispr#2810 OpenWhispr#2655
  test/helpers/voicePillPresentation.test.js: OpenWhispr#2693
  test/helpers/whisperGpuUpgradeReset.test.js: OpenWhispr#2585
  test/stores/policyEffectiveVisionOverride.test.js: OpenWhispr#2958

Our features upstream touched:
  Self-hosting and build info (our 54d560e, e83ad4e): OpenWhispr#2565 OpenWhispr#2818
    check: node scripts/selfhost.js check, then ship: the installed build-info.json names the commit and Settings > System > This build shows it
  Logs you control (our 2d60c94, 4200336, 13fbdfe): OpenWhispr#2565 OpenWhispr#2818 OpenWhispr#2585
    check: Storage > Logs: switch Problems only / Everything / Off and dictate; the level follows and Clear logs works. node --import tsx --test test/helpers/logTextCanary.test.js: no dictated text in any log line, upstream's new ones included
  Bluetooth start cue (our 2582c67): OpenWhispr#2693
    check: With Bluetooth headphones idle, start a dictation: the start cue is heard in full
  Floating icon placement (our 79c292d): OpenWhispr#2624 OpenWhispr#2855
    check: Settings > Floating Icon: lock to a monitor and a grid spot; dictate with the mouse on another monitor, the pill stays put; sleep a monitor, it comes back
  Recording look, edge strip and tray badge (our bf28e6f, c1189f1, 5b6f156, bef2f90): OpenWhispr#2644 OpenWhispr#2693 OpenWhispr#2624 OpenWhispr#2654 OpenWhispr#2988 OpenWhispr#2842 OpenWhispr#2855
    check: Beacon red, Full frame and tray badge on: dictate; the pill, frame and tray dot show red while recording and go at stop and cancel; REC shows the time
  OpenWhispr account & Cloud switch (our 89be0f8): OpenWhispr#2517 OpenWhispr#2824 OpenWhispr#2842
    check: Privacy & Data: turn Cloud off; no request reaches openwhispr.com (log shows [cloud-off] blocked), no account surfaces show, local dictation works
  GitHub Copilot provider (our 5a317eb, 655b9a1, 5a45572): OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2634 OpenWhispr#2790 OpenWhispr#2845 OpenWhispr#2988 OpenWhispr#2842
    check: Cleanup on GitHub Copilot: dictate twice; the log shows 'Copilot runtime prewarmed' and 'Copilot request finished' under 6 s, the cleaned text pastes
  Glossary corrections (our b6c9ce0): OpenWhispr#2436 OpenWhispr#2586 OpenWhispr#2634 OpenWhispr#2718 OpenWhispr#2678 OpenWhispr#2694 OpenWhispr#2559 OpenWhispr#2654 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2855
    check: Add a glossary rule (e.g. 'little body' -> 'Little Buddy'), dictate the spelling; the pasted text and View steps show the rule fired
  Prompt library and test set (our 4e0739a, 8479b17): OpenWhispr#2694 OpenWhispr#2790 OpenWhispr#2654 OpenWhispr#2842 OpenWhispr#2933 OpenWhispr#2855
    check: Prompt Studio: switch prompts, edit and save a draft, run the test set A vs B; the prompt in use is what a dictation sends
  Step history and the History view (our 97f52cc, 4999528, 042d889, 616ead3, 3184e26, 368e6ac): OpenWhispr#2634 OpenWhispr#2718 OpenWhispr#2622 OpenWhispr#2654 OpenWhispr#2742 OpenWhispr#2988 OpenWhispr#2989 OpenWhispr#2933 OpenWhispr#3010
    check: Dictate, open History: the row shows its steps, 👎 opens the review side by side with the player, Re-run cleanup replaces the text, the review hotkey opens the last dictation
  Dictation styles and style hotkeys (our b7314cc, 195aaec): OpenWhispr#2656 OpenWhispr#2586 OpenWhispr#2693 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2622 OpenWhispr#2886 OpenWhispr#2855
    check: Turn a style on with a hotkey; dictate with it: the badge shows, the text is restyled, View steps shows the style step
  Settings layer, pages and search (our 5bdceae, 8f19d0d, e265357, e44c006, 91a2e0e, cf152fb): OpenWhispr#2652
    check: Open Settings: every page in the menu renders, search ('hotkey', 'logs') lands on the row, Esc leaves, the view underneath keeps running
  AI Connections, model pickers and connection checks (our b792f1c, 6501a60, b1b4773, a44dac3, 6f43147, 33a15ca): OpenWhispr#2694 OpenWhispr#2652 OpenWhispr#2585
    check: AI Connections: every saved connection checks green, the pickers on Cleanup and Recognition list each connection's models, a pick routes the next dictation there
  Storage page and recording quality (our 53f63d6): OpenWhispr#2656
    check: Settings > Storage: sizes measure, Recordings 'Smaller' records at 32 kbps (file size), Clear app cache works, Privacy & Data's summaries link here
  Transcription request deadline and retry (our 323f0d6): OpenWhispr#2656 OpenWhispr#2693 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2988
    check: Point the custom endpoint at a server that never answers: 'Taking longer than usual · Esc to cancel' after 8 s, Esc cancels, Retry re-sends the recording
  Azure Speech MAI-Transcribe-2 (our 85a04c3): OpenWhispr#2683 OpenWhispr#2685 OpenWhispr#2634 OpenWhispr#2845 OpenWhispr#2988 OpenWhispr#2855
    check: Recognition on Azure Speech with a Foundry endpoint: dictate, it transcribes; Re-transcribe from History works
  Dictionary v2 (our a16ea57, 6ca5f92, d41bdf7, 9d098d6, 0fd61bf, 36d7f41, be53cf7): OpenWhispr#2436 OpenWhispr#2586 OpenWhispr#2634 OpenWhispr#2678 OpenWhispr#2654 OpenWhispr#2879 OpenWhispr#2880 OpenWhispr#2824 OpenWhispr#2842 OpenWhispr#2933 OpenWhispr#2855 OpenWhispr#3010
    check: Dictionary: usage counts, coverage dots and pin order show; Tidy up applies and Undo restores; a learned word appears as a suggestion; Re-transcribe sends the hint list
  Graphics mode and tray warning (our 50c8790, b7c580b): OpenWhispr#2988 OpenWhispr#2842
    check: Start with --use-gl=desktop: 'Graphics mode' warns fallback, the tray shows the amber dot and Restart; a normal start logs hardware
  Quiet empty dictations (our bc86fe7): OpenWhispr#2656 OpenWhispr#2693 OpenWhispr#2678 OpenWhispr#2842
    check: Start a dictation and say nothing for 2 s: the pill spins down red, no error card, no History row
  Upstream strings we reworded (all locales): OpenWhispr#2643 OpenWhispr#2654 OpenWhispr#2655 OpenWhispr#2822 OpenWhispr#2958 OpenWhispr#2824 OpenWhispr#2855
    check: npm run i18n:check passes; the Settings pages and pill read the same in English after the merge
  CLAUDE.md sections we rewrote: OpenWhispr#2824
    check: CLAUDE.md keeps upstream's new sections and ours; nothing describes code that no longer exists

Tracked topics upstream touched:
  Workspaces, teams and guests in Settings: OpenWhispr#2876 OpenWhispr#2921 OpenWhispr#3010

Full review: C:\Git\Notes\obsidian\Work\Repo Specific Knowledge 🤔\OpenWhispr\Upstream reviews\2026-11-10 upstream b9e9ebe.md

Conflicts resolved to keep upstream v1.11.0 behaviour and every fork feature; per-file notes and the decisions taken are in the session brief files/upstream-merge-v1.11.0-brief.md and the resolver's report.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: cb886a50-814f-49fb-938d-e5716305752a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chat: "Streaming request timed out" with local model, the 60 s streaming timeout is a total deadline, not an idle timeout

2 participants