Repository navigation
[FEATURE]: Voice/Speech Input Support for Desktop App #33300
Description
Feature hasn't been suggested before.
- I have verified this feature I'm about to request hasn't been suggested before.
Describe the enhancement you want to request
Problem
Currently, OpenCode Desktop does not support voice/speech input. Users must type all prompts manually, which is inefficient for long or complex instructions. Most mainstream AI coding agents (e.g., Cursor, GitHub Copilot Chat, Claude Desktop) already support voice input natively.
Proposed Solution
Add built-in voice/speech-to-text input capability to the OpenCode Desktop app, similar to what other AI coding tools offer:
- Push-to-talk button or always-on wake word in the chat input area
- Speech-to-text using local or cloud-based ASR (e.g., Whisper, browser Web Speech API)
- Configurable: Allow users to choose between local processing (privacy-first) or cloud-based transcription (accuracy-first)
- Cross-platform: Support macOS, Windows, and Linux
Use Cases
- Hands-free coding — describe feature requests verbally while reviewing code
- Accessibility — users with RSI or other conditions benefit greatly from voice input
- Long prompts — faster to speak complex multi-step instructions than type them
- Non-native speakers — easier to articulate ideas in spoken language for many users
Alternatives Considered
- OS-level dictation (macOS/Windows built-in) — works but lacks context awareness and requires switching modes
- Third-party tools — adds friction, not integrated into the workflow
Context
Voice input is becoming a standard feature in AI-powered developer tools. Adding this to OpenCode Desktop would significantly improve the user experience and accessibility, especially since OpenCode emphasizes being "the open source AI coding agent" — voice input aligns with making coding more accessible to everyone.
Reacted by HermiG, Tõnis Ormisson, Jose Hernandez, Kenzen92, Ilyes, Chavali V S S Chaitanya, Renzo Damian, Pascal, Codeman X, NightCrewSec and 4 more
Metadata
Metadata
Assignees
Labels
No labels
Activity
LauraGPT commentedon Jul 7, 2026
For the first desktop voice-input slice, I would keep the UI small and make the STT backend pluggable from day one instead of hard-wiring browser/OS dictation.
A practical MVP shape:
POST /v1/audio/transcriptionsendpoint plus optional model/language fieldsThat provider boundary would let users run Whisper-compatible services, FunASR/SenseVoice, or other local ASR engines without OpenCode needing to embed each model runtime. It also matters for CJK/local/offline workflows where browser dictation is usually the wrong default.
Tests I would want around the first pass:
x-lixu commentedon Aug 5, 2026
For anyone who wants voice with OpenCode today: qwen-audio-agent connects to OpenCode natively over ACP (one-click install inside the app) and provides full-duplex voice with barge-in, so the conversation keeps going while OpenCode runs tasks in the background. It ships a macOS desktop app, a TUI, and a web UI. It is external to OpenCode itself, but it covers the use case end to end until first-party support lands.
liudongyan13701205717-source commentedon Aug 9, 2026
Supporting this request as a Chinese-speaking Windows desktop user (was tracked in #41364, closed as duplicate of this one). Strong demand: typing long prompts in CJK is slow, and voice would help a lot.
Additional context beyond the original proposal:
SpeechSynthesizer) or Whisper local instead of cloud — keeps with privacy expectations of a desktop IDE-adjacent tool.This one issue (#33300) is the consolidation point for #35219, #37742, #39688, #38300, #18226, #31542 and my #41364 — worth a clean spec + assignment so all the voice threads can converge.
x-lixu commentedon Aug 9, 2026
+1 to the consolidation — and a quick mapping for anyone who wants parts of this today, since qwen-audio-agent (ACP client running alongside OpenCode) already ships several of these points:
None of this replaces the native feature — just noting the overlap so the spec work can borrow what's already been validated. (Disclosure: maintainer, as before.)
liudongyan13701205717-source commentedon Aug 15, 2026
+1, would love voice/speech input on the Desktop app; happy to test on Windows once available.
NightCrewSec commentedon Oct 8, 2026
Accessibility and hands-free input
Following the pointer from #53977, since the accessibility angle is not covered in detail yet.
For people who cannot type comfortably or quickly, voice input is not a convenience feature, it is the difference between using an AI coding assistant and not using it at all. That includes people with motor impairments, injuries, or visual limitations, and older users who find typing slow or tiring. For them, the absence of a microphone is an accessibility barrier rather than a missing nicety.
On that basis I would argue for three things beyond what is already described.
1. Hands-free should not be optional polish. Push-to-talk assumes the user can press and hold a key on demand. That works for most people and excludes people whose hands are unavailable or unreliable. Voice activity detection to close the utterance automatically removes the keypress entirely. If this lands only as a convenience feature, the accessibility case is only partly served.
2. Speech output matters for the same reason dictation does. This is the part that appears least often in the thread. For a user who cannot read a terminal comfortably, or who is screen-reader dependent, hearing the response is the equivalent of being able to read it. Any voice input feature that cannot speak back leaves a one-way conversation. Streaming per sentence as the response arrives also keeps it usable interactively rather than making it a batch playback.
3. Local transcription is an accessibility-adjacent concern too. Users who cannot send audio off the machine should not have to give up voice entirely. A local Whisper option removes that trade-off, and it is what makes the feature viable on machines with no third-party account.
There is already a working implementation of point 1 at the push-to-talk level as a TUI plugin, roughly 170 lines, using
pw-recordpluswhisper-large-v3-turbo. It captures, transcribes, shows the text for review and submits it to the session. I can share it if that is useful as a starting point, but the parts that matter most for the accessibility case, hands-free and speech output, are exactly what a plugin cannot provide today.Also worth noting that the plugin API already supports everything needed for dictation. The gap is a supported, discoverable path, not a missing capability.
reynold-hu commentedon Oct 16, 2026
Sharing a concrete shape for the speech-output half of this thread, since the pieces now exist in the repo:
Current state
@opencode/aialready ships speech generation routes for OpenAI, ElevenLabs, Cartesia, Deepgram, Google, and xAI (the latter in feat(ai): add xAI Grok speech and transcription routes #56042), and word-level transcription routes for most of the same providers. The synthesis capability is merged and tested.packages/tui/src/audio.ts, with the stop-when-done fix from TUI keeps audio output active after notification playback #54432), so local playback plumbing exists and is platform-aware.Proposed MVP (TUI first, default off)
POST /api/experimental/voice/speech, taking{ text, model, voice? }and returning the provider's audio stream. A thin wrapper overSpeech.streamwith the configured provider; no session state, no retention.prompt.voice.read_aloud) that speaks the latest assistant message by playing the returned audio through the existing attention-sound player; pressing it again stops playback.voice.speech = { model, voice }selection so users choose the provider/voice they already have credentials for; disabled by default.attention.sound: "speak"to announce a short spoken summary when a long task completes, which is the form raised earlier in this thread for long tasks and accessibility.Scope boundaries
afplay, Linuxffplay/paplay, Windows counterpart), so no new native dependencies.Testing
Happy to prototype this if maintainers want it; otherwise this records the intended shape for whoever picks it up.
Silverstar187 commentedon Oct 18, 2026
Disclosure: I built this, so take it with a grain of salt.
Until there's native voice in the desktop app, one workaround that works with opencode today: voicehook, a remote MCP server. You add
https://voicehook.ai/mcpas an MCP server, ask the agent to "start a voicehook call with me", and you get a browser link. From then on you talk, the agent hears you as text and answers by voice, and you see what it's doing on a status board while it works.It's not dictation into the prompt box, it's a voice channel next to the session, so it also covers the speech-output half discussed above. Free minutes, no signup: https://voicehook.ai/?utm_source=github&utm_medium=comment&utm_campaign=opencode-33300
opencodejr commentedon Oct 29, 2026
Thanks for the detailed request, @dayesann! You're asking for built-in speech-to-text in the OpenCode Desktop composer: a push-to-talk button, with transcription done either locally or by a cloud service, on all platforms. Maintainers will consider it.
This issue is now the main place where voice input for Desktop is tracked, and several related requests have been merged into it. Related items:
Desktop also has an experimental "Voice calls" mode (Settings → Experiments). It's a spoken conversation with OpenCode, not dictation into the prompt box, so it doesn't fully cover this request.
kVizir0 commentedon Nov 8, 2026
Disclosure: I built this, so take it with a grain of salt.
For Windows users who want voice input today, I made OpenCode Voice — a local plugin for OpenCode Desktop:
https://github.com/kVizir0/opencode-voice-win
Ctrl+Shift+Space), review before insert,Escto cancel;faster-whisper— no cloud, no API keys; Russian and auto-detect;127.0.0.1only, no audio cache on disk.Requirements: Windows 10/11, Python 3.10–3.12, OpenCode Desktop 2.2.x. Install with the repository's
install.ps1or the tarball from the v1.17.0 release. MIT.It is Windows-only and desktop-only, so it does not solve the cross-platform native request, but it may be useful for Windows users or as a reference until first-party support lands.
Aitooor commentedon Nov 10, 2026
Proposal: local, real-time, cross-platform voice chat for OpenCode
I am asking for more than speech-to-text in the prompt box. I want to keep a natural, hands-free conversation with OpenCode while I work: speak instructions, hear the agent's responses, interrupt it, and continue without stopping to type. The voice experience should work across Linux, macOS, and Windows, and one voice interface should be able to work with more than one OpenCode session.
User-facing behavior
How this can fit the current code
The current Desktop Voice Calls work is a useful base for lifecycle and UI, but its audio backend is provider-backed. The experimental entry point and floating panel are in packages/app/src/voice/composer.tsx; call state and session navigation are in context.tsx, and captions, turn handling, and delegation are in call.ts. The current transport.ts captures the microphone with getUserMedia, meters input/output through Web Audio, and negotiates a WebRTC session. The app currently requires the GPT-Live model for this call path.
The server path is explicitly experimental and server-brokered: packages/core/src/voice.ts, the realtime engine list, the protocol routes, and the server handler create OpenAI, xAI, Google, or OpenCode-hosted realtime sessions using credentials on the server. That is useful as an optional realtime-provider backend, but it cannot be the required path for a local-first mode.
A concrete implementation could build on those boundaries as follows:
Related requests and prior work
Earlier requests include #35219 (hold-spacebar push-to-talk), #37742 (Desktop microphone button), #18226 and #31542 (speech-to-text), #41364 (Desktop voice input and optional spoken summaries), #53977 (first-party voice input/output), plus #39688 and #38300.
PR #29663 and PR #53492 explored dictation and are closed. PR #55951 was merged for an experimental server voice-session endpoint. Comments in this issue already cover local, pluggable STT and reviewable transcripts, full-duplex/barge-in, VAD and accessibility, TTS architecture, and the distinction between dictation and experimental Voice calls. A Windows local voice plugin is another reference for local speech, though it does not cover the cross-platform, multi-session product flow.
Could the maintainers advise whether this belongs in #33300 or should be tracked/design-reviewed separately, and how I should proceed? I would be willing to prepare a PR and contribute code to OpenCode if this direction is useful. I would appreciate guidance on the intended scope and design—particularly the local runtime boundary, session routing, and keeping this distinct from dictation and provider-backed Voice calls—before starting implementation. The earlier OpenCode bot response and related discussion are linked above for context.
atomicdjt commentedon Nov 11, 2026
I'd like to contribute built-in, offline dictation to the Desktop composer, distinct from Voice Calls and the endpoint-based approach explored in #53492. Would this scope fit the planned design?
Would maintainers prefer Hex's reusable runtime for this, and is a staged Desktop-first implementation acceptable? I'd appreciate design guidance on the runtime and initial platform/model scope before implementation.
boesbo commentedon Nov 11, 2026
+1 — built-in audio-to-text prompt input is a very important feature, and I would really like to see it prioritized.
Being able to speak a longer instruction instead of typing it would make prompt creation faster and more natural, and would also improve accessibility. A simple workflow would already be valuable: record audio from the composer, stop, and have the transcript inserted into the prompt for review and editing before sending.
The experimental Voice Calls work is encouraging, and dictation into the existing prompt deserves support as well. Please keep this on the roadmap and prioritize a first-party implementation. Thanks to everyone working on it.
akaRem commentedon Nov 13, 2026
Proposal: Desktop dictation — design proposal for review
After rereading this thread and the linked attempts, here is a consolidated design. Everything below is proposed for review, not agreed, and I try to be accurate about what prior work does and does not decide:
@opencode/aiCore/Protocol/Server/generated-Client layers, andIntegration.get+connection.resolveas the single credential source. It does not decide the local Whisper host or the batch media path — see question 2.I also have a working Desktop plugin prototype using a user-installed
whisper-cli, preflight before microphone access, a decorative waveform, batch insertion, and pending-on-conflict. This proposal turns those parts into a first-party contract, adds server-backed providers, and extends dictation to every Desktop text input.Scope: Desktop only. TUI, Web, TTS, wake word detection, VAD, and realtime streaming are out of scope for this proposal.
Universal field coverage
Every text input on every Desktop screen is supported through a shared input contract, covering existing and future text fields.
This is not opt-in per screen:
ComposerEditor(including shell text mode), quote/review create/editLineCommentEditor, question customTextInput, elicitation text-capable inputs, fork/rename, project/worktree text fields, settings, search, connect — and all other current and future screens.Every supported field shows a microphone icon. One recording attempt can be active per window; other microphone icons are visibly disabled with an explanation. There is no auto-send or form submission — the result is plain text that the user can edit as one edit/undo operation before confirming.
Source-bound insertion and pending
A dictation target is a generic
TargetRef: screen/form/draft identity, stable control identity, content revision, opaque caret, and analiveflag. Every field registers through the shared primitive or an adapter. Targets are not required to belong to a Session — many fields have none.Server/backend architecture
This proposal uses the active
ServerConnection/Locationfor three batch operations: prepare, transcribe, and cancel. The backend runs where the active OpenCode Server runs. Dictation is not part ofSessionExecution.Flow after Stop: Desktop → generated Client → active Server → local Whisper process or remote provider → text. One batch call per Stop, with thin layers throughout.
Today's ModelResolver/catalog has no transcription model type or adapter, so this requires a bounded transcription capability extension. That capability is not claimed to exist today.
Registered providers resolve credentials via
Integration.get+connection.resolve, with no custom credential-list logic. A custom OpenAI-compatible provider is a static config profile containing a base URL, a model, and the name of an environment variable resolved server-side. It contains no secret value, and requests cannot provide arbitrary URLs, keys, or headers. HTTPS is required except for an actual loopback target; redirects are not followed.Destination honesty. Prepare runs before the microphone opens and fixes the backend/server destination. The UI shows that destination before recording starts:
<server>— audio leaves this device”.A reconnect never silently switches backend or Server. Remote-host execution is never described as “offline” or “on this computer”. There is no hidden fallback between backends.
Audio and waveform
The recording UI shows a stylized waveform. The capture contract exposes PCM/RMS/peak values and real samples, allowing a future true waveform without changing the transcription backend API.
After Stop, audio is finalized as canonical WAV — 16 kHz, mono, signed 16-bit little-endian PCM — and sent as one batch.
Configuration
Config-only, behind a separate experimental flag. No Settings UI in this proposal.
All schema below is illustrative and proposed, not an existing API.
Local backend:
{ "experimental": { "dictation": true }, "dictation": { "limits": { "maxDuration": "5m", "transcriptionTimeout": "2m" }, "backend": { "type": "local", "executable": "/opt/homebrew/bin/whisper-cli", "preset": "base", "presets": { "base": { "model_path": "/absolute/path/to/ggml-base.bin", "options": { "threads": 4, "beam_size": 5 } } } } } }Registered remote provider:
{ "experimental": { "dictation": true }, "dictation": { "backend": { "type": "remote", "model": "openai/gpt-4o-mini-transcribe" } } }Language is auto-detected by default. OpenCode passes no language option to the transcription backend unless the user explicitly configures an override.
The user installs
whisper-cliand models themselves; there are no managed downloads. When a limit expires, the run terminates: the local process tree for a local backend, or the outbound request for a remote one. Server-side hard ceilings on request body, response size, and concurrency cannot be raised through config. User-configured limits remain capped by them.Privacy and lifecycle
A pure state machine owns explicit lifecycle phases. An attempt is claimed synchronously before any asynchronous work, and late or stale results are discarded.
Prepare validates the executable, model, provider, and destination before microphone access. Microphone access uses the existing native OS permission flow, including its first-use prompt when required; there is no separate custom dictation permission prompt.
Cancellation covers the capture track, audio buffers, outbound request, and local process tree. No audio or raw transcripts are written to logs, history, analytics, or debug output. Local Whisper may use a bounded private temporary WAV with restrictive permissions and best-effort cleanup on success, failure, and cancellation.
Provider timeouts and error responses are sanitized before display. Server authentication uses the normal connection authentication; the renderer holds no provider secrets.
Voice Calls and Dictation are mutually exclusive. Starting one while the other is active is refused; neither automatically cancels the other.
Non-goals
TUI, Web, TTS, wake word detection, VAD, realtime streaming, Settings UI, managed model downloads, auto-send, automatic fallback, and automatic retries are outside this proposal.
Questions for maintainers
Locationacceptable, including when that Server is remote, or must local audio processing always execute on the user's device?Integrationproviders plus an env-var-only custom credential contract the right shape here?Happy to adjust the proposal around these decisions.