Repository navigation
feat(opencode): make local turns survive, and make failures visible - #48
Merged
Merged
Conversation
Investigation of "OpenCode stops mid-task on a local model and I have to type continue", grounded in OpenCode's own session database (58k assistant turns), its log, the meter's telemetry, and a throwaway OpenAI-compatible endpoint that recorded exactly what OpenCode puts on the wire. The stalls were overwhelmingly the endpoint dying, not OpenCode losing its place. Turns that never completed over 7 days: glm-meter 0.9%, local-orch 0.0%, local-dev 38.1%. By hour, 2026-08-25 00:00-04:00 shows 40 local turns and 40 failures -- exactly the M2 wedge window. Historically, 5,167 of 5,430 local-review stream errors were Bad Gateway clustered on Aug 2-6, the M2 outage that went unreported for four days. What made it look like an OpenCode fault is that it fails silently: three retries, an AI_RetryError line in its log, then the turn ends with nothing on screen. Changes, each tied to something measured: - limit.output 32000 (was 16384, set earlier today). local-orch has produced 32,000-token turns and been truncated at exactly that ceiling three times; 2 of the 34 recorded "continue" nudges followed such a turn. 16384 was my error -- it halved a ceiling the model actually uses. OpenCode clamps the wire value at 32000 regardless, verified against the recorder. - options.timeout / chunkTimeout per local provider. Worst observed time-to-first-token on these endpoints is 209s, and OpenCode does not retry a chunk timeout at all (anomalyco/opencode#20466), so a default in the tens of seconds would silently kill healthy turns. - configs/opencode/plugins/local-stall-alert.mjs raises a desktop notification on session.error, with a one-minute cooldown so an outage does not fire forty times. It fixes nothing upstream; it makes a silent failure announce itself. - Dropped the `instructions` entry. ~/.config/opencode/AGENTS.md is a symlink to the same file OpenCode auto-loads, so every house rule was being sent twice: 12,555 duplicated characters in every request, 15% of the system prompt. Verified end to end -- the system prompt fell from 81,046 to 67,421 chars and the whole request from 121,076 to 106,121 bytes. - Disabled local-ai-delegate_run_local_plan: never called once in 47,240 recorded tool calls. Three more never-called tools (list_mcp_resources, list_mcp_resource_templates, read_mcp_resource) CANNOT be disabled from config -- `mcp*`, `*resource*` and `*mcp_resource*` were each tested against the recorder and left all three in place. A test now rejects those dead patterns so the config never claims coverage it does not have. - small_model moved off the paid cloud; 21,734 title generations were going there. - The mlx launchers set --max-tokens (server default is 512, and applies to any request that omits it -- a trap for curl probes and new clients, not for OpenCode which sends its own) and gain optional --chat-template-args. DELIBERATELY unset by default: thinking off measured 89% -> 78% recall on the code-review eval, so it belongs on a bulk lane, never on the reviewer. Takes effect on the next server restart; nothing restarted here. - rotate-logs.sh now covers OpenCode's log directory. Its log was one unrotated 374 MB file, mostly 137,653 "duplicate skill name" warnings. Rotated in place. Not done, deliberately: `steps` stays unset, because capping it forces a text-only summary mid-task, which looks exactly like the stall being fixed. And the duplicate skills themselves are spread across ~/.agents, ~/.claude, ~/.opencode and more -- shared with four other agents, so consolidating them is the operator's call, not a change to make from inside this repo. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Answers "why does OpenCode stop mid-task on a local model and need a typed continue", and acts on it.
Evidence
OpenCode's own session database (58k assistant turns), its log, the meter's telemetry, and a throwaway OpenAI-compatible endpoint that recorded exactly what OpenCode puts on the wire.
It was the endpoint dying, not OpenCode losing its place. Turns that never completed, last 7 days:
By hour, 2026-08-25 00:00–04:00 shows 40 local turns and 40 failures — exactly the M2 wedge window from #45. Historically, 5,167 of 5,430
local-reviewstream errors wereBad Gatewayclustered on Aug 2–6, the M2 outage that went unreported for four days.What made it look like an OpenCode fault: it fails silently. Three retries, an
AI_RetryErrorline in its log, then the turn ends with nothing on screen.Changes
limit.output→ 32000.local-orchhas produced 32,000-token turns and been truncated at exactly that ceiling three times; 2 of the 34 recorded "continue" nudges followed such a turn. I set 16384 earlier today, which halved a ceiling the model actually uses — that was my error. OpenCode clamps the wire value at 32000 regardless, verified.timeout/chunkTimeoutper local provider. Worst observed time-to-first-token here is 209s, and OpenCode does not retry a chunk timeout at all (#20466), so a default in the tens of seconds silently kills healthy turns.A plugin that speaks up.
local-stall-alert.mjsnotifies onsession.error, one-minute cooldown so an outage doesn't fire forty times. It fixes nothing upstream — it makes a silent failure announce itself.Dropped the duplicated instructions.
~/.config/opencode/AGENTS.mdis a symlink to the file OpenCode already auto-loads, soinstructionssent every house rule twice — 12,555 duplicated characters per request, 15% of the system prompt. Measured before and after against the recorder: system prompt 81,046 → 67,421 chars, whole request 121,076 → 106,121 bytes.small_modeloff the paid cloud — 21,734 title generations were going there.mlx launchers gain
--max-tokens(the server default is 512 and applies to any request omitting it — a trap for curl probes and new clients, not for OpenCode) and optional--chat-template-args, deliberately unset because thinking-off measured 89% → 78% recall on the code-review eval. Takes effect on the next server restart; nothing was restarted here.rotate-logs.shcovers OpenCode's log dir. It was one unrotated 374 MB file, mostly 137,653 duplicate-skill warnings. Rotated in place.What could not be fixed from our side
Three tools that were never called once in 47,240 recorded tool calls —
list_mcp_resources,list_mcp_resource_templates,read_mcp_resource— cannot be disabled. They're OpenCode built-ins, not MCP-prefixed. I testedmcp*,*resource*and*mcp_resource*against the recorder; all three tools survived every one. A test now rejects those dead patterns so the config never claims coverage it doesn't have.Also unfixable here: #20466 (chunk timeouts not retryable) and #31430 (zero-content response reads as done). The second doesn't currently affect this stack — zero of 4,315 captured local responses were empty.
Deliberately not done
stepsstays unset: capping it forces a text-only summary mid-task, which looks exactly like the stall being fixed. And the duplicate skill directories are spread across~/.agents,~/.claude,~/.opencodeand more, shared with four other agents — consolidating them is the operator's call, not a change to make from inside this repo.Verification
73 tests pass, 13 new. shellcheck clean at warning level. Live smoke test through the meter on the orchestrator and developer endpoints after the final config.
🤖 Generated with Claude Code