[BUG] format.json_schema.retryCount is ignored - structured output requests fail without retry #25430

Activity

1. 21pounder commented on May 3, 2026

   

   21pounder

   on May 3, 2026

   Contributor

   More actions

   Hi, I've reproduced this issue and confirmed the bug.

   retryCount is defined in the schema (message-v2.ts:74) but never read in prompt.ts. The StructuredOutputError is always created with retries: 0 regardless of what the caller passes (prompt.ts:1599-1602).

   Reproduction:

   Step 1 — Send a prompt with retryCount: 5:

   Step 2 — Server logs confirm the prompt was processed, but `retryCount` was never used: I'd like to take this one — the fix is in `prompt.ts`, reading `format.retryCount` and implementing the retry loop.

   👍React with 👍1Reacted by Mustafa Göksever
2. thdxr

   assigned

   rekram1-node

   and

   kitlangton

   and unassigned

   rekram1-node

   on May 3, 2026
3. mustafagoksever commented on May 3, 2026

   

   mustafagoksever

   on May 3, 2026

   Last edited by mustafagoksever

   Author

   More actions

   OpenCode creates a tool for structured output. However, the LLM sometimes does not call it, or calls it incorrectly.

   Does it guarantee calling the tool now? Does it validate the JSON structure?

   It might also be worth testing this with a smaller model that may struggle to fill the JSON schema correctly.

   Thanks. @21pounder
4. 21pounder commented on May 3, 2026

   

   21pounder

   on May 3, 2026

   Contributor

   More actions

   Thanks for the detailed write-up — I dug into this and wanted to share what I found.

   There are actually two bugs stacked on top of each other. The retry counter is declared but the loop's early-exit condition fires before the retry check is ever reached on the second iteration, so retries never actually happen. And even if they did, the model gets the exact same system prompt each time with no feedback that its previous response was wrong — so it'd probably just do the same thing again.

   That said, fixing retryCount felt like patching the symptom. The underlying issue is that structured output is implemented by wrapping the schema in a fake StructuredOutput tool and hoping the model calls it. toolChoice: "required" only forces some tool call, not that specific one — that's why it's flaky in the first place.

   Most providers have native structured output APIs (OpenAI's response_format, Anthropic's forced tool use with input_schema, Google's responseMimeType) that enforce schema conformance at the API level with no retries needed. That feels like the right fix here, so I didn't want to ship something that makes retryCount "work" while leaving the fragile foundation in place.

   Flagging this in case the team wants to tackle the root cause instead. Happy to help if @mustafagoksever
5. mustafagoksever commented on May 3, 2026

   

   mustafagoksever

   on May 3, 2026

   Last edited by mustafagoksever

   Author

   More actions

   Thanks for the detailed analysis. This clarifies the situation a lot. @21pounder

   It seems like this goes beyond a simple retry fix and points to a deeper issue in how structured output is currently implemented. Given how critical structured output is for reliability, I think this is something that should be addressed at the foundation level rather than patched incrementally.

   I’d really appreciate the team’s input and support on tackling this properly. Happy to contribute on testing if needed.
   @kitlangton
6. trevorWieland commented on May 27, 2026

   

   trevorWieland

   on May 27, 2026

   More actions

   Spent some time digging into this. Two things stacked here:

   1. retryCount is unread — confirmed at prompt.ts:1466-1474, the StructuredOutputError is constructed unconditionally with retries: 0 and the loop breaks immediately. A simple counter-increment + return "continue" (as in PR fix: respect retryCount in json_schema structured output #25534) doesn't actually retry, because prompt.ts:1273-1291's loop-exit check fires on the next iteration whenever lastAssistant.finish is truthy and there are no pending tool calls. The fix needs to insert a feedback user message with the prior format carried forward so lastUser.id > lastAssistant.id and toolChoice: "required" stays in force.

   2. Encoding bug entangled with this — GET /session/:id/message returns 400 when messages contain format with inline JSON schema #26929 (GET /session/:id/message returns 400 with Expected OutputFormatJsonSchema, got {...}) is the same area: OutputFormatText / OutputFormatJsonSchema are Schema.Class instances, which Effect's encoder can't round-trip from plain DB objects. Switching them to Schema.Struct (matching the rest of the file) fixes it.

   On the architectural question @21pounder raised in their earlier comment: I think you're right that native provider APIs (response_format, etc.) would be a stronger foundation. But packages/llm/src/llm.ts:151-162 documents the synthetic-tool approach as the intentional design ("provider-native JSON modes are intentionally avoided so behaviour is uniform"). Improving the existing path seems like the smaller, more reviewable change; it also makes the retryCount field do what its name says. If a native-API rewrite is preferred, happy to defer.

   I have a PR ready that closes both this issue and #26929. Posting shortly.

   👍React with 👍2Reacted by Cameron DeMille and hertznsk
7. trevorWieland

   added a commit that references this issue on Jun 18, 2026

   fix(session): respect format.retryCount and fix OutputFormat encoding

   ...

   This issue will close once commit f692d53 is merged into the 'automation' branch.

   f692d53
8. hertznsk commented on Jul 7, 2026

   

   hertznsk

   on Jul 7, 2026

   More actions

   We are still seeing this on current OpenCode structured-output sessions.

   In our case, a client sends a prompt with:

   {
     "format": {
       "type": "json_schema",
       "retryCount": 2,
       "schema": {
         "type": "object",
         "required": ["action", "reason", "sha"]
       }
     }
   }

   The raw session history shows several assistant turns ending with finish: "tool-calls", then OpenCode repeats the formatted user prompt, and the final assistant message ends with:

   {
     "finish": "stop",
     "error": {
       "name": "StructuredOutputError",
       "data": {
         "message": "Model did not produce structured output",
         "retries": 0
       }
     }
   }

   This matches the implementation gap described here: retryCount is present on the user message and defaults to 2 in the schema, but the runtime structured-output path does not appear to read it before constructing StructuredOutputError({ retries: 0 }).

   The current source still appears to have the same behavior:

   • packages/schema/src/v1/session.ts defines OutputFormatJsonSchema.retryCount with default 2.
   • packages/opencode/src/session/prompt.ts injects the StructuredOutput tool for json_schema.
   • But when the assistant finishes without structured output, it immediately sets StructuredOutputError with hardcoded retries: 0.

   PR #29632 looks like it already addressed the right root cause: preserve the original format on a synthetic reminder message and continue the run loop so toolChoice: "required" and the StructuredOutput tool remain active during retries, then report the actual retry count after the retry budget is exhausted.

   That PR was closed by automated cleanup, not because the approach was rejected. Could it be reopened or folded into the current structured-output/session runtime work?
9. github-actions commented on Sep 6, 2026

   

   github-actionsBot

   on Sep 6, 2026 – with GitHub Actions

   Contributor

   More actions

   To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.
10. github-actions

    closed this as not plannedon Sep 6, 2026