Skip to content

[BUG]: structured output forces tool_choice "required", which newer Anthropic models reject with a 400 #46735

Closed as not planned

Description

@iceteaSA

Description

SessionPrompt.loop sets toolChoice: "required" whenever a request carries format: { type: "json_schema" }:

toolChoice: format.type === "json_schema" ? "required" : undefined,

The Anthropic adapter lowers required to {"type":"any"}, and claude-fable-5-1 rejects that outright. Every structured-output request against it fails.

This is the same defect as #15226, which was closed by the inactivity bot in July with "if the issue is still relevant please open a new one" — it is, so here it is with a live reproduction. The fix PR for it, #29565, was closed unmerged by the automated PR cleanup without a maintainer review.

What I measured

Direct API probe. auto and none are controls — they pass, which is what makes the failures attributable to tool_choice rather than to auth or request shape:

model auto none {"type":"any"} {"type":"tool",name}
claude-fable-5-1 200 200 400 400
claude-fable-5 200 200 200 200
tool_choice: type "tool" and "any" are not supported for this model.

So this is model-gated, not an API-wide change — claude-fable-5 accepts all four.

Then end-to-end through the AI SDK with opencode's exact shape (a StructuredOutput tool plus toolChoice: "required"), capturing the outgoing body:

claude-fable-5-1: FAILED   wire tool_choice={"type":"any"}
    tool_choice: type "tool" and "any" are not supported for this model.
claude-fable-5:   200 OK   wire tool_choice={"type":"any"}

The chain is required → {"type":"any"} → 400, in one run rather than inferred across two.

Not Anthropic-specific

#15226 collected the same failure across providers, each with its own error string:

  • Kimi K2.5 / Moonshot — tool_choice 'required' is incompatible with thinking enabled
  • DeepSeek v4 flash — Thinking mode does not support this tool_choice
  • Qwen3.5-9B
  • Anthropic with thinking: true — reported in that thread and now hard-failing on 5-1

The common factor is thinking-enabled models rejecting a forced tool choice. The newer Anthropic models make it unconditional rather than conditional on a thinking flag.

Suggested fix

The AI SDK's Anthropic adapter already supports native structured output and uses it in preference to a synthetic tool. generateObject against both models emits no tool and no tool choice at all:

wire keys: [model, max_tokens, output_config, messages, system]
tools=[]  tool_choice=undefined     -> 200 on claude-fable-5-1 and claude-fable-5

So the model-agnostic fix is for the json_schema path to use the provider's native structured output where available, instead of hand-rolling a StructuredOutput tool and forcing a choice. That removes the conflict everywhere at once and needs no per-model capability tracking. The tool-based path stays as the fallback for providers without native support.

A narrower alternative — downgrading required to auto when the model reasons — is what #29565 implemented (compatibleToolChoice in session/llm.ts, credit @serejaris). It is a smaller diff and would fix the 400. Worth noting it does weaken the guarantee: with auto the model may not call the StructuredOutput tool at all, so the request can come back without structured output and lean on format.retryCount, which #25430 reports as unwired. Native structured output gets both properties at once.

Happy to open the PR for either shape — say which you'd prefer and I'll build it.

Steps to reproduce

  1. Send a session prompt with format: { type: "json_schema", schema: {...} }
  2. Target anthropic/claude-fable-5-1
  3. The request fails with tool_choice: type "tool" and "any" are not supported for this model.

Substituting claude-fable-5 succeeds, which isolates it to the model.

Additional context

One adjacent fragility, for context rather than as a second ask: the Anthropic adapter decides whether to use native structured output from getModelCapabilities(modelId), and claude-fable-5-1 only qualifies because it matches the claude-fable-5 entry as a substring. A future model id that matches no known prefix would fall through to the forced-tool path, which would turn agent.ts's generateObject/streamObject calls into the same 400. That is upstream in @ai-sdk/anthropic, not here.

Also related: #45953 (same forced-choice class in the v2 SessionRunner, different code path) and #37672.

OpenCode version

dev; probed against claude-fable-5-1 and claude-fable-5 on 2026-09-02.

Activity

  1. foglerek commented on Sep 2, 2026

    @foglerek

    We've been monkeypatching this for months at this point, a combination of auto + retry shim to get structured output with thinking. It has been sufficiently reliable with capable models and 3 coercion attempts.

    Honoring the provider specific shape seems like the preferable path here. But we would also be happy with the retry being implemented properly and not forcing the tool call.

  2. rscambler commented on Sep 2, 2026

    @rscambler

    It would be great to get this merged. It has led me to turn thinking off and use lesser models. It is very frustrating. It also causes issues with the server UI. I cannot view the sessions on the server because I get an error message about the schema. This makes debugging a nightmare.

  3. opencodejr commented on Oct 28, 2026

    @opencodejr
    Contributor

    Thanks for the detailed write-up and the wire-level probes.

    The code path you're describing (SessionPrompt.loop in packages/opencode/src/session/prompt.ts, which forces toolChoice: "required" for format: { type: "json_schema" }) belongs to OpenCode v1. That was still dev when you probed it on 2026-09-02, but dev has since moved to v2, which is a separate implementation. v1 now only gets fixes for critical bugs such as data loss, security problems or breakage with no workaround. Structured output failing on one model while other models still work doesn't meet that bar, so this is unlikely to be fixed in v1. That's also why #44634 was closed with a request to port it to v2.

    In v2 (@opencode/cli, 2.x) I couldn't find this path: there's no StructuredOutput tool and no forced required tool choice for json_schema. The only forced tool choice v2 sends is "none" when an agent reaches its steps limit, which is tracked in #45953. We'd encourage upgrading to v2.

    Related issues and PRs:

    If you think this is critical, or you can reproduce a forced-tool-choice 400 with structured output on v2, please reply here with the v2 version and steps and we'll take another look.

  4. opencodejr commented on Oct 29, 2026

    @opencodejr
    Contributor

    Closing this as not planned since we haven't heard back. The forced tool_choice: "required" for json_schema structured output is in OpenCode v1 (SessionPrompt.loop), and v1 only gets critical fixes. Current v2 doesn't have this structured-output path.

    Thanks again for the thorough investigation. If you can reproduce a forced-tool-choice 400 on v2 (@opencode/cli, 2.x), please open a new issue with the version and steps, or follow #45953 for the related v2 tool_choice problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions