Skip to content

Commit 45a788c

Browse files
lu-zeroclaude
andcommitted
fix(opencode): native output_format structured output (works with thinking)
opencode implements structured output (`format: { type: "json_schema" }`) by injecting a `StructuredOutput` tool and forcing `tool_choice: "required"`. Anthropic and several reasoning providers reject thinking + forced tool use with `400 "Thinking may not be enabled when tool_choice forces tool use."`, so structured output either 400s or the model can't reason. Route `json_schema` structured output through the AI SDK's native structured output for providers that support it (Anthropic via `@ai-sdk/anthropic`'s `structuredOutputMode: "outputFormat"`), exactly as the Anthropic SDK does it (`output_format`, not a forced tool). The forced-tool path is kept as a fallback for non-native providers, so their behavior is unchanged (strict superset). - llm.ts: add `structuredOutput` to `StreamInput`; on native-capable providers pass `output: Output.object(jsonSchema(...))` and merge `{ anthropic: { structuredOutputMode: "outputFormat" } }` into provider options; defensively downgrade a forced `tool_choice` to `auto` on the native path so the two can never collide. Add `supportsNativeStructuredOutput`. - prompt.ts: skip the `StructuredOutput` tool, the coercion system prompt, and the forced `tool_choice` on the native path; thread `structuredOutput` through `handle.process`; parse the assistant text into `message.structured`, degrading to the existing `StructuredOutputError` on parse failure. - test: structured output on a thinking-enabled Anthropic model requests `output_config.format` and does NOT force `tool_choice`. Refs anomalyco#15226. Alternative to anomalyco#29565 (which relaxes the forced tool_choice required->auto for thinking surfaces); this is the native output_format approach. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
1 parent c101347 commit 45a788c

3 files changed

Lines changed: 177 additions & 6 deletions

File tree

‎packages/opencode/src/session/llm.ts‎

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ import { SessionV1 } from "@opencode-ai/core/v1/session"
66
import { serviceUse } from "@opencode-ai/core/effect/service-use"
77
import { Context, Effect, Layer } from "effect"
88
import * as Stream from "effect/Stream"
9-
import { streamText, wrapLanguageModel, type ModelMessage, type Tool } from "ai"
9+
import { Output, jsonSchema, streamText, wrapLanguageModel, type ModelMessage, type Tool } from "ai"
10+
import type { JSONSchema7 } from "@ai-sdk/provider"
1011
import type { LLMEvent } from "@opencode-ai/llm"
1112
import { LLMClient } from "@opencode-ai/llm/route"
1213
import type { LLMClientService } from "@opencode-ai/llm/route"
@@ -45,6 +46,13 @@ export type StreamInput = {
4546
tools: Record<string, Tool>
4647
retries?: number
4748
toolChoice?: "auto" | "required" | "none"
49+
// When set on a provider with native structured output (e.g. Anthropic's
50+
// `output_format`), the request uses the AI SDK `output` option instead of a
51+
// forced `StructuredOutput` tool. This avoids `tool_choice: "required"`, which
52+
// reasoning providers reject alongside thinking ("Thinking may not be enabled
53+
// when tool_choice forces tool use."). Non-native providers ignore this and
54+
// keep the forced-tool path.
55+
structuredOutput?: { schema: JSONSchema7 }
4856
}
4957

5058
export type StreamRequest = StreamInput & {
@@ -273,11 +281,34 @@ const live: Layer.Layer<
273281
"llm.provider": input.model.providerID,
274282
"llm.model": input.model.id,
275283
})
284+
285+
// Native structured output: route `json_schema` through the provider's own
286+
// structured-output mode (Anthropic `output_format`) instead of a forced
287+
// `StructuredOutput` tool. This keeps `tool_choice` untouched so the
288+
// request coexists with thinking. Providers without native support fall
289+
// back to the forced-tool path handled in prompt.ts (strict superset).
290+
const useNativeStructuredOutput =
291+
input.structuredOutput !== undefined && supportsNativeStructuredOutput(input.model)
292+
const providerOptions = ProviderTransform.providerOptions(input.model, prepared.params.options)
293+
if (useNativeStructuredOutput && input.model.api.npm === "@ai-sdk/anthropic") {
294+
providerOptions.anthropic = {
295+
...providerOptions.anthropic,
296+
structuredOutputMode: "outputFormat",
297+
}
298+
}
299+
// Native structured output and a forced tool choice are mutually exclusive
300+
// on reasoning providers (Anthropic rejects thinking + forced tool use).
301+
// Defensively drop the forced choice so the two never collide.
302+
const toolChoice = useNativeStructuredOutput && input.toolChoice === "required" ? "auto" : input.toolChoice
303+
276304
// Default runtime path: AI SDK owns provider execution and tool dispatch;
277305
// LLMAISDK.toLLMEvents below normalizes fullStream parts for the processor.
278306
return {
279307
type: "ai-sdk" as const,
280308
result: streamText({
309+
output: useNativeStructuredOutput
310+
? Output.object({ schema: jsonSchema(input.structuredOutput!.schema) })
311+
: undefined,
281312
onError(error) {
282313
bridge.fork(
283314
Effect.logError("stream error", {
@@ -313,10 +344,10 @@ const live: Layer.Layer<
313344
temperature: prepared.params.temperature,
314345
topP: prepared.params.topP,
315346
topK: prepared.params.topK,
316-
providerOptions: ProviderTransform.providerOptions(input.model, prepared.params.options),
347+
providerOptions,
317348
activeTools: Object.keys(prepared.tools).filter((x) => x !== "invalid"),
318349
tools: prepared.tools,
319-
toolChoice: input.toolChoice,
350+
toolChoice,
320351
maxOutputTokens: prepared.params.maxOutputTokens,
321352
abortSignal: input.abort,
322353
headers: prepared.headers,
@@ -386,6 +417,13 @@ const live: Layer.Layer<
386417

387418
export const hasToolCalls = LLMRequestPrep.hasToolCalls
388419

420+
// Providers whose AI SDK package exposes a native structured-output mode that is
421+
// NOT a forced tool call (so it coexists with thinking). Kept conservative: only
422+
// providers we've verified. Anything else keeps the forced-tool path untouched.
423+
export function supportsNativeStructuredOutput(model: Provider.Model): boolean {
424+
return model.api.npm === "@ai-sdk/anthropic"
425+
}
426+
389427
export const node = LayerNode.make({
390428
service: Service,
391429
layer: live,

‎packages/opencode/src/session/prompt.ts‎

Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1240,7 +1240,12 @@ const layer = Layer.effect(
12401240
Effect.provideService(RuntimeFlags.Service, flags),
12411241
)
12421242

1243-
if (lastUser.format?.type === "json_schema") {
1243+
// Native structured output (e.g. Anthropic `output_format`) coexists
1244+
// with thinking, so prefer it. Only inject the forced `StructuredOutput`
1245+
// tool for providers without native support.
1246+
const useNativeStructuredOutput =
1247+
lastUser.format?.type === "json_schema" && LLM.supportsNativeStructuredOutput(model)
1248+
if (lastUser.format?.type === "json_schema" && !useNativeStructuredOutput) {
12441249
tools["StructuredOutput"] = createStructuredOutputTool({
12451250
schema: lastUser.format.schema,
12461251
onSuccess(output) {
@@ -1268,7 +1273,10 @@ const layer = Layer.effect(
12681273
...(skills ? [skills] : []),
12691274
]
12701275
const format = lastUser.format ?? { type: "text" as const }
1271-
if (format.type === "json_schema") system.push(STRUCTURED_OUTPUT_SYSTEM_PROMPT)
1276+
// The forced-tool path needs the system prompt to coerce the model
1277+
// into calling StructuredOutput. The native path drives the schema
1278+
// through the provider's structured-output mode instead.
1279+
if (format.type === "json_schema" && !useNativeStructuredOutput) system.push(STRUCTURED_OUTPUT_SYSTEM_PROMPT)
12721280
const result = yield* handle.process({
12731281
user: lastUser,
12741282
agent,
@@ -1282,9 +1290,30 @@ const layer = Layer.effect(
12821290
],
12831291
tools,
12841292
model,
1285-
toolChoice: format.type === "json_schema" ? "required" : undefined,
1293+
// Native structured output must not force tool choice — reasoning
1294+
// providers reject `tool_choice: "required"` alongside thinking.
1295+
toolChoice: format.type === "json_schema" && !useNativeStructuredOutput ? "required" : undefined,
1296+
structuredOutput:
1297+
useNativeStructuredOutput && lastUser.format?.type === "json_schema"
1298+
? { schema: lastUser.format.schema as JSONSchema7 }
1299+
: undefined,
12861300
})
12871301

1302+
// Native path: the model returns the structured object as its text
1303+
// content. Parse it here so it flows into `message.structured`. On
1304+
// parse failure we fall through to the StructuredOutputError below,
1305+
// matching the forced-tool path's graceful degradation.
1306+
if (useNativeStructuredOutput && structured === undefined && !handle.message.error) {
1307+
const withParts = yield* sessions.messages({ sessionID }).pipe(Effect.orElseSucceed(() => []))
1308+
const assistant = withParts.find((m) => m.info.id === handle.message.id)
1309+
const text = (assistant?.parts ?? [])
1310+
.filter((p): p is SessionV1.TextPart => p.type === "text" && !p.synthetic)
1311+
.map((p) => p.text)
1312+
.join("")
1313+
const parsed = parseStructuredOutput(text)
1314+
if (parsed !== undefined) structured = parsed
1315+
}
1316+
12881317
if (structured !== undefined) {
12891318
handle.message.structured = structured
12901319
handle.message.finish = handle.message.finish ?? "stop"
@@ -1561,6 +1590,22 @@ export const CommandInput = Schema.Struct({
15611590
})
15621591
export type CommandInput = Schema.Schema.Type<typeof CommandInput>
15631592

1593+
// With native structured output the model returns the object as its text
1594+
// content. Anthropic returns bare JSON, but tolerate a stray markdown fence in
1595+
// case a provider wraps it. Returns undefined when the text isn't parseable so
1596+
// the caller can degrade to StructuredOutputError, matching the tool path.
1597+
function parseStructuredOutput(text: string): unknown {
1598+
const trimmed = text.trim()
1599+
if (!trimmed) return undefined
1600+
const fenced = trimmed.match(/^```(?:json)?\s*([\s\S]*?)\s*```$/)
1601+
const candidate = fenced ? fenced[1] : trimmed
1602+
try {
1603+
return JSON.parse(candidate)
1604+
} catch {
1605+
return undefined
1606+
}
1607+
}
1608+
15641609
/** @internal Exported for testing */
15651610
export function createStructuredOutputTool(input: {
15661611
schema: Record<string, any>

‎packages/opencode/test/session/llm.test.ts‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:tes
44
import { SessionV1 } from "@opencode-ai/core/v1/session"
55
import path from "path"
66
import { tool, type ModelMessage } from "ai"
7+
import type { JSONSchema7 } from "@ai-sdk/provider"
78
import { Cause, Effect, Exit, Fiber, Layer, Stream } from "effect"
89
import { InstanceRef } from "../../src/effect/instance-ref"
910
import { HttpClientRequest, HttpClientResponse } from "effect/unstable/http"
@@ -1214,6 +1215,93 @@ describe("session.llm.stream", () => {
12141215
},
12151216
)
12161217

1218+
const anthropicThinkingFixture = { providerID: "anthropic", modelID: "claude-sonnet-4-5" }
1219+
it.instance(
1220+
"uses native output_format (not forced tool_choice) for structured output on a thinking Anthropic model",
1221+
() =>
1222+
Effect.gen(function* () {
1223+
const fixture = loadFixture(anthropicThinkingFixture.providerID, anthropicThinkingFixture.modelID)
1224+
// Sanity: this only matters for reasoning-capable models.
1225+
expect(fixture.model.reasoning).toBe(true)
1226+
1227+
// Interrupt before consuming the response body — the request is captured
1228+
// on arrival, so the (unparseable) response never has to be drained.
1229+
const pending = waitStreamingRequest("/messages")
1230+
1231+
const resolved = yield* Provider.use.getModel(
1232+
ProviderV2.ID.make(anthropicThinkingFixture.providerID),
1233+
ModelV2.ID.make(fixture.model.id),
1234+
)
1235+
const sessionID = SessionID.make("session-test-structured-thinking")
1236+
const agent = {
1237+
name: "test",
1238+
mode: "primary",
1239+
options: {},
1240+
permission: [{ permission: "*", pattern: "*", action: "allow" }],
1241+
} satisfies Agent.Info
1242+
const user = {
1243+
id: MessageID.make("msg_user-structured-thinking"),
1244+
sessionID,
1245+
role: "user",
1246+
time: { created: Date.now() },
1247+
agent: agent.name,
1248+
// "high" variant enables Anthropic thinking for this model.
1249+
model: { providerID: ProviderV2.ID.make(anthropicThinkingFixture.providerID), modelID: resolved.id, variant: "high" },
1250+
} satisfies SessionV1.User
1251+
1252+
const schema: JSONSchema7 = {
1253+
type: "object",
1254+
properties: { description: { type: "string" } },
1255+
required: ["description"],
1256+
additionalProperties: false,
1257+
}
1258+
1259+
const fiber = yield* drain({
1260+
user,
1261+
sessionID,
1262+
model: resolved,
1263+
agent,
1264+
system: ["Return structured output."],
1265+
messages: [{ role: "user", content: "Describe the repo as JSON" }],
1266+
tools: {},
1267+
// Even if the caller forces "required" (the legacy structured-output
1268+
// path), native structured output must override it — Anthropic rejects
1269+
// thinking + forced tool use.
1270+
toolChoice: "required",
1271+
structuredOutput: { schema },
1272+
}).pipe(Effect.exit, Effect.forkScoped)
1273+
1274+
const capture = yield* Effect.promise(() => pending.request)
1275+
yield* Fiber.interrupt(fiber)
1276+
1277+
const body = capture.body
1278+
expect(body.model).toBe(resolved.api.id)
1279+
1280+
// Thinking is enabled (this is the conflicting surface).
1281+
expect(body.thinking).toBeDefined()
1282+
1283+
// Native structured output is requested via output_config.format, not a
1284+
// forced StructuredOutput tool.
1285+
const outputConfig = body.output_config as { format?: { type?: string } } | undefined
1286+
expect(outputConfig?.format?.type).toBe("json_schema")
1287+
1288+
// The forced tool choice must NOT survive — Anthropic uses { type: "any" }
1289+
// / { type: "tool" } for forced calls; native output must leave it unforced.
1290+
const toolChoice = body.tool_choice as { type?: string } | undefined
1291+
expect(toolChoice?.type === "any" || toolChoice?.type === "tool").toBe(false)
1292+
}),
1293+
{
1294+
config: () => ({
1295+
enabled_providers: [anthropicThinkingFixture.providerID],
1296+
provider: {
1297+
[anthropicThinkingFixture.providerID]: {
1298+
options: { apiKey: "test-key", baseURL: `${state.server!.url.origin}/v1` },
1299+
},
1300+
},
1301+
}),
1302+
},
1303+
)
1304+
12171305
it.instance(
12181306
"keeps tools enabled by prompt permissions",
12191307
() =>

0 commit comments

Comments
 (0)