refactor(ai): structured output via schema on generate, stream and loop by rekram1-node · Pull Request #60783 · anomalyco/opencode

rekram1-node · 2026-10-31T03:20:47Z

Change

Structured output moves from a separate generateObject call to a schema option on ai.generate, ai.stream and ai.loop, in both the Promise and Effect clients. How it works doesn't change: the model still answers through the same forced hidden tool call on every protocol.

// Before
const { object } = await ai.generateObject({ model, prompt: "Pick a city", schema: City })

// After
const res = await ai.generate({ model, prompt: "Pick a city", schema: City })
res.object.city // typed from the schema

schema takes an Effect Schema (typed object) or a raw JSON Schema object (object: unknown). The old jsonSchema: input is now schema:.
The result is the normal LLMResponse plus object, so res.response.* becomes res.*.
stream: the finish event carries object.
loop: user tools run as usual. With tools present, the hidden tool is only required, not named, so the model can call user tools first. loop never runs the hidden tool itself. A step that calls it ends the loop, even if it also called other tools, and that step's finish carries object.
Errors work as before: a missing call, the token limit or a decode failure is an AIError with the response in partial.
generateObject is gone, along with its types, README section and tutorial step.

Follow-up

Native JSON modes (OpenAI json_schema, Gemini response schema, Anthropic output format) where the protocol supports them.
Streaming partial objects.

Verification

bun run test in packages/ai: 2003 pass, 0 fail.
bun typecheck and typecheck:tsc pass in packages/ai and packages/core.
bun run check from the root passes.

Added by the opencode bot (AI-generated).

Public API: removes generateObject in favour of a schema option on generate/stream/loop with object on the result and finish events (breaking for generateObject callers)

opencodejr

The design holds up: a canonical responseFormat, an explicit supportsStructuredOutput on each protocol, and the tool-call fallback rewritten into text events in the client. Undeclared protocols fall back safely, so the schema is never silently dropped. I ran the packages/ai suite (2007 pass, 0 fail) and typecheck:tsc in packages/ai and packages/core, and checked strictJsonSchema on the schemas Effect Schema produces. Live providers weren't available, and nothing in OpenCode calls this API yet, so there was nothing to run end to end.

Public API (@opencode/cli packages/ai), not backwards compatible for generateObject callers:

Removed: ai.generateObject / LLM.generateObject (Promise and Effect clients), GenerateObjectResponse, GenerateObjectOptions, GenerateObjectDynamicOptions, and the jsonSchema input. Callers move to ai.generate({ ..., schema }), where jsonSchema: x becomes schema: x. res.response.* becomes res.*: the result is now an LLMResponse with object added.
Added: a schema option on generate, stream and loop; object on finish events; the ResponseFormat export and LLMRequest.responseFormat; Protocol.supportsStructuredOutput / Route.supportsStructuredOutput; the model option compatibility.supportsStructuredOutput.
Changed: the fallback tool is now structured_output (was generate_object). Its call comes back as text instead of a tool call.

Concerns, inline: the native format is now the default for every OpenAI Chat-compatible and Anthropic-compatible route. The old forced tool call worked on all models, so endpoints without json_schema support will now return errors unless each model opts out.

opencodejr · 2026-10-31T03:23:07Z

    schema: OpenAIChatBody,
    from: fromRequest,
  },
+  supportsStructuredOutput: true,


OpenAICompatibleChat reuses this protocol as is, so every generic OpenAI-compatible endpoint (DeepSeek, older Ollama or llama.cpp builds, many gateways) now gets response_format: { type: "json_schema" } by default. generateObject used to work there, and some of these reject json_schema (DeepSeek, for example, only accepts json_object), so they break until every model is opted out. I know this is listed as out of scope, but defaulting the generic compatible route to the fallback and keeping native for first-party OpenAI/Azure would avoid the regression.

Fixed in fcc5e82: the generic OpenAICompatibleChat route and DeepSeek now set supportsStructuredOutput: false (new route.with option) and use the forced tool call. First-party OpenAI, and named providers that reuse the protocol, keep the native format. A model can still opt out with compatibility.supportsStructuredOutput: false.

opencodejr · 2026-10-31T03:23:07Z

    step,
  },
  supportsEffortUpdates,
+  supportsStructuredOutput: true,


This turns on output_config.format for every Claude model on every Messages route, including Bedrock Mantle and the Anthropic-compatible providers (MiniMax, Moonshot, Z.ai and others). As far as I know, only recent Claude models accept it, so older ones that worked with the forced tool call may now return a 400. A model-version check, or defaulting to the fallback for compatible providers, would keep the old behaviour where native support is uncertain.

Partly fixed: the Anthropic-compatible, MiniMax, Moonshot and Bedrock Mantle Messages routes now use the fallback. First-party Anthropic stays native. I'm not adding a model-version check because the brief rules out guessing from model names. Older Claude models can opt out per model.

opencodejr · 2026-10-31T03:23:07Z

+      }),
+    ],
+    // With other tools available the model may still call them first, as long as it calls some tool.
+    toolChoice:


The fallback replaces any toolChoice the caller set. "none" or a named tool becomes required (or structured_output) without a word. It's probably fine to keep it, but a named or none choice from the caller should either be kept or rejected clearly, not dropped silently.

Kept on purpose: with the fallback, the answer has to arrive as that tool call, so a caller's none or named choice can't be honored. The comment now says so, and the PR body documents it.

opencodejr

The new commits fix the OpenAI-compatible regression: the generic OpenAI-compatible chat route and DeepSeek now use the fallback, and the route-level override carries through .with(). On the Anthropic side, the compatible routes (anthropic-compatible, MiniMax, Moonshot, Bedrock Mantle Messages) now fall back too. Older first-party Claude models still get output_config.format by default, which the description now lists as out of scope with a per-model opt-out. The toolChoice replacement on the fallback is now documented rather than changed.

All tests in packages/ai pass (2008 pass, 0 fail) and typecheck:tsc passes. I couldn't run anything against real providers.

Public API: in addition to the earlier list, this adds supportsStructuredOutput to RoutePatch (route.with({ supportsStructuredOutput })), which is backwards compatible.

opencodejr

With the native JSON modes dropped, this is now a narrower refactor and it looks sound, apart from one new bug in loop (inline). Threads 1 and 2 are moot now that the native formats are gone. Thread 3 still applies: a toolChoice the caller sets is still replaced without notice when schema is set. All tests in packages/ai pass and typecheck:tsc passes. I didn't run anything against real providers.

Public API (packages/ai), breaking for generateObject callers:

Removed: ai.generateObject / LLM.generateObject (Promise and Effect clients), GenerateObjectResponse, GenerateObjectOptions, GenerateObjectDynamicOptions, and the jsonSchema input.
Added: a schema option on generate, stream and loop; object on the result and on finish events; the OutputSchema, OutputType, ObjectInput, ObjectResponse, ObjectEvent and LoopObjectEvent types.
Migration: generateObject({ schema }) becomes generate({ schema }), jsonSchema: becomes schema:, and res.response.* becomes res.*.

opencodejr · 2026-10-31T04:02:41Z

  const { tools, until, ...rest } = input
-  const step = (request: LLMRequest, steps: ReadonlyArray<LLMResponse>): Stream.Stream<LLMEvent, AIError, Service> =>
+  const clientCalls = (response: LLMResponse) =>
+    response.toolCalls.filter((call) => call.providerExecuted !== true && call.name !== OBJECT_TOOL)


The model can call generate_object and a user tool in the same step (parallel tool calls). clientCalls leaves the hidden call out, so the loop runs only the user tool and continues. The next request then has response.message with the generate_object tool call but no tool result for it. OpenAI and Anthropic reject that with a 400, and the answer from that step is lost. Two options: if generate_object is called, end the loop and decode its input; or add a result message for the hidden call before continuing.

Fixed: a step that calls generate_object now ends the loop and its input is decoded, even if it also called other tools. Added a test.

opencodejr

Thread 4 is fixed. A loop step that calls generate_object now ends the loop, even if it also called other tools, so no next request goes out with an unanswered tool call. The new test covers that case directly. All of packages/ai passes (0 fail) and typecheck:tsc passes. I didn't run it against a real provider. The public API impact is unchanged since my last review: generateObject is removed in favour of a schema option on generate, stream and loop. Nothing new to flag.

github-actions Bot added the contributor label Oct 31, 2026

opencodejr Bot reviewed Oct 31, 2026

View reviewed changes

rekram1-node force-pushed the ai-schema-output branch from 604bf89 to fcc5e82 Compare October 31, 2026 03:28

opencodejr Bot reviewed Oct 31, 2026

View reviewed changes

rekram1-node changed the title ~~feat(ai): schema on generate, stream and loop; remove generateObject~~ refactor(ai): structured output via schema on generate, stream and loop Oct 31, 2026

rekram1-node force-pushed the ai-schema-output branch from fcc5e82 to e347266 Compare October 31, 2026 03:56

opencodejr Bot reviewed Oct 31, 2026

View reviewed changes

rekram1-node added 2 commits October 30, 2026 23:05

refactor(ai): structured output via schema on generate, stream and loop

3cbb82c

fix(ai): end the loop at a step that calls the answer tool

96653ef

rekram1-node force-pushed the ai-schema-output branch from e347266 to 96653ef Compare October 31, 2026 04:05

opencodejr Bot reviewed Oct 31, 2026

View reviewed changes

rekram1-node merged commit e8238ba into dev Oct 31, 2026
24 checks passed

rekram1-node deleted the ai-schema-output branch October 31, 2026 04:35

Navigation Menu

Repository navigation

refactor(ai): structured output via schema on generate, stream and loop - #60783

refactor(ai): structured output via schema on generate, stream and loop#60783
rekram1-node merged 2 commits into
devfrom
ai-schema-output

rekram1-node commented Oct 31, 2026 •

edited

Loading

Uh oh!

opencodejr Bot left a comment

Uh oh!

opencodejr Bot Oct 31, 2026

Uh oh!

rekram1-node Oct 31, 2026

Uh oh!

opencodejr Bot Oct 31, 2026

Uh oh!

rekram1-node Oct 31, 2026

Uh oh!

opencodejr Bot Oct 31, 2026

Uh oh!

rekram1-node Oct 31, 2026

Uh oh!

opencodejr Bot left a comment

Uh oh!

opencodejr Bot left a comment

Uh oh!

opencodejr Bot Oct 31, 2026

Uh oh!

rekram1-node Oct 31, 2026

Uh oh!

opencodejr Bot left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

rekram1-node commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change

Follow-up

Verification

Uh oh!

opencodejr Bot left a comment

Choose a reason for hiding this comment

Uh oh!

opencodejr Bot Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

rekram1-node Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

opencodejr Bot Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

rekram1-node Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

opencodejr Bot Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

rekram1-node Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

opencodejr Bot left a comment

Choose a reason for hiding this comment

Uh oh!

opencodejr Bot left a comment

Choose a reason for hiding this comment

Uh oh!

opencodejr Bot Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

rekram1-node Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

opencodejr Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rekram1-node commented Oct 31, 2026 •

edited

Loading