Skip to content

Commit c472ad0

Browse files
ntdatt812claude
andauthored
fix(chat): keep the structured-output opt-out exact on the native chat wire (#2042)
`noStructuredOutputModels` is documented, in every locale, as "Exact model IDs whose `openai-chat` endpoint rejects `response_format`. Only an exact requested-model match omits the field; structured-output translation stays enabled for every other `openai-chat` model." The Responses ingress enforces that, and tests/openai-chat-hardening.test.ts already pins a `:tag` sibling keeping the field there. The native Chat passthrough added in #1467 matched through `modelInList` instead, which also matches the pre-colon prefix. On a provider that serves Ollama-style tags -- ollama-cloud ships `gpt-oss:120b`, `qwen3-coder:480b`, `qwen3.5:397b`, `gemma4:31b` -- a `noStructuredOutputModels: ["gpt-oss"]` entry therefore stripped `response_format` from `gpt-oss:120b` on /v1/chat/completions while /v1/responses kept it. The caller asked for JSON and silently got prose, on a model the operator never opted out. That is the failure #1424 called out when it chose the exact boundary: a wider match "would silently return prose for siblings that support JSON Schema". The sibling gates on the lines above keep `modelInList` -- `noVisionModels` is documented as tolerating an Ollama `:size` tag, this one is not -- so the comment now says why this gate differs. Four tests, next to the existing Responses-side assertions so the two ingresses read as a pair: exact id opts out, a `:tag` sibling does not, the full `:tag` id does when listed, and an unrelated model is untouched. The `:tag` sibling case fails on current dev. Co-authored-by: Claude Opus 5 <[email protected]>
1 parent 0161a66 commit c472ad0

2 files changed

Lines changed: 38 additions & 2 deletions

File tree

‎src/adapters/openai-chat.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,11 @@ export function buildOpenAIChatPassthroughRequest(
116116
delete body.presence_penalty;
117117
delete body.frequency_penalty;
118118
}
119-
if (modelInList(provider.noStructuredOutputModels, modelId)) delete body.response_format;
119+
// Exact match, unlike the gates above: `noStructuredOutputModels` is documented as
120+
// "only an exact requested-model match omits the field" (#1424), and the Responses
121+
// ingress enforces exactly that. A prefix match here would strip response_format from
122+
// `<listed>:<tag>` siblings the operator never opted out, silently returning prose.
123+
if (provider.noStructuredOutputModels?.includes(modelId)) delete body.response_format;
120124

121125
if (provider.chatServiceTier && rawBody.service_tier !== undefined) {
122126
body.service_tier = rawBody.service_tier;

‎tests/openai-chat-hardening.test.ts‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { afterEach, describe, expect, test } from "bun:test";
2-
import { createOpenAIChatAdapter as createOpenAIChatAdapterProduction } from "../src/adapters/openai-chat";
2+
import { buildOpenAIChatPassthroughRequest, createOpenAIChatAdapter as createOpenAIChatAdapterProduction } from "../src/adapters/openai-chat";
33
import { stripResponsesOnlyEncryptedMarker } from "../src/adapters/responses-tool-schema";
44
import { getDebugLogEntries, resetDebugLogBufferForTests } from "../src/lib/debug-log-buffer";
55
import { resetDebugSettingsForTests } from "../src/lib/debug-settings";
@@ -792,4 +792,36 @@ describe("openai-chat response_format emission", () => {
792792
json_schema: { name: "answer", schema: { type: "object" }, strict: true },
793793
});
794794
});
795+
796+
// The native Chat ingress reads the same provider option and must draw the same
797+
// boundary. It used to match through modelInList, so a `<listed>:<tag>` sibling
798+
// lost response_format on this wire while keeping it on Responses.
799+
describe("native chat passthrough draws the same exact boundary", () => {
800+
const passthrough = (modelId: string, noStructuredOutputModels: string[]) =>
801+
JSON.parse(buildOpenAIChatPassthroughRequest(
802+
provider({ noStructuredOutputModels }),
803+
{ messages: [{ role: "user", content: "hi" }], response_format: { type: "json_object" } },
804+
modelId,
805+
false,
806+
).body as string) as Record<string, unknown>;
807+
808+
test("omits response_format for the exact listed id", () => {
809+
expect(passthrough("test-model", ["test-model"]).response_format).toBeUndefined();
810+
});
811+
812+
test("keeps response_format for a :tag sibling the operator never listed", () => {
813+
expect(passthrough("test-model:structured", ["test-model"]).response_format)
814+
.toEqual({ type: "json_object" });
815+
});
816+
817+
test("listing the full :tag id opts that id out", () => {
818+
expect(passthrough("test-model:structured", ["test-model:structured"]).response_format)
819+
.toBeUndefined();
820+
});
821+
822+
test("leaves an unrelated model untouched", () => {
823+
expect(passthrough("supported-model", ["test-model"]).response_format)
824+
.toEqual({ type: "json_object" });
825+
});
826+
});
795827
});

0 commit comments

Comments
 (0)