Skip to content

Commit 9f545f2

Browse files
authored
fix: add per-model structured output opt-out (#1424)
* fix: add per-model structured output opt-out * fix: honor exact structured-output opt-outs * docs(config): clarify exact structured-output opt-outs --------- Co-authored-by: Ingwannu <[email protected]>
1 parent dd00784 commit 9f545f2

15 files changed

Lines changed: 304 additions & 19 deletions

File tree

‎docs-site/src/content/docs/ja/reference/configuration/providers.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ account を削除しても mapping は保持され、同じ id を再追加す
9393
| `noTemperatureModels?` | `string[]` |発信者指定の`temperature`を拒否するモデル。 |
9494
| `noTopPModels?` | `string[]` |発信者指定の`top_p`を拒否するモデル。 |
9595
| `noPenaltyModels?` | `string[]` |存在/周波数ペナルティを拒否するモデル。 |
96+
| `noStructuredOutputModels?` | `string[]` | `openai-chat` エンドポイントが `response_format` を拒否する正確なモデル ID。要求モデルが項目と完全一致する場合だけフィールドを省略し、その他の `openai-chat` モデルでは structured-output 変換を維持します。 |
9697
| `parallelToolCalls?` | `boolean` |並列ツール呼び出しを切り替えます。 OpenAI Chat はデフォルトでオンになっています。非チャット アダプターは明示的な `true` でのみアドバタイズします。 |
9798
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` |正確なプレースホルダー ID、欠落している端末 ID、および(`repairInvalidIds` で)正規の `msg_`/`rs_` 接頭辞を欠く message/reasoning ID に対するダウンストリーム SSE 修復はデフォルトで無効になっています。関数呼び出し ID は決して書き換えられません。組み込み DeepSeek は最後の 2 つをデフォルトで有効にします。 |
9899
| `responsesSnapshotRepair?` | `boolean` | デフォルトで無効のクライアント向け修復です。SSE と JSON の Responses ライフサイクルで欠落した status、output、ツールメタデータを補完し、raw 検査と永続化は変更しません。 |

‎docs-site/src/content/docs/ko/reference/configuration/providers.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ managed map을 활성화하면 privacy-safe selector를 만들고, 이후 계정
9393
| `noTemperatureModels?` | `string[]` | 호출자가 지정한 `temperature`를 거부하는 모델입니다. |
9494
| `noTopPModels?` | `string[]` | 호출자가 지정한 `top_p`를 거부하는 모델입니다. |
9595
| `noPenaltyModels?` | `string[]` | presence/frequency penalty를 허용하지 않는 모델입니다. |
96+
| `noStructuredOutputModels?` | `string[]` | `openai-chat` 엔드포인트가 `response_format`을 거부하는 정확한 모델 ID입니다. 요청 모델이 항목과 정확히 일치할 때만 필드를 생략하며, 그 외 `openai-chat` 모델에서는 structured-output 변환을 유지합니다. |
9697
| `parallelToolCalls?` | `boolean` | 병렬 도구 호출을 켜거나 끕니다. OpenAI Chat은 기본으로 켜져 있고, 비-chat 어댑터는 명시적으로 `true`일 때만 이를 노출합니다. |
9798
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | 기본값이 꺼진 downstream SSE 복구입니다. 정확한 자리표시자 id, 누락된 종료 id, 그리고(`repairInvalidIds`) 정규 `msg_`/`rs_` 접두사가 없는 message/reasoning id를 복구합니다. function-call id는 다시 쓰지 않습니다. 내장 DeepSeek은 마지막 두 가지를 기본으로 켭니다. |
9899
| `responsesSnapshotRepair?` | `boolean` | 기본값이 꺼진 클라이언트용 복구입니다. SSE와 JSON의 Responses 수명 주기에서 누락된 status, output, 도구 메타데이터를 채우며 raw 검사와 영속화는 변경하지 않습니다. |

‎docs-site/src/content/docs/reference/configuration/providers.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids.
103103
| `noTemperatureModels?` | `string[]` | Models that reject caller-specified `temperature`. |
104104
| `noTopPModels?` | `string[]` | Models that reject caller-specified `top_p`. |
105105
| `noPenaltyModels?` | `string[]` | Models that reject presence/frequency penalties. |
106+
| `noStructuredOutputModels?` | `string[]` | Exact model IDs whose `openai-chat` endpoint rejects `response_format`. Only an exact requested-model match omits the field; structured-output translation stays enabled for every other `openai-chat` model. |
106107
| `parallelToolCalls?` | `boolean` | Toggle parallel tool calls. OpenAI Chat defaults on; non-chat adapters advertise only on explicit `true`. |
107108
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | Disabled-by-default downstream SSE repair for exact placeholder ids, missing terminal ids, and (with `repairInvalidIds`) message/reasoning ids missing the canonical `msg_`/`rs_` prefix. Function-call ids are never rewritten. Built-in DeepSeek enables the last two by default. |
108109
| `responsesSnapshotRepair?` | `boolean` | Disabled-by-default client-facing repair for sparse Responses lifecycle snapshots in SSE and JSON. Fills missing canonical status, output, and tool metadata while raw inspection and persistence remain unchanged. |

‎docs-site/src/content/docs/reference/proxy-formats.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -149,8 +149,9 @@ Structured output is part of that translation: `response_format` with `json_obje
149149
`json_schema` is forwarded to routed `openai-chat` models. On `POST /v1/responses` the
150150
equivalent request field is `text.format`: native Responses routes preserve it in the raw
151151
Responses body, and it is translated to `response_format` when the model routes to an
152-
`openai-chat` provider. A backend without structured-output support returns its own error
153-
instead of the proxy rejecting the request locally.
152+
`openai-chat` provider. A model listed in the provider's `noStructuredOutputModels` omits
153+
`response_format` on that chat wire; sibling models keep the translation. Unclassified backends
154+
receive the field and return their own error instead of the proxy guessing their capability.
154155

155156
Non-streaming output has `object: "chat.completion"`. Streaming output uses SSE objects with
156157
`object: "chat.completion.chunk"`, choice deltas, a terminal choice with `finish_reason`, and
@@ -287,4 +288,4 @@ structurally valid Fernet runs unchanged. If an `agent_message` loses all encryp
287288
that repair, it becomes a normal user message. If a current v2 task remains genuinely encrypted
288289
but the selected routed target cannot read native ChatGPT ciphertext, opencodex fails with
289290
`unreadable_encrypted_agent_task` instead of sending unreadable bytes to that provider. See
290-
[Sub-agent Surface](/guides/sub-agent-surface/) for the client behavior around worker tasks.
291+
[Sub-agent Surface](/guides/sub-agent-surface/) for the client behavior around worker tasks.

‎docs-site/src/content/docs/ru/reference/configuration/providers.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@ cross-route credential fallback не существует. Строки API GPT-
106106
| `noTemperatureModels?` | `string[]` | Модели, отвергающие переданный вызывающей стороной `temperature`. |
107107
| `noTopPModels?` | `string[]` | Модели, отвергающие переданный вызывающей стороной `top_p`. |
108108
| `noPenaltyModels?` | `string[]` | Модели, отвергающие penalty presence/frequency. |
109+
| `noStructuredOutputModels?` | `string[]` | Точные идентификаторы моделей, чей endpoint `openai-chat` отклоняет `response_format`. Поле опускается только при точном совпадении запрошенной модели; для остальных моделей `openai-chat` преобразование structured output остаётся включённым. |
109110
| `parallelToolCalls?` | `boolean` | Переключатель parallel tool call'ов. Для OpenAI Chat по умолчанию включено; не-chat adapter'ы рекламируют это только при явном `true`. |
110111
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | По умолчанию выключенная downstream SSE-repair для exact placeholder-id, отсутствующих terminal-id и (с `repairInvalidIds`) message/reasoning id без канонического префикса `msg_`/`rs_`. Function-call id никогда не переписываются. Встроенный DeepSeek включает последние два по умолчанию. |
111112
| `responsesSnapshotRepair?` | `boolean` | По умолчанию выключенная клиентская repair для неполных lifecycle snapshot'ов Responses в SSE и JSON. Добавляет отсутствующие status, output и tool metadata, не меняя raw inspection и persistence. |

‎docs-site/src/content/docs/zh-cn/reference/configuration/providers.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ selector,而不是分配一个新名称。
9393
| `noTemperatureModels?` | `string[]` | 会拒绝调用方指定 `temperature` 的模型。 |
9494
| `noTopPModels?` | `string[]` | 会拒绝调用方指定 `top_p` 的模型。 |
9595
| `noPenaltyModels?` | `string[]` | 会拒绝 presence/frequency penalty 的模型。 |
96+
| `noStructuredOutputModels?` | `string[]` | `openai-chat` 端点拒绝 `response_format` 的精确模型 ID。仅当请求模型与条目完全匹配时才省略该字段;其他 `openai-chat` 模型仍启用 structured-output 转换。 |
9697
| `parallelToolCalls?` | `boolean` | 切换并行工具调用。OpenAI Chat 默认开启;非 chat 适配器只有显式 `true` 时才会声明支持。 |
9798
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | 默认关闭的下游 SSE 修复,用于精确占位 id、缺失的终止 id,以及(`repairInvalidIds`)缺少规范 `msg_`/`rs_` 前缀的 message/reasoning id。function-call id 永远不会被重写。内置 DeepSeek 默认启用后两项。 |
9899
| `responsesSnapshotRepair?` | `boolean` | 默认关闭的客户端修复,用于补全 SSE 与 JSON 中稀疏 Responses 生命周期快照缺失的 status、output 和工具元数据;原始检查与持久化保持不变。 |

‎src/adapters/openai-chat.ts‎

Lines changed: 19 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -813,19 +813,25 @@ export function createOpenAIChatAdapter(provider: OcxProviderConfig): ProviderAd
813813
if (provider.promptCacheKey && parsed.options.promptCacheKey !== undefined) {
814814
body.prompt_cache_key = parsed.options.promptCacheKey;
815815
}
816-
const textFormat = parsed.options.textFormat;
817-
if (textFormat?.type === "json_object") {
818-
body.response_format = { type: "json_object" };
819-
} else if (textFormat?.type === "json_schema") {
820-
body.response_format = {
821-
type: "json_schema",
822-
json_schema: {
823-
name: textFormat.name ?? "response",
824-
...(textFormat.description !== undefined ? { description: textFormat.description } : {}),
825-
...(textFormat.schema !== undefined ? { schema: textFormat.schema } : {}),
826-
...(textFormat.strict !== undefined ? { strict: textFormat.strict } : {}),
827-
},
828-
};
816+
// Structured-output support varies by the physical upstream model even when one
817+
// gateway exposes a uniform OpenAI-compatible endpoint. Keep the #1137 translation
818+
// as the default, but let an exact model opt out instead of forcing a provider-wide
819+
// rollback that would silently return prose for siblings that support JSON Schema.
820+
if (!provider.noStructuredOutputModels?.includes(parsed.modelId)) {
821+
const textFormat = parsed.options.textFormat;
822+
if (textFormat?.type === "json_object") {
823+
body.response_format = { type: "json_object" };
824+
} else if (textFormat?.type === "json_schema") {
825+
body.response_format = {
826+
type: "json_schema",
827+
json_schema: {
828+
name: textFormat.name ?? "response",
829+
...(textFormat.description !== undefined ? { description: textFormat.description } : {}),
830+
...(textFormat.schema !== undefined ? { schema: textFormat.schema } : {}),
831+
...(textFormat.strict !== undefined ? { strict: textFormat.strict } : {}),
832+
},
833+
};
834+
}
829835
}
830836

831837
if (tools) {

‎src/config.ts‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -629,6 +629,9 @@ const providerConfigSchema = z.object({
629629
supportsServiceTier: z.boolean().optional(),
630630
preserveResponsesReasoningContent: z.boolean().optional(),
631631
allowPrivateNetwork: z.boolean().optional(),
632+
noStructuredOutputModels: z.array(z.string().min(1))
633+
.transform(normalizeNonBlankStringArray)
634+
.optional(),
632635
retryOn429: retryOn429PolicySchema.optional(),
633636
codexAccountMode: z.enum(["pool", "direct"]).optional(),
634637
responsesItemIdRepair: z.object({
@@ -820,6 +823,26 @@ export function positiveIntegerConfigError(value: unknown, field: string): strin
820823
return null;
821824
}
822825

826+
export function nonBlankStringArrayConfigError(value: unknown, field: string): string | null {
827+
if (value === undefined) return null;
828+
if (!Array.isArray(value)) return `${field} must be an array`;
829+
for (const [index, entry] of value.entries()) {
830+
if (typeof entry !== "string" || !entry.trim()) {
831+
return `${field}.${index} must be a nonblank model id`;
832+
}
833+
}
834+
return null;
835+
}
836+
837+
/**
838+
* Keep hand-edited config and management writes on one canonical model-id list.
839+
* Validation happens separately so an all-whitespace value is rejected rather than
840+
* normalized into a model id that can never match at runtime.
841+
*/
842+
export function normalizeNonBlankStringArray(value: readonly string[]): string[] {
843+
return [...new Set(value.map(entry => entry.trim()))];
844+
}
845+
823846
export function booleanRecordConfigError(value: unknown, field: string): string | null {
824847
if (value === undefined) return null;
825848
if (!value || typeof value !== "object" || Array.isArray(value)) return `${field} must be a plain object`;
@@ -1408,6 +1431,17 @@ const configSchema = z.object({
14081431
message: maxOutputError,
14091432
});
14101433
}
1434+
const structuredOutputOptOutError = nonBlankStringArrayConfigError(
1435+
(provider as { noStructuredOutputModels?: unknown }).noStructuredOutputModels,
1436+
"noStructuredOutputModels",
1437+
);
1438+
if (structuredOutputOptOutError) {
1439+
ctx.addIssue({
1440+
code: "custom",
1441+
path: ["providers", redactSecretString(name), "noStructuredOutputModels"],
1442+
message: structuredOutputOptOutError,
1443+
});
1444+
}
14111445
if (Object.hasOwn(provider, "codexAccountMode") && provider.codexAccountMode !== undefined) {
14121446
// Persisted account mode is valid ONLY on the canonical built-in `openai` forward provider.
14131447
// Old openai-multi rows stay parseable (they never carry a mode) so startup can migrate them.

‎src/server/auth-cors.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import {
66
modelAdapterRecordConfigError,
77
modelPreferHostedToolsConfigError,
88
codexAutoStartEnabled,
9+
nonBlankStringArrayConfigError,
910
positiveIntegerConfigError,
1011
positiveIntegerRecordConfigError,
1112
providerBaseUrlConfigError,
@@ -509,6 +510,11 @@ export function providerManagementConfigError(name: unknown, provider: unknown):
509510
if (defaultMaxOutputError) return `provider ${name} ${defaultMaxOutputError}`;
510511
const maxOutputError = positiveIntegerRecordConfigError(raw.modelMaxOutputTokens, "modelMaxOutputTokens");
511512
if (maxOutputError) return `provider ${name} ${maxOutputError}`;
513+
const structuredOutputOptOutError = nonBlankStringArrayConfigError(
514+
raw.noStructuredOutputModels,
515+
"noStructuredOutputModels",
516+
);
517+
if (structuredOutputOptOutError) return `provider ${name} ${structuredOutputOptOutError}`;
512518
const openRouterError = openRouterRoutingConfigError(typed);
513519
if (openRouterError) return `provider ${name} ${openRouterError}`;
514520
if (typed.authMode === "local") {
@@ -589,6 +595,7 @@ export function safeConfigDTO(config: OcxConfig): unknown {
589595
"noTemperatureModels",
590596
"noTopPModels",
591597
"noPenaltyModels",
598+
"noStructuredOutputModels",
592599
"autoToolChoiceOnlyModels",
593600
"preserveReasoningContentModels",
594601
"requiresReasoningPlaceholderModels",

‎src/server/management/provider-routes.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ import {
88
hasOwnProvider,
99
isValidProviderName,
1010
multiAgentGuidanceEnabled,
11+
nonBlankStringArrayConfigError,
12+
normalizeNonBlankStringArray,
1113
providerBaseUrlConfigError,
1214
providerHeadersConfigError,
1315
saveConfigPreservingClaudeCode,
@@ -205,6 +207,19 @@ function applyProviderPatchFields(
205207
}
206208
touched = true;
207209
}
210+
if (Object.hasOwn(rawBody, "noStructuredOutputModels")) {
211+
const value = rawBody.noStructuredOutputModels;
212+
if (value === null) {
213+
delete next.noStructuredOutputModels;
214+
} else {
215+
const error = nonBlankStringArrayConfigError(value, "noStructuredOutputModels");
216+
if (error) return { error };
217+
const models = normalizeNonBlankStringArray(value as string[]);
218+
if (models.length > 0) next.noStructuredOutputModels = models;
219+
else delete next.noStructuredOutputModels;
220+
}
221+
touched = true;
222+
}
208223

209224
// headers is the one object-valued field in the mask. PATCH semantics merge it
210225
// shallowly into the existing block so a single fingerprint header can be added
@@ -291,6 +306,7 @@ export async function handleProviderRoutes(ctx: ManagementContext): Promise<Resp
291306
models: p.models ?? [],
292307
contextWindow: p.contextWindow,
293308
modelContextWindows: p.modelContextWindows,
309+
noStructuredOutputModels: p.noStructuredOutputModels,
294310
authMode: p.authMode,
295311
apiKeyTransport: p.apiKeyTransport,
296312
disabled: p.disabled === true,

0 commit comments

Comments
 (0)