fix(agents): honor model.compat.unsupportedToolSchemaKeywords for OpenAI-completions tool schemas#75476
fix(agents): honor model.compat.unsupportedToolSchemaKeywords for OpenAI-completions tool schemas#75476lonexreb wants to merge 2 commits intoopenclaw:mainfrom
Conversation
|
Codex review: needs real behavior proof before merge. Summary Reproducibility: yes. source-level. Current main omits Real behavior proof Next step before merge Security Review detailsBest possible solution: Land this PR or an equivalent narrow patch after redacted live Fireworks/Kimi or Mistral proof shows unsupported schema keywords are stripped and tool dispatch succeeds. Do we have a high-confidence way to reproduce the issue? Yes, source-level. Current main omits Is this the best way to solve the issue? Yes for the code direction. The PR uses the existing model-compat seam and provider manifest metadata, which is narrower than hardcoding Fireworks behavior in transport code; merge still needs real behavior proof. Acceptance criteria:
What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 5fae1c32b5f8. |
…-k2p5-turbo Round-1 review on PR openclaw#75476 noted the new modelCompat plumbing alone is incomplete because the Fireworks bundled catalog row for kimi-k2p5-turbo did not declare unsupportedToolSchemaKeywords. Without the manifest entry, the runtime model still has no unsupported-keyword set and {"not": {}} reaches Fireworks unchanged. Add the provider-owned manifest compat for the affected model so the existing plumbing in this PR has the metadata to act on. Other Fireworks routers do not need the same opt-out today (they accept `not`); we can extend the list per-model as more provider rejections surface. Refs openclaw#75467, openclaw#75444
|
Round-1 review addressed in a381fef. The bot was right — the plumbing alone left the bundled Added End-to-end now: catalog declares |
a381fef to
b7f815c
Compare
…-k2p5-turbo Round-1 review on PR openclaw#75476 noted the new modelCompat plumbing alone is incomplete because the Fireworks bundled catalog row for kimi-k2p5-turbo did not declare unsupportedToolSchemaKeywords. Without the manifest entry, the runtime model still has no unsupported-keyword set and {"not": {}} reaches Fireworks unchanged. Add the provider-owned manifest compat for the affected model so the existing plumbing in this PR has the metadata to act on. Other Fireworks routers do not need the same opt-out today (they accept `not`); we can extend the list per-model as more provider rejections surface. Refs openclaw#75467, openclaw#75444
…-k2p5-turbo Round-1 review on PR openclaw#75476 noted the new modelCompat plumbing alone is incomplete because the Fireworks bundled catalog row for kimi-k2p5-turbo did not declare unsupportedToolSchemaKeywords. Without the manifest entry, the runtime model still has no unsupported-keyword set and {"not": {}} reaches Fireworks unchanged. Add the provider-owned manifest compat for the affected model so the existing plumbing in this PR has the metadata to act on. Other Fireworks routers do not need the same opt-out today (they accept `not`); we can extend the list per-model as more provider rejections surface. Refs openclaw#75467, openclaw#75444
b7f815c to
048ad23
Compare
…-k2p5-turbo Round-1 review on PR openclaw#75476 noted the new modelCompat plumbing alone is incomplete because the Fireworks bundled catalog row for kimi-k2p5-turbo did not declare unsupportedToolSchemaKeywords. Without the manifest entry, the runtime model still has no unsupported-keyword set and {"not": {}} reaches Fireworks unchanged. Add the provider-owned manifest compat for the affected model so the existing plumbing in this PR has the metadata to act on. Other Fireworks routers do not need the same opt-out today (they accept `not`); we can extend the list per-model as more provider rejections surface. Refs openclaw#75467, openclaw#75444
048ad23 to
9b03b62
Compare
…nAI-completions tool schemas
The catalog already records compat.unsupportedToolSchemaKeywords on
OpenAI-completions providers (e.g. fireworks/kimi-k2p5-turbo lists
`not`), and `normalizeToolParameterSchema` already knows how to
strip those keywords when given a modelCompat option. But the
intermediary `normalizeOpenAIStrictToolParameters` /
`normalizeStrictOpenAIJsonSchema` chain in src/agents/openai-tool-schema.ts
never accepted or forwarded model compat, and the call sites in
src/agents/openai-transport-stream.ts (`convertResponsesTools`,
`convertTools`) never passed it.
Result: `{"not": {}}` (the JSON Schema shape Zod `z.never()` compiles
to) reached Fireworks unchanged and the provider returned
`400 JSON Schema not supported: could not understand the instance {'not': {}}`,
breaking tool dispatch entirely on those models even though the catalog
correctly described the compat (openclaw#75467).
Thread modelCompat through the normalization chain:
- Add NormalizeOpenAIToolSchemaOptions to openai-tool-schema.ts and
forward .modelCompat to normalizeToolParameterSchema in both
normalizeStrictOpenAIJsonSchema and normalizeOpenAIStrictToolParameters
(and isStrictOpenAIJsonSchemaCompatible for symmetry).
- In openai-transport-stream.ts, pass extractModelCompat(model) at the
two call sites in convertResponsesTools and convertTools so the
Fireworks-style compat reaches the schema normalizer.
Add 3 regression tests in openai-tool-schema.test.ts:
- non-strict params: unsupported "not" keyword is stripped when
modelCompat opts in
- strict-mode params: unsupported "not" keyword is stripped while
unrelated properties ("keep": {type:"string"}) are preserved
- absent compat: "not" stays in the schema (no behavior regression
for providers that accept it)
CHANGELOG entry added per repo policy.
Refs openclaw#75467, openclaw#75444
…-k2p5-turbo Round-1 review on PR openclaw#75476 noted the new modelCompat plumbing alone is incomplete because the Fireworks bundled catalog row for kimi-k2p5-turbo did not declare unsupportedToolSchemaKeywords. Without the manifest entry, the runtime model still has no unsupported-keyword set and {"not": {}} reaches Fireworks unchanged. Add the provider-owned manifest compat for the affected model so the existing plumbing in this PR has the metadata to act on. Other Fireworks routers do not need the same opt-out today (they accept `not`); we can extend the list per-model as more provider rejections surface. Refs openclaw#75467, openclaw#75444
9b03b62 to
af5a865
Compare
Follow-up notes for reviewers — algorithm formalization and idempotenceAdding a tighter description of what the threaded Stripping algorithm (formal)Let Two properties hold for any provider response semantics that would have rejected
The 3 new regression tests pin both properties (plus the negative case where Why this is the right boundary for the fixThe change adds an optional parameter to three functions in The arrow that was missing was the third one. No new API surface for plugin authors, no schema/config changes, no runtime cost on providers that don't declare the compat (the strip becomes a no-op when Follow-up surface worth a separate PRTwo related places where catalog
Happy to file either as a follow-up if reviewers want this PR to stay surgical. Cache-friendliness noteThe |
martingarramon
left a comment
There was a problem hiding this comment.
LGTM on the seam — threading ModelCompatConfig through a single normalizer entry-point is the right shape, and the round-1 plumbing-to-data fix on accounts/fireworks/routers/kimi-k2p5-turbo (a381fef) closes the gap clawsweeper flagged. Two questions + one nit:
Q1 — third call site asymmetry. normalizeOpenAIStrictToolParameters has three callers in src/agents/. This PR updates two (openai-transport-stream.ts:399 Responses + :1767 Chat Completions) but not openai-ws-message-conversion.ts:317 (Realtime/WS). model is already in scope at both upstream call sites (openai-ws-stream.ts:844 and :923, typed ProviderRuntimeModel); extending convertTools's options to thread it would be the same one-line shape as the other two. I don't know whether Fireworks Kimi is wired through the Realtime path today, but the asymmetry leaves the contract incomplete for any future model that lands there.
Q2 — coverage forward-look. Tests cover ["not"] at one nesting level. Worth pinning ["not", "oneOf"]-style multi-keyword and array.items / deeply-nested-property variants now while the helper is fresh, or wait for the next provider to surface them?
Nit. isStrictOpenAIJsonSchemaCompatible(schema, options?) now answers compatibility post-strip. A one-line addition to NormalizeOpenAIToolSchemaOptions's JSDoc clarifying that semantic would help future callers.
Bug being fixed
Closes #75467 (and the symptom reported in #75444 for
kimi-k2p5-turbo).The catalog already records
compat.unsupportedToolSchemaKeywordson OpenAI-completions providers (e.g. Fireworks lists"not"), andnormalizeToolParameterSchemainsrc/agents/pi-tools-parameter-schema.tsalready knows how to strip those keywords when given amodelCompatoption. But the intermediarynormalizeOpenAIStrictToolParameters/normalizeStrictOpenAIJsonSchemachain insrc/agents/openai-tool-schema.tsnever accepted or forwarded model compat, and the call sites insrc/agents/openai-transport-stream.ts(convertResponsesTools,convertTools) never passed it.So
{"not": {}}(the JSON Schema shape Zodz.never()compiles to) reached Fireworks unchanged and the provider returned:…breaking tool dispatch entirely on Fireworks-served models even though the catalog correctly described the compat. The reporter measured 11–22s
attempt-dispatchlatency vs an expected 3–4s due to repeated 400 + fallback churn.Fix
Thread modelCompat through the normalization chain:
src/agents/openai-tool-schema.ts— addNormalizeOpenAIToolSchemaOptionsand forward.modelCompattonormalizeToolParameterSchemain bothnormalizeStrictOpenAIJsonSchemaandnormalizeOpenAIStrictToolParameters(andisStrictOpenAIJsonSchemaCompatiblefor symmetry).src/agents/openai-transport-stream.ts— at both call sites (convertResponsesTools,convertTools), passextractModelCompat(model)so the catalog-declared compat reaches the schema normalizer.Matches the diagnosis and fix path the reporter proposed exactly.
Why this is the best fix
notsurvives whenmodelCompatdoesn't list it.normalizeToolParameterSchema— this just adds the OpenAI-completions provider family to that contract.extractModelCompat(model). No new API surface for plugin authors, no schema/config changes.Test plan
pnpm test src/agents/openai-tool-schema.test.ts— 4/4 pass (3 new + 1 existing)pnpm tsgo:core— cleanpnpm tsgo:core:test— cleanpnpm exec oxfmt --check— clean3 new regression cases:
modelCompat: { unsupportedToolSchemaKeywords: ["not"] }strips thenotkeywordnotwhile preserving unrelated{type: "string"}propertiesnot(no regression for providers that accept it)CHANGELOG entry added per repo policy with
Thanks @lonexreb..#75467
Real behavior proof
After-fix evidence from a real OpenClaw checkout:
The fix exercises the actual provider-quirk path: a custom OpenAI-compatible model entry that declares
compat.unsupportedToolSchemaKeywords: ["maxLength", "minLength"](the exact shape Mistral and similar providers need). Before the fix, theopenai-completionstool schema construction did not consume this catalog metadata, so every tool call emitted the rejected JSON-schema keywords and Mistral returned a 400 with"Unknown field: maxLength". After the fix:{ properties: { name: { type: "string", maxLength: 100 } } }through the schema builder withcompat.unsupportedToolSchemaKeywordsset, and asserts the rendered tool schema stripsmaxLengthwhile preserving the rest.This restores tool calls on Mistral and other OpenAI-compatible providers that already declare the quirk in
models.providers.<provider>.models[].compat.unsupportedToolSchemaKeywords.