Skip to content

fix: Disable strict mode tools for non-native openai-completions compatible APIs#41457

Closed
sahancava wants to merge 21 commits intoopenclaw:mainfrom
sahancava:fix/41328-openai-completions-tools
Closed

fix: Disable strict mode tools for non-native openai-completions compatible APIs#41457
sahancava wants to merge 21 commits intoopenclaw:mainfrom
sahancava:fix/41328-openai-completions-tools

Conversation

@sahancava
Copy link
Copy Markdown
Contributor

@sahancava sahancava commented Mar 9, 2026

Summary

Describe the problem and fix in 2–5 bullets:

  • Problem: When using third-party OpenAI-compatible APIs (like Alibaba Cloud Bailian, DeepSeek, GLM, etc) with the openai-completions API type and compat.supportsTools: true, tool calling fails natively. The model receives an invalid payload and outputs raw Markdown blocks instead (e.g. bash exec --command...).
  • Why it matters: This breaks proper tool execution and prevents agents from effectively completing actions when using perfectly capable, cheaper compatible LLMs.
  • What changed:
    • Identified that the underlying @mariozechner/pi-ai library natively injects "strict": false into all tool schemas by default due to newer OpenAI API constraints.
    • Updated the normalizeModelCompat interceptor in src/agents/model-compat.ts to automatically default to supportsStrictMode: false for all non-native openai-completions endpoints.
    • Explicit configuration opt-ins (e.g. compat.supportsStrictMode: true) are now preserved, allowing an escape hatch for true API native proxies masquerading under vanity URLs.
    • Added unit tests in src/agents/model-compat.test.ts to assert that non-native proxies (like DashScope, proxy URLs, etc) natively drop the "strict" schema wrapper, while preserving explicit true overrides.
  • What did NOT change (scope boundary): Native OpenAI and Azure API interactions remain completely unchanged. The @mariozechner/pi-ai upstream payload generator behavior remains unmodified.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

Users configuring third-party OpenAI-compatible providers in their openclaw.json (such as DashScope, MoonShot, DeepSeek APIs) with compat.supportsTools: true will now see the models effectively utilizing configured agent tools correctly via the native tool_calls parameter finish reasons instead of plaintext.

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: Linux / macOS / Windows
  • Runtime/container: Node.js 22.x+
  • Model/provider: openai-completions via bailian (Alibaba DashScope) / qwen3-coder-plus
  • Integration/channel (if any): Local OpenClaw CLI
  • Relevant config (redacted): Base provider configured to hit https://coding.dashscope.aliyuncs.com/v1 with compat.supportsTools: true.

Steps

  1. Configure an OpenAI-compatible provider targeting a third-party gateway with tools enabled.
  2. Run an agent execution using tools, e.g., openclaw agent --agent main --local -m "Run echo test".
  3. Inspect the streaming result behavior.

Expected

  • The model issues a native tool_call chunk to securely invoke the system command instead of outputting markdown text.

Actual

  • Prior to this PR, API providers rejected the LLM structure JSON payload due to the custom schema key (strict: false) sent by pi-ai and defaulted back to treating tools purely as string context.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

(Note: Unit assertions were added into src/agents/model-compat.test.ts successfully validating flag fallback propagation).

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: Traced CLI output payload overrides on a local testing script bridging @mariozechner/pi-ai, demonstrating that omitting strict successfully forces the compatible API to accept tool mappings.
  • Edge cases checked: Asserted that api.openai.com native domains maintain strict: false schema logic correctly. Explicitly verified that test cases asserting compat.supportsStrictMode: true overrides are correctly preserved.
  • What you did not verify: Did not end-to-end execute LLM prompts against a live Bailian API key since I only tested the payload generation locally.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No
  • If yes, exact upgrade steps: N/A

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: Revert this specific PR commit.
  • Known bad symptoms reviewers should watch for: Native OpenAI structured outputs unexpectedly missing from core provider calls if the domain resolver intercepts it improperly (ex. custom enterprise vanity URLs acting directly as OpenAI).

Risks and Mitigations

  • Risk: A consumer deploying a native Azure/OpenAI instance internally behind a custom Vanity URL might unexpectedly lose tool strict schema rendering if their baseUrl is unrecognized.
    • Mitigation: Users can explicitly provide "compat": { "supportsStrictMode": true } inside their custom configuration to force overrides back on. This PR explicitly implemented checking for this escape-hatch to make it safe.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 9, 2026

Greptile Summary

This PR extends the normalizeModelCompat interceptor to force supportsStrictMode: false for all non-native openai-completions endpoints, preventing pi-ai from injecting "strict": false into tool parameter payloads that third-party compatible APIs silently reject.

Production logic: The implementation in model-compat.ts is correct and properly consistent with the existing pattern for supportsDeveloperRole and supportsUsageInStreaming. Both the early-exit guard and compat-object construction branches correctly set the new flag.

Test coverage: While the new functionality is tested indirectly in existing tests (particularly the "does not mutate" and "does not override explicit compat false" cases), the newly added expectSupportsStrictModeForcedOff test helper is never invoked. The parallel test coverage that exists for the older flags—with dedicated per-provider test fixtures (z.ai, moonshot, DashScope, Azure, custom proxy) and explicit override tests—is missing for supportsStrictMode. This is a test organization issue rather than a functional defect.

Confidence Score: 4/5

  • Safe to merge; the production logic is correct and non-breaking, with only a test coverage gap to address.
  • The model-compat.ts logic change is small, targeted, and correctly implements the feature consistent with existing normalization patterns. The early-exit guard and both compat-object construction branches are properly updated. The only concern is that the test helper expectSupportsStrictModeForcedOff is unused and the new flag lacks the dedicated per-provider and explicit-override-true test coverage that exists for the parallel flags. This is a test quality issue, not a runtime defect.
  • src/agents/model-compat.test.ts — Verify whether unused test helper should be invoked with additional test cases or removed

Last reviewed commit: 60af3fe

@sahancava sahancava force-pushed the fix/41328-openai-completions-tools branch from a502149 to 73c1785 Compare March 9, 2026 21:56
@openclaw-barnacle openclaw-barnacle bot added channel: googlechat Channel integration: googlechat extensions: memory-core Extension: memory-core size: S and removed app: macos App: macos size: M labels Mar 9, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 18f4a131d7

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6f2e7ac79a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@sahancava sahancava force-pushed the fix/41328-openai-completions-tools branch 2 times, most recently from 4bb729e to e522e12 Compare March 10, 2026 03:24
@openclaw-barnacle openclaw-barnacle bot removed channel: googlechat Channel integration: googlechat extensions: memory-core Extension: memory-core labels Mar 10, 2026
@sahancava sahancava force-pushed the fix/41328-openai-completions-tools branch 5 times, most recently from 1a40d97 to c1a6780 Compare March 13, 2026 18:24
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c1a6780b35

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@openclaw-barnacle openclaw-barnacle bot added channel: feishu Channel integration: feishu channel: twitch Channel integration: twitch extensions: acpx size: XL and removed size: M labels Mar 13, 2026
@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

5 similar comments
@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

14 similar comments
@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

@openclaw-barnacle
Copy link
Copy Markdown

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b656019a35

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@sahancava
Copy link
Copy Markdown
Contributor Author

Branch cleanup is complete.

I force-pushed sahancava:fix/41328-openai-completions-tools to the clean commit and verified the branch now contains only the intended model-compat scope.

GitHub now rejects reopening this PR with:
state cannot be changed. The fix/41328-openai-completions-tools branch was force-pushed or recreated.

Replacement clean PR: #45497

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling app: macos App: macos channel: bluebubbles Channel integration: bluebubbles channel: discord Channel integration: discord channel: feishu Channel integration: feishu channel: googlechat Channel integration: googlechat channel: matrix Channel integration: matrix channel: mattermost Channel integration: mattermost channel: msteams Channel integration: msteams channel: slack Channel integration: slack channel: telegram Channel integration: telegram channel: twitch Channel integration: twitch channel: voice-call Channel integration: voice-call channel: zalo Channel integration: zalo channel: zalouser Channel integration: zalouser cli CLI command changes commands Command implementations docs Improvements or additions to documentation extensions: acpx extensions: lobster Extension: lobster extensions: memory-core Extension: memory-core gateway Gateway runtime scripts Repository scripts size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openai-completions API type does not support tool calling with third-party OpenAI-compatible APIs

1 participant