Skip to content

fix(openai): Enable gpt-5.4 support via Chat Completions fallback on scope error#38026

Closed
yfge wants to merge 1 commit intoopenclaw:mainfrom
yfge:fix/responses-api-scope-fallback-24927
Closed

fix(openai): Enable gpt-5.4 support via Chat Completions fallback on scope error#38026
yfge wants to merge 1 commit intoopenclaw:mainfrom
yfge:fix/responses-api-scope-fallback-24927

Conversation

@yfge
Copy link
Copy Markdown
Contributor

@yfge yfge commented Mar 6, 2026

Resolves #24927

Problem

Models like openai-codex/gpt-5.4 currently fail with a 401 Unauthorized error when used with an OAuth token (e.g., from the Codex app app_EMoamEEZ73f0CkXaXp7hrann).

This is because OpenClaw defaults to the newer OpenAI Responses API (/v1/responses), which requires an api.responses.write scope that is not present in existing OAuth tokens. The current system treats this as a generic authentication error and triggers a model failover, which prevents gpt-5.4 from being used as intended.

Solution

To provide a seamless experience and enable gpt-5.4 support for users on existing authentication schemes, this PR introduces a more graceful fallback mechanism.

When a model using the openai-responses API fails specifically with a "missing scopes: api.responses.write" error, this change implements a one-time retry. The retry automatically uses the openai-completions API type for the same model, effectively routing the request to the compatible /v1/chat/completions endpoint.

Key Changes:

  • Specific Error Detection: A new matcher isMissingScopeResponsesWrite() precisely identifies the scope-related failure.
  • Intelligent Retry Logic: Instead of failing immediately, the system attempts a targeted fix by switching to a compatible API for the same model.
  • Improved UX: Users can now use gpt-5.4 without interruption. A warning is logged to inform about the automatic API fallback, maintaining transparency.

@openclaw-barnacle openclaw-barnacle bot added agents Agent runtime and tooling size: S labels Mar 6, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 6, 2026

Greptile Summary

This PR adds a targeted fallback mechanism for openai-responses API calls that fail due to a missing api.responses.write OAuth scope, transparently retrying the same request using openai-completions rather than surfacing the error to the user.

  • Adds isMissingScopeResponsesWrite(raw) in failover-matches.ts that gates on both "missing scopes" and a "responses.write" scope string in the error message.
  • Re-exports the helper through errors.ts and the top-level pi-embedded-helpers.ts barrel.
  • In the runEmbeddedPiAgent run loop, a new responsesScopeFallbackRetried flag and forcedApiModelForRetry variable enable a single one-shot retry with api: "openai-completions" while leaving all other model settings intact.
  • Tests cover the main positive and negative cases, though the singular form ("missing scope:") is not tested and the current implementation would miss it.
  • The api.responses.write branch in the || expression inside isMissingScopeResponsesWrite is redundant since "responses.write" is a substring of "api.responses.write" — the check can be simplified.

Confidence Score: 4/5

  • Safe to merge; the fallback logic is well-guarded and the only issues are minor style/coverage gaps.
  • The core logic is sound: the one-shot retry is correctly bounded by responsesScopeFallbackRetried, the model object is spread from the stable model reference, and normal failover is not disrupted. The two findings (redundant OR condition and untested singular scope form) are non-blocking style issues.
  • src/agents/pi-embedded-helpers/failover-matches.ts (redundant condition) and src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts (missing singular-form test coverage).

Last reviewed commit: 0d9ab6f

Comment thread src/agents/pi-embedded-helpers/failover-matches.ts
Comment thread src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts
@yfge yfge changed the title fix: fallback openai-responses calls when api.responses.write scope is missing fix(openai): Enable gpt-5.4 support via Chat Completions fallback on scope error Mar 6, 2026
@Takhoffman
Copy link
Copy Markdown
Contributor

Closing as superseded: linked issue #24927 was already resolved by merged PR #27501. This PR is now redundant; if you still see a gap after latest main, open a fresh PR/issue with updated repro.

@Takhoffman Takhoffman closed this Mar 7, 2026
@yfge
Copy link
Copy Markdown
Contributor Author

yfge commented Mar 7, 2026

Thanks for the review @Takhoffman. I'd like to respectfully push back — I believe this PR addresses a gap that #27501 does not cover.

What #27501 does: Adds openai-codex-responses as a valid api value in ModelApiSchema, allowing users to explicitly configure it.

What it doesn't do: Fix the default behavior. When users configure the openai-codex provider (which naturally uses api: "openai-responses"), models like gpt-5.4 hit a 401 Missing scopes: api.responses.write error and silently failover to a completely different provider. The user sees "working" responses but is unknowingly not using the model they selected.

What this PR does: Catches the specific api.responses.write scope error at runtime and transparently retries with openai-completions (Chat Completions API), which doesn't require that scope. This is a runtime-level fallback — it works without requiring users to know about openai-codex-responses or manually reconfigure.

Real-world impact I'm seeing right now (OpenClaw 2026.3.2):

[diagnostic] lane task error: error="FailoverError: HTTP 401: Missing scopes: api.responses.write"

Every openai-codex/gpt-5.4 request fails → silently falls back to Anthropic Claude via the general failover chain. The user has no idea gpt-5.4 never actually ran.

#27501 gives power users a config escape hatch. This PR makes it Just Work™ for everyone. They're complementary, not redundant.

Could you please reopen this?

@yfge
Copy link
Copy Markdown
Contributor Author

yfge commented Mar 7, 2026

Adding more context on why this PR is still needed — I dug into the source code and found the root cause.

Why gpt-5.3-codex works but gpt-5.4 doesn't

OpenClaw has a hardcoded forward-compat function that only handles one specific model ID:

// src/agents/model-forward-compat.ts
const OPENAI_CODEX_GPT_53_MODEL_ID = "gpt-5.3-codex";

function resolveOpenAICodexGpt53FallbackModel(provider, modelId, modelRegistry) {
    if (trimmedModelId.toLowerCase() !== OPENAI_CODEX_GPT_53_MODEL_ID) return; // ← only gpt-5.3-codex
    // ...
    return { api: "openai-codex-responses", baseUrl: "https://chatgpt.com/backend-api", ... };
}

This function intercepts gpt-5.3-codex and automatically routes it through openai-codex-responses (ChatGPT backend API), which doesn't require the api.responses.write scope.

But gpt-5.4 (and any future model) doesn't get this treatment. It falls through to the user-configured api: "openai-responses", which hits api.openai.com/v1/responses and requires the api.responses.write scope that Codex OAuth tokens don't have → HTTP 401 → silent failover to a completely different provider.

The real problem

This is a hardcoded allowlist pattern. Every time OpenAI releases a new model, someone has to manually add it to the forward-compat function. That's fragile and guarantees breakage for early adopters.

#27501 added openai-codex-responses as a valid config value, but:

  • Users don't know they need to change their config
  • The default behavior still breaks for any model not in the hardcoded list
  • The existing gpt-5.3-codex workaround masks the issue — users think "Codex OAuth works" because 5.3 works, then hit a wall on 5.4

What this PR does differently

Instead of adding another model ID to the allowlist, this PR catches the specific scope error at runtime and retries with openai-completions (Chat Completions API). This works for gpt-5.4, gpt-5.5, and any future model — no code changes needed.

Could you please reconsider reopening this?

@darbsllim
Copy link
Copy Markdown

I think it's actually more related to this issue:

#37558

@sene1337
Copy link
Copy Markdown

sene1337 commented Mar 8, 2026

Filed a follow-up issue for a legacy-config case that can still leave Codex OAuth broken after upgrade: #40066\n\nIn short: older manual overrides can shadow the built-in Codex OAuth provider even after the fixes in this PR.

@sene1337
Copy link
Copy Markdown

sene1337 commented Mar 8, 2026

Correcting my previous comment formatting:

Filed a follow-up issue for a legacy-config case that can still leave Codex OAuth broken after upgrade: #40066

In short: older manual models.providers.openai-codex overrides can shadow the built-in Codex OAuth provider even after the fixes in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sub-agents fail with OpenAI OAuth: Missing scope api.responses.write

4 participants