fix(media): route OAuth image defaults through Codex by bek91 · Pull Request #92824 · openclaw/openclaw

bek91 · 2026-06-14T01:38:45Z

Summary

AI-assisted bug fix for #87168.

This PR makes implicit image media-understanding model selection auth-mode-aware for OpenAI-family defaults:

keeps implicit openai/... image media candidates only when direct OpenAI API-key-capable auth is available for the direct openai-responses route
substitutes codex/gpt-5.5 for implicit OpenAI image defaults and auto candidates only when canonical OpenAI OAuth/token auth plus the Codex app-server route are both available
applies the OpenAI direct-auth rule to implicit OpenAI image fallback candidates even when the primary provider is not OpenAI
preserves explicit openai/... and codex/... image overrides
forwards agentDir into isolated Codex media app-server client startup so agent-scoped OAuth is available at execution
intentionally leaves PDF routing unchanged; PDF has separate native/document/text-extraction behavior and is out of scope for this image-focused fix

Root Cause

The old image selection path checked provider-level auth and treated canonical OpenAI OAuth as enough to select direct OpenAI image media models. Runtime execution later requires API-key auth for openai-responses, producing the misleading OPENAI_API_KEY error. Codex image media can use the Codex app-server route, but that route must be selected deliberately and only when the auth signal is real.

Verification

node scripts/run-vitest.mjs src/agents/tools/model-config.helpers.test.ts src/agents/tools/image-tool.test.ts extensions/codex/media-understanding-provider.test.ts
node scripts/run-vitest.mjs src/agents/tools/pdf-tool.model-config.test.ts
git diff --check
.agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main --stream-engine-output clean: no accepted/actionable findings reported

Real behavior proof

Behavior addressed: OAuth/token-only OpenAI auth no longer selects unusable direct OpenAI image media defaults or implicit image fallbacks; image auto-selection uses Codex when the Codex media route is actually available.

Real environment tested: Local OpenClaw PR worktree with the locally installed Codex CLI authenticated by ChatGPT/Codex token credentials. The subprocess explicitly removed OPENAI_API_KEY.

Exact steps or command run after this patch: Ran an inline tsx script with env -u OPENAI_API_KEY that imported buildCodexMediaUnderstandingProvider, generated a 64x64 PNG using OpenClaw's PNG encoder, and called provider.describeImage with provider: "codex", model: "gpt-5.5", and a temporary agent directory.

Evidence after fix:

{
  "ok": true,
  "provider": "codex",
  "model": "gpt-5.5",
  "openaiApiKeyPresent": false,
  "image": {
    "mime": "image/png",
    "width": 64,
    "height": 64,
    "bytes": 208
  },
  "durationMs": 49654,
  "textPreview": "Solid red square."
}

Observed result after fix: The real Codex media provider completed image understanding through codex/gpt-5.5 with no OPENAI_API_KEY in the subprocess environment. Regression coverage also proves OAuth/token-only image defaults select codex/gpt-5.5, non-OpenAI primaries do not retain bad implicit OpenAI image fallback candidates, direct OpenAI API-key auth keeps openai/..., explicit overrides remain unchanged, legacy openai-codex:* profiles do not trigger Codex behavior, and external CLI Codex OAuth survives the candidate auth filter.

What was not tested: Crabbox remote proof. PDF live UX proof is not part of this PR because PDF routing is intentionally unchanged.

Fixes #87168

clawsweeper · 2026-06-14T01:40:29Z

Codex review: needs maintainer review before merge. Reviewed June 14, 2026, 5:05 AM ET / 09:05 UTC.

Summary
The PR makes implicit OpenAI image media candidates auth-mode-aware, substitutes Codex image media when canonical OpenAI OAuth/token auth and the Codex app-server route are available, preserves explicit overrides, and carries scoped auth into isolated Codex media startup.

PR surface: Source +538, Tests +1306. Total +1844 across 19 files.

Reproducibility: yes. source-reproducible from current main and the linked issue logs: broad OpenAI provider auth can admit an implicit direct OpenAI image candidate while openai-responses image execution requires API-key material. I did not run a fresh OAuth-only install in this read-only pass.

Review metrics: 2 noteworthy metrics.

Implicit image default routing: 1 default-routing surface changed. The image tool now changes implicit OpenAI candidate selection based on direct API-key versus Codex OAuth availability, which maintainers should notice before merge.
Runtime auth provenance: 1 runtime-only auth-store field added. The new provenance marker controls whether isolated Codex media startup uses scoped OAuth credentials or persisted credentials.

Stored data model
Persistent data-model change detected: serialized state: extensions/codex/src/app-server/auth-bridge.test.ts, serialized state: src/agents/auth-profiles/oauth-shared.test.ts, serialized state: src/agents/auth-profiles/persisted-boundary.test.ts, serialized state: src/agents/auth-profiles/profiles.test.ts, serialized state: src/agents/tools/image-tool.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Risk before merge

[P1] Merging changes implicit image media defaults for existing OAuth-only OpenAI/Codex setups: implicit direct OpenAI image candidates can now become Codex or be dropped when no direct API-key auth exists.
[P1] The patch also broadens runtime auth-profile provenance and Codex app-server token refresh handling, so maintainer review should confirm the upgrade-visible auth lifecycle is intentional.
[P1] The real behavior proof is copied local live output for the Codex provider path, not packaged or Crabbox proof through the full user-facing image tool auto-selection flow.

Maintainer options:

Accept image-only OAuth routing after owner review (recommended)
Maintainers can land if they accept that implicit OpenAI image candidates now require direct API-key auth and OAuth-only installs route to Codex when available.
Require packaged auto-selection proof
Maintainers can ask for a packaged or Crabbox run that exercises the full image tool auto-selection path before merge.
Pause for provider-routing policy
If core should not special-case OpenAI/Codex image defaults, pause this PR and design a broader provider capability contract instead.

Next step before merge

[P2] Manual review is needed because the PR has a protected maintainer label and changes compatibility-sensitive auth-provider/default image routing; there is no narrow automated repair remaining.

Security
Cleared: Cleared: the diff changes auth routing and tests but does not add dependencies, workflows, package scripts, secret logging, or a concrete supply-chain regression.

Review details

Best possible solution:

Land or revise this PR after maintainer acceptance of the image-only auth-aware implicit routing behavior and the scoped Codex app-server auth lifecycle, while keeping explicit direct OpenAI image choices intact.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible from current main and the linked issue logs: broad OpenAI provider auth can admit an implicit direct OpenAI image candidate while openai-responses image execution requires API-key material. I did not run a fresh OAuth-only install in this read-only pass.

Is this the best way to solve the issue?

Yes, with maintainer acceptance: applying the auth-mode decision at implicit image candidate selection and carrying scoped auth into isolated Codex media startup are the right boundaries. The remaining question is upgrade-visible default-routing policy, not a line-level patch defect.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 439a9e97fd61.

Label changes

Label justifications:

P2: This is a normal-priority auth-provider/media routing bugfix with limited but real impact on Codex OAuth-only image understanding.
merge-risk: 🚨 compatibility: The PR changes implicit default image model selection and auth-store provenance behavior that existing OAuth-only setups may observe during upgrade.
merge-risk: 🚨 auth-provider: The diff changes how OpenAI OAuth/token auth, direct OpenAI API-key auth, and the Codex app-server route are selected and refreshed for image media tools.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): Sufficient: the PR body includes copied local live output from a Codex CLI-authenticated image-provider run with OPENAI_API_KEY removed and an observed image description result.
proof: sufficient: Contributor real behavior proof is sufficient. Sufficient: the PR body includes copied local live output from a Codex CLI-authenticated image-provider run with OPENAI_API_KEY removed and an observed image description result.

Evidence reviewed

PR surface:

Source +538, Tests +1306. Total +1844 across 19 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	10	582	44	+538
Tests	9	1342	36	+1306
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	19	1924	80	+1844

What I checked:

Repository policy applied: Root and scoped AGENTS.md files were read; the review applied OpenClaw guidance that auth-provider routing, provider defaults, and Codex dependency behavior are compatibility-sensitive and require maintainer judgment. (AGENTS.md:18, 439a9e97fd61)
Live PR state: Live GitHub state shows this PR is open, contributor-authored, labeled maintainer, assigned to vincentkoc, mergeable but unstable, and linked with closing syntax to the open auth-provider bug. (f9bb9da17452)
Current main admits broad OpenAI provider auth for tool candidates: On current main, hasProviderAuthForTool returns true for any OpenAI auth profile before falling back to custom API-key config, which explains how OAuth-only OpenAI auth can make an implicit direct OpenAI image candidate look usable. (src/agents/tools/model-config.helpers.ts:94, 439a9e97fd61)
Current main direct image runtime needs API-key material: The direct image media runtime resolves provider auth with getApiKeyForModel and then calls requireApiKey, so openai-responses image execution cannot be satisfied by Codex/ChatGPT OAuth alone. (src/media-understanding/image.ts:231, 439a9e97fd61)
PR adds auth-mode decision for implicit OpenAI image candidates: The new resolveOpenAiImageMediaCandidate keeps direct OpenAI only with direct API-key-capable auth, substitutes codex/ only with canonical OpenAI OAuth/token plus Codex route availability, and otherwise drops the implicit OpenAI image candidate. (src/agents/tools/model-config.helpers.ts:319, f9bb9da17452)
PR applies the decision to primary and auto image candidates: The image tool routes implicit OpenAI primary defaults and OpenAI auto candidates through the new decision helper while preserving explicit imageModel configuration. (src/agents/tools/image-tool.ts:259, f9bb9da17452)

Likely related people:

steipete: Recent history shows repeated work on image media routing, Codex media-understanding support, and auth-profile store documentation adjacent to this behavior. (role: feature-history owner; confidence: high; commits: 6290ed52ff9f, cec34821754a, 417022864b3b; files: src/agents/tools/image-tool.ts, extensions/codex/media-understanding-provider.ts, src/agents/auth-profiles/store.ts)
vincentkoc: Recent merged history touches Codex app-server startup, auth bridge aliases, shared-client lifecycle, and this PR is currently assigned to vincentkoc for review. (role: recent Codex/auth area contributor; confidence: high; commits: 5a0d9d6326f7, 07870dff45eb, 5ef812293b08; files: extensions/codex/src/app-server/shared-client.ts, extensions/codex/src/app-server/auth-bridge.ts)
hxy91819: Auth-aware media tool preflight work in model-config.helpers.ts is directly adjacent to the candidate filtering changed by this PR. (role: recent tool-auth helper contributor; confidence: medium; commits: 31c269f0ed7b; files: src/agents/tools/model-config.helpers.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

bek91 · 2026-06-14T02:30:19Z

@clawsweeper review

clawsweeper · 2026-06-14T02:30:22Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/27486003009
Updated: 2026-06-14T02:39:15.524Z

vincentkoc · 2026-06-14T09:11:03Z

Land-ready verification

Best-fix judgment: yes. This routes OAuth-backed implicit OpenAI image understanding through the owning Codex plugin, preserves scoped versus persisted auth provenance, and keeps isolated app-server startup plus refresh callbacks on one prepared store. The maintainer fixes also prevent rotating-token reuse and guard concurrent account replacement.

Fresh local and full-branch autoreview: clean after fixing one accepted P2 prepared-store concurrency finding.
Focused touched-and-sibling Vitest suite: 282 passed across agent auth, image routing, overflow compaction, Codex media, and app-server auth/client paths.
Final Codex media provider test: 12 passed.
Exact extension test-type lane: passed via node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/extensions-test.tsbuildinfo.
Exact head SHA: a3807ce2adf6ca7086cb349eaeac742f3ea5112f.
Exact-SHA GitHub CI: success, including the repaired check-test-types lane: https://github.com/openclaw/openclaw/actions/runs/27494131797
Remote check:changed could not provision because the local Azure CLI session is expired (AADSTS50132); no remote box or test run was created.

Direct upstream Codex contract checked at openai/codex@b39f943a634a6e7ba86c3d6e8cf6d5f35e612566:

External ChatGPT tokens are memory-only and host-refreshed: https://github.com/openai/codex/blob/b39f943a634a6e7ba86c3d6e8cf6d5f35e612566/codex-rs/app-server-protocol/src/protocol/common.rs#L28-L33 and https://github.com/openai/codex/blob/b39f943a634a6e7ba86c3d6e8cf6d5f35e612566/codex-rs/app-server-protocol/src/protocol/common.rs#L1428-L1431
App-server installs external auth then reloads the auth manager: https://github.com/openai/codex/blob/b39f943a634a6e7ba86c3d6e8cf6d5f35e612566/codex-rs/app-server/src/request_processors/account_processor.rs#L590-L606
App-server bridges refresh callbacks to the external host: https://github.com/openai/codex/blob/b39f943a634a6e7ba86c3d6e8cf6d5f35e612566/codex-rs/app-server/src/message_processor.rs#L110-L130
External auth precedes persisted fallback and remains ephemeral: https://github.com/openai/codex/blob/b39f943a634a6e7ba86c3d6e8cf6d5f35e612566/codex-rs/login/src/auth/manager.rs#L814-L852 and https://github.com/openai/codex/blob/b39f943a634a6e7ba86c3d6e8cf6d5f35e612566/codex-rs/login/src/auth/manager.rs#L1075-L1083

No new live-provider OAuth media call was run in this maintainer pass; exact-SHA CI, focused behavior proof, and direct dependency-contract proof are green.

fix: route OpenAI OAuth media defaults

f6f829d

openclaw-barnacle Bot added agents Agent runtime and tooling extensions: codex size: L maintainer Maintainer-authored PR labels Jun 14, 2026

fix: close OpenAI OAuth media fallback gaps

2037298

test: remove stale pdf codex mock

2416628

clawsweeper Bot mentioned this pull request Jun 14, 2026

Codex-authenticated installs can auto-select direct OpenAI for image media understanding without OPENAI_API_KEY #87168

Closed

Narrow OpenAI OAuth media fix to image

57f9e6e

fix: satisfy media routing test checks

dbab29b

vincentkoc self-assigned this Jun 14, 2026

vincentkoc added 5 commits June 14, 2026 14:30

fix(codex): preserve media auth boundaries

98875a2

fix(media): honor eligible auth routes

d740005

fix(codex): preserve scoped media auth lifecycle

cdfe800

fix(auth): preserve persisted profile provenance

ee5d56d

fix(codex): refresh persisted isolated auth

f9bb9da

vincentkoc requested a review from a team as a code owner June 14, 2026 09:00

fix(codex): reuse prepared isolated auth

4da0ac1

openclaw-barnacle Bot added size: XL and removed size: L labels Jun 14, 2026

test(codex): provide media request config

a3807ce

vincentkoc changed the title ~~[codex] Fix OpenAI OAuth media routing~~ fix(media): route OAuth image defaults through Codex Jun 14, 2026

github-actions Bot mentioned this pull request Jun 14, 2026

📡 Upstream Digest — 2026-06-14 11:49 UTC curtismercier/openclaw-mods#1074

Open

This was referenced Jun 14, 2026

[Feature]: Allow disabling external-cli-sync for auth-profiles via config #70055

Open

fix(agents): null-guard baseUrl in getAttributionHeaders #93013

Closed

clawsweeper Bot mentioned this pull request Jun 22, 2026

fix: bound Codex Telegram turns fail after /codex bind on OAuth refresh #95289

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(media): route OAuth image defaults through Codex#92824

fix(media): route OAuth image defaults through Codex#92824
vincentkoc merged 15 commits into
mainfrom
chore/review-87168

bek91 commented Jun 14, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 14, 2026 •

edited

Loading

Uh oh!

bek91 commented Jun 14, 2026

Uh oh!

clawsweeper Bot commented Jun 14, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jun 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

bek91 commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root Cause

Verification

Real behavior proof

Uh oh!

clawsweeper Bot commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bek91 commented Jun 14, 2026

Uh oh!

clawsweeper Bot commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jun 14, 2026

Land-ready verification

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bek91 commented Jun 14, 2026 •

edited

Loading

clawsweeper Bot commented Jun 14, 2026 •

edited

Loading

clawsweeper Bot commented Jun 14, 2026 •

edited

Loading