fix(talk): diagnose invalid OpenAI realtime auth#87273
Conversation
openclaw-barnacle
Bot
added
docs
openclaw-barnacle
Bot
added
the
proof: supplied
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6111a4aabe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Codex review: needs maintainer review before merge. Reviewed July 25, 2026, 6:49 AM ET / 10:49 UTC. ClawSweeper reviewWhat this changesThis PR changes the shared OpenAI Realtime voice provider to reject known Azure/proxy credentials on direct Merge readinessThe PR has recent activity, proof, and a bounded auth-diagnostic goal, but it should stay open for a maintainer decision on whether direct OpenAI Realtime may reject unknown key formats. The available review context does not support a safe cleanup close. Priority: P1 Review scores
Verification
How this fits togetherOpenClaw’s shared OpenAI Realtime voice provider resolves credentials for browser client-secret sessions and server-side WebSocket bridges used by Talk and other voice surfaces. It chooses between direct OpenAI and configured custom/Azure endpoints, then passes the resulting session or error to the calling voice flow. flowchart LR
A[Voice surface config] --> B[OpenAI Realtime provider]
C[Configured key or profile] --> B
B --> D{Direct OpenAI endpoint?}
D -->|Yes| E[Validate and mint session]
D -->|Custom or Azure| F[Use configured endpoint]
E --> G[Voice session or safe diagnostic]
F --> G
Decision needed
Why: This is an upgrade-sensitive auth contract: rejecting unknown formats can block valid future OpenAI Platform keys, while allowing them preserves the existing opaque-key behavior and relies on sanitized provider failures. Before merge
Findings
Agent review detailsSecurityNeeds attention: The patch’s safe-error goal is security-positive, but the new environment override and credential classifier require maintainer review of their authentication boundary. PR surfaceSource +92, Tests +220, Docs +16. Total +328 across 4 files. View PR surface stats
Review metrics
Merge-risk optionsMaintainer options:
Copy recommended automerge instructionTechnical reviewBest possible solution: Keep this PR open for a maintainer decision on the direct-key classification boundary; if accepted, retain only positive Azure/proxy detection and avoid a permanent operator override for unknown direct OpenAI key formats. Do we have a high-confidence way to reproduce the issue? No high-confidence current-main reproduction was established in this review because the repository inspection command could not run. The PR body supplies a plausible local source execution for a proxy-looking key, but it does not establish the current-main behavior independently. Is this the best way to solve the issue? Unclear. The safe diagnostic and secret-redaction goal are sound, but the correct key-classification boundary requires maintainer intent because the proposed broad rejection can affect valid unknown direct-key formats. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against b8bb08a1ad37. LabelsLabel changes:
Label justifications:
EvidenceSecurity concerns:
What I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (3 earlier review cycles) |
clawsweeper
Bot
added
proof: sufficient
|
ClawSweeper PR egg ✨ Hatched: 🥚 common Frosted Proofling Hatch commandComment Hatchability rules:
Rarity: 🥚 common. What is this egg doing here?
|
openclaw-barnacle
Bot
removed
the
proof: sufficient
|
Addressed the P1 in 5f14335. Endpoint-only custom Realtime routes now bypass the direct-api.openai.com key-shape rejection and still send configured/env proxy-looking keys to the configured custom endpoint. Added regression coverage for both configured and env proxy-looking keys on a custom endpoint, and documented OPENCLAW_OPENAI_REALTIME_ALLOW_UNVALIDATED_KEY as an emergency override. Re-ran targeted Talk/OpenAI Realtime tests and check:changed successfully. @clawsweeper re-review |
|
🦞👀 Command router queued. I will update this comment with the next step. |
clawsweeper
Bot
added
proof: sufficient
|
Opened follow-up feature issue for Azure Foundry / Azure OpenAI GPT Realtime Talk via Gateway Relay: #87325. Keeping this PR scoped to diagnostics so Azure support can be reviewed separately with Azure-specific auth, relay, and compliance coverage. |
clawsweeper
Bot
added
rating: 🌊 off-meta tidepool
clawsweeper
Bot
added
the
rating: 🦞 diamond lobster
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e310c50658
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| function openAIRealtimeDirectAuthError(cause?: unknown): Error { | ||
| return cause instanceof Error | ||
| ? new Error(OPENAI_REALTIME_DIRECT_AUTH_MESSAGE, { cause }) |
There was a problem hiding this comment.
When the Realtime client-secret endpoint returns invalid_api_key with the rejected key in the provider message, this wrapper keeps that original error as cause. The actual talk.client.create/gateway-relay handlers return errors through formatForLog, and src/gateway/ws-log.ts deliberately walks the cause chain, so users still see the upstream Incorrect API key provided... detail/secret preview appended to the new diagnostic instead of only the safe auth guidance. Avoid attaching the provider error as a public cause here, or strip the cause before it reaches gateway RPC formatting.
Useful? React with 👍 / 👎.
| isDirectOpenAIRealtimeStartupAuthFailure(url, startupError) | ||
| ? openAIRealtimeDirectAuthError(startupError) | ||
| : startupError, |
There was a problem hiding this comment.
This new direct-OpenAI startup wrapper also changes the error surfaced by the existing keeps a retried connection ready after delayed startup failure close case: that test still sends Incorrect API key provided and asserts the old raw message at extensions/openai/realtime-voice-provider.test.ts:1254. Since this branch now rejects with the direct-auth diagnostic before the deferred close is emitted, the targeted extension Vitest lane will fail unless that expectation is updated to the new message while keeping the retry/close assertions intact.
Useful? React with 👍 / 👎.
| const OPENAI_REALTIME_DEFAULT_MIN_BARGE_IN_AUDIO_END_MS = 250; | ||
| const OPENAI_REALTIME_ALLOW_UNVALIDATED_KEY_ENV = "OPENCLAW_OPENAI_REALTIME_ALLOW_UNVALIDATED_KEY"; | ||
| const OPENAI_REALTIME_DIRECT_AUTH_MESSAGE = | ||
| "Talk Realtime provider 'openai' requires a direct OpenAI API key. Azure AI Foundry / Azure OpenAI model credentials and OpenAI-compatible proxy keys are not valid for OpenAI Realtime sessions against api.openai.com. Configure talk.realtime.providers.openai.apiKey or OPENAI_API_KEY with a direct OpenAI Platform key, or use Talk gateway relay with an Azure Realtime deployment (azureEndpoint + azureDeployment)."; |
There was a problem hiding this comment.
This provider is registered as the shared OpenAI realtime voice provider, so the same error is reachable from non-Talk paths such as Voice Call and Discord realtime voice. In those contexts an invalid OpenAI key will now tell the operator to configure talk.realtime.providers.openai.apiKey or use Talk gateway relay, even though Voice Call uses plugins.entries.voice-call.config.realtime.providers.openai.apiKey and Discord has its own voice.realtime config. Make the message name the generic OpenAI Realtime voice provider and include the non-Talk config surface so these users get actionable guidance.
Useful? React with 👍 / 👎.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Addressed the open Codex review findings from review
Validation:
|
clawsweeper
Bot
added
rating: 🐚 platinum hermit
|
This pull request has been automatically marked as stale due to inactivity. |
3f174de to
99e956b
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 99e956b820
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| ```json5 | ||
| { | ||
| <Note> |
There was a problem hiding this comment.
This <Note> is inserted after the opening { of the speech-to-text json5 fence, so Mintlify treats the callout as literal code and the copied configuration is invalid; users also never see the Realtime authentication guidance as a rendered callout. Move it outside the fenced block, preferably into the Realtime voice accordion.
AGENTS.md reference: docs/AGENTS.md:L5-L7
Useful? React with 👍 / 👎.
| OpenAI Realtime Talk auth is not shared with Azure AI Foundry or Azure | ||
| OpenAI model providers. Browser WebRTC sessions and native OpenAI backend | ||
| bridges need a direct OpenAI Platform API key; OpenAI-compatible proxy keys are not valid for the |
There was a problem hiding this comment.
For OAuth-only installations, this new direct-Platform-key requirement conflicts with the Realtime voice note at docs/providers/openai.md:912-914, which still promises an openai OAuth profile or external Codex login as fallback. The resolver restricts profiles to profileTypes: ["api_key"] in extensions/openai/realtime-voice-provider.ts:408-412, and extensions/openai/realtime-voice-provider.test.ts:770-779 explicitly verifies that OAuth is unsupported, so update the stale fallback text to leave one actionable authentication contract.
Useful? React with 👍 / 👎.
Detect keys that cannot be direct OpenAI Platform keys (Azure model keys, OpenAI-compatible proxy keys) before minting api.openai.com Realtime client secrets or opening direct Realtime websockets, and surface an actionable auth diagnostic. Preserves proxy-looking keys for custom OpenAI-compatible Realtime endpoints (azureEndpoint), Azure deployment auth failures, and the existing post-request client-secret rejection message. Adds OPENCLAW_OPENAI_REALTIME_ALLOW_UNVALIDATED_KEY escape hatch and docs. Co-authored-by: Copilot <[email protected]>
99e956b to
95de986
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 95de986175
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const startupError = new Error(readRealtimeErrorDetail(event.error)); | ||
| rejectStartup( | ||
| isDirectOpenAIRealtimeStartupAuthFailure(url, startupError) |
There was a problem hiding this comment.
When OpenAI reports startup authentication through structured fields such as { code: "invalid_api_key", message: "Invalid API key" }, this converts the payload to a plain Error before classification, so isOpenAIRealtimeAuthFailure loses the code/status fields and its remaining message checks do not match that wording. The session therefore surfaces the raw provider error instead of the new safe direct-auth diagnostic; classify event.error itself while retaining startupError only for the fallback display error.
Useful? React with 👍 / 👎.
| if (!normalized.startsWith("sk-")) { | ||
| return true; |
There was a problem hiding this comment.
When a valid OpenAI Platform credential uses a format other than the currently recognized sk-* convention, this blanket check classifies it as invalid and blocks both browser client-secret creation and the default backend bridge before OpenAI can authenticate it; the pre-change path treated the credential as opaque, and the newly added docs explicitly acknowledge that unusual direct-key formats can be rejected. Requiring a new process-wide override to preserve those sessions turns provider key-format evolution into an operator-visible outage, so reject only positively identified Azure/proxy formats or rely on the sanitized provider 401 diagnostic for unknown formats.
AGENTS.md reference: AGENTS.md:L31-L31
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦐 gold shrimp
2 participants
Summary
What problem does this PR solve?
Incorrect API key providedwhen the available credential was an Azure AI Foundry / Azure OpenAI model key or OpenAI-compatible proxy key rather than a direct OpenAI Platform credential.Why does this matter now?
What is the intended outcome?
openai-codexOAuth, or Azure gateway relay configuration.What is intentionally out of scope?
CHANGELOG.md.What does success look like?
Incorrect API key providederror when enabling OpenAI Realtime Talk.What should reviewers focus on?
extensions/openai/realtime-voice-provider.ts, especially preserving valid direct OpenAI Realtime setups andopenai-codexOAuth fallback.AI-assisted: yes. I used GitHub Copilot CLI to implement and validate this fix, and I understand the resulting code changes.
Linked context
Which issue does this close?
Which issues, PRs, or discussions are related?
Was this requested by a maintainer or owner?
Real behavior proof (required for external PRs)
OPENAI_API_KEY=<redacted proxy key> node --import tsx -e "import { buildOpenAIRealtimeVoiceProvider } from './extensions/openai/realtime-voice-provider.ts'; const provider = buildOpenAIRealtimeVoiceProvider(); try { await provider.createBrowserSession?.({ providerConfig: {}, instructions: 'diagnostic smoke test' }); console.log('unexpected success'); process.exitCode = 1; } catch (error) { const message = error instanceof Error ? error.message : String(error); console.log(message.replace(/REDACTED_KEY_PATTERN/g, '<redacted key>')); console.log('secret leaked:', message.includes(process.env.OPENAI_API_KEY ?? '')); }"node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts src/gateway/server-methods/talk.test.ts --runcorepack pnpm check:changedgit diff --checkAdditional local validation output:
check:changedcompleted with typecheck extensions, typecheck extension tests, lint extensions, media download helper guard, runtime sidecar loader guard, and runtime import cycles all passing. Re-run after both Codex follow-up fixes and the ClawSweeper P1 custom-endpoint fix.Incorrect API key providedstartup failure; this PR updates that behavior to the clearer Talk Realtime auth diagnostic.Tests and validation
Which commands did you run?
OPENAI_API_KEY=<redacted proxy key> node --import tsx -e "..."node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts src/gateway/server-methods/talk.test.ts --runcorepack pnpm check:changedgit diff --checkWhat regression coverage was added or updated?
What failed before this fix, if known?
Incorrect API key provided: <redacted OpenAI-like key>when Talk used OpenAI Realtime with a non-direct OpenAI credential.If no test was added, why not?
Risk checklist
Did user-visible behavior change? (
Yes/No)Yes.
Did config, environment, or migration behavior change? (
Yes/No)No migration. There is a diagnostic-only auth behavior change for clearly invalid OpenAI Realtime credentials, plus an escape hatch
OPENCLAW_OPENAI_REALTIME_ALLOW_UNVALIDATED_KEYfor unusual direct-key formats.Did security, auth, secrets, network, or tool execution behavior change? (
Yes/No)Yes.
What is the highest-risk area?
How is that risk mitigated?
openai-codexOAuth is preferred/fallback capable for GPT Realtime models, and an environment escape hatch allows unvalidated keys.Current review state
What is the next action?
What is still waiting on author, maintainer, CI, or external proof?
codex review --base origin/mainwas not run because thecodexCLI is not available in this environment.Which bot or reviewer comments were addressed?
azureEndpointroutes, with regression tests.OPENCLAW_OPENAI_REALTIME_ALLOW_UNVALIDATED_KEYas an emergency override instead of leaving it undocumented.