Skip to content

Commit 38455fb

Browse files
authored
fix(live): join client-created voice calls as the caller that owns them (#6678)
* docs(devlog): plan realtime ExistingCall sideband join compatibility * docs(devlog): fold plan audit residuals for ExistingCall join * fix(live): join client-created voice calls as the caller that owns them ChatGPT voice can hand a live call to a Codex thread, and Codex Desktop creates the call itself when its renderer owns it. In both cases the WebRTC call is created with the caller's own ChatGPT login and only the V3 existingCall sideband join reaches opencodex. In Pool mode the join was re-authenticated with a Pool-selected account, so it reached the upstream as a different ChatGPT account and Codex reported 502 realtime websocket handshake failed. Record the call ids handleLive creates (extracted from the create Location the way openai/codex reads it) and, for a join to any other call, forward the caller's explicit ChatGPT credential through the Direct passthrough. Calls created through opencodex keep Pool selection and thread affinity (#35830). * docs: describe caller-owned voice sideband joins * test(layout): keep layout.json under the file-size threshold * test(live): require Pool selection for a mismatched caller credential
1 parent f494403 commit 38455fb

15 files changed

Lines changed: 618 additions & 15 deletions

File tree

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Realtime voice: sideband joins for calls opencodex did not create
2+
3+
ChatGPT voice ("GPT Live") can now hand a call to a Codex thread on the user's Mac: the ChatGPT client creates the WebRTC call with its own ChatGPT login, and the local Codex app-server joins the call's control sideband (`V3`, `transport=ExistingCall`). With opencodex injected, that join arrives at `ws://127.0.0.1:<port>/v1/live/<call_id>`, and opencodex re-authenticates it with whichever Pool account it selects. The join therefore reaches the upstream as an account other than the caller that started the call, and Codex reports `502 Bad Gateway: realtime websocket handshake failed` after five attempts. The credential overwrite is confirmed in source and in the user's ledger; that the upstream refused the join because of the account (rather than another handshake fault) is a strong inference, because Bun hides the upstream handshake status.
4+
5+
- Archetype: satisfy-spec (compatibility with the current Codex/ChatGPT voice clients).
6+
- Trigger: user report on 2026-10-06 that live voice through opencodex stopped working after the Codex/ChatGPT update.
7+
- Class: C3 relay repair with C4 care, because it changes which credential a sideband join carries.
8+
- Goal: one ordinary PR to `dev` so a native sideband join for a call opencodex did not create is authenticated as the caller that owns it, while joins for calls opencodex created keep the existing Pool account continuity.
9+
- Non-goals: GPT-Live public `/v1/live/sessions` API family, external audio-key aliases, Codex config injection for the provider-table mode, merge, release, restarting the user's proxy, live paid probes.
10+
- Verifier: focused regression tests for the new join decision and the call registry, the unchanged #35830 continuity test, and the existing scope/external-audio/layout/ratchet/structure/privacy/Lab gates, all run by hosted CI only. Explicit security review is required (credential handling, `MAINTAINERS.md`). The user instructed no local test runs ("로컬 테스트 하지 말고"); typecheck, suite, lint and build are left to exact-head CI and named as not run locally.
11+
- Stop: PR open against `dev`, exact-head CI inspected, independent review findings folded.
12+
- Artifacts: `.tmp/realtime-investigation/` (ignored) holds the seven investigation reports; Aside's public-web report is under the Aside account artifact directory.
13+
- Resources: this worktree, git/gh for this branch, gpt-6.1-sol leaves, Aside exec for public research. No token or wall-clock budget was set by the user.
14+
15+
## Evidence
16+
17+
Runtime (read-only, identifiers omitted): Codex `logs_2.sqlite` shows four `RealtimeConversationStart` operations with `version=V3; transport=ExistingCall; output_modality=Audio` from `codex_chatgpt_ios_remote` 1.2026.266 (2026-10-03, three sessions) and 1.2026.267 (2026-10-05, one session). Each dialed `/v1/live/<call-id>` on the loopback proxy five times and failed with 502. opencodex `usage.jsonl` holds the matching twenty `gpt-live` rows (`upstream_server_error`, `failureStage=headers-only`).
18+
19+
The account labels on those rows differ: the last voice session that completed its handshake (2026-09-30, create 201 then join 101; audibility not recorded) ran on one Pool account for both legs, the 2026-10-03 joins on a second account and the 2026-10-05 joins on a third. The configured Pool holds five ChatGPT accounts. A local sanitized comparison (booleans only, no identifiers printed) found that none of the five Pool credentials carries the ChatGPT account id of the Codex login on this Mac, which is the credential Codex attaches to an ExistingCall join; both accounts used for the failing joins are non-main Pool accounts.
20+
21+
Source: `resolveLiveRelay` (`src/server/live.ts`) resolves every native join through `resolveFirstUsableOpenAiSidecar`, which in Pool mode selects by Pool policy and thread affinity and replaces the caller's `authorization`/`chatgpt-account-id`. Continuity for calls opencodex created is carried only by thread affinity (`tests/server/server-live.test.ts`, "call-create and its sideband join bind to the same pool account (openai/codex #35830)"). Nothing records which calls opencodex created, so an externally created call is joined under an unrelated account. Upstream Codex attaches its own configured-provider ChatGPT auth to the ExistingCall join (`codex-rs/core/src/realtime_conversation.rs` ExistingCall branch; `client.rs` `current_client_setup`), which is the account that owns the call.
22+
23+
The rest of the Codex 6cfe29984..41acdad24 realtime range is transparent to opencodex; see `010_incompatibility_matrix.md`.
24+
25+
## Work phases
26+
27+
1. wp1 (this document set): investigation and roadmap, docs only.
28+
2. wp2: implement `020_existing_call_join.md`, add the regression test, update the owning structure doc, publish the PR, inspect exact-head CI.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Incompatibility matrix: Codex realtime 6cfe29984..41acdad24 and Desktop 26.930.31730 vs opencodex
2+
3+
Each row is a request or behavior the current Codex clients produce, checked against opencodex at `fde8eebd61`. Status values: **confirmed-broken** (source plus runtime evidence), **ok** (opencodex relays it unchanged), **unresolved** (not proven either way; outside this unit unless stated).
4+
5+
| # | Client behavior | opencodex behavior | Status | Evidence |
6+
|---|---|---|---|---|
7+
| 1 | V3 `existingCall` sideband join `GET /v1/live/<call_id>` (WebSocket upgrade) for a call the client created itself: ChatGPT iOS remote 1.2026.266/267, and Desktop when the renderer owns the call (`clientOwnsCall`, creating it at `chatgpt.com/backend-api/wham/realtime/calls`) | Replaces the caller's `authorization`/`chatgpt-account-id` with a Pool-selected account; the call belongs to the caller's account, so the upstream handshake fails and opencodex answers 502 | **confirmed-broken** (overwrite confirmed; refusal cause inferred) | Codex `logs_2.sqlite` V3 ExistingCall starts with five 502 retries each; `~/.opencodex/usage.jsonl` twenty `gpt-live` 502 rows on two different Pool accounts, both unlike the 2026-09-30 working account; `src/server/live.ts` `resolveLiveRelay`; `src/providers/openai-sidecar.ts` Pool selection; Desktop `app-initial-576fc7ca620e.js:4605,4611` |
8+
| 2 | V3 WebRTC call-create `POST /v1/live` through opencodex, then join on the same thread | Pool account chosen at create; thread affinity keeps the join on the same account | ok | `tests/server/server-live.test.ts` #35830 continuity case; 2026-09-30 201 then 101 on one account |
9+
| 3 | `thread/realtime/listVoices` (#49073 now aborts voice on failure) | Not a network request: the app-server answers from a built-in list | ok | `codex-rs/core/src/session/handlers.rs:66`; `app-server/src/request_processors/turn_processor.rs:328` |
10+
| 4 | V3 `backend_reasoning_status` (#47377) as `delegation.context.append` on `commentary` | Sideband frames relayed byte for byte, no event allowlist | ok | `src/server/index/websocket-handler.ts:146-183`; `src/server/index/live-sideband.ts:580-600` |
11+
| 5 | `partial_answer` phase (#51241, #51260), per-request realtime context (#47596), transcript tail flush (#50531), stale answer filter (#47975), live tool-call metadata (#49401) | Client-local, or carried inside existing frames and Responses requests | ok | L3 report; no realtime serializer/parser file changed in the range |
12+
| 6 | RTP timestamps on a 20 ms grid (#48824), microphone channel selection (#49836), device selection (#49437) | WebRTC media does not pass through opencodex | ok | `structure/data-planes/inbound-compat.md` (no media relay); `codex-rs/voice-host` |
13+
| 7 | Realtime WebSocket network-policy errors (#47101 and follow-ups) | Client-side error mapping; loopback still dialed | ok | `codex-rs/codex-api/src/endpoint/realtime_websocket/methods.rs` diff |
14+
| 8 | `in_app_voice` managed feature gate (#49683) | Requirements-only client gate | ok | commit `6996cde69` |
15+
| 9 | Desktop 26.930.21537 → 26.930.31730 voice transport selection, start RPC, call-create endpoint and headers | No contract change found between the two bundles | ok (bounded) | L4 paired-bundle comparison |
16+
| 10 | Provider-table injection mode and the Design B fallback file omit `experimental_realtime_ws_base_url` | Sideband would bypass opencodex in those modes | unresolved, out of scope | L7; the user's logs show the sideband reaching the loopback proxy |
17+
| 11 | External audio-key create on the `/backend-api/codex/live` alias tests the raw path | Non-Frameless defaults for that alias | unresolved, out of scope | L5; affects opencodex audio keys, not Codex clients |
18+
| 12 | Public GPT-Live API `/v1/live/sessions` family | Not used by Codex clients | out of scope | Aside report; Codex URL builders unchanged |
19+
| 13 | Upstream handshake status behind opencodex's 502 | Bun hides the upstream handshake status; the precise upstream code (likely 404) is not recorded | unresolved | `src/server/index/live-sideband.ts` handshake failure mapping |
20+
21+
Row 1 is the only confirmed incompatibility and is the scope of `020_existing_call_join.md`.
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Diff plan: authenticate ExistingCall joins as the call owner
2+
3+
## Behavior
4+
5+
A native sideband join (`/v1/live/<id>`, `/v1/realtime/calls/<id>`, `/v1/realtime?call_id=<id>`) is split by who created the call:
6+
7+
- The call was created through this opencodex process (native `POST /v1/live` or `/v1/realtime/calls` returned a Location naming it): unchanged. Pool selection and thread affinity choose the account, as the #35830 case requires.
8+
- Otherwise the client created the call with its own ChatGPT login (V3 `existingCall`). When the caller presents an explicit ChatGPT bearer whose token claim matches its `chatgpt-account-id` and passes forward-admission validation, the join carries the caller's own credentials, exactly as Codex Direct mode does. Without such a caller credential the current behavior stays.
9+
10+
Standalone sessions (`/v1/live?model=`, `/v1/realtime?model=`) and the external audio-key path are untouched.
11+
12+
## Changes
13+
14+
1. `src/providers/openai-sidecar.ts`: export `resolveCallerOwnedOpenAiSidecar(candidates, incomingHeaders, config, { admission })`. It returns the first forward candidate with `authContext: { kind: "main", accountId: null }` and `directSidecarHeaders(...)` when the caller credential is explicit, forwardable (`validateForwardAdmissionCredential`) and a Codex bearer; otherwise `undefined`. Materialization keeps the existing main-account hard-lock and credits checks, whose errors extend `CodexAccountCooldownError` and are already mapped by `resolveLiveRelay`.
15+
2. `src/server/live-native-calls.ts` (new, small): bounded process-local set of upstream call ids created through the native call-create path. `recordNativeLiveCall(location)` extracts the call id the way Codex does (`decode_call_id_from_location` in `codex-rs/codex-api/src/endpoint/realtime_call.rs`: drop the query, scan path segments from the end, take the first `rtc_<nonempty>` or 36-character hyphenated UUID), bounded to the join grammar of 128 characters; entries expire after six hours, at most 1024 are kept, oldest evicted first; `isNativeLiveCall(id)`; `clearNativeLiveCallsForTests()`. Ids only, no account or credential.
16+
3. `src/server/live.ts`:
17+
- `handleLive`: after a 2xx upstream response, record the Location.
18+
- `LiveScopeDestination` gains `callerOwnedCallId?: string`.
19+
- `resolveLiveRelay`: when `callerOwnedCallId` is set and not a native call, try `resolveCallerOwnedOpenAiSidecar` first inside the existing `try`; if it returns a sidecar, use it as `forward` (no Pool selection, no outcome recording against a Pool account). Otherwise fall through to the current selection.
20+
- `resolveLiveSidebandUpgrade`: pass `callerOwnedCallId` for the three join styles.
21+
4. `tests/server/server-live-existing-call.test.ts` (new sibling; `server-live.test.ts` has 16 lines of ratchet headroom), Pool mode with two accounts and round-robin:
22+
- a join for a call never created through opencodex reaches the upstream with the caller's `authorization` and `chatgpt-account-id` and the session/thread headers, on all three join forms;
23+
- a call created through opencodex (API `/v1/live/<id>` Location and backend `/v1/realtime/calls/calls/<id>` Location) keeps the Pool account on join;
24+
- no caller bearer keeps the current Pool selection; a claim/header account mismatch and the proxy admission secret as bearer are refused with 401 before any upstream dispatch, as today;
25+
- an active main-account hard lock refuses the caller-owned join (cooldown response) without falling back to a Pool account;
26+
- a configured key with a provider scope that excludes `openai` is refused on the caller-owned branch, as on the Pool branch;
27+
- the caller-owned join records no Pool outcome.
28+
`tests/server/live-native-calls.test.ts` (new): Location extraction for API, backend (`/calls/calls/<id>`), UUID, query, trailing-slash and trailing-suffix shapes and rejection of malformed values; TTL expiry, oldest-first eviction at capacity, and clear, with an injected clock. Both files registered in `scripts/test-layout/layout.json` and `tests/fixtures/test-layout-expected.json`. The #35830 case stays as the regression guard for opencodex-created calls.
29+
5. Docs: `structure/data-planes/inbound-compat.md` (admission paragraph: a native join now consults call-id membership, still no model/account binding; streaming-audio section: the ownership rule); review the `src/providers/` and `src/server/` owner docs from `structure/INDEX.md` and update only those whose explanation changes; `docs-site/src/content/docs/guides/codex-integration.md` voice paragraph gains the client-created call case, and translated locales are checked for contradiction.
30+
31+
## Risks
32+
33+
- Continuity for opencodex-created calls now depends on the call id being retained: a restart, six-hour expiry, or eviction past 1024 calls turns it into an unknown call, and a later rejoin then carries the caller's credentials. The relay socket does not survive a restart, joins happen right after create, and the effect is limited to Pool users whose caller account differs from the creating account. Persisting call-to-account bindings is broader scope and not needed for the #35830 contract.
34+
- Model/provider admission scope still runs after either resolution; a native join still names no model, so model-scoped keys stay denied for joins as today.
35+
- Explicit security review is required on the PR (credential selection).
36+
- The caller bearer is forwarded only to the canonical sideband root or the configured `experimental_realtime_ws_base_url`, the same destinations that already receive Pool bearers.
37+
- No local test run (user instruction); hosted CI is the verifier.

‎docs-site/src/content/docs/guides/codex-integration.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,11 @@ WebSocket at `api.openai.com` directly unless `experimental_realtime_ws_base_url
5757
Pool mode the call is created under the account opencodex selects, so a direct join under the app's
5858
own login fails with `realtime websocket handshake failed` (404). The injected key sends the join
5959
back through opencodex (`GET /v1/live/{callId}`), where the Pool reuses the account it bound to that
60-
session/thread pair (a process-local binding). In Direct mode both legs already use the caller's
60+
session/thread pair (a process-local binding). A call the client created itself is the exception:
61+
when ChatGPT voice hands a call to a Codex thread, or Codex Desktop creates the call on its own,
62+
the call belongs to your ChatGPT login and only its sideband join reaches opencodex, so opencodex
63+
forwards that join with the caller's own ChatGPT credential rather than a Pool account. In Direct
64+
mode both legs already use the caller's
6165
current bearer, so the key only keeps the join on the proxy path. It is written only on the loopback
6266
`openai_base_url` form, is removed together with it, and a user-owned
6367
`experimental_realtime_ws_base_url` is never overwritten.

‎docs-site/src/content/docs/ko/guides/codex-integration.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,10 @@ codex 0.146(openai/codex#35830)부터는 `experimental_realtime_ws_base_url`이
3535
WebSocket을 `api.openai.com`에 직접 붙입니다. Pool 모드에서는 통화가 opencodex가 고른 계정으로
3636
만들어지므로, 앱 자체 로그인으로 직접 붙는 join은 `realtime websocket handshake failed`(404)로
3737
실패합니다. 주입된 키는 join을 다시 opencodex(`GET /v1/live/{callId}`)로 보내고, Pool은 그
38-
session/thread 쌍에 묶어 둔 계정(프로세스 로컬 바인딩)을 그대로 씁니다. Direct 모드는 두 요청 모두
38+
session/thread 쌍에 묶어 둔 계정(프로세스 로컬 바인딩)을 그대로 씁니다. 클라이언트가 직접 만든
39+
통화는 예외입니다. ChatGPT 음성이 통화를 Codex 스레드로 넘기거나 Codex Desktop이 통화를 직접
40+
만들면 그 통화는 내 ChatGPT 로그인 소유이고 sideband join만 opencodex에 도착하므로, opencodex는
41+
그 join을 Pool 계정이 아니라 호출자 자신의 ChatGPT 자격 증명으로 보냅니다. Direct 모드는 두 요청 모두
3942
호출자의 현재 bearer를 쓰므로, 이 키는 join을 프록시 경로에 붙잡아 두는 역할만 합니다. 이 키는
4043
loopback `openai_base_url` 형태에서만 쓰이고, 그 키와 함께 제거되며, 사용자가 직접 적은
4144
`experimental_realtime_ws_base_url`은 덮어쓰지 않습니다.

‎scripts/test-layout/layout.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1135,7 +1135,7 @@
11351135
"lab-read-surfaces.test.ts": "lab",
11361136
"launchd-repair.test.ts": "service",
11371137
"legacy-shell-compat.test.ts": "responses",
1138-
"live-call-bindings.test.ts": "server",
1138+
"live-call-bindings.test.ts": "server", "live-native-calls.test.ts": "server",
11391139
"live-service-manager-guard.test.ts": "service",
11401140
"linux-desktop-packaged-e2e.test.ts": "ci-workflows",
11411141
"local-aside-sync-capability.test.ts": "server",
@@ -1677,7 +1677,7 @@
16771677
"server-jev-combo-e2e.test.ts": "server",
16781678
"server-kiro-completion-e2e.test.ts": "server",
16791679
"server-kiro-device-login.test.ts": "server", "server-kiro-oauth-401-replay.test.ts": "server",
1680-
"server-kiro-refusal-e2e.test.ts": "server",
1680+
"server-kiro-refusal-e2e.test.ts": "server", "server-live-existing-call.test.ts": "server",
16811681
"server-live-frame-log.test.ts": "server",
16821682
"server-live-realtime-fixtures.test.ts": "server",
16831683
"server-live.test.ts": "server",

‎src/providers/openai-sidecar.ts‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,37 @@ function directSidecarHeaders(
125125
return selected;
126126
}
127127

128+
/**
129+
* The caller's own ChatGPT credential for a call the caller created itself.
130+
*
131+
* A V3 `existingCall` sideband join attaches to a WebRTC call that ChatGPT voice or Codex Desktop
132+
* created with the caller's login, so only that login can join it; a Pool-selected account is a
133+
* different ChatGPT account and the upstream refuses the handshake. This hands the join the same
134+
* caller-owned headers Codex Direct mode forwards: an explicit bearer whose token claim matches
135+
* its `chatgpt-account-id`, never the proxy admission secret, materialized through the same main
136+
* policy (hard lock, credits), whose refusals surface as `CodexAccountCooldownError`.
137+
* Returns undefined when the caller presents no such credential.
138+
*/
139+
export function resolveCallerOwnedOpenAiSidecar(
140+
candidates: readonly OpenAiForwardSidecarCandidate[],
141+
incomingHeaders: Headers,
142+
config: OcxConfig,
143+
options: { admission?: Pick<DataPlaneAdmission, "source">; codexAuthPolicy?: CodexAuthPolicyConfig } = {},
144+
): ResolvedOpenAiForwardSidecar | undefined {
145+
const candidate = candidates[0];
146+
if (!candidate) return undefined;
147+
try {
148+
validateForwardAdmissionCredential(incomingHeaders, config);
149+
} catch (error) {
150+
if (error instanceof ForwardAdmissionCredentialError) return undefined;
151+
throw error;
152+
}
153+
if (!hasCallerCodexBearer(incomingHeaders)) return undefined;
154+
const headers = directSidecarHeaders(incomingHeaders, options.codexAuthPolicy ?? config, options.admission);
155+
if (!headers) return undefined;
156+
return { ...candidate, authContext: { kind: "main", accountId: null }, headers };
157+
}
158+
128159
export async function resolveFirstUsableOpenAiSidecar(
129160
candidates: readonly OpenAiForwardSidecarCandidate[],
130161
incomingHeaders: Headers,

0 commit comments

Comments
 (0)