Skip to content

Commit 5a3d32c

Browse files
olddonkeyclaude
andcommitted
fix(codex): keep oversized Responses turns off the WS transport
The Codex backend closes the socket on any inbound message of 16 MiB or more without sending a Responses terminal event, which reached clients as a bare 502 upstream_server_error. Because the wrapper only fell back to SSE when the *upgrade* failed, a thread that crossed the ceiling could never recover: every retry resent the same oversized frame. Measured against the live endpoint on 2026-08-23: 16,777,000 B completed, 16,777,300 B closed the socket in ~1s, reproducibly. The same body still succeeds over HTTP SSE, so the limit belongs to this transport alone. Size the `response.create` frame before dialing and take the SSE path when it does not fit. Deciding before the socket opens is what keeps the resend safe -- after open the caller already holds a streaming Response, and a retry there could double-generate the turn. Two supporting changes: - Carry the WS close code and reason into the stream error. A 1009 was previously indistinguishable from a network drop, and nothing in usage.jsonl or /api/logs recorded the real cause. - Apply the provider's `upstreamHttpVersion` pin to the SSE fallback. The fallback is a routine path now, and serving a turn over HTTP while silently dropping the operator's protocol pin is wrong. Co-Authored-By: Claude Fable 5 <[email protected]>
1 parent c44e43f commit 5a3d32c

3 files changed

Lines changed: 181 additions & 7 deletions

File tree

‎src/server/responses/fetch-helpers.ts‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -64,14 +64,26 @@ export function providerFetch(
6464
options: ProviderFetchOptions = {},
6565
): ProviderFetch {
6666
const base = (provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch ?? globalThis.fetch;
67+
const preconnect = (...args: Parameters<typeof globalThis.fetch.preconnect>): void => {
68+
base.preconnect?.(...args);
69+
};
70+
const httpFetch = Object.assign(
71+
(input: Parameters<typeof globalThis.fetch>[0], init?: RequestInit) =>
72+
base(input, withUpstreamHttpVersion(input, init, provider)),
73+
{ preconnect },
74+
) as typeof globalThis.fetch;
6775
// ChatGPT Codex backend: streaming turns ride the responses_websockets
6876
// transport (measured ~3s faster TTFT than the SSE POST queue); everything
6977
// else keeps the provider's HTTP fetch. See ws-upstream.ts for the details.
7078
const unpaced = async (input: Parameters<typeof globalThis.fetch>[0], init?: RequestInit) => {
7179
if (typeof input === "string" && init && shouldUseCodexWsUpstream(input, init, runtime)) {
72-
return codexWsUpstreamFetch(input, init, base, runtime);
80+
// The fallback has to be the same HTTP fetch the non-WS branch would have
81+
// used, protocol pin included: a WS turn that falls back is serving the
82+
// request over HTTP, and dropping the provider's `upstreamHttpVersion`
83+
// there would silently negotiate a transport the operator ruled out.
84+
return codexWsUpstreamFetch(input, init, httpFetch, runtime);
7385
}
74-
return base(input, withUpstreamHttpVersion(input, init, provider));
86+
return httpFetch(input, init);
7587
};
7688
let pacingSlotAcquired = options.pacingSlotAcquired === true;
7789
const waitForPacing = (signal?: AbortSignal) => {
@@ -87,9 +99,6 @@ export function providerFetch(
8799
await waitForPacing(init?.signal ?? undefined);
88100
return unpaced(input, init);
89101
};
90-
const preconnect = (...args: Parameters<typeof globalThis.fetch.preconnect>): void => {
91-
base.preconnect?.(...args);
92-
};
93102
return Object.assign(wrapped, {
94103
preconnect,
95104
waitForPacing,

‎src/server/responses/ws-upstream.ts‎

Lines changed: 65 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,22 @@ const UPGRADE_DEADLINE_MS = 10_000;
2828
export const MAX_CODEX_WS_FRAME_BYTES = MAX_CLIENT_SSE_FRAME_BYTES;
2929
export const MAX_CODEX_WS_QUEUE_BYTES = 8 * 1024 * 1024;
3030
export const MIN_BOUNDED_CODEX_WS_BUN_VERSION = "1.4.0";
31+
// The backend drops any inbound message of 16 MiB or more: it closes the socket
32+
// (1009) without a Responses terminal event, which reaches clients as a bare
33+
// 502 upstream_server_error. Measured against the live endpoint 2026-08-23:
34+
// 16,777,000 B completed, 16,777,300 B closed in ~1s, every time. The same
35+
// request body succeeds over HTTP SSE, so the ceiling belongs to this transport
36+
// alone (see #2426). A full-replay thread reaches it with ~11 pasted
37+
// screenshots, and then never recovers, because each retry resends the frame.
38+
export const MAX_CODEX_WS_CREATE_FRAME_BYTES = 16 * 1024 * 1024;
39+
// Bun frames the payload it is handed, so the send-side budget is the JSON text
40+
// itself. The margin only absorbs the difference between the measurement here
41+
// and what a future caller might append after it.
42+
const CODEX_WS_CREATE_FRAME_MARGIN_BYTES = 64 * 1024;
43+
export const CODEX_WS_CREATE_FRAME_LIMIT_BYTES =
44+
MAX_CODEX_WS_CREATE_FRAME_BYTES - CODEX_WS_CREATE_FRAME_MARGIN_BYTES;
45+
/** Close code the backend uses for an oversized message (RFC 6455 "message too big"). */
46+
const WS_CLOSE_MESSAGE_TOO_BIG = 1009;
3147

3248
export type BunRuntimeIdentity = {
3349
version: string;
@@ -102,6 +118,45 @@ export function shouldUseCodexWsUpstream(
102118
}
103119
}
104120

121+
const CLOSED_BEFORE_TERMINAL = "codex websocket closed before a Responses terminal event";
122+
123+
/**
124+
* The close code is the only thing that separates "the backend refused this
125+
* payload" from "the network dropped", and both used to reach the logs as the
126+
* same bare 502. Naming the oversized case here is what makes it diagnosable
127+
* without reading the client's own rollout files.
128+
*/
129+
function closedBeforeTerminalMessage(event: unknown): string {
130+
const detail = event as { code?: unknown; reason?: unknown } | null | undefined;
131+
const code = typeof detail?.code === "number" ? detail.code : null;
132+
const reason = typeof detail?.reason === "string" ? detail.reason.trim() : "";
133+
if (code === null) return CLOSED_BEFORE_TERMINAL;
134+
const suffix = reason ? ` ${code} ${reason}` : ` ${code}`;
135+
if (code === WS_CLOSE_MESSAGE_TOO_BIG) {
136+
return `codex websocket rejected the request frame as too large (close${suffix});`
137+
+ ` requests at or above ${MAX_CODEX_WS_CREATE_FRAME_BYTES} bytes must use the HTTP SSE transport`;
138+
}
139+
return `${CLOSED_BEFORE_TERMINAL} (close${suffix})`;
140+
}
141+
142+
/**
143+
* True when the `response.create` frame is at or above the backend's inbound
144+
* message ceiling, so this turn must take the HTTP SSE path instead.
145+
*
146+
* Sizing a 16 MiB string should not cost a 16 MiB copy. UTF-8 never encodes
147+
* below one byte per UTF-16 code unit and never above three, so both tails are
148+
* settled from the string length alone; only the narrow band between them pays
149+
* for a real byte count, and `Buffer.byteLength` measures without allocating.
150+
*/
151+
export function codexWsCreateFrameExceedsLimit(
152+
frameText: string,
153+
limitBytes: number = CODEX_WS_CREATE_FRAME_LIMIT_BYTES,
154+
): boolean {
155+
if (frameText.length >= limitBytes) return true;
156+
if (frameText.length * 3 < limitBytes) return false;
157+
return Buffer.byteLength(frameText, "utf8") >= limitBytes;
158+
}
159+
105160
export function codexWsUpstreamFetch(
106161
url: string,
107162
init: RequestInit,
@@ -127,6 +182,14 @@ export function codexWsUpstreamFetch(
127182
return sseFallback(url, init);
128183
}
129184

185+
// Decide before dialing. Once the socket is open the caller already holds a
186+
// streaming Response, so the oversized close can only be surfaced as a stream
187+
// error — and a resend at that point could double-generate. Measuring the
188+
// frame we are about to send keeps the whole failure mode unreachable.
189+
if (codexWsCreateFrameExceedsLimit(frameText)) {
190+
return sseFallback(url, init);
191+
}
192+
130193
const headers: Record<string, string> = {};
131194
new Headers(init.headers ?? {}).forEach((value, key) => {
132195
// HTTP-body framing headers do not apply to a WS handshake.
@@ -279,7 +342,7 @@ export function codexWsUpstreamFetch(
279342
}
280343
});
281344

282-
ws.addEventListener("close", () => {
345+
ws.addEventListener("close", (event: unknown) => {
283346
signal?.removeEventListener("abort", onAbort);
284347
if (!opened) {
285348
if (settledPreOpen) return;
@@ -297,7 +360,7 @@ export function codexWsUpstreamFetch(
297360
// here would reach clients with no response.completed/failed at all —
298361
// relaySseWithFailedTail() only synthesizes a failed terminal when the
299362
// body read THROWS. Error the stream like a reset TCP socket.
300-
try { controller.error(new Error("codex websocket closed before a Responses terminal event")); } catch { /* stream already done */ }
363+
try { controller.error(new Error(closedBeforeTerminalMessage(event))); } catch { /* stream already done */ }
301364
}
302365
});
303366

‎tests/ws-upstream.test.ts‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,12 @@ import { isEagerRelaySseResponse } from "../src/server/relay";
55
import { isWin32EagerRewrite } from "../src/lib/bun-stream-caps";
66
import {
77
bunSupportsBoundedCodexWsRelay,
8+
CODEX_WS_CREATE_FRAME_LIMIT_BYTES,
9+
codexWsCreateFrameExceedsLimit,
810
codexWsUpstreamFetch as rawCodexWsUpstreamFetch,
911
currentBunRuntimeIdentity,
1012
isCodexWsUpstreamResponse,
13+
MAX_CODEX_WS_CREATE_FRAME_BYTES,
1114
MAX_CODEX_WS_FRAME_BYTES,
1215
MAX_CODEX_WS_QUEUE_BYTES,
1316
shouldUseCodexWsUpstream as rawShouldUseCodexWsUpstream,
@@ -672,3 +675,102 @@ describe("codexWsUpstreamFetch", () => {
672675
expect(FakeWebSocket.instances[0].closed).toBe(true);
673676
});
674677
});
678+
679+
describe("codexWsCreateFrameExceedsLimit", () => {
680+
test("measures the frame in UTF-8 bytes, not code units", () => {
681+
// Two UTF-8 bytes per code unit, so half the limit in "é" is exactly the limit.
682+
const halfLimit = CODEX_WS_CREATE_FRAME_LIMIT_BYTES / 2;
683+
expect(codexWsCreateFrameExceedsLimit("é".repeat(halfLimit))).toBe(true);
684+
expect(codexWsCreateFrameExceedsLimit("é".repeat(halfLimit - 1))).toBe(false);
685+
});
686+
687+
test("holds the boundary at the limit itself", () => {
688+
expect(codexWsCreateFrameExceedsLimit("x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES))).toBe(true);
689+
expect(codexWsCreateFrameExceedsLimit("x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES - 1))).toBe(false);
690+
});
691+
692+
test("keeps a margin under the backend's measured ceiling", () => {
693+
// Measured against the live endpoint: 16,777,000 B completed, 16,777,300 B
694+
// closed the socket. The gate has to trip below the smaller of those.
695+
expect(MAX_CODEX_WS_CREATE_FRAME_BYTES).toBe(16 * 1024 * 1024);
696+
expect(CODEX_WS_CREATE_FRAME_LIMIT_BYTES).toBeLessThan(16_777_000);
697+
});
698+
});
699+
700+
describe("oversized Codex create frames", () => {
701+
test("takes the HTTP SSE path instead of dialing a socket the backend would close", async () => {
702+
installFake(() => { throw new Error("WS must not be dialed for an oversized frame"); });
703+
const sentinel = new Response("sse");
704+
let fallbackCalls = 0;
705+
const fallback = (async () => {
706+
fallbackCalls += 1;
707+
return sentinel;
708+
}) as unknown as typeof fetch;
709+
710+
const oversized = streamingInit({ padding: "x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES) });
711+
expect(await codexWsUpstreamFetch(CODEX_URL, oversized, fallback)).toBe(sentinel);
712+
expect(fallbackCalls).toBe(1);
713+
// Nothing was sent, so the SSE resend cannot double-generate a turn.
714+
expect(FakeWebSocket.instances).toHaveLength(0);
715+
});
716+
717+
test("keeps the provider's HTTP version pin on the fallback", async () => {
718+
installFake(() => { throw new Error("WS must not be dialed for an oversized frame"); });
719+
const seen: RequestInit[] = [];
720+
const provider = {
721+
upstreamHttpVersion: "http1.1",
722+
fetch: (async (_input: unknown, init: RequestInit) => {
723+
seen.push(init);
724+
return new Response("sse");
725+
}) as unknown as typeof fetch,
726+
} as unknown as OcxProviderConfig;
727+
const wrapped = providerFetch(provider, BOUNDED_WS_RUNTIME);
728+
729+
await wrapped(CODEX_URL, streamingInit({ padding: "x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES) }));
730+
731+
expect(FakeWebSocket.instances).toHaveLength(0);
732+
// Falling back means serving the turn over HTTP, so the operator's pin has
733+
// to survive the transport switch.
734+
expect((seen[0] as { protocol?: string }).protocol).toBe("http1.1");
735+
});
736+
737+
test("still uses WS for a frame that fits", async () => {
738+
installFake(ws => {
739+
ws.emit("open", {});
740+
ws.emit("message", { data: JSON.stringify({ type: "response.completed", response: {} }) });
741+
});
742+
const fallback = (() => {
743+
throw new Error("fallback must not run for a frame that fits");
744+
}) as unknown as typeof fetch;
745+
746+
const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit({ padding: "x".repeat(1024) }), fallback);
747+
expect(isCodexWsUpstreamResponse(response)).toBe(true);
748+
expect(FakeWebSocket.instances).toHaveLength(1);
749+
});
750+
751+
test("names the oversized close instead of reporting a bare drop", async () => {
752+
installFake(ws => {
753+
ws.emit("open", {});
754+
ws.emit("close", { code: 1009, reason: "Message Too Big" });
755+
});
756+
const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => {
757+
throw new Error("fallback must not run after open");
758+
}) as unknown as typeof fetch);
759+
760+
await expect(response.text()).rejects.toThrow(
761+
/rejected the request frame as too large \(close 1009 Message Too Big\)/,
762+
);
763+
});
764+
765+
test("carries the close code for any other pre-terminal drop", async () => {
766+
installFake(ws => {
767+
ws.emit("open", {});
768+
ws.emit("close", { code: 1006 });
769+
});
770+
const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => {
771+
throw new Error("fallback must not run after open");
772+
}) as unknown as typeof fetch);
773+
774+
await expect(response.text()).rejects.toThrow("closed before a Responses terminal event (close 1006)");
775+
});
776+
});

0 commit comments

Comments
 (0)