Skip to content

Commit 84ade0f

Browse files
olddonkeyclaudelidge-jun
authored
fix(codex): keep oversized Responses turns off the WS transport (#2473)
* fix(codex): keep oversized Responses turns off the WS transport The Codex backend closes the socket on any inbound message of 16 MiB or more without sending a Responses terminal event, which reached clients as a bare 502 upstream_server_error. Because the wrapper only fell back to SSE when the *upgrade* failed, a thread that crossed the ceiling could never recover: every retry resent the same oversized frame. Measured against the live endpoint on 2026-08-23: 16,777,000 B completed, 16,777,300 B closed the socket in ~1s, reproducibly. The same body still succeeds over HTTP SSE, so the limit belongs to this transport alone. Size the `response.create` frame before dialing and take the SSE path when it does not fit. Deciding before the socket opens is what keeps the resend safe -- after open the caller already holds a streaming Response, and a retry there could double-generate the turn. Two supporting changes: - Carry the WS close code and reason into the stream error. A 1009 was previously indistinguishable from a network drop, and nothing in usage.jsonl or /api/logs recorded the real cause. - Apply the provider's `upstreamHttpVersion` pin to the SSE fallback. The fallback is a routine path now, and serving a turn over HTTP while silently dropping the operator's protocol pin is wrong. Co-Authored-By: Claude Fable 5 <[email protected]> * test(codex): pin the transport boundary at the adjacent byte The sizing helper already had unit tests, but nothing proved the real serialized frame routes correctly one byte on each side of the limit. That gap matters because the request body is not the frame: `stream` is deleted and `type` is added before sending, so padding sized against the body sits eleven bytes away from what is actually transmitted. An off-by-one would live exactly there and pass every existing test. These two build padding so the serialized frame is exactly limit-1 and exactly limit, then assert the whole path: one socket and one send of the expected byte length under, zero sockets and one SSE call at it. Flipping the gate from >= to > fails the second one, so it catches a real off-by-one at the transport level rather than only in the helper. Two comment corrections while here. The close-code comment claimed the named 1009 message makes the failure diagnosable from the logs; it does not. The eager relay turns any stream error into a generic `upstream_reset` synthetic terminal without feeding it back through the inspector, so `/api/logs` retains only `streamAborted`. The message reaches the client and stops there, and the comment now says so rather than promising observability the code does not deliver. The margin comment described 64 KiB as absorbing a future append. There is no append. It is a conservative cushion, and the useful thing to record is what it actually covers: RFC 6455 framing is 14 bytes at this payload size — an 8-byte extended length plus a 4-byte client mask — so even a backend counting frame headers has ~65.5 KiB of room. Tests: 59 pass, 1 skip across ws-upstream, sse-failed-tail, and upstream-http-version. tsc --noEmit clean. --------- Co-authored-by: Claude Fable 5 <[email protected]> Co-authored-by: bitkyc08-arch <[email protected]>
1 parent a60d517 commit 84ade0f

3 files changed

Lines changed: 254 additions & 7 deletions

File tree

‎src/server/responses/fetch-helpers.ts‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -64,14 +64,26 @@ export function providerFetch(
6464
options: ProviderFetchOptions = {},
6565
): ProviderFetch {
6666
const base = (provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch ?? globalThis.fetch;
67+
const preconnect = (...args: Parameters<typeof globalThis.fetch.preconnect>): void => {
68+
base.preconnect?.(...args);
69+
};
70+
const httpFetch = Object.assign(
71+
(input: Parameters<typeof globalThis.fetch>[0], init?: RequestInit) =>
72+
base(input, withUpstreamHttpVersion(input, init, provider)),
73+
{ preconnect },
74+
) as typeof globalThis.fetch;
6775
// ChatGPT Codex backend: streaming turns ride the responses_websockets
6876
// transport (measured ~3s faster TTFT than the SSE POST queue); everything
6977
// else keeps the provider's HTTP fetch. See ws-upstream.ts for the details.
7078
const unpaced = async (input: Parameters<typeof globalThis.fetch>[0], init?: RequestInit) => {
7179
if (typeof input === "string" && init && shouldUseCodexWsUpstream(input, init, runtime)) {
72-
return codexWsUpstreamFetch(input, init, base, runtime);
80+
// The fallback has to be the same HTTP fetch the non-WS branch would have
81+
// used, protocol pin included: a WS turn that falls back is serving the
82+
// request over HTTP, and dropping the provider's `upstreamHttpVersion`
83+
// there would silently negotiate a transport the operator ruled out.
84+
return codexWsUpstreamFetch(input, init, httpFetch, runtime);
7385
}
74-
return base(input, withUpstreamHttpVersion(input, init, provider));
86+
return httpFetch(input, init);
7587
};
7688
let pacingSlotAcquired = options.pacingSlotAcquired === true;
7789
const waitForPacing = (signal?: AbortSignal) => {
@@ -87,9 +99,6 @@ export function providerFetch(
8799
await waitForPacing(init?.signal ?? undefined);
88100
return unpaced(input, init);
89101
};
90-
const preconnect = (...args: Parameters<typeof globalThis.fetch.preconnect>): void => {
91-
base.preconnect?.(...args);
92-
};
93102
return Object.assign(wrapped, {
94103
preconnect,
95104
waitForPacing,

‎src/server/responses/ws-upstream.ts‎

Lines changed: 75 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,25 @@ const UPGRADE_DEADLINE_MS = 10_000;
2828
export const MAX_CODEX_WS_FRAME_BYTES = MAX_CLIENT_SSE_FRAME_BYTES;
2929
export const MAX_CODEX_WS_QUEUE_BYTES = 8 * 1024 * 1024;
3030
export const MIN_BOUNDED_CODEX_WS_BUN_VERSION = "1.4.0";
31+
// The backend drops any inbound message of 16 MiB or more: it closes the socket
32+
// (1009) without a Responses terminal event, which reaches clients as a bare
33+
// 502 upstream_server_error. Measured against the live endpoint 2026-08-23:
34+
// 16,777,000 B completed, 16,777,300 B closed in ~1s, every time. The same
35+
// request body succeeds over HTTP SSE, so the ceiling belongs to this transport
36+
// alone (see #2426). A full-replay thread reaches it with ~11 pasted
37+
// screenshots, and then never recovers, because each retry resends the frame.
38+
export const MAX_CODEX_WS_CREATE_FRAME_BYTES = 16 * 1024 * 1024;
39+
// Bun frames the payload it is handed, so the send-side budget is the JSON text
40+
// itself, and nothing is appended between the check and the send. The margin is
41+
// a conservative cushion, not a computed requirement: it covers RFC 6455 frame
42+
// overhead in case the backend counts it (14 bytes at this payload size — an
43+
// 8-byte extended length plus a 4-byte client mask, leaving ~65.5 KiB spare),
44+
// and it leaves room for a future caller that appends to the frame.
45+
const CODEX_WS_CREATE_FRAME_MARGIN_BYTES = 64 * 1024;
46+
export const CODEX_WS_CREATE_FRAME_LIMIT_BYTES =
47+
MAX_CODEX_WS_CREATE_FRAME_BYTES - CODEX_WS_CREATE_FRAME_MARGIN_BYTES;
48+
/** Close code the backend uses for an oversized message (RFC 6455 "message too big"). */
49+
const WS_CLOSE_MESSAGE_TOO_BIG = 1009;
3150

3251
export type BunRuntimeIdentity = {
3352
version: string;
@@ -102,6 +121,52 @@ export function shouldUseCodexWsUpstream(
102121
}
103122
}
104123

124+
const CLOSED_BEFORE_TERMINAL = "codex websocket closed before a Responses terminal event";
125+
126+
/**
127+
* The close code is the only thing that separates "the backend refused this
128+
* payload" from "the network dropped", and both used to reach the caller as the
129+
* same bare 502. Naming the oversized case here puts that distinction in the
130+
* message the client receives.
131+
*
132+
* It does NOT reach the request log as a typed code. The eager relay turns any
133+
* stream error into a generic `upstream_reset` synthetic terminal
134+
* (`relay.ts`, `relay-eager.ts`) without feeding that frame back through the
135+
* inspector, so `/api/logs` keeps neither this message nor a specific code —
136+
* only `streamAborted`. Machine-readable typing would mean changing the error
137+
* taxonomy, which is deliberately out of scope for this transport fix.
138+
*/
139+
function closedBeforeTerminalMessage(event: unknown): string {
140+
const detail = event as { code?: unknown; reason?: unknown } | null | undefined;
141+
const code = typeof detail?.code === "number" ? detail.code : null;
142+
const reason = typeof detail?.reason === "string" ? detail.reason.trim() : "";
143+
if (code === null) return CLOSED_BEFORE_TERMINAL;
144+
const suffix = reason ? ` ${code} ${reason}` : ` ${code}`;
145+
if (code === WS_CLOSE_MESSAGE_TOO_BIG) {
146+
return `codex websocket rejected the request frame as too large (close${suffix});`
147+
+ ` requests at or above ${MAX_CODEX_WS_CREATE_FRAME_BYTES} bytes must use the HTTP SSE transport`;
148+
}
149+
return `${CLOSED_BEFORE_TERMINAL} (close${suffix})`;
150+
}
151+
152+
/**
153+
* True when the `response.create` frame is at or above the backend's inbound
154+
* message ceiling, so this turn must take the HTTP SSE path instead.
155+
*
156+
* Sizing a 16 MiB string should not cost a 16 MiB copy. UTF-8 never encodes
157+
* below one byte per UTF-16 code unit and never above three, so both tails are
158+
* settled from the string length alone; only the narrow band between them pays
159+
* for a real byte count, and `Buffer.byteLength` measures without allocating.
160+
*/
161+
export function codexWsCreateFrameExceedsLimit(
162+
frameText: string,
163+
limitBytes: number = CODEX_WS_CREATE_FRAME_LIMIT_BYTES,
164+
): boolean {
165+
if (frameText.length >= limitBytes) return true;
166+
if (frameText.length * 3 < limitBytes) return false;
167+
return Buffer.byteLength(frameText, "utf8") >= limitBytes;
168+
}
169+
105170
export function codexWsUpstreamFetch(
106171
url: string,
107172
init: RequestInit,
@@ -127,6 +192,14 @@ export function codexWsUpstreamFetch(
127192
return sseFallback(url, init);
128193
}
129194

195+
// Decide before dialing. Once the socket is open the caller already holds a
196+
// streaming Response, so the oversized close can only be surfaced as a stream
197+
// error — and a resend at that point could double-generate. Measuring the
198+
// frame we are about to send keeps the whole failure mode unreachable.
199+
if (codexWsCreateFrameExceedsLimit(frameText)) {
200+
return sseFallback(url, init);
201+
}
202+
130203
const headers: Record<string, string> = {};
131204
new Headers(init.headers ?? {}).forEach((value, key) => {
132205
// HTTP-body framing headers do not apply to a WS handshake.
@@ -279,7 +352,7 @@ export function codexWsUpstreamFetch(
279352
}
280353
});
281354

282-
ws.addEventListener("close", () => {
355+
ws.addEventListener("close", (event: unknown) => {
283356
signal?.removeEventListener("abort", onAbort);
284357
if (!opened) {
285358
if (settledPreOpen) return;
@@ -297,7 +370,7 @@ export function codexWsUpstreamFetch(
297370
// here would reach clients with no response.completed/failed at all —
298371
// relaySseWithFailedTail() only synthesizes a failed terminal when the
299372
// body read THROWS. Error the stream like a reset TCP socket.
300-
try { controller.error(new Error("codex websocket closed before a Responses terminal event")); } catch { /* stream already done */ }
373+
try { controller.error(new Error(closedBeforeTerminalMessage(event))); } catch { /* stream already done */ }
301374
}
302375
});
303376

‎tests/ws-upstream.test.ts‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,12 @@ import { isEagerRelaySseResponse } from "../src/server/relay";
55
import { isWin32EagerRewrite } from "../src/lib/bun-stream-caps";
66
import {
77
bunSupportsBoundedCodexWsRelay,
8+
CODEX_WS_CREATE_FRAME_LIMIT_BYTES,
9+
codexWsCreateFrameExceedsLimit,
810
codexWsUpstreamFetch as rawCodexWsUpstreamFetch,
911
currentBunRuntimeIdentity,
1012
isCodexWsUpstreamResponse,
13+
MAX_CODEX_WS_CREATE_FRAME_BYTES,
1114
MAX_CODEX_WS_FRAME_BYTES,
1215
MAX_CODEX_WS_QUEUE_BYTES,
1316
shouldUseCodexWsUpstream as rawShouldUseCodexWsUpstream,
@@ -672,3 +675,165 @@ describe("codexWsUpstreamFetch", () => {
672675
expect(FakeWebSocket.instances[0].closed).toBe(true);
673676
});
674677
});
678+
679+
describe("codexWsCreateFrameExceedsLimit", () => {
680+
test("measures the frame in UTF-8 bytes, not code units", () => {
681+
// Two UTF-8 bytes per code unit, so half the limit in "é" is exactly the limit.
682+
const halfLimit = CODEX_WS_CREATE_FRAME_LIMIT_BYTES / 2;
683+
expect(codexWsCreateFrameExceedsLimit("é".repeat(halfLimit))).toBe(true);
684+
expect(codexWsCreateFrameExceedsLimit("é".repeat(halfLimit - 1))).toBe(false);
685+
});
686+
687+
test("holds the boundary at the limit itself", () => {
688+
expect(codexWsCreateFrameExceedsLimit("x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES))).toBe(true);
689+
expect(codexWsCreateFrameExceedsLimit("x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES - 1))).toBe(false);
690+
});
691+
692+
test("keeps a margin under the backend's measured ceiling", () => {
693+
// Measured against the live endpoint: 16,777,000 B completed, 16,777,300 B
694+
// closed the socket. The gate has to trip below the smaller of those.
695+
expect(MAX_CODEX_WS_CREATE_FRAME_BYTES).toBe(16 * 1024 * 1024);
696+
expect(CODEX_WS_CREATE_FRAME_LIMIT_BYTES).toBeLessThan(16_777_000);
697+
});
698+
});
699+
700+
describe("oversized Codex create frames", () => {
701+
test("takes the HTTP SSE path instead of dialing a socket the backend would close", async () => {
702+
installFake(() => { throw new Error("WS must not be dialed for an oversized frame"); });
703+
const sentinel = new Response("sse");
704+
let fallbackCalls = 0;
705+
const fallback = (async () => {
706+
fallbackCalls += 1;
707+
return sentinel;
708+
}) as unknown as typeof fetch;
709+
710+
const oversized = streamingInit({ padding: "x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES) });
711+
expect(await codexWsUpstreamFetch(CODEX_URL, oversized, fallback)).toBe(sentinel);
712+
expect(fallbackCalls).toBe(1);
713+
// Nothing was sent, so the SSE resend cannot double-generate a turn.
714+
expect(FakeWebSocket.instances).toHaveLength(0);
715+
});
716+
717+
test("keeps the provider's HTTP version pin on the fallback", async () => {
718+
installFake(() => { throw new Error("WS must not be dialed for an oversized frame"); });
719+
const seen: RequestInit[] = [];
720+
const provider = {
721+
upstreamHttpVersion: "http1.1",
722+
fetch: (async (_input: unknown, init: RequestInit) => {
723+
seen.push(init);
724+
return new Response("sse");
725+
}) as unknown as typeof fetch,
726+
} as unknown as OcxProviderConfig;
727+
const wrapped = providerFetch(provider, BOUNDED_WS_RUNTIME);
728+
729+
await wrapped(CODEX_URL, streamingInit({ padding: "x".repeat(CODEX_WS_CREATE_FRAME_LIMIT_BYTES) }));
730+
731+
expect(FakeWebSocket.instances).toHaveLength(0);
732+
// Falling back means serving the turn over HTTP, so the operator's pin has
733+
// to survive the transport switch.
734+
expect((seen[0] as { protocol?: string }).protocol).toBe("http1.1");
735+
});
736+
737+
test("still uses WS for a frame that fits", async () => {
738+
installFake(ws => {
739+
ws.emit("open", {});
740+
ws.emit("message", { data: JSON.stringify({ type: "response.completed", response: {} }) });
741+
});
742+
const fallback = (() => {
743+
throw new Error("fallback must not run for a frame that fits");
744+
}) as unknown as typeof fetch;
745+
746+
const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit({ padding: "x".repeat(1024) }), fallback);
747+
expect(isCodexWsUpstreamResponse(response)).toBe(true);
748+
expect(FakeWebSocket.instances).toHaveLength(1);
749+
});
750+
751+
// The unit tests above measure the helper; these two measure the REAL serialized
752+
// frame, one byte on each side of the limit. That distinction matters because the
753+
// request body is not the frame: `stream` is deleted and `type` is added before
754+
// sending, so padding sized against the body would sit at a different offset than
755+
// the bytes actually transmitted. An off-by-one lives exactly here and nowhere else.
756+
describe("the adjacent-byte transport boundary", () => {
757+
// Build padding such that the serialized frame is EXACTLY `target` bytes.
758+
function initForFrameBytes(target: number): RequestInit {
759+
const probe = frameTextFor(0);
760+
const padding = "x".repeat(target - Buffer.byteLength(probe, "utf8"));
761+
const init = streamingInit({ padding });
762+
const actual = Buffer.byteLength(frameTextFor(padding.length), "utf8");
763+
if (actual !== target) throw new Error(`frame sizing is wrong: wanted ${target}, built ${actual}`);
764+
return init;
765+
}
766+
767+
function frameTextFor(paddingLength: number): string {
768+
const body = JSON.parse(streamingInit({ padding: "x".repeat(paddingLength) }).body as string) as Record<string, unknown>;
769+
delete body.stream;
770+
return JSON.stringify({ ...body, type: "response.create" });
771+
}
772+
773+
test("a frame one byte under the limit goes over WS", async () => {
774+
installFake(ws => {
775+
ws.emit("open", {});
776+
ws.emit("message", { data: JSON.stringify({ type: "response.completed", response: {} }) });
777+
});
778+
const fallback = (() => {
779+
throw new Error("fallback must not run one byte under the limit");
780+
}) as unknown as typeof fetch;
781+
782+
const response = await codexWsUpstreamFetch(
783+
CODEX_URL,
784+
initForFrameBytes(CODEX_WS_CREATE_FRAME_LIMIT_BYTES - 1),
785+
fallback,
786+
);
787+
expect(isCodexWsUpstreamResponse(response)).toBe(true);
788+
expect(FakeWebSocket.instances).toHaveLength(1);
789+
expect(FakeWebSocket.instances[0]!.sent).toHaveLength(1);
790+
// Byte length, not code units: the limit is a byte budget, and this
791+
// assertion should keep meaning the same thing if the fixture ever
792+
// carries non-ASCII text.
793+
expect(Buffer.byteLength(FakeWebSocket.instances[0]!.sent[0]!, "utf8"))
794+
.toBe(CODEX_WS_CREATE_FRAME_LIMIT_BYTES - 1);
795+
});
796+
797+
test("the very next byte takes SSE without dialing", async () => {
798+
installFake(() => { throw new Error("WS must not be dialed at the limit"); });
799+
const sentinel = new Response("sse");
800+
let fallbackCalls = 0;
801+
const fallback = (async () => { fallbackCalls += 1; return sentinel; }) as unknown as typeof fetch;
802+
803+
const response = await codexWsUpstreamFetch(
804+
CODEX_URL,
805+
initForFrameBytes(CODEX_WS_CREATE_FRAME_LIMIT_BYTES),
806+
fallback,
807+
);
808+
expect(response).toBe(sentinel);
809+
expect(fallbackCalls).toBe(1);
810+
expect(FakeWebSocket.instances).toHaveLength(0);
811+
});
812+
});
813+
814+
test("names the oversized close instead of reporting a bare drop", async () => {
815+
installFake(ws => {
816+
ws.emit("open", {});
817+
ws.emit("close", { code: 1009, reason: "Message Too Big" });
818+
});
819+
const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => {
820+
throw new Error("fallback must not run after open");
821+
}) as unknown as typeof fetch);
822+
823+
await expect(response.text()).rejects.toThrow(
824+
/rejected the request frame as too large \(close 1009 Message Too Big\)/,
825+
);
826+
});
827+
828+
test("carries the close code for any other pre-terminal drop", async () => {
829+
installFake(ws => {
830+
ws.emit("open", {});
831+
ws.emit("close", { code: 1006 });
832+
});
833+
const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => {
834+
throw new Error("fallback must not run after open");
835+
}) as unknown as typeof fetch);
836+
837+
await expect(response.text()).rejects.toThrow("closed before a Responses terminal event (close 1006)");
838+
});
839+
});

0 commit comments

Comments
 (0)