Skip to content

Commit 118c66a

Browse files
authored
fix(responses): keep custom websocket upstreams on bounded HTTP SSE (#5141)
Bun's client WebSocket API assembles complete messages before handlers run and has no enforceable inbound payload limit, so a malicious configured upstream could force unbounded allocations. Restrict the WS lane to the first-party endpoints: the canonical ChatGPT backend unconditionally and api.openai.com under the existing upstreamWebsocket opt-in. Arbitrary provider endpoints stay on bounded HTTP/SSE. Co-authored-by: luvs01 <[email protected]>
1 parent dfb76e7 commit 118c66a

3 files changed

Lines changed: 34 additions & 57 deletions

File tree

‎docs-site/src/content/docs/reference/configuration/providers.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity
146146
| `responsesPath?` | `string` | Relative resource path for key-auth `openai-responses` requests. It must start with `/` and contain no scheme, query, or fragment. |
147147
| `chatCompletionsPath?` | `string` | Relative resource path for `openai-chat` requests, the mirror of `responsesPath` and subject to the same shape rules. Needed when one upstream serves Chat Completions and Responses under different prefixes: a per-model wire override changes the adapter and leaves `baseUrl` alone, so without this an opted-in Chat request would be sent to the Responses base. Z.AI is the shipped example. |
148148
| `allowEncryptedV2AgentTasks?` | `boolean` | Disabled by default. Trust a direct key-auth `openai-responses` provider to consume or relay opaque encrypted V2 sub-agent tasks unchanged. Eligible routes skip `agentTaskRecovery`; all other routes keep the existing recovery or fail-closed behavior. OpenCodex does not decrypt, translate, or recover tasks sent through this opt-in. |
149-
| `upstreamWebsocket?` | `boolean` | Opt-in upstream Responses WebSocket transport for `openai-responses` requests (default false). When the upstream supports the Responses WebSocket protocol, streaming POST requests to the configured Responses path (default `/v1/responses`) are dialed as WSS over an HTTPS base URL and re-encoded to SSE for the usual pipeline. Forward providers use `{baseUrl}/responses`; key-auth providers use `responsesPath`, or the legacy `/v1/responses` fallback. This mirrors the canonical ChatGPT backend optimization for OpenAI-compatible gateways (for example sub2api) whose WebSocket ingress is measurably faster than its SSE queue. Plain HTTP remains on SSE; non-Responses paths and `openai-chat` requests stay on HTTP. |
149+
| `upstreamWebsocket?` | `boolean` | Opt-in upstream Responses WebSocket transport for `openai-responses` requests (default false). Honored only for the first-party `https://api.openai.com/v1` upstream; custom-provider endpoints always use bounded HTTP/SSE because Bun cannot enforce an inbound WebSocket message limit before allocating the complete message. The canonical ChatGPT transport is unaffected. Plain HTTP remains on SSE; non-Responses paths and `openai-chat` requests stay on HTTP. |
150150
| `supportsServiceTier?` | `boolean` | Tri-state canonical Fast capability fallback. `true` publishes Fast in the catalog, satisfies service-tier routing requirements, contributes a supported fingerprint, and lets fast mode inject the provider's canonical wire value on a compatible final adapter. `false` strips the field and never injects, and exact model declarations cannot reopen it. Absent leaves the provider unclassified: fast mode does not inject or normalize a canonical caller value, and caller values obey the final wire's forwarding permission (`chatServiceTier` on Chat; passthrough on Responses). The registry classifies canonical OpenAI (`true`), DeepSeek, and Volcengine Ark (`false`); set it explicitly only for custom gateways that genuinely support tiers. |
151151
| `modelSupportsServiceTier?` | `Record<string, boolean>` | Exact upstream model capability overrides. Exact `true` enables canonical Fast for that model; exact `false` narrows provider defaults. An explicit provider-level `supportsServiceTier: false` remains fail-closed and cannot be reopened. Exact `true` does not authorize foreign caller-tier forwarding on Chat. Undeclared models fall back to provider-wide behavior. Management `PATCH /api/providers` merges entries and accepts `null` to clear one. |
152152
| `chatServiceTier?` | `boolean` | Provider-wide Chat-wire opt-in for forwarding caller `service_tier` values. On a classified route it governs foreign values such as `flex`, not proxy-owned canonical Fast after capability validation; on an unclassified route it governs every caller value because no Fast capability has been validated. Exact model capability does not authorize foreign forwarding. Responses routes retain their capability-based caller forwarding behavior. |

‎src/server/responses/ws-upstream.ts‎

Lines changed: 14 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -29,33 +29,14 @@ export { CODEX_WS_LIVENESS_PING_INTERVAL_MS, CODEX_WS_RESPONSE_PRELUDE_TIMEOUT_M
2929
export const MIN_BOUNDED_CODEX_WS_BUN_VERSION = "1.4.0";
3030

3131
/**
32-
* Dial URL for a request URL. The canonical ChatGPT backend keeps its constant;
33-
* an operator-opted OpenAI-compatible upstream swaps https for wss on the same
34-
* path so gateways that serve the Responses WebSocket protocol on their
35-
* /v1/responses path get the same fast lane. Plain HTTP remains on SSE because
36-
* a provider WS handshake would otherwise send credentials and request data
37-
* without transport encryption.
32+
* Dial URL for a first-party Responses endpoint. The canonical ChatGPT backend
33+
* keeps its constant; the api.openai.com Responses endpoint swaps https for wss
34+
* on the same path. No other upstream may enter the WebSocket lane.
3835
*/
3936
function wsUpstreamUrlFor(httpUrl: string): string {
4037
if (httpUrl === CODEX_RESPONSES_HTTP_URL) return CODEX_RESPONSES_WS_URL;
41-
return httpUrl.replace(/^http(s?):/, "ws$1:");
42-
}
43-
44-
/**
45-
* An operator-opted OpenAI-compatible upstream only joins the WS lane for
46-
* Responses endpoints: the WebSocket path speaks the Responses event protocol,
47-
* and every downstream consumer (adapter parsers, usage sniffing, SSE relay)
48-
* assumes that wire. Other paths (chat completions, images, search) stay HTTP.
49-
*/
50-
function isResponsesWebsocketEligibleUrl(url: string): boolean {
51-
let parsed: URL;
52-
try {
53-
parsed = new URL(url);
54-
} catch {
55-
return false;
56-
}
57-
return parsed.protocol === "https:"
58-
&& parsed.pathname.endsWith("/responses");
38+
if (httpUrl === OPENAI_API_RESPONSES_URL) return httpUrl.replace(/^http(s?):/, "ws$1:");
39+
throw new Error("unsupported Codex WebSocket upstream");
5940
}
6041
export type BunRuntimeIdentity = {
6142
version: string;
@@ -108,8 +89,13 @@ export function shouldUseCodexWsUpstream(
10889
): boolean {
10990
if (!bunSupportsBoundedCodexWsRelay(runtime)) return false;
11091
if (socks5ProxyFromEnv()) return false;
111-
if (url !== CODEX_RESPONSES_HTTP_URL && !upstreamWebsocketConfigured) return false;
112-
if (upstreamWebsocketConfigured && !isResponsesWebsocketEligibleUrl(url)) return false;
92+
// Bun's client WebSocket API delivers only fully assembled messages and has
93+
// no enforceable inbound payload limit. Keep arbitrary provider endpoints on
94+
// bounded HTTP/SSE until the client can reject fragmented text and binary
95+
// messages during ingestion rather than after allocation. The first-party
96+
// api.openai.com lane still requires the operator opt-in.
97+
if (url !== CODEX_RESPONSES_HTTP_URL
98+
&& !(upstreamWebsocketConfigured && url === OPENAI_API_RESPONSES_URL)) return false;
11399
if ((init?.method ?? "GET").toUpperCase() !== "POST") return false;
114100
const body = init?.body;
115101
if (typeof body !== "string") return false;
@@ -139,7 +125,8 @@ export function codexWsUpstreamFetch(
139125
const prepared = prepareCodexWsRequest(url, init);
140126
if (!prepared) return sseFallback(url, prepareCodexHttpInit(url, init));
141127
init = prepared.httpInit;
142-
if (!bunSupportsBoundedCodexWsRelay(runtime)) {
128+
if ((url !== CODEX_RESPONSES_HTTP_URL && url !== OPENAI_API_RESPONSES_URL)
129+
|| !bunSupportsBoundedCodexWsRelay(runtime)) {
143130
return sseFallback(url, init);
144131
}
145132
const signal = init.signal ?? undefined;

‎tests/responses/ws-upstream.test.ts‎

Lines changed: 19 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -150,11 +150,15 @@ describe("shouldUseCodexWsUpstream", () => {
150150
expect(shouldUseCodexWsUpstream(CODEX_URL, { method: "POST", body: "{\"stream\":true" })).toBe(false);
151151
});
152152

153-
test("opt-in upstream WebSocket only for configured OpenAI-compatible Responses endpoints", () => {
153+
test("keeps configured provider endpoints on bounded HTTP SSE", () => {
154154
// The canonical backend ignores the flag.
155155
expect(shouldUseCodexWsUpstream(CODEX_URL, streamingInit(), false)).toBe(true);
156-
// Configured providers join the WS lane on their own /v1/responses path.
157-
expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/responses", streamingInit(), true)).toBe(true);
156+
// Bun cannot reject oversized messages before assembling them, so even an
157+
// opted-in provider cannot join the WebSocket lane.
158+
expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/responses", streamingInit(), true)).toBe(false);
159+
// The first-party api.openai.com lane still honors the operator opt-in.
160+
expect(shouldUseCodexWsUpstream("https://api.openai.com/v1/responses", streamingInit(), true)).toBe(true);
161+
expect(shouldUseCodexWsUpstream("https://api.openai.com/v1/responses", streamingInit(), false)).toBe(false);
158162
// Plain HTTP stays on SSE; never send credentials or request data through ws://.
159163
expect(shouldUseCodexWsUpstream("http://10.0.0.5:8080/v1/responses", streamingInit(), true)).toBe(false);
160164
expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/responses", streamingInit(), false)).toBe(false);
@@ -297,11 +301,7 @@ describe("providerFetch routing", () => {
297301
expect(FakeWebSocket.instances).toHaveLength(1);
298302
});
299303

300-
test("routes an opt-in provider's Responses streams over its upstream WS", async () => {
301-
installFake(ws => {
302-
ws.emit("open", {});
303-
ws.emit("message", { data: JSON.stringify({ type: "response.completed", response: { id: "r1" } }) });
304-
});
304+
test("routes an opt-in provider's Responses streams over bounded HTTP SSE", async () => {
305305
const baseCalls: string[] = [];
306306
const sentinel = new Response("base");
307307
const provider = {
@@ -313,16 +313,15 @@ describe("providerFetch routing", () => {
313313
} as unknown as OcxProviderConfig;
314314
const wrapped = providerFetch(provider, BOUNDED_WS_RUNTIME);
315315

316-
const wsResponse = await wrapped("https://sub2api.example.com/v1/responses", streamingInit());
317-
expect(wsResponse.headers.get("content-type")).toContain("text/event-stream");
318-
expect(baseCalls).toHaveLength(0);
319-
expect(FakeWebSocket.instances).toHaveLength(1);
320-
expect(FakeWebSocket.instances[0]!.url).toBe("wss://sub2api.example.com/v1/responses");
316+
const response = await wrapped("https://sub2api.example.com/v1/responses", streamingInit());
317+
expect(await response.text()).toBe("base");
318+
expect(baseCalls).toHaveLength(1);
319+
expect(FakeWebSocket.instances).toHaveLength(0);
321320

322321
// The same provider's non-Responses paths (images/search/chat) stay on the base fetch.
323322
await wrapped("https://sub2api.example.com/v1/images", streamingInit());
324-
expect(baseCalls).toHaveLength(1);
325-
expect(FakeWebSocket.instances).toHaveLength(1);
323+
expect(baseCalls).toHaveLength(2);
324+
expect(FakeWebSocket.instances).toHaveLength(0);
326325
});
327326
});
328327

@@ -1064,12 +1063,12 @@ describe("codexWsUpstreamFetch", () => {
10641063
});
10651064

10661065
test("keeps noncanonical providers on the stream path", async () => {
1067-
const response = await receive(refusal, [], "https://gateway.example/v1/responses");
1066+
const response = await receive(refusal, [], "https://api.openai.com/v1/responses");
10681067
expect(response.status).toBe(200);
10691068
expect(await response.text()).toContain("event: error");
10701069
});
10711070

1072-
test.each([CODEX_URL, "https://gateway.example/v1/responses"])(
1071+
test.each([CODEX_URL, "https://api.openai.com/v1/responses"])(
10731072
"settles synchronous error/send-throw/close races and detaches deadlines for %s", async url => {
10741073
jest.useFakeTimers();
10751074
const abort = new AbortController();
@@ -1953,24 +1952,15 @@ describe("oversized Codex create frames", () => {
19531952
expect(stage?.relayedEvents).toBeGreaterThan(0);
19541953
});
19551954

1956-
test("dials the configured provider's own wss URL for an opt-in upstream", async () => {
1957-
process.env.HTTPS_PROXY = "http://proxy.example:8080";
1958-
process.env.NO_PROXY = "sub2api.example.com:443";
1959-
installFake(ws => {
1960-
ws.emit("open", {});
1961-
ws.emit("message", { data: JSON.stringify({ type: "response.completed", response: { id: "r-ws" } }) });
1962-
});
1955+
test("falls back before dialing a custom upstream URL", async () => {
19631956
const sentinel = new Response("fallback");
19641957
const response = await codexWsUpstreamFetch(
19651958
"https://sub2api.example.com/v1/responses",
19661959
streamingInit(),
19671960
(async () => sentinel) as typeof fetch,
19681961
);
1969-
expect(FakeWebSocket.instances).toHaveLength(1);
1970-
expect(FakeWebSocket.instances[0]!.url).toBe("wss://sub2api.example.com/v1/responses");
1971-
expect(FakeWebSocket.instances[0]!.options?.proxy).toBeUndefined();
1972-
expect(response.headers.get("content-type")).toContain("text/event-stream");
1973-
expect(await response.text()).toContain("response.completed");
1962+
expect(FakeWebSocket.instances).toHaveLength(0);
1963+
expect(response).toBe(sentinel);
19741964
});
19751965

19761966
test("response.done normalization keeps unknown usage fields (#41980 parity)", async () => {

0 commit comments

Comments
 (0)