Skip to content

Commit 5a75e57

Browse files
committed
fix(grok): switch to Responses backend and backfill required annotations
Grok CLI was pinned to api_backend = "chat_completions" because opencodex emitted response.heartbeat as a typed SSE event. That is not a valid Responses variant, so Grok-build's strict enum deserializer crashed with "unknown variant response.heartbeat". The keep-alive now emits an SSE comment line instead, which re-arms the idle timer without triggering deserialization on any client. With heartbeats fixed, Grok can finally use the Responses passthrough path. This gives Grok clients the same protocol fidelity Codex already enjoys and removes the chat to responses translation layer from the hot path. Some upstream relays (e.g. sub2api) omit annotations on output_text content parts even though the Responses spec marks it as a required Vec field. Strict clients, including Grok-build's async-openai fork, fail with "missing field annotations". A new stateless SSE/JSON backfill adds annotations: [] on any output_text part that lacks it, on both the streaming and bounded-JSON passthrough paths. The rewrite is unconditional and safe for all clients because the field is always valid on the wire. The /v1/responses handler now surfaces grok-tagged requests as surface=grok in the log context, matching the chat-completions handler. Stale comments referencing grok-build's decoder and the old chat_completions pin have been corrected in the tests.
1 parent dec332c commit 5a75e57

10 files changed

Lines changed: 392 additions & 25 deletions

‎src/bridge.ts‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -325,10 +325,12 @@ export function bridgeToResponsesSSE(
325325
clearOwnedWatchdog();
326326
};
327327
// RC3 keep-alive: Codex's idle timer is timeout(idle_timeout, stream.next()) over an
328-
// eventsource_stream; ANY received event re-arms it, while an unknown type is ignored
329-
// (responses.rs `_ => Ok(None)`). Emit a parser-ignored `response.heartbeat` whenever the
328+
// eventsource_stream; ANY received bytes re-arm it. An SSE comment line (a line starting
329+
// with `:`) is discarded by every eventsource parser without producing an event, so it
330+
// keeps the wire alive without triggering deserialization. Emit a comment line whenever the
330331
// *wire* has been silent, even if invisible adapter heartbeats are still flowing (web-search
331-
// buffering + raw-byte progress). Upstream activity only resets the stall watchdog.
332+
// buffering + raw-byte progress). Upstream activity only resets the stall watchdog. Parity
333+
// with the passthrough relay's `: opencodex keepalive` (relay.ts).
332334
let upstreamActivity = false;
333335
let wireActivity = false;
334336
let beat: unknown;
@@ -395,7 +397,7 @@ export function bridgeToResponsesSSE(
395397
...(endTurn !== undefined ? { end_turn: endTurn } : {}),
396398
});
397399

398-
const heartbeatFrame = encoder.encode('event: response.heartbeat\ndata: {"type":"response.heartbeat"}\n\n');
400+
const heartbeatFrame = encoder.encode(': opencodex heartbeat\n\n');
399401
let stallTicks = 0;
400402
const stallSec = resolveStallTimeoutSec(options?.stallTimeoutSec);
401403
const maxStallTicks = Math.ceil((stallSec * 1000) / heartbeatMs);

‎src/grok/inject.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -324,7 +324,7 @@ export function buildGrokManagedBlock(
324324
`[model.${alias}]`,
325325
`model = ${tomlString(model.id)}`,
326326
`base_url = ${tomlString(baseUrl)}`,
327-
'api_backend = "chat_completions"',
327+
'api_backend = "responses"',
328328
'api_key = "opencodex-loopback"',
329329
`name = ${tomlString(model.name ?? `OCX ${model.id}`)}`,
330330
// Best-effort attribution tag for the usage dashboard. Upstream Grok sends

‎src/server/index.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1201,6 +1201,7 @@ export function startServer(port?: number, deps: StartServerDeps = {}): Server<W
12011201
...admissionFields(admission),
12021202
inboundProtocol: "responses",
12031203
};
1204+
if (req.headers.get("x-opencodex-grok") === "1") logCtx.surface = "grok";
12041205
let logged = false;
12051206
const finalizeNativePassthroughLog = (
12061207
status: number,

‎src/server/responses/core.ts‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
import type { Server } from "bun";
22
import { bridgeToResponsesSSE, buildResponseJSON, formatErrorResponse, type ResponsesTerminalStatus } from "../../bridge";
33
import { formatPassthroughUpstreamError } from "./passthrough-error";
4+
import {
5+
createResponsesFieldBackfillBlockRewrite,
6+
backfillResponsesFieldsJson,
7+
} from "./responses-field-backfill";
48
import { checkInputAdmission } from "./input-admission";
59
import { nativeContextLimits } from "../../codex/catalog";
610
import { describeUpstreamConnectFailure } from "./upstream-error";
@@ -2799,6 +2803,7 @@ async function handleResponsesInner(
27992803
snapshotRepairEnabled
28002804
? createResponsesSnapshotBlockRewrite(outboundRequestBody, translatorBudget)
28012805
: undefined,
2806+
createResponsesFieldBackfillBlockRewrite(),
28022807
// Last: every rewrite above can still rename or reshape a call item, so the guard must
28032808
// compare the names the client will actually receive against the declared catalog.
28042809
undeclaredToolGuardActive
@@ -2993,8 +2998,8 @@ async function handleResponsesInner(
29932998
? repairResponsesSnapshotJson(restored, outboundRequestBody)
29942999
: restored;
29953000
return parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId
2996-
? rewriteResponsesModelJson(repaired, parsed._responseModelId)
2997-
: repaired;
3001+
? rewriteResponsesModelJson(backfillResponsesFieldsJson(repaired), parsed._responseModelId)
3002+
: backfillResponsesFieldsJson(repaired);
29983003
})();
29993004
// #1700: same fail-closed policy as the SSE relay above. Both the plain JSON answer and
30003005
// the reframed-SSE branch below are built from this body, so one check covers them. This
Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
/**
2+
* Stateless SSE block rewrite that backfills the `annotations` field on
3+
* `output_text` content parts in Responses SSE events.
4+
*
5+
* The Responses API spec defines `annotations` as a required field on
6+
* `OutputTextContent` (it is `Vec<Annotation>`, not `Option<Vec>`). Some
7+
* upstream relays omit it when there are no annotations, which is technically
8+
* spec-non-compliant. Strict deserializers — any client that follows the
9+
* schema without `#[serde(default)]` on that field — fail with
10+
* `missing field `annotations`` when the field is absent.
11+
*
12+
* This rewrite scans every SSE event for output_text content parts — whether
13+
* they appear in item.content[], part, or response.output[].content[] — and
14+
* adds annotations: [] if missing.
15+
*
16+
* Stateless: no retained buffers, no lifecycle tracking, no fail-closed.
17+
* Existing values are always authoritative; only absent fields are added.
18+
* The field is always valid on the wire, so adding it when absent is safe
19+
* for all clients including Codex CLI/App.
20+
*/
21+
22+
import {
23+
replaceSseDataPayload,
24+
sseDataPayload,
25+
type SseBlockRewrite,
26+
} from "../sse-payload-rewrite";
27+
28+
function isPlainObject(value: unknown): value is Record<string, unknown> {
29+
return !!value && typeof value === "object" && !Array.isArray(value);
30+
}
31+
32+
/**
33+
* Backfill annotations: [] on an output_text content part if missing.
34+
* Returns the same object reference if no change is needed.
35+
*/
36+
function backfillOutputTextPart(part: Record<string, unknown>): Record<string, unknown> {
37+
if (part.type !== "output_text") return part;
38+
// Only add annotations when the key is entirely absent — preserve any
39+
// existing value (even null or a malformed type) so we never overwrite
40+
// what the upstream actually sent.
41+
if ("annotations" in part) return part;
42+
return { ...part, annotations: [] };
43+
}
44+
45+
/**
46+
* Walk a content array and backfill output_text parts.
47+
* Returns the same array reference if nothing changed.
48+
*/
49+
function backfillContentArray(content: unknown): unknown {
50+
if (!Array.isArray(content)) return content;
51+
let changed = false;
52+
const repaired = content.map((part) => {
53+
if (!isPlainObject(part)) return part;
54+
const next = backfillOutputTextPart(part);
55+
if (next !== part) changed = true;
56+
return next;
57+
});
58+
return changed ? repaired : content;
59+
}
60+
61+
/**
62+
* Walk an output item and backfill output_text parts in its content.
63+
* Returns the same object reference if nothing changed.
64+
*/
65+
function backfillOutputItem(item: unknown): unknown {
66+
if (!isPlainObject(item)) return item;
67+
const content = item.content;
68+
const repaired = backfillContentArray(content);
69+
if (repaired === content) return item;
70+
return { ...item, content: repaired };
71+
}
72+
73+
/**
74+
* Walk a response object's output[] and backfill output_text parts.
75+
* Returns the same object reference if nothing changed.
76+
*/
77+
function backfillResponseOutput(response: unknown): unknown {
78+
if (!isPlainObject(response)) return response;
79+
const output = response.output;
80+
if (!Array.isArray(output)) return response;
81+
let changed = false;
82+
const repaired = output.map((item) => {
83+
if (!isPlainObject(item)) return item;
84+
const next = backfillOutputItem(item);
85+
if (next !== item) changed = true;
86+
return next;
87+
});
88+
return changed ? { ...response, output: repaired } : response;
89+
}
90+
91+
/**
92+
* Statelessly rewrite one SSE event: backfill annotations
93+
* on any output_text content part found in the event payload.
94+
*/
95+
function rewriteEvent(event: Record<string, unknown>): Record<string, unknown> {
96+
const type = typeof event.type === "string" ? event.type : "";
97+
let next = event;
98+
let changed = false;
99+
100+
// output_item.added / output_item.done: item.content[] -> output_text parts
101+
if ((type === "response.output_item.added" || type === "response.output_item.done")
102+
&& isPlainObject(event.item)) {
103+
const item = backfillOutputItem(event.item);
104+
if (item !== event.item) {
105+
next = { ...next, item };
106+
changed = true;
107+
}
108+
}
109+
110+
// content_part.added / content_part.done: part -> output_text
111+
if ((type === "response.content_part.added" || type === "response.content_part.done")
112+
&& isPlainObject(event.part)) {
113+
const part = backfillOutputTextPart(event.part);
114+
if (part !== event.part) {
115+
next = { ...next, part };
116+
changed = true;
117+
}
118+
}
119+
120+
// response.created / in_progress / completed / incomplete / failed:
121+
// response.output[].content[] -> output_text parts
122+
if (isPlainObject(event.response)) {
123+
const response = backfillResponseOutput(event.response);
124+
if (response !== event.response) {
125+
next = { ...next, response };
126+
changed = true;
127+
}
128+
}
129+
130+
return changed ? next : event;
131+
}
132+
133+
/**
134+
* Create a stateless SSE block rewrite that backfills annotations and
135+
* on output_text content parts. Unconditional: the field is a required
136+
* canonical Responses field, so adding it when absent is safe for all
137+
* clients.
138+
*/
139+
export function createResponsesFieldBackfillBlockRewrite(): SseBlockRewrite {
140+
const rewrite: SseBlockRewrite = (block: string): readonly string[] => {
141+
const payload = sseDataPayload(block);
142+
if (payload === null) return [block];
143+
let event: unknown;
144+
try {
145+
event = JSON.parse(payload);
146+
} catch {
147+
return [block];
148+
}
149+
if (!isPlainObject(event)) return [block];
150+
const rewritten = rewriteEvent(event);
151+
if (rewritten === event) return [block];
152+
return [replaceSseDataPayload(block, JSON.stringify(rewritten))];
153+
};
154+
return rewrite;
155+
}
156+
157+
/**
158+
* Backfill annotations on a non-streaming Responses JSON
159+
* object. Mirrors the SSE block rewrite for the bounded-JSON passthrough
160+
* path. Returns the original string if no change is needed.
161+
*/
162+
export function backfillResponsesFieldsJson(payload: string): string {
163+
let response: unknown;
164+
try {
165+
response = JSON.parse(payload);
166+
} catch {
167+
return payload;
168+
}
169+
if (!isPlainObject(response)) return payload;
170+
const repaired = backfillResponseOutput(response);
171+
if (repaired === response) return payload;
172+
return JSON.stringify(repaired);
173+
}

‎tests/bridge.test.ts‎

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -729,7 +729,7 @@ describe("Responses bridge reasoning and usage parity", () => {
729729
// Regression for the Cursor parallel-tool-call stall: while the upstream silently assembles tool
730730
// calls, the adapter emits `heartbeat` events. They must keep the stall watchdog alive (no
731731
// upstream_stall_timeout). Adapter heartbeats themselves are not translated into Responses
732-
// protocol items; wire keepalives use a separate `response.heartbeat` frame (see next test).
732+
// protocol items; wire keepalives use a separate SSE comment line (see next test).
733733
//
734734
// resolveStallTimeoutSec ceils to a minimum of 1s, so sub-second stallTimeoutSec values cannot
735735
// prove the reset. Drive the beat loop through a test clock seam and run adapter-only progress
@@ -803,10 +803,11 @@ describe("Responses bridge reasoning and usage parity", () => {
803803
expect(frames.some(f => f.data.type === "heartbeat")).toBe(false);
804804
});
805805

806-
test("wire response.heartbeat keeps firing while only adapter heartbeats flow", async () => {
806+
test("wire keepalive comment keeps firing while only adapter heartbeats flow", async () => {
807807
// Issue #521: web-search buffers semantic events and yields invisible adapter heartbeats from
808808
// raw-byte progress. Those must not suppress wire keepalives, or Codex Desktop idle-timeouts
809-
// (~5 min) while OCX still considers the upstream alive.
809+
// (~5 min) while OCX still considers the upstream alive. The wire keepalive is an SSE comment
810+
// line (": opencodex heartbeat") so it never triggers deserialization on any client.
810811
const heartbeatMs = 50;
811812
const stallTimeoutSec = 1;
812813
const cycles = 4;
@@ -842,7 +843,7 @@ describe("Responses bridge reasoning and usage parity", () => {
842843
yield { type: "done" };
843844
}
844845

845-
const framesPromise = collectSse(bridgeToResponsesSSE(
846+
const stream = bridgeToResponsesSSE(
846847
adapterHeartbeatsOnly(),
847848
"model",
848849
undefined,
@@ -851,7 +852,8 @@ describe("Responses bridge reasoning and usage parity", () => {
851852
undefined,
852853
heartbeatMs,
853854
{ stallTimeoutSec, timers },
854-
));
855+
);
856+
const rawTextPromise = new Response(stream).text();
855857

856858
await flush();
857859
for (let i = 0; i < cycles; i++) {
@@ -860,12 +862,24 @@ describe("Responses bridge reasoning and usage parity", () => {
860862
releaseDelay();
861863
await flush();
862864
}
865+
const rawText = await rawTextPromise;
866+
const frames: { event?: string; data: Record<string, unknown> }[] = [];
867+
for (const frame of rawText.split("\n\n")) {
868+
const trimmed = frame.trim();
869+
if (!trimmed || trimmed === "data: [DONE]") continue;
870+
const lines = trimmed.split("\n");
871+
const event = lines.find(l => l.startsWith("event: "))?.slice(7);
872+
const dataLine = lines.find(l => l.startsWith("data: "));
873+
// Skip comment-only frames (e.g. ": opencodex heartbeat"); they have no data
874+
// line and must not become fake deserializable events.
875+
if (!dataLine) continue;
876+
frames.push({ event, data: JSON.parse(dataLine?.slice(6) ?? "{}") as Record<string, unknown> });
877+
}
863878

864-
const frames = await framesPromise;
865-
const wireHeartbeats = frames.filter(f =>
866-
f.event === "response.heartbeat" && f.data.type === "response.heartbeat"
867-
);
868-
expect(wireHeartbeats.length).toBeGreaterThan(1);
879+
// Wire keepalives are SSE comment lines (": opencodex heartbeat") — they keep the
880+
// idle timer alive without producing a typed event any client must deserialize.
881+
const keepaliveCount = (rawText.match(/^: opencodex heartbeat$/gm) ?? []).length;
882+
expect(keepaliveCount).toBeGreaterThan(1);
869883
expect(frames.some(f => f.event === "response.completed")).toBe(true);
870884
expect(frames.some(f => (f.data.response as Record<string, unknown> | undefined)?.incomplete_details)).toBe(false);
871885
// Reject every adapter-shaped heartbeat payload, regardless of event name or field count.

‎tests/chat-completions-endpoint.test.ts‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -349,10 +349,9 @@ test("chatCompletionsUsage always emits detail objects with zero defaults", () =
349349
});
350350

351351
test("responsesSseToChatCompletionsSse consumes response.heartbeat without forwarding a raw frame", async () => {
352-
// grok-build's strict Responses decoder dies on unknown variants (response.heartbeat),
353-
// which is why the injected Grok config pins api_backend = "chat_completions". This
354-
// regression pins the safety property: heartbeats never surface as raw frames here —
355-
// at most a valid role chunk is emitted.
352+
// Upstream responses SSE may contain heartbeat events (SSE comment keep-alive in
353+
// bridge.ts, but some upstreams emit them as typed frames). The chat-completions
354+
// converter must drop them rather than forwarding raw Responses-vocab frames.
356355
const { responsesSseToChatCompletionsSse } = budgetedChatOutbound(await import("../src/chat/outbound"));
357356
const upstream = new Response([
358357
`event: response.heartbeat\ndata: ${JSON.stringify({ type: "response.heartbeat" })}\n\n`,

‎tests/grok-config-inject.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ describe("Grok config injection", () => {
7070
const table = block.slice(block.indexOf("[model.ocx-cursor-grok-4-5]"));
7171
expect(table).toContain('model = "cursor/grok-4.5"');
7272
expect(table).toContain('base_url = "http://127.0.0.1:10190/v1"');
73-
expect(table).toContain('api_backend = "chat_completions"');
73+
expect(table).toContain('api_backend = "responses"');
7474
expect(table).toContain('api_key = "opencodex-loopback"');
7575
expect(table).toContain("context_window = 500000");
7676
});

‎tests/grok-orphan-adoption.test.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ describe("Grok orphan adoption (#511)", () => {
4545
"[model.ocx-gpt-5-6-sol]",
4646
'model = "gpt-5.6-sol"',
4747
'base_url = "http://127.0.0.1:10100/v1"',
48-
'api_backend = "chat_completions"',
48+
'api_backend = "responses"',
4949
'api_key = "opencodex-loopback"',
5050
'name = "OCX gpt-5.6-sol"',
5151
"",
@@ -140,7 +140,7 @@ describe("Grok orphan adoption (#511)", () => {
140140
expect(content).toContain("[ui]");
141141
expect(content).toContain('theme = "dark"');
142142
// No key from the removed table leaked into [ui].
143-
expect(content).not.toContain('api_backend = "chat_completions"\ntheme');
143+
expect(content).not.toContain('api_backend = "responses"\ntheme');
144144
});
145145

146146
// F5: an orphan with no replacement stays, and its reference is not rewritten to
@@ -272,7 +272,7 @@ describe("Grok orphan adoption (#511)", () => {
272272
"[model.ocx-gpt-5-6-sol]", // stale generation: dead port
273273
'model = "gpt-5.6-sol"',
274274
'base_url = "http://127.0.0.1:4179/v1"',
275-
'api_backend = "chat_completions"',
275+
'api_backend = "responses"',
276276
'api_key = "opencodex-loopback"',
277277
"context_window = 372000",
278278
"",

0 commit comments

Comments
 (0)