Skip to content

Commit 644a412

Browse files
committed
fix(grok): switch to Responses backend and backfill required annotations
Grok CLI was pinned to api_backend = "chat_completions" because opencodex emitted response.heartbeat as a typed SSE event. That is not a valid Responses variant, so Grok-build's strict enum deserializer crashed with "unknown variant response.heartbeat". The keep-alive now emits an SSE comment line instead, which re-arms the idle timer without triggering deserialization on any client. With heartbeats fixed, Grok can finally use the Responses passthrough path. This gives Grok clients the same protocol fidelity Codex already enjoys and removes the chat to responses translation layer from the hot path. Some upstream relays (e.g. sub2api) omit annotations on output_text content parts even though the Responses spec marks it as a required Vec field. Strict clients, including Grok-build's async-openai fork, fail with "missing field annotations". A new stateless SSE/JSON backfill adds annotations: [] on any output_text part that lacks it, on both the streaming and bounded-JSON passthrough paths. The rewrite is unconditional and safe for all clients because the field is always valid on the wire. The /v1/responses handler now surfaces grok-tagged requests as surface=grok in the log context, matching the chat-completions handler. Stale comments referencing grok-build's decoder and the old chat_completions pin have been corrected in the tests.
1 parent 4ef1fca commit 644a412

9 files changed

Lines changed: 342 additions & 24 deletions

‎src/bridge.ts‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -325,10 +325,12 @@ export function bridgeToResponsesSSE(
325325
clearOwnedWatchdog();
326326
};
327327
// RC3 keep-alive: Codex's idle timer is timeout(idle_timeout, stream.next()) over an
328-
// eventsource_stream; ANY received event re-arms it, while an unknown type is ignored
329-
// (responses.rs `_ => Ok(None)`). Emit a parser-ignored `response.heartbeat` whenever the
328+
// eventsource_stream; ANY received bytes re-arm it. An SSE comment line (a line starting
329+
// with `:`) is discarded by every eventsource parser without producing an event, so it
330+
// keeps the wire alive without triggering deserialization. Emit a comment line whenever the
330331
// *wire* has been silent, even if invisible adapter heartbeats are still flowing (web-search
331-
// buffering + raw-byte progress). Upstream activity only resets the stall watchdog.
332+
// buffering + raw-byte progress). Upstream activity only resets the stall watchdog. Parity
333+
// with the passthrough relay's `: opencodex keepalive` (relay.ts).
332334
let upstreamActivity = false;
333335
let wireActivity = false;
334336
let beat: unknown;
@@ -395,7 +397,7 @@ export function bridgeToResponsesSSE(
395397
...(endTurn !== undefined ? { end_turn: endTurn } : {}),
396398
});
397399

398-
const heartbeatFrame = encoder.encode('event: response.heartbeat\ndata: {"type":"response.heartbeat"}\n\n');
400+
const heartbeatFrame = encoder.encode(': opencodex heartbeat\n\n');
399401
let stallTicks = 0;
400402
const stallSec = resolveStallTimeoutSec(options?.stallTimeoutSec);
401403
const maxStallTicks = Math.ceil((stallSec * 1000) / heartbeatMs);

‎src/grok/inject.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -324,7 +324,7 @@ export function buildGrokManagedBlock(
324324
`[model.${alias}]`,
325325
`model = ${tomlString(model.id)}`,
326326
`base_url = ${tomlString(baseUrl)}`,
327-
'api_backend = "chat_completions"',
327+
'api_backend = "responses"',
328328
'api_key = "opencodex-loopback"',
329329
`name = ${tomlString(model.name ?? `OCX ${model.id}`)}`,
330330
// Best-effort attribution tag for the usage dashboard. Upstream Grok sends

‎src/server/index.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1201,6 +1201,7 @@ export function startServer(port?: number, deps: StartServerDeps = {}): Server<W
12011201
...admissionFields(admission),
12021202
inboundProtocol: "responses",
12031203
};
1204+
if (req.headers.get("x-opencodex-grok") === "1") logCtx.surface = "grok";
12041205
let logged = false;
12051206
const finalizeNativePassthroughLog = (
12061207
status: number,

‎src/server/responses/core.ts‎

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
import type { Server } from "bun";
22
import { bridgeToResponsesSSE, buildResponseJSON, formatErrorResponse, type ResponsesTerminalStatus } from "../../bridge";
33
import { formatPassthroughUpstreamError } from "./passthrough-error";
4+
import {
5+
createResponsesFieldBackfillBlockRewrite,
6+
backfillResponsesFieldsJson,
7+
} from "./responses-field-backfill";
48
import { checkInputAdmission } from "./input-admission";
59
import { nativeContextLimits } from "../../codex/catalog";
610
import { describeUpstreamConnectFailure } from "./upstream-error";
@@ -2794,10 +2798,11 @@ async function handleResponsesInner(
27942798
githubCopilotRepairEnabled
27952799
? createGithubCopilotResponsesBlockRewrite(translatorBudget)
27962800
: undefined,
2797-
snapshotRepairEnabled
2798-
? createResponsesSnapshotBlockRewrite(outboundRequestBody, translatorBudget)
2799-
: undefined,
2800-
// Last: every rewrite above can still rename or reshape a call item, so the guard must
2801+
snapshotRepairEnabled
2802+
? createResponsesSnapshotBlockRewrite(outboundRequestBody, translatorBudget)
2803+
: undefined,
2804+
createResponsesFieldBackfillBlockRewrite(),
2805+
// Last: every rewrite above can still rename or reshape a call item, so the guard must
28012806
// compare the names the client will actually receive against the declared catalog.
28022807
undeclaredToolGuardActive
28032808
? createUndeclaredToolCallGuardBlockRewrite(declaredWireToolNames)
@@ -2987,13 +2992,13 @@ async function handleResponsesInner(
29872992
restoreImageGenCallsInJson(text, imageGenCallAliases),
29882993
routedCustomToolNames,
29892994
);
2990-
const repaired = hasResponsesSnapshotRepair(route.provider.responsesSnapshotRepair)
2991-
? repairResponsesSnapshotJson(restored, outboundRequestBody)
2992-
: restored;
2993-
return parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId
2994-
? rewriteResponsesModelJson(repaired, parsed._responseModelId)
2995-
: repaired;
2996-
})();
2995+
const repaired = hasResponsesSnapshotRepair(route.provider.responsesSnapshotRepair)
2996+
? repairResponsesSnapshotJson(restored, outboundRequestBody)
2997+
: restored;
2998+
return parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId
2999+
? rewriteResponsesModelJson(backfillResponsesFieldsJson(repaired), parsed._responseModelId)
3000+
: backfillResponsesFieldsJson(repaired);
3001+
})();
29973002
// #1700: same fail-closed policy as the SSE relay above. Both the plain JSON answer and
29983003
// the reframed-SSE branch below are built from this body, so one check covers them. This
29993004
// runs BEFORE the continuation cache write below: a refused turn must not become state a
Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
/**
2+
* Stateless SSE block rewrite that backfills the `annotations` field on
3+
* `output_text` content parts in Responses SSE events.
4+
*
5+
* The Responses API spec defines `annotations` as a required field on
6+
* `OutputTextContent` (it is `Vec<Annotation>`, not `Option<Vec>`). Some
7+
* upstream relays omit it when there are no annotations, which is technically
8+
* spec-non-compliant. Strict deserializers — any client that follows the
9+
* schema without `#[serde(default)]` on that field — fail with
10+
* `missing field `annotations`` when the field is absent.
11+
*
12+
* This rewrite scans every SSE event for output_text content parts — whether
13+
* they appear in item.content[], part, or response.output[].content[] — and
14+
* adds annotations: [] if missing.
15+
*
16+
* Stateless: no retained buffers, no lifecycle tracking, no fail-closed.
17+
* Existing values are always authoritative; only absent fields are added.
18+
* The field is always valid on the wire, so adding it when absent is safe
19+
* for all clients including Codex CLI/App.
20+
*/
21+
22+
import {
23+
replaceSseDataPayload,
24+
sseDataPayload,
25+
type SseBlockRewrite,
26+
} from "../sse-payload-rewrite";
27+
28+
function isPlainObject(value: unknown): value is Record<string, unknown> {
29+
return !!value && typeof value === "object" && !Array.isArray(value);
30+
}
31+
32+
/**
33+
* Backfill annotations: [] on an output_text content part if missing.
34+
* Returns the same object reference if no change is needed.
35+
*/
36+
function backfillOutputTextPart(part: Record<string, unknown>): Record<string, unknown> {
37+
if (part.type !== "output_text") return part;
38+
if (Array.isArray(part.annotations)) return part;
39+
return { ...part, annotations: [] };
40+
}
41+
42+
/**
43+
* Walk a content array and backfill output_text parts.
44+
* Returns the same array reference if nothing changed.
45+
*/
46+
function backfillContentArray(content: unknown): unknown {
47+
if (!Array.isArray(content)) return content;
48+
let changed = false;
49+
const repaired = content.map((part) => {
50+
if (!isPlainObject(part)) return part;
51+
const next = backfillOutputTextPart(part);
52+
if (next !== part) changed = true;
53+
return next;
54+
});
55+
return changed ? repaired : content;
56+
}
57+
58+
/**
59+
* Walk an output item and backfill output_text parts in its content.
60+
* Returns the same object reference if nothing changed.
61+
*/
62+
function backfillOutputItem(item: unknown): unknown {
63+
if (!isPlainObject(item)) return item;
64+
const content = item.content;
65+
const repaired = backfillContentArray(content);
66+
if (repaired === content) return item;
67+
return { ...item, content: repaired };
68+
}
69+
70+
/**
71+
* Walk a response object's output[] and backfill output_text parts.
72+
* Returns the same object reference if nothing changed.
73+
*/
74+
function backfillResponseOutput(response: unknown): unknown {
75+
if (!isPlainObject(response)) return response;
76+
const output = response.output;
77+
if (!Array.isArray(output)) return response;
78+
let changed = false;
79+
const repaired = output.map((item) => {
80+
if (!isPlainObject(item)) return item;
81+
const next = backfillOutputItem(item);
82+
if (next !== item) changed = true;
83+
return next;
84+
});
85+
return changed ? { ...response, output: repaired } : response;
86+
}
87+
88+
/**
89+
* Statelessly rewrite one SSE event: backfill annotations
90+
* on any output_text content part found in the event payload.
91+
*/
92+
function rewriteEvent(event: Record<string, unknown>): Record<string, unknown> {
93+
const type = typeof event.type === "string" ? event.type : "";
94+
let next = event;
95+
let changed = false;
96+
97+
// output_item.added / output_item.done: item.content[] -> output_text parts
98+
if ((type === "response.output_item.added" || type === "response.output_item.done")
99+
&& isPlainObject(event.item)) {
100+
const item = backfillOutputItem(event.item);
101+
if (item !== event.item) {
102+
next = { ...next, item };
103+
changed = true;
104+
}
105+
}
106+
107+
// content_part.added / content_part.done: part -> output_text
108+
if ((type === "response.content_part.added" || type === "response.content_part.done")
109+
&& isPlainObject(event.part)) {
110+
const part = backfillOutputTextPart(event.part);
111+
if (part !== event.part) {
112+
next = { ...next, part };
113+
changed = true;
114+
}
115+
}
116+
117+
// response.created / in_progress / completed / incomplete / failed:
118+
// response.output[].content[] -> output_text parts
119+
if (isPlainObject(event.response)) {
120+
const response = backfillResponseOutput(event.response);
121+
if (response !== event.response) {
122+
next = { ...next, response };
123+
changed = true;
124+
}
125+
}
126+
127+
return changed ? next : event;
128+
}
129+
130+
/**
131+
* Create a stateless SSE block rewrite that backfills annotations and
132+
* on output_text content parts. Unconditional: the field is a required
133+
* canonical Responses field, so adding it when absent is safe for all
134+
* clients.
135+
*/
136+
export function createResponsesFieldBackfillBlockRewrite(): SseBlockRewrite {
137+
const rewrite: SseBlockRewrite = (block: string): readonly string[] => {
138+
const payload = sseDataPayload(block);
139+
if (payload === null) return [block];
140+
let event: unknown;
141+
try {
142+
event = JSON.parse(payload);
143+
} catch {
144+
return [block];
145+
}
146+
if (!isPlainObject(event)) return [block];
147+
const rewritten = rewriteEvent(event);
148+
if (rewritten === event) return [block];
149+
return [replaceSseDataPayload(block, JSON.stringify(rewritten))];
150+
};
151+
return rewrite;
152+
}
153+
154+
/**
155+
* Backfill annotations on a non-streaming Responses JSON
156+
* object. Mirrors the SSE block rewrite for the bounded-JSON passthrough
157+
* path. Returns the original string if no change is needed.
158+
*/
159+
export function backfillResponsesFieldsJson(payload: string): string {
160+
let response: unknown;
161+
try {
162+
response = JSON.parse(payload);
163+
} catch {
164+
return payload;
165+
}
166+
if (!isPlainObject(response)) return payload;
167+
const repaired = backfillResponseOutput(response);
168+
if (repaired === response) return payload;
169+
return JSON.stringify(repaired);
170+
}

‎tests/chat-completions-endpoint.test.ts‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -349,10 +349,9 @@ test("chatCompletionsUsage always emits detail objects with zero defaults", () =
349349
});
350350

351351
test("responsesSseToChatCompletionsSse consumes response.heartbeat without forwarding a raw frame", async () => {
352-
// grok-build's strict Responses decoder dies on unknown variants (response.heartbeat),
353-
// which is why the injected Grok config pins api_backend = "chat_completions". This
354-
// regression pins the safety property: heartbeats never surface as raw frames here —
355-
// at most a valid role chunk is emitted.
352+
// Upstream responses SSE may contain heartbeat events (SSE comment keep-alive in
353+
// bridge.ts, but some upstreams emit them as typed frames). The chat-completions
354+
// converter must drop them rather than forwarding raw Responses-vocab frames.
356355
const { responsesSseToChatCompletionsSse } = budgetedChatOutbound(await import("../src/chat/outbound"));
357356
const upstream = new Response([
358357
`event: response.heartbeat\ndata: ${JSON.stringify({ type: "response.heartbeat" })}\n\n`,

‎tests/grok-config-inject.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ describe("Grok config injection", () => {
7070
const table = block.slice(block.indexOf("[model.ocx-cursor-grok-4-5]"));
7171
expect(table).toContain('model = "cursor/grok-4.5"');
7272
expect(table).toContain('base_url = "http://127.0.0.1:10190/v1"');
73-
expect(table).toContain('api_backend = "chat_completions"');
73+
expect(table).toContain('api_backend = "responses"');
7474
expect(table).toContain('api_key = "opencodex-loopback"');
7575
expect(table).toContain("context_window = 500000");
7676
});

‎tests/grok-orphan-adoption.test.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ describe("Grok orphan adoption (#511)", () => {
4545
"[model.ocx-gpt-5-6-sol]",
4646
'model = "gpt-5.6-sol"',
4747
'base_url = "http://127.0.0.1:10100/v1"',
48-
'api_backend = "chat_completions"',
48+
'api_backend = "responses"',
4949
'api_key = "opencodex-loopback"',
5050
'name = "OCX gpt-5.6-sol"',
5151
"",
@@ -140,7 +140,7 @@ describe("Grok orphan adoption (#511)", () => {
140140
expect(content).toContain("[ui]");
141141
expect(content).toContain('theme = "dark"');
142142
// No key from the removed table leaked into [ui].
143-
expect(content).not.toContain('api_backend = "chat_completions"\ntheme');
143+
expect(content).not.toContain('api_backend = "responses"\ntheme');
144144
});
145145

146146
// F5: an orphan with no replacement stays, and its reference is not rewritten to
@@ -272,7 +272,7 @@ describe("Grok orphan adoption (#511)", () => {
272272
"[model.ocx-gpt-5-6-sol]", // stale generation: dead port
273273
'model = "gpt-5.6-sol"',
274274
'base_url = "http://127.0.0.1:4179/v1"',
275-
'api_backend = "chat_completions"',
275+
'api_backend = "responses"',
276276
'api_key = "opencodex-loopback"',
277277
"context_window = 372000",
278278
"",

0 commit comments

Comments
 (0)