Skip to content

Commit d6723f7

Browse files
fix(responses): recover routed continuations after replay misses
Carry #4086 from 0c39bc6 onto current dev. Stateless routed destinations and unmappable lowered custom results return previous_response_not_found before dispatch so the client can resend full history. Local product tests NOT RUN per explicit user instruction; hosted final-tip CI follows. Co-authored-by: Eleven-is-cool <[email protected]>
1 parent b09ef15 commit d6723f7

10 files changed

Lines changed: 247 additions & 7 deletions

File tree

‎docs-site/src/content/docs/guides/codex-integration.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -288,6 +288,15 @@ The cache remains bounded; this does not extend retention or recover history the
288288
longer has. HTTP clients must handle the error explicitly and resend their full context without
289289
`previous_response_id`. Retrying only the same ID cannot recover missing state.
290290

291+
The same recovery signal applies to routed Responses providers configured with
292+
`statelessResponses: true`, and to routed requests where a custom tool was lowered to a function
293+
but a delta result has no local call to establish its original type. Full replay preserves the
294+
call, result, and reasoning together; opencodex does not guess the result type or drop it.
295+
Stateful providers still resolve native function and native-only custom continuations themselves.
296+
These checks follow the selected wire protocol and tool declarations, not the model name. For a
297+
gateway that cannot resolve stored response IDs, enable `statelessResponses` on that provider;
298+
other providers keep their defaults.
299+
291300
### Client-side compaction (opt-in)
292301

293302
Authenticated loopback routing normally keeps Codex on its built-in `openai` provider identity.

‎docs-site/src/content/docs/ko/guides/codex-integration.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,14 @@ upstream 요청 전에 `previous_response_not_found`를 반환합니다. Codex W
134134
`previous_response_id` 없이 전체 컨텍스트를 다시 보내야 합니다. 같은 ID만 재시도해서는
135135
누락된 상태를 복구할 수 없습니다.
136136

137+
`statelessResponses: true`로 설정한 routed Responses provider에도 같은 복구 신호가 적용됩니다.
138+
routed 경로에서 custom 도구를 function으로 낮췄는데 증분 결과에 대응하는 로컬 호출 기록이
139+
없을 때도 전체 기록을 다시 요청합니다. 호출과 결과, reasoning을 함께 재생하며 결과 유형을
140+
추측하거나 버리지 않습니다. 상태를 저장하는 provider의 네이티브 function 및 네이티브 custom
141+
전용 continuation은 그대로 전달됩니다. 이 검사는 모델명이 아니라 선택된 wire protocol과 도구
142+
선언을 따릅니다. 저장된 response ID를 복원하지 못하는 gateway에는 해당 provider의
143+
`statelessResponses`를 명시적으로 켜세요. 다른 provider의 기본값은 바뀌지 않습니다.
144+
137145
## 스레드 식별자와 대화 기록
138146

139147
기본 loopback 형식은 새 thread에 네이티브 `openai` provider 태그를 유지하므로 일반적인 resume history는 다시 매핑할 필요가 없습니다. sync와 restore는 일치하는 백업 manifest만 적용하여 각 thread의 원래 provider, source, event marker를 정확히 복원합니다. manifest가 없는 `opencodex` row는 변경하지 않으며, legacy 재태깅을 명시적으로 강제하려는 경우에만 `ocx recover-history --legacy-openai --yes`를 사용합니다. 이 명령은 의도적으로 범위가 넓습니다. 사용자 메시지가 있고 현재 `opencodex`로 표시된 모든 thread를 `openai`로 바꾸고, `exec`를 `cli`로 정규화하며 event marker를 설정합니다. 정상적인 dedicated-provider history도 포함됩니다. 상태를 백업하고 이 전체 범위를 의도한 경우에만 사용하세요. non-loopback 전용 provider 모드는 활성 상태일 때만 history를 `opencodex` provider 아래로 미러링하고, 종료할 때는 백업된 메타데이터를 복원합니다. history를 건드리지 않으려면 `syncResumeHistory: false`로 설정하세요.

‎docs-site/src/content/docs/reference/configuration/providers.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -958,7 +958,8 @@ representation returned to the client, including the visible content-to-summary
958958
echoing full history with `previous_response_id` does not duplicate that history. Hidden-summary
959959
requests and opaque reasoning blobs retain their existing representation. Cache hits can also
960960
supply earlier history for delta continuations;
961-
after a cache miss, resend the complete conversation without `previous_response_id`. Stateless
961+
after a cache miss, the proxy returns `previous_response_not_found` before upstream dispatch so the
962+
client can resend the complete conversation without `previous_response_id`. Stateless
962963
repair labels orphan results and missing tool results; it cannot reconstruct lost history or
963964
prove whether a missing tool execution succeeded.
964965

‎src/responses/custom-tool-compat.ts‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -261,6 +261,29 @@ export function rewriteRoutedCustomToolsForUpstream(
261261
return { body: rewriteForUpstream(body, conversionNames, callIds), names, repairNames };
262262
}
263263

264+
/**
265+
* A delta result has no tool name. Without its call, lowering cannot tell whether it belongs
266+
* to a converted function or a native custom tool. Request full replay instead of guessing.
267+
*/
268+
export function hasUnmappedRoutedCustomToolOutput(
269+
body: unknown,
270+
supportsResponsesCustomTools?: boolean,
271+
): boolean {
272+
if (!isPlainObject(body) || !Array.isArray(body.input)) return false;
273+
if (collectRoutedCustomToolNames(body, supportsResponsesCustomTools).size === 0) return false;
274+
const callIds = new Set<string>();
275+
for (const item of body.input) {
276+
if (isPlainObject(item)
277+
&& (item.type === "custom_tool_call" || item.type === "function_call")
278+
&& typeof item.call_id === "string") callIds.add(item.call_id);
279+
}
280+
return body.input.some(item => isPlainObject(item)
281+
&& item.type === "custom_tool_call_output"
282+
&& typeof item.call_id === "string"
283+
&& item.call_id.length > 0
284+
&& !callIds.has(item.call_id));
285+
}
286+
264287
export function restoreRoutedCustomCalls(
265288
value: unknown,
266289
names: ReadonlySet<string>,

‎src/server/responses/core.ts‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -413,7 +413,7 @@ import {
413413
payloadRewriteAsBlockRewrite,
414414
relaySseWithBlockRewrite,
415415
} from "../sse-payload-rewrite";
416-
import { restoreRoutedCustomCalls, restoreRoutedCustomCallsInJson } from "../../responses/custom-tool-compat";
416+
import { hasUnmappedRoutedCustomToolOutput, restoreRoutedCustomCalls, restoreRoutedCustomCallsInJson } from "../../responses/custom-tool-compat";
417417
import { createRoutedCustomToolRestoreBlockRewrite } from "../responses-custom-tool-repair";
418418
import { collectFunctionCallRepairSchemas, repairFunctionCallsInJson } from "../../responses/function-call-compat";
419419
import { createResponsesFunctionToolRepairBlockRewrite } from "../responses-function-tool-repair";
@@ -3902,6 +3902,22 @@ async function handleResponsesInner(
39023902
);
39033903
}
39043904

3905+
if (hasUnexpandedPreviousResponse) {
3906+
const continuationProvider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire);
3907+
// Stateless destinations cannot resolve the omitted prefix. Stateful destinations may,
3908+
// but a lowered custom result still needs its call to recover the original wire type.
3909+
// Native function/custom continuations without lowering keep their upstream-owned state.
3910+
if (continuationProvider.adapter === "openai-responses"
3911+
&& (continuationProvider.statelessResponses === true
3912+
|| hasUnmappedRoutedCustomToolOutput(parsed._rawBody, continuationProvider.supportsResponsesCustomTools))) {
3913+
return formatErrorResponse(
3914+
400,
3915+
"previous_response_not_found",
3916+
"Routed continuation requires unavailable local history; resend the full conversation without previous_response_id.",
3917+
);
3918+
}
3919+
}
3920+
39053921
// Captured before normalization: whether the CLIENT asked for SSE. The
39063922
// transport-neutral upstream-streaming policy below may force a bounded JSON
39073923
// upstream for reliability (#875); the answer must then be reframed to SSE

‎src/types/provider.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -292,8 +292,8 @@ export interface OcxProviderConfig {
292292
/**
293293
* Responses upstream that stores nothing server-side (DeepSeek documents "the API
294294
* is stateless"). Stateful request parameters are dropped, `store` is pinned false,
295-
* and orphaned tool results left by a replay miss are repaired rather than
296-
* forwarded to an upstream that cannot resolve their pair.
295+
* and missing local continuation history returns previous_response_not_found so
296+
* clients can resend full input. Explicit input still receives orphan-item repair.
297297
*/
298298
statelessResponses?: boolean;
299299
/**

‎structure/runtime.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,8 @@ an unowned client remains disconnected. OpenCodex omits Raycast API-key fields
182182
and exports only to eligible local targets. Pro detection is an advisory hint,
183183
not an authentication or entitlement decision.
184184

185+
Routed Responses continuations whose local replay state is missing resolve their recovery decision from the selected wire protocol, not the model name; the contract lives in [Responses transport](transports/responses.md).
186+
185187
## Remote Hub hardening ownership
186188

187189
`src/remote/protocol.ts` owns pure interval/feature negotiation. `src/remote/hub-state.ts` owns the `GET|HEAD /v1/hub-state` contract, its caps, and the parser both sides share. `src/client/hub-client.ts` owns bounded, schema-validated remote catalog consumption, hub-state reads, and key-id probes; `src/client/hub-state.ts` owns the resolution and the owner-stamped 0600 cache, and a failed read reports "unavailable" rather than degrading to the client's own local provider and login state. `src/client/hub-relay.ts` is a fixed-authority management relay with URL, header, body, redirect, and stream bounds. The public data listener remains the direct client→hub path; the loopback management ingress never serves data-plane routes.

‎structure/transports/responses.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -303,7 +303,13 @@ logic fills absent values and preserves explicit false; renamed custom configura
303303
no new destination-based migration. The existing stateless pass sets `store: false`, removes
304304
stored continuation parameters, and repairs orphan calls/results without claiming execution
305305
success. A local replay-cache hit supplies history; a miss cannot reconstruct it, so callers
306-
must resend complete history without `previous_response_id`. This flag also enables the existing
306+
receive `previous_response_not_found` before upstream dispatch and must resend complete history
307+
without `previous_response_id`. Routed custom-tool lowering requires the same recovery when a delta
308+
custom result has no local call, because its original wire type cannot be established and guessing it
309+
would send an unmatched result upstream. The check resolves the selected wire protocol and the
310+
request's own tool declarations after final route selection, so stateful destinations keep their
311+
upstream-owned native function and native-only custom continuations. Explicit input still receives
312+
orphan repair; this path asks the client to replay rather than reconstructing history. This flag also enables the existing
307313
visible content-to-summary rewrite for SSE and JSON; summary-channel items and opaque reasoning
308314
blobs keep their existing response handling. The shared recording callback applies the same
309315
reasoning rewrite under the exact client-visible predicate before caching output, after tool

‎tests/codex-integration/issue-702-expired-replay-state.test.ts‎

Lines changed: 131 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -286,6 +286,126 @@ afterEach(() => {
286286
else process.env.OPENCODEX_API_AUTH_TOKEN = previousApiToken;
287287
});
288288

289+
describe("routed replay recovery", () => {
290+
test.each([
291+
{ stateless: true, custom: false, expired: false },
292+
{ stateless: true, custom: false, expired: true },
293+
{ stateless: false, custom: true, expired: false },
294+
{ stateless: false, custom: true, expired: true },
295+
{ stateless: false, custom: true, expired: false, authMode: "forward" },
296+
{ stateless: true, custom: false, expired: false, adapter: "openai-chat" },
297+
] as const)("recovers missing history without orphaning tool results: %j", async scenario => {
298+
const { stateless, custom, expired } = scenario;
299+
const upstreamRequests: Record<string, unknown>[] = [];
300+
const realNow = Date.now;
301+
const toolCall = custom
302+
? { type: "custom_tool_call", call_id: "call_replay", name: "exec", input: "text(1)", status: "completed" }
303+
: { type: "function_call", call_id: "call_replay", name: "lookup", arguments: "{}", status: "completed" };
304+
const toolResult = {
305+
type: custom ? "custom_tool_call_output" : "function_call_output",
306+
call_id: "call_replay", output: "1",
307+
};
308+
const reasoning = { type: "reasoning", content: [{ type: "reasoning_text", text: "Use the lookup result." }] };
309+
const history = [inputMessage(HISTORICAL_USER_SENTINEL), reasoning, toolCall];
310+
const tools = custom
311+
? [{ type: "custom", name: "exec", description: "Run JavaScript", format: { type: "text" } }]
312+
: [{ type: "function", name: "lookup", parameters: { type: "object", properties: {} } }];
313+
const upstream = Bun.serve({
314+
port: 0,
315+
async fetch(request) {
316+
const body = await request.json() as Record<string, unknown>;
317+
upstreamRequests.push(body);
318+
const items = body.input as Array<Record<string, unknown>>;
319+
// A strict upstream cannot resolve a tool result without its matching function call.
320+
if (!items.some(item => item.type === "function_call" && item.call_id === "call_replay")
321+
|| !items.some(item => item.type === "function_call_output" && item.call_id === "call_replay")) {
322+
return Response.json({ error: { type: "invalid_request_error", message: "No matching tool call" } }, { status: 400 });
323+
}
324+
return new Response(completedSse("resp_routed_recovered", "recovered"), {
325+
headers: { "content-type": "text/event-stream" },
326+
});
327+
},
328+
});
329+
let server: ReturnType<typeof startServer> | null = null;
330+
let socket: WebSocket | null = null;
331+
try {
332+
if (expired) {
333+
Date.now = () => realNow() - EXPIRED_AGE_MS;
334+
rememberResponseState(
335+
{ input: [history[0]], tools, store: false },
336+
{ id: FIRST_RESPONSE_ID, status: "completed", output: [reasoning, toolCall] },
337+
undefined, { force: true },
338+
);
339+
Date.now = realNow;
340+
}
341+
saveConfig({
342+
port: 0, hostname: "127.0.0.1", websockets: true, defaultProvider: "routed-test",
343+
providers: {
344+
"routed-test": {
345+
adapter: "adapter" in scenario ? scenario.adapter : "openai-responses",
346+
modelAdapters: { "test-model": "openai-responses" },
347+
baseUrl: upstream.url.toString(), allowPrivateNetwork: true,
348+
authMode: "authMode" in scenario ? scenario.authMode : "key", apiKey: "synthetic-key", defaultModel: "test-model",
349+
statelessResponses: stateless, preserveResponsesReasoningContent: true,
350+
},
351+
},
352+
} as OcxConfig);
353+
server = startServer(0);
354+
const deltaRequest = {
355+
model: "routed-test/test-model", previous_response_id: FIRST_RESPONSE_ID,
356+
input: [toolResult], tools, store: false,
357+
};
358+
const httpRejected = await originalFetch(new URL("/v1/responses", server.url), {
359+
method: "POST", headers: { "content-type": "application/json" },
360+
body: JSON.stringify(deltaRequest),
361+
});
362+
expect(httpRejected.status).toBe(400);
363+
expect(await httpRejected.json()).toMatchObject({ error: { code: "previous_response_not_found" } });
364+
expect(upstreamRequests).toHaveLength(0);
365+
socket = await openResponseSocket(server.url, {});
366+
const rejected = await sendSocketTurn(socket, deltaRequest);
367+
expect(rejected).toMatchObject({
368+
type: "error", status: 400,
369+
error: { type: "invalid_request_error", code: "previous_response_not_found" },
370+
});
371+
expect(upstreamRequests).toHaveLength(0);
372+
373+
socket.close();
374+
socket = await openResponseSocket(server.url, {});
375+
const recovered = await sendSocketTurn(socket, {
376+
model: "routed-test/test-model", input: [...history, toolResult], tools, store: false,
377+
});
378+
expect(recovered.type).toBe("response.completed");
379+
expect(upstreamRequests).toHaveLength(1);
380+
expect(upstreamRequests[0]!.previous_response_id).toBeUndefined();
381+
expect(upstreamRequests[0]!.input).toEqual([
382+
history[0], reasoning,
383+
{ type: "function_call", call_id: "call_replay", name: custom ? "exec" : "lookup",
384+
arguments: custom ? JSON.stringify({ input: "text(1)" }) : "{}", status: "completed" },
385+
{ ...toolResult, type: "function_call_output" },
386+
]);
387+
await waitForRecordedResponseState();
388+
389+
// Recovery must seed complete local history so the next delta does not repeat the miss.
390+
const continued = await sendSocketTurn(socket, {
391+
model: "routed-test/test-model", previous_response_id: "resp_routed_recovered",
392+
input: [inputMessage(CURRENT_USER_SENTINEL)], tools, store: false,
393+
});
394+
expect(continued.type).toBe("response.completed");
395+
expect(upstreamRequests).toHaveLength(2);
396+
expect(upstreamRequests[1]!.previous_response_id).toBeUndefined();
397+
const items = upstreamRequests[1]!.input as Array<Record<string, unknown>>;
398+
expect(items.filter(item => item.call_id === "call_replay")).toHaveLength(2);
399+
expect(items.at(-1)).toEqual(inputMessage(CURRENT_USER_SENTINEL));
400+
} finally {
401+
Date.now = realNow;
402+
socket?.close();
403+
await server?.stop(true);
404+
await upstream.stop(true);
405+
}
406+
}, SERVER_BUDGET_MS);
407+
});
408+
289409
describe("Issue #702 expired forward replay state", () => {
290410
test("known continuation spill failure returns terminal structured previous_response_not_found before upstream I/O", async () => {
291411
const responseId = "resp_issue_702_missing_spill";
@@ -512,7 +632,7 @@ describe("Issue #702 expired forward replay state", () => {
512632
expect(JSON.stringify(scenario.upstreamRequests[0]!.body)).toContain(HISTORICAL_USER_SENTINEL);
513633
});
514634

515-
test("API-key Responses providers can still forward native previous_response_id state", async () => {
635+
test.each(["message", "function", "custom"] as const)("API-key Responses providers can still forward native %s continuation state", async kind => {
516636
const upstreamRequests: Record<string, unknown>[] = [];
517637
const realNow = Date.now;
518638
let upstream: ReturnType<typeof Bun.serve> | null = null;
@@ -539,6 +659,7 @@ describe("Issue #702 expired forward replay state", () => {
539659
allowPrivateNetwork: true,
540660
apiKey: "provider-key",
541661
defaultModel: "gpt-5.5",
662+
statelessResponses: false,
542663
},
543664
},
544665
} as OcxConfig);
@@ -551,13 +672,22 @@ describe("Issue #702 expired forward replay state", () => {
551672
model: "test-openai/gpt-5.5",
552673
previous_response_id: "resp_upstream_native_state",
553674
input: [inputMessage(CURRENT_USER_SENTINEL)],
675+
...(kind === "message" ? {} : {
676+
tools: [kind === "custom"
677+
? { type: "custom", name: "apply_patch", description: "Apply patch", format: { type: "text" } }
678+
: { type: "function", name: "lookup", parameters: { type: "object", properties: {} } }],
679+
input: [{ type: kind === "custom" ? "custom_tool_call_output" : "function_call_output", call_id: "call_native", output: "done" }],
680+
}),
554681
stream: true,
555682
}),
556683
});
557684
expect(response.status).toBe(200);
558685
await response.text();
559686
expect(upstreamRequests).toHaveLength(1);
560687
expect(upstreamRequests[0]!.previous_response_id).toBe("resp_upstream_native_state");
688+
if (kind !== "message") expect(upstreamRequests[0]!.input).toEqual([
689+
{ type: kind === "custom" ? "custom_tool_call_output" : "function_call_output", call_id: "call_native", output: "done" },
690+
]);
561691
} finally {
562692
Date.now = realNow;
563693
globalThis.fetch = originalFetch;

0 commit comments

Comments
 (0)