Skip to content

Commit dea7d2c

Browse files
committed
fix(responses): keep a mid-stream refusal out of continuation state
Checking only the terminal snapshot left a hole. An upstream can announce the undeclared call in `response.output_item.added` — which trips the client guard and sends `response.failed` — and then close with a `response.completed` whose `output` is empty. The terminal check then sees nothing undeclared and the refused turn enters `previous_response_id` replay state anyway. Rejection is now sticky for the whole turn, set from every parsed payload on the inspection side rather than derived from the terminal snapshot. That required a seam: `SseInspectorHandlers` had no per-payload callback, so the flag could never have been set. `onParsedPayload` is added there, invoked in `scanPayload` before any terminal classification, and carried on `InspectionConsumerOptions` so both `consumeForInspection` and `consumeForResponseLogMetadata` wire it — adding it to the handler type alone would have left the tee path silently inert. The flag is gated on `undeclaredToolGuardActive`, the same condition as the guard. Without that gate a no-catalog or forward-auth stream would mark every call undeclared and stop recording continuation state for exactly the passthrough traffic the guard deliberately stands down for. Regression: a stream whose only undeclared item is in `output_item.added` and whose terminal `output` is empty must give the client `response.failed` and leave nothing for a follow-up to inherit. Driven red against the unfixed flag before landing.
1 parent cfbb3cd commit dea7d2c

3 files changed

Lines changed: 82 additions & 0 deletions

File tree

‎src/server/relay.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -603,6 +603,13 @@ export type SseInspectorHandlers = {
603603
onTerminal?: (status: ResponsesTerminalStatus, httpStatusOverride?: number) => void;
604604
logCtx?: RequestLogContext;
605605
onCompletedResponse?: (response: { id?: unknown; output?: unknown; status?: unknown }) => void;
606+
/**
607+
* Every parsed SSE payload, delivered BEFORE any onCompletedResponse derived from that same
608+
* payload. A caller that must decide on the whole turn -- not just its terminal snapshot --
609+
* needs to see the incremental events, because a stream can announce an item and then close
610+
* with an empty `output`.
611+
*/
612+
onParsedPayload?: (payload: unknown) => void;
606613
onFirstOutput?: () => void;
607614
/**
608615
* Provider-scoped compatibility: persist the completed snapshot under the
@@ -786,6 +793,11 @@ export function createSseInspector(handlers: SseInspectorHandlers): SseInspector
786793
if (!reported && handlers.logCtx) {
787794
inspectResponseLogSsePayloadParsed(handlers.logCtx, payload, parsed);
788795
}
796+
// Before any terminal handling: a consumer deciding on the whole turn must observe this
797+
// payload even when the terminal snapshot that follows no longer mentions it.
798+
if (handlers.onParsedPayload && parsed !== undefined) {
799+
try { handlers.onParsedPayload(parsed); } catch { /* inspection must never throw into the pump */ }
800+
}
789801
reportFirstOutput.parsed(parsed);
790802
const status = terminalStatusFromParsed(parsed);
791803
if (status) sawTerminal = true;
@@ -953,6 +965,8 @@ export type InspectionConsumerOptions = {
953965
now?: () => number;
954966
/** Forward provider-scoped response-id pinning to the owned inspector. */
955967
pinCompletedResponseIdToFirstSeen?: boolean;
968+
/** Observe every parsed SSE payload on the inspection side; see SseInspectorHandlers. */
969+
onParsedPayload?: (payload: unknown) => void;
956970
/** Test seam for proving both public consumers dispose their owned inspector. */
957971
inspectorFactory?: (handlers: SseInspectorHandlers) => SseInspector;
958972
};
@@ -1108,6 +1122,7 @@ export function consumeForInspection(
11081122
onTerminal,
11091123
logCtx,
11101124
onCompletedResponse,
1125+
onParsedPayload: options?.onParsedPayload,
11111126
onFirstOutput,
11121127
pinCompletedResponseIdToFirstSeen: options?.pinCompletedResponseIdToFirstSeen,
11131128
});
@@ -1158,6 +1173,7 @@ export function consumeForResponseLogMetadata(
11581173
const inspector = (options?.inspectorFactory ?? createSseInspector)({
11591174
logCtx,
11601175
onCompletedResponse,
1176+
onParsedPayload: options?.onParsedPayload,
11611177
onFirstOutput,
11621178
pinCompletedResponseIdToFirstSeen: options?.pinCompletedResponseIdToFirstSeen,
11631179
});

‎src/server/responses/core.ts‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,7 @@ import {
241241
collectDeclaredWireToolNames,
242242
createUndeclaredToolCallGuardBlockRewrite,
243243
undeclaredToolCallMessage,
244+
undeclaredToolCallName,
244245
undeclaredToolCallNameInResponse,
245246
} from "../responses-undeclared-tool-guard";
246247
import { createGithubCopilotResponsesBlockRewrite } from "../github-copilot-responses-repair";
@@ -2308,8 +2309,25 @@ async function handleResponsesInner(
23082309
// untouched upstream stream, so it can still observe a `response.completed` the client never
23092310
// received; checking the payload itself rather than a flag shared with the client relay keeps
23102311
// this free of tee ordering races.
2312+
//
2313+
// Checking only the terminal snapshot is not enough. An upstream can announce the undeclared
2314+
// call in `response.output_item.added`, which trips the client guard, and then close with a
2315+
// `response.completed` whose `output` is empty. The client gets `response.failed`, the terminal
2316+
// check sees nothing undeclared, and the refused turn enters continuation state anyway. So the
2317+
// rejection is sticky for the whole turn, set from every parsed payload on the inspection side.
2318+
let inspectionSawUndeclaredTool = false;
2319+
const noteInspectedPayload = (payload: unknown) => {
2320+
// Gated on the same flag as the guard itself: with no readable catalog (or a forward-auth
2321+
// provider) every name looks undeclared, and flipping this would stop recording continuation
2322+
// state for exactly the passthrough traffic the guard deliberately stands down for.
2323+
if (!undeclaredToolGuardActive || inspectionSawUndeclaredTool) return;
2324+
if (undeclaredToolCallName(payload, declaredWireToolNames) !== undefined) {
2325+
inspectionSawUndeclaredTool = true;
2326+
}
2327+
};
23112328
const rememberPassthroughResponseChecked = rememberPassthroughResponse
23122329
? (response: { id?: unknown; output?: unknown; status?: unknown }) => {
2330+
if (inspectionSawUndeclaredTool) return;
23132331
if (
23142332
undeclaredToolGuardActive
23152333
&& undeclaredToolCallNameInResponse(response, declaredWireToolNames) !== undefined
@@ -2775,6 +2793,7 @@ async function handleResponsesInner(
27752793
onTerminal: reportNativeTerminal,
27762794
logCtx,
27772795
onCompletedResponse: rememberPassthroughResponseChecked,
2796+
onParsedPayload: noteInspectedPayload,
27782797
onFirstOutput: options.onFirstOutput,
27792798
pinCompletedResponseIdToFirstSeen: githubCopilotRepairEnabled,
27802799
});
@@ -2825,6 +2844,7 @@ async function handleResponsesInner(
28252844
drainBounds: { ms: 15_000, bytes: 32 * 1024 * 1024 },
28262845
upstream,
28272846
pinCompletedResponseIdToFirstSeen: githubCopilotRepairEnabled,
2847+
onParsedPayload: noteInspectedPayload,
28282848
};
28292849
if (recordTerminalOutcomes) {
28302850
// A real terminal was parsed from the (teed) inspection stream — record it as the outcome

‎tests/responses-undeclared-tool-guard.test.ts‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -414,6 +414,52 @@ describe("a refused turn does not become continuation state", () => {
414414
// Nothing to inherit: the follow-up keeps only its own single input item.
415415
expect(expandedInputLength("resp_refused")).toBe(1);
416416
});
417+
418+
test("a streamed turn refused mid-stream is not remembered, even when the terminal snapshot is empty", async () => {
419+
// The terminal-snapshot check alone misses this shape: the undeclared call is announced in
420+
// `response.output_item.added` (which trips the client guard) and the stream then closes with
421+
// a `response.completed` carrying an EMPTY output. The client sees `response.failed`, the
422+
// terminal check sees nothing undeclared, and the refused turn would enter continuation state.
423+
const responseId = "resp_stream_refused";
424+
const sse = [
425+
`data: ${JSON.stringify({ type: "response.created", response: { id: responseId, status: "in_progress" } })}\n\n`,
426+
`data: ${JSON.stringify({
427+
type: "response.output_item.added",
428+
output_index: 0,
429+
item: { type: "function_call", id: "fc_bad", call_id: "call_bad", name: "apply_patch", arguments: "{}" },
430+
})}\n\n`,
431+
`data: ${JSON.stringify({ type: "response.completed", response: { id: responseId, status: "completed", output: [] } })}\n\n`,
432+
"data: [DONE]\n\n",
433+
].join("");
434+
435+
const savedFetch = globalThis.fetch;
436+
globalThis.fetch = (async () => new Response(sse, {
437+
headers: { "content-type": "text/event-stream" },
438+
})) as typeof fetch;
439+
let response: Response;
440+
try {
441+
response = await handleResponses(new Request("http://localhost/v1/responses", {
442+
method: "POST",
443+
headers: { "content-type": "application/json" },
444+
body: JSON.stringify({
445+
model: "fixture/deepseek-v4-flash",
446+
stream: true,
447+
input: [{ role: "user", content: [{ type: "input_text", text: "edit the file" }] }],
448+
tools: declaredTools,
449+
}),
450+
}), config, { model: "", provider: "" });
451+
} finally {
452+
globalThis.fetch = savedFetch;
453+
}
454+
455+
// Drain so the inspection side observes the whole stream before we assert on its effect.
456+
const clientStream = await response.text();
457+
expect(clientStream).toContain("response.failed");
458+
await Bun.sleep(50);
459+
460+
// Nothing to inherit: the follow-up keeps only its own single input item.
461+
expect(expandedInputLength(responseId)).toBe(1);
462+
});
417463
});
418464

419465
describe("a request that declares no tools has no catalog to police", () => {

0 commit comments

Comments
 (0)