Skip to content

Commit 799330b

Browse files
committed
fix(claude): keep mid-conversation system messages in the timeline
Claude Code sends role:"system" entries inside `messages`, and every one of them was folded into `instructions`. The parser pushes `data.instructions` onto the system prompt before anything else, so each injected reminder rewrote the head of the prompt and invalidated the upstream KV cache prefix. Where no metadata.user_id is present, the Desktop prompt_cache_key fallback hashes the same system parts, so the cache key rotated along with it. Emit each non-empty in-messages system message as a chronological { type: "message", role: "developer", content: [{ type: "input_text", text }] } input item instead. Top-level Anthropic `system` keeps flowing through systemToInstructions into body.instructions unchanged, so `instructions` now has exactly one source and stops moving mid-conversation. Not role:"system" in `input`. The schema allows it, but ChatGPT Codex rejects it, parseRequest re-hoists it onto the system prompt, and the canonical Responses forward folds text-only system items back into instructions - so it would land back where it started. `developer` is first-class in the schema and keeps timeline order. The fold was deliberate in origin (cee918c) so native ChatGPT would not 400 on a system input item. That constraint is real and still respected; the 2026-07-11 note that folding is "the only shape that works on every route" is what turned out to be stale. systemParts keeps its array shape in the cache-key hash on purpose. Joining it to a string would rotate every existing cohort key for a reason unrelated to this bug. Closes #4148.
1 parent 4498fb9 commit 799330b

2 files changed

Lines changed: 94 additions & 13 deletions

File tree

‎src/claude/inbound.ts‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -150,10 +150,17 @@ function blockedSkillCallIds(messages: readonly unknown[], blocked: readonly str
150150

151151
/**
152152
* Claude Code (observed 2026-07-11, real CLI smoke) sends `role:"system"` entries in
153-
* `messages` despite the published API having no system role. Map them to Responses
154-
* instructions text: the native ChatGPT backend rejects system message items in
155-
* `input` ("System messages are not allowed", verified live), so folding into
156-
* `instructions` is the only shape that works on every route.
153+
* `messages` despite the published API having no system role. They are emitted as
154+
* chronological `role:"developer"` input items, which keeps the timeline intact and
155+
* leaves `instructions` owned solely by the top-level Anthropic `system` field.
156+
*
157+
* The original mapping folded them into `instructions` because the native ChatGPT
158+
* backend rejects `role:"system"` items in `input` ("System messages are not allowed",
159+
* verified live). That constraint is real and still respected — but it only rules out
160+
* `system`, not `developer`, which every Responses route accepts. Folding meant each
161+
* mid-conversation reminder mutated the prompt head, invalidating the upstream KV
162+
* prefix and rotating the Desktop `prompt_cache_key` fallback below on every turn
163+
* (#4148).
157164
*/
158165
function systemMessageText(content: unknown): string {
159166
if (typeof content === "string") return content;
@@ -333,7 +340,12 @@ function translateAnthropicRequest(raw: unknown, cc: OcxClaudeCodeConfig | undef
333340
else if (msg.role === "assistant") assistantMessageToItems(msg.content, input, budget);
334341
else if (msg.role === "system") {
335342
const text = systemMessageText(msg.content);
336-
if (text.length > 0) systemParts.push(text);
343+
// Keep it where the client put it. `developer` is first-class in the Responses
344+
// schema and survives parseRequest as a chronological message, where `system`
345+
// would be re-hoisted back onto the system prompt and defeat the point.
346+
if (text.length > 0) {
347+
input.push({ type: "message", role: "developer", content: [{ type: "input_text", text }] });
348+
}
337349
}
338350
else throw new AnthropicRequestError(`unsupported message role: ${String(msg.role)}`);
339351
}

‎tests/claude-integration/claude-inbound.test.ts‎

Lines changed: 77 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,23 @@ import { createResponsesPassthroughAdapter } from "../../src/adapters/openai-res
99
import { withTestTranslatorBudget } from "../helpers/translator-budget";
1010
import type { OcxProviderConfig } from "../../src/types";
1111

12+
// The translator returns an untyped wire body. These aliases name just the fields the
13+
// system-message cases assert on, so the assertions read as a contract instead of a cast.
14+
type TranslatedInputItem = {
15+
type?: string;
16+
role?: string;
17+
content?: Array<{ type?: string; text?: string }>;
18+
};
19+
type TranslatedBody = {
20+
instructions?: string;
21+
prompt_cache_key?: string;
22+
input: TranslatedInputItem[];
23+
};
24+
25+
function translatedBody(raw: Record<string, unknown>): TranslatedBody {
26+
return anthropicToResponsesBody(raw) as TranslatedBody;
27+
}
28+
1229
// Full Claude Code-shaped request: system array, tool cycle, image, thinking, options.
1330
function claudeCodeRequest(): Record<string, unknown> {
1431
return {
@@ -310,25 +327,77 @@ describe("claude inbound translation", () => {
310327
expect(() => parseRequest(body)).not.toThrow();
311328
});
312329

313-
test("system role messages fold into instructions (real Claude Code sends them; native backend rejects system items)", () => {
314-
const body = anthropicToResponsesBody({
330+
// Claude Code sends role:"system" entries in `messages`. They used to be folded into
331+
// `instructions` alongside the top-level system field; now each one becomes a
332+
// chronological role:"developer" input item, so `instructions` belongs to the
333+
// top-level Anthropic `system` field alone and the prompt head stops moving
334+
// mid-conversation.
335+
test("in-messages system role becomes a chronological developer item, never a system item", () => {
336+
const body = translatedBody({
315337
model: "m", max_tokens: 10,
316338
system: "top-level",
317339
messages: [
318340
{ role: "system", content: "be terse" },
319341
{ role: "system", content: [{ type: "text", text: "block form" }] },
320342
{ role: "user", content: "hi" },
321343
],
322-
}) as any;
323-
expect(body.instructions).toBe("top-level\n\nbe terse\n\nblock form");
324-
// No system message items in input — native ChatGPT backend 400s on them.
325-
expect((body.input as any[]).every(item => item.role !== "system")).toBe(true);
326-
expect(body.input).toHaveLength(1);
327-
expect(body.input[0].role).toBe("user");
344+
});
345+
// Only the top-level system reaches instructions now.
346+
expect(body.instructions).toBe("top-level");
347+
// Still no system message items in input — the native ChatGPT backend 400s on them.
348+
expect(body.input.every(item => item.role !== "system")).toBe(true);
349+
expect(body.input.map(item => item.role)).toEqual(["developer", "developer", "user"]);
350+
expect(body.input[0]?.content).toEqual([{ type: "input_text", text: "be terse" }]);
351+
expect(body.input[1]?.content).toEqual([{ type: "input_text", text: "block form" }]);
328352
expect(() => responsesRequestSchema.parse(body)).not.toThrow();
329353
expect(() => parseRequest(body)).not.toThrow();
330354
});
331355

356+
// #4148: a client that injects a fresh reminder each turn used to rewrite the prompt
357+
// head every time, which invalidates the upstream KV prefix and — with no
358+
// metadata.user_id — rotated the Desktop prompt_cache_key fallback along with it.
359+
test("a mid-conversation system message leaves the cache prefix and cache key alone", () => {
360+
const turn = (messages: unknown[]) =>
361+
translatedBody({ model: "m", max_tokens: 10, system: "S", messages });
362+
363+
const turn1 = turn([
364+
{ role: "user", content: "u1" },
365+
{ role: "system", content: "r1" },
366+
]);
367+
const turn2 = turn([
368+
{ role: "user", content: "u1" },
369+
{ role: "system", content: "r1" },
370+
{ role: "assistant", content: "a1" },
371+
{ role: "user", content: "u2" },
372+
{ role: "system", content: "r2" },
373+
]);
374+
375+
// The prompt head is the whole point: identical across turns, and equal to the
376+
// top-level system field on its own.
377+
expect(turn1.instructions).toBe("S");
378+
expect(turn2.instructions).toBe("S");
379+
380+
expect(turn1.input.map(item => item.role)).toEqual(["user", "developer"]);
381+
expect(turn2.input.map(item => item.role))
382+
.toEqual(["user", "developer", "assistant", "user", "developer"]);
383+
expect(turn2.input.map(item => item.content?.[0]?.text))
384+
.toEqual(["u1", "r1", "a1", "u2", "r2"]);
385+
expect(turn2.input.every(item => item.role !== "system")).toBe(true);
386+
387+
// Turn 1's items are still a prefix of turn 2's, which is what the KV cache matches on.
388+
expect(turn2.input.slice(0, 2)).toEqual(turn1.input);
389+
390+
// No metadata.user_id, so the Desktop cohort fallback applies. It hashes the
391+
// post-translation system text, which no longer absorbs the injected reminders.
392+
expect(turn1.prompt_cache_key).toBeDefined();
393+
expect(turn2.prompt_cache_key).toBe(turn1.prompt_cache_key);
394+
395+
for (const body of [turn1, turn2]) {
396+
expect(() => responsesRequestSchema.parse(body)).not.toThrow();
397+
expect(() => parseRequest(body)).not.toThrow();
398+
}
399+
});
400+
332401
test("tool_result is_error and string content", () => {
333402
const body = anthropicToResponsesBody({
334403
model: "m", max_tokens: 10,

0 commit comments

Comments
 (0)