Skip to content

Commit a88f406

Browse files
committed
fix: hoist <system-reminder> to system role for non-Anthropic models + warning false-positive
Two related bugs the trace finally revealed, after the qwen.txt theory in b07da4e didn't actually fix the refusal in production: 1. **Root cause of GPT-5.x refusals**: prompt.ts:2049 attaches PROMPT_PLAN (wrapped in `<system-reminder>...</system-reminder>` tags) as a text part on the latest user message. Claude is trained to recognize that tag as authoritative system-level instructions even when delivered in a user-role message. OpenAI / Google / etc. models are NOT — they see a user message containing a `<system-reminder>` block with aggressive constraint language (STRICTLY FORBIDDEN / ZERO exceptions / ABSOLUTE CONSTRAINT / MUST) and pattern-match it as a prompt-injection attempt. GPT-5.x via `altimate-backend/altimate-default` reliably returned "I'm sorry, but I cannot assist with that request." for benign developer requests like "plan a feature to add a button…". Confirmed by reading the actual LLM payload via temporary debug instrumentation in `llm.ts` (since removed). Fix in `llm.ts`: for non-Anthropic-like models, walk `input.messages` and extract any `<system-reminder>`-wrapped text parts from user-role messages, then append them to the system array so they arrive as proper `role:"system"` messages. User messages that become empty after extraction are dropped. Anthropic-like models keep the existing behavior. Detection of Anthropic-like uses providerID + family + api.npm + the existing api.id substring patterns, matching the heuristics already used in `provider/transform.ts:284-295` for cache-control handling. 2. **Counter scope bug in plan-no-tool warning**: `sessionToolCallsMade` in `processor.ts:57` was claimed (comment line 207) to be session- scoped, but `SessionProcessor.create()` is called per-step by `loop()` in `prompt.ts:485`, so the counter resets each step. A multi-step plan session (read → grep → read → … → final text) false-positives on the last text-only step. The warning fires *after* the agent has successfully explored the codebase and produced a plan — confusing the user into thinking the session failed. Fix: at warning-check time, also scan `streamInput.messages` for any prior assistant tool-call content. The session has used tools iff either the per-step counter is non-zero OR the history contains a tool-call. Regression test added in `processor.test.ts`. The earlier `system.ts` family-based routing (commit b07da4e) stays. It was based on a wrong root-cause theory, but the change itself is correct: a model with `family: "openai"` should not fall through to qwen.txt (an Anthropic-style fallback). Worth keeping as defense in depth even though it wasn't the load-bearing fix. Touches: - `packages/opencode/src/session/llm.ts`: system-reminder hoist. - `packages/opencode/src/session/processor.ts`: warning false-positive fix + comment block explaining the per-step scoping. - `packages/opencode/test/session/processor.test.ts`: regression test for the warning false-positive. Typecheck clean. 124 affected tests pass. This is the actual fix for #887 / AI-6957.
1 parent b07da4e commit a88f406

3 files changed

Lines changed: 91 additions & 2 deletions

File tree

‎packages/opencode/src/session/llm.ts‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,48 @@ export namespace LLM {
6464
])
6565
const isCodex = provider.id === "openai" && auth?.type === "oauth"
6666

67+
// altimate_change start — for non-Anthropic models, hoist `<system-reminder>`-wrapped
68+
// synthetic text parts out of user messages and into the system array.
69+
//
70+
// Plan mode (and a few other paths) attaches PROMPT_PLAN as a text part on the latest
71+
// user message, wrapped in `<system-reminder>...</system-reminder>` tags — an Anthropic-
72+
// Claude convention that Claude is trained to recognize as legitimate system-level
73+
// instructions even when it arrives in a user-role message. OpenAI / Google / etc. are
74+
// NOT trained on that convention. When GPT-5.x via the altimate gateway sees a user
75+
// message that contains a `<system-reminder>` block with aggressive constraint language
76+
// (STRICTLY FORBIDDEN, ZERO exceptions, MUST, ABSOLUTE CONSTRAINT), it pattern-matches
77+
// it as a prompt-injection attempt and emits a content-policy refusal ("I'm sorry, but
78+
// I cannot assist with that request.") — reliably, even on benign requests like
79+
// "add a button". GH #887 / AI-6957. Hoisting these parts into actual system-role
80+
// messages routes around the misinterpretation.
81+
const isAnthropicLike =
82+
input.model.providerID === "anthropic" ||
83+
input.model.providerID === "google-vertex-anthropic" ||
84+
input.model.family === "anthropic" ||
85+
input.model.api.npm === "@ai-sdk/anthropic" ||
86+
input.model.api.id.includes("anthropic") ||
87+
input.model.api.id.includes("claude")
88+
const hoistedReminders: string[] = []
89+
if (!isAnthropicLike) {
90+
input.messages = input.messages.map((m) => {
91+
if (m.role !== "user" || !Array.isArray(m.content)) return m
92+
const kept: typeof m.content = []
93+
for (const part of m.content) {
94+
if (part.type === "text" && typeof part.text === "string" && part.text.trimStart().startsWith("<system-reminder>")) {
95+
hoistedReminders.push(part.text)
96+
} else {
97+
kept.push(part)
98+
}
99+
}
100+
return { ...m, content: kept }
101+
})
102+
// Drop user messages that became empty (had only system-reminder parts).
103+
input.messages = input.messages.filter(
104+
(m) => !(m.role === "user" && Array.isArray(m.content) && m.content.length === 0),
105+
)
106+
}
107+
// altimate_change end
108+
67109
const system = []
68110
system.push(
69111
[
@@ -72,6 +114,10 @@ export namespace LLM {
72114
...(input.agent.prompt ? [input.agent.prompt] : isCodex ? [] : SystemPrompt.provider(input.model)),
73115
// any custom prompt passed into this call
74116
...input.system,
117+
// altimate_change start — hoisted reminders appended after the base provider
118+
// prompt so the model sees them as authoritative session-level instructions.
119+
...hoistedReminders,
120+
// altimate_change end
75121
// any custom prompt from last user message
76122
...(input.user.system ? [input.user.system] : []),
77123
]

‎packages/opencode/src/session/processor.ts‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -348,10 +348,27 @@ export namespace SessionProcessor {
348348
// Users read the text, see no progress, and abandon. Surface a
349349
// warning + telemetry so the pattern is measurable and the user
350350
// knows to try a different model.
351+
//
352+
// sessionToolCallsMade tracks tool calls in the CURRENT step only
353+
// — SessionProcessor.create() is called per-step by loop() (see
354+
// prompt.ts), so the closure variable resets each step. A multi-
355+
// step plan-mode session (read → grep → read → … → final text)
356+
// would then false-positive on the final text-only step. Also
357+
// scan streamInput.messages for any prior assistant tool-call
358+
// content; if found, the session has used tools and the warning
359+
// should be suppressed.
360+
const sessionHasPriorToolCalls =
361+
sessionToolCallsMade > 0 ||
362+
streamInput.messages.some(
363+
(m) =>
364+
m.role === "assistant" &&
365+
Array.isArray(m.content) &&
366+
m.content.some((p) => p.type === "tool-call"),
367+
)
351368
if (
352369
input.assistantMessage.agent === "plan" &&
353370
value.finishReason === "stop" &&
354-
sessionToolCallsMade === 0 &&
371+
!sessionHasPriorToolCalls &&
355372
!planNoToolWarningEmitted
356373
) {
357374
planNoToolWarningEmitted = true

‎packages/opencode/test/session/processor.test.ts‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -474,11 +474,19 @@ describe("plan-agent no-tool-generation detection", () => {
474474
modelID: string
475475
providerID: string
476476
tokensOutput: number
477+
/**
478+
* Whether the conversation history (streamInput.messages) contains any prior
479+
* assistant tool-call content. Mirrors the source check at processor.ts that
480+
* compensates for `sessionToolCallsMade` being per-step (SessionProcessor.create
481+
* is called per-step by loop()).
482+
*/
483+
priorToolCallsInHistory?: boolean
477484
}): { event: Telemetry.Event | null; warningEmitted: boolean } {
485+
const sessionHasPriorToolCalls = opts.sessionToolCallsMade > 0 || opts.priorToolCallsInHistory === true
478486
if (
479487
opts.agent === "plan" &&
480488
opts.finishReason === "stop" &&
481-
opts.sessionToolCallsMade === 0 &&
489+
!sessionHasPriorToolCalls &&
482490
!opts.planNoToolWarningEmitted
483491
) {
484492
return {
@@ -535,6 +543,24 @@ describe("plan-agent no-tool-generation detection", () => {
535543
expect(result.warningEmitted).toBe(false)
536544
})
537545

546+
// Regression guard: SessionProcessor.create() is called per-step by loop(),
547+
// so sessionToolCallsMade is per-step in practice (not session-wide as the
548+
// 2026-06-04 comment claimed). A multi-step plan-mode session that runs many
549+
// tools and then produces a final text-only step would false-positive without
550+
// also consulting the conversation history. See processor.ts comment block.
551+
test("does not fire on final text-only step when prior steps used tools", () => {
552+
const result = simulateFinishStep({
553+
...baseOpts,
554+
agent: "plan",
555+
finishReason: "stop",
556+
sessionToolCallsMade: 0, // resets each step — final text-only step is 0
557+
priorToolCallsInHistory: true, // earlier steps populated tool-call parts
558+
planNoToolWarningEmitted: false,
559+
})
560+
expect(result.event).toBeNull()
561+
expect(result.warningEmitted).toBe(false)
562+
})
563+
538564
test("does not fire when finish_reason is tool-calls", () => {
539565
const result = simulateFinishStep({
540566
...baseOpts,

0 commit comments

Comments
 (0)