Skip to content

Commit 098a646

Browse files
committed
fix: J1 + J2 from PR #888 multi-LLM review
J1 — exact family-string match misses real gateway values `system.ts` (and `prompt.ts:isAnthropicLikeModel`) compared `model.family` exactly against `"anthropic"` / `"gemini"`. But the model registry (`models-api.json`) uses *specific* family values like `claude-sonnet`, `claude-haiku`, `gemini-pro`, `gemini-flash`, `gpt-codex`, etc. — never the coarse literals. Any config-added Claude or Gemini model on the altimate-backend gateway therefore silently fell through to `PROMPT_CODEX` — recreating the GH #887 misrouting class this PR was meant to fix. Latent today because the only statically-registered altimate-backend model is `altimate-default` with `family: "openai"` (coarse). Fix: shared `familyVendor(family)` helper in `src/provider/family.ts` that maps `claude` / `claude-*` / `anthropic` → `"anthropic"`, `gemini` / `gemini-*` → `"gemini"`, `openai` / `openai-compatible` / `gpt` / `gpt-*` → `"openai"`. Used by both `SystemPrompt.provider` (altimate-backend block) and `isAnthropicLikeModel`. `isAnthropicLikeModel` also tightens the api.id substring matches (lowercased; `claude-` anchored to the last URL segment so `foo-claude-bench` doesn't false-match; `anthropic-`/`anthropic/` anchored to the path). J2 — experimental plan-mode reminders replay as user-role `<system-reminder>` on turn 2+ Under `OPENCODE_EXPERIMENTAL_PLAN_MODE=true`, `insertReminders` calls `Session.updatePart(...)` to persist the entering-plan-mode and build-switch reminder parts. The persisted row was written *without* `ignored: true`. The caller mutated `part.ignored = true` only on the in-memory copy, so on the next turn the DB-reloaded part had `ignored: undefined`, slipped past the `!part.ignored` filter in `toModelMessages`, and re-entered the user-role payload as `<system-reminder>...</system-reminder>` — re-triggering the GPT-5.x refusal class this PR fixes. Bug was flag-gated so the default path was unaffected, but the PR ships tests pinned to the experimental branches so it shipped the bug latent under the flag. Fix: push the hoist decision *into* `insertReminders`. The function now takes `model`, computes `nonAnthropic = !isAnthropicLikeModel(...)` once, and stamps `ignored: true` onto every trusted reminder at insertion time — including the `Session.updatePart` payloads, so the persisted row carries the flag across turns. Caller in `loop()` no longer mutates `part.ignored`; the contract is now owned by `insertReminders`. The hoist branch in `loop()` reduces to a simple `.map(p => p.text)` over `trustedReminderParts`. Tests - New: `test/provider/family.test.ts` table-drives `familyVendor` against every coarse-vs-specific case (claude/claude-haiku/ claude-opus/claude-sonnet/gemini/gemini-pro/gemini-flash/gpt/ gpt-codex), case-insensitivity, and the unknown→undefined contract. - `test/session/plan-layer-e2e.test.ts`: regression guards that (a) `system.ts` and `prompt.ts:isAnthropicLikeModel` both consume `familyVendor` and no exact `family === "anthropic"` shape lingers in the altimate-backend block; (b) `insertReminders` takes `model` and stamps `nonAnthropic ? { ignored: true } : {}` onto persisted reminder writes; (c) the caller no longer mutates `part.ignored`. 154 affected tests pass. Typecheck clean. Addresses #888 review J1, J2, and N1 (case-sensitivity / over-broad api.id substring) and N2 in spirit (`isAnthropicLikeModel` now has its classifier under direct test via `familyVendor`).
1 parent 64f72f2 commit 098a646

5 files changed

Lines changed: 175 additions & 37 deletions

File tree

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// altimate_change start — shared vendor classifier for `model.family` values.
2+
//
3+
// `model.family` is a free-form string in the Model schema; the model registry
4+
// (`packages/opencode/test/tool/fixtures/models-api.json`) uses *specific*
5+
// values like `claude-sonnet`, `claude-haiku`, `gemini-pro`, `gemini-flash`,
6+
// `gpt-codex`, etc. — never the coarse `anthropic`/`gemini`/`openai` literals.
7+
// Routing code that exact-matches against the coarse names silently misses
8+
// real models, recreating the GH #887 misrouting class on any altimate-backend
9+
// gateway path that exposes Claude or Gemini models. Use this helper to map a
10+
// specific family value to its vendor bucket.
11+
//
12+
// Reviewer-driven addition (PR #888 review thread, J1 from the multi-LLM
13+
// panel). Keep this the single source of truth so the prompt-routing and the
14+
// hoist-decision both interpret family the same way.
15+
16+
export type Vendor = "anthropic" | "gemini" | "openai"
17+
18+
export function familyVendor(family: string | undefined): Vendor | undefined {
19+
if (!family) return undefined
20+
const f = family.toLowerCase()
21+
if (f === "anthropic" || f === "claude" || f.startsWith("claude-")) return "anthropic"
22+
if (f === "gemini" || f.startsWith("gemini-")) return "gemini"
23+
if (f === "openai" || f === "openai-compatible" || f === "gpt" || f.startsWith("gpt-")) return "openai"
24+
return undefined
25+
}
26+
// altimate_change end

‎packages/opencode/src/session/prompt.ts‎

Lines changed: 45 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ import { Session } from "."
1111
import { Agent } from "../agent/agent"
1212
import { Provider } from "../provider/provider"
1313
import { ModelID, ProviderID } from "../provider/schema"
14+
// altimate_change start — shared family→vendor classifier (#888 J1)
15+
import { familyVendor } from "../provider/family"
16+
// altimate_change end
1417
import { type Tool as AITool, tool, jsonSchema, type ToolCallOptions, asSchema } from "ai"
1518
import { SessionCompaction } from "./compaction"
1619
import { Instance } from "../project/instance"
@@ -688,25 +691,23 @@ export namespace SessionPrompt {
688691
const agent = await Agent.get(lastUser.agent)
689692
const maxSteps = agent.steps ?? Infinity
690693
const isLastStep = step >= maxSteps
691-
// altimate_change start — insertReminders now returns the trusted reminder
692-
// parts it appended; for non-Anthropic-like models we hoist them into the
693-
// system prompt and mark the parts ignored so they don't ALSO appear in
694-
// the user message. The returned-parts list is the trust boundary —
695-
// never the `synthetic` flag (other code paths set it on file/resource
696-
// expansions, which are user-derived). See #887/#888 review thread.
694+
// altimate_change start — insertReminders returns the trusted reminder parts
695+
// it appended. The function now also pre-applies `ignored: true` to those
696+
// parts (and to the persisted rows under experimental plan mode) for
697+
// non-Anthropic-like models, so `toModelMessages` skips them on every turn
698+
// — not just this one (#888 J2). The returned-parts list is the trust
699+
// boundary; we never infer trust from the `synthetic` flag (other code
700+
// paths set it on user-derived file/resource expansions). See #887/#888.
697701
const reminderResult = await insertReminders({
698702
messages: msgs,
699703
agent,
700704
session,
705+
model,
701706
})
702707
msgs = reminderResult.messages
703-
const hoistedReminders: string[] = []
704-
if (!isAnthropicLikeModel(model)) {
705-
for (const part of reminderResult.trustedReminderParts) {
706-
hoistedReminders.push(part.text)
707-
part.ignored = true
708-
}
709-
}
708+
const hoistedReminders = isAnthropicLikeModel(model)
709+
? []
710+
: reminderResult.trustedReminderParts.map((p) => p.text)
710711
// altimate_change end
711712

712713
// altimate_change start — plan refinement detection and telemetry
@@ -2055,16 +2056,22 @@ export namespace SessionPrompt {
20552056
}
20562057
}
20572058

2058-
// altimate_change start — model-family helper used by the trust-aware hoist below
2059+
// altimate_change start — model-family helper used by the trust-aware hoist below.
2060+
// Uses `familyVendor` so specific family values (`claude-sonnet`, `claude-haiku`,
2061+
// `gemini-pro`, etc.) classify correctly — an exact `family === "anthropic"`
2062+
// check would miss the gateway-emitted specific names (#888 J1). The api.id
2063+
// checks are lowercased and tightened to a `claude-` / `anthropic-` /
2064+
// `anthropic/...` shape so a model named `foo-claude-bench` doesn't false-match.
20592065
function isAnthropicLikeModel(model: Provider.Model): boolean {
2060-
return (
2061-
model.providerID === "anthropic" ||
2062-
model.providerID === "google-vertex-anthropic" ||
2063-
model.family?.toLowerCase() === "anthropic" ||
2064-
model.api.npm === "@ai-sdk/anthropic" ||
2065-
model.api.id.includes("anthropic") ||
2066-
model.api.id.includes("claude")
2067-
)
2066+
if (model.providerID === "anthropic") return true
2067+
if (model.providerID === "google-vertex-anthropic") return true
2068+
if (familyVendor(model.family) === "anthropic") return true
2069+
if (model.api.npm === "@ai-sdk/anthropic") return true
2070+
const apiId = model.api.id.toLowerCase()
2071+
const lastSeg = apiId.split("/").pop() ?? apiId
2072+
if (/^claude[-_.]/.test(lastSeg)) return true
2073+
if (/^anthropic[-_/]/.test(apiId)) return true
2074+
return false
20682075
}
20692076
// altimate_change end
20702077

@@ -2080,8 +2087,20 @@ export namespace SessionPrompt {
20802087
messages: MessageV2.WithParts[]
20812088
agent: Agent.Info
20822089
session: Session.Info
2090+
// altimate_change start — used to bake `ignored` into the persisted experimental
2091+
// plan-mode reminders so they don't replay as user-role `<system-reminder>` on
2092+
// turn 2+ on non-Anthropic models (#888 J2).
2093+
model: Provider.Model
2094+
// altimate_change end
20832095
}): Promise<InsertRemindersResult> {
20842096
const trustedReminderParts: MessageV2.TextPart[] = []
2097+
// altimate_change start — pre-compute the hoist decision once so it can be
2098+
// applied at insertion time (including to persisted rows). For non-Anthropic
2099+
// models, every trusted reminder is marked `ignored: true` immediately so
2100+
// `toModelMessages` will skip it (the caller no longer needs to mutate the
2101+
// flag, and DB-persisted rows survive the contract across turns).
2102+
const nonAnthropic = !isAnthropicLikeModel(input.model)
2103+
// altimate_change end
20852104
const userMessage = input.messages.findLast((msg) => msg.info.role === "user")
20862105
if (!userMessage) return { messages: input.messages, trustedReminderParts }
20872106

@@ -2095,6 +2114,7 @@ export namespace SessionPrompt {
20952114
type: "text",
20962115
text: PROMPT_PLAN,
20972116
synthetic: true,
2117+
...(nonAnthropic ? { ignored: true } : {}),
20982118
}
20992119
userMessage.parts.push(part)
21002120
trustedReminderParts.push(part)
@@ -2108,6 +2128,7 @@ export namespace SessionPrompt {
21082128
type: "text",
21092129
text: BUILD_SWITCH,
21102130
synthetic: true,
2131+
...(nonAnthropic ? { ignored: true } : {}),
21112132
}
21122133
userMessage.parts.push(part)
21132134
trustedReminderParts.push(part)
@@ -2131,6 +2152,7 @@ export namespace SessionPrompt {
21312152
text:
21322153
BUILD_SWITCH + "\n\n" + `A plan file exists at ${plan}. You should execute on the plan defined within it`,
21332154
synthetic: true,
2155+
...(nonAnthropic ? { ignored: true } : {}),
21342156
})
21352157
userMessage.parts.push(part)
21362158
trustedReminderParts.push(part as MessageV2.TextPart)
@@ -2235,6 +2257,7 @@ This is critical - your turn should only end with either asking the user a quest
22352257
NOTE: At any point in time through this workflow you should feel free to ask the user questions or clarifications. Don't make large assumptions about user intent. The goal is to present a well researched plan to the user, and tie any loose ends before implementation begins.
22362258
</system-reminder>`,
22372259
synthetic: true,
2260+
...(nonAnthropic ? { ignored: true } : {}),
22382261
})
22392262
userMessage.parts.push(part)
22402263
trustedReminderParts.push(part as MessageV2.TextPart)

‎packages/opencode/src/session/system.ts‎

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ import PROMPT_ANTHROPIC from "./prompt/anthropic.txt"
1010
import PROMPT_ANTHROPIC_WITHOUT_TODO from "./prompt/qwen.txt"
1111
import PROMPT_BEAST from "./prompt/beast.txt"
1212
import PROMPT_GEMINI from "./prompt/gemini.txt"
13+
// altimate_change start — shared family→vendor classifier (see #888 J1)
14+
import { familyVendor } from "../provider/family"
15+
// altimate_change end
1316

1417
import PROMPT_CODEX from "./prompt/codex_header.txt"
1518
import PROMPT_TRINITY from "./prompt/trinity.txt"
@@ -33,16 +36,23 @@ export namespace SystemPrompt {
3336
// before the api.id string-match fallthrough. The gateway's model.api.id is
3437
// the opaque alias `altimate-default` (kept stable for backward compat —
3538
// users persist it in model.json), which matches none of the patterns below.
36-
// Defense in depth: without this guard a future provider misconfiguration
37-
// could route an OpenAI-backed gateway model to PROMPT_ANTHROPIC_WITHOUT_TODO.
38-
// Family lookup is case-insensitive since `family` is a free-form string in
39-
// the Model schema. Unknown families default to PROMPT_CODEX because the
40-
// gateway is registered as `@ai-sdk/openai-compatible`.
39+
// Use the shared `familyVendor` classifier so specific family values
40+
// (`claude-sonnet`, `gemini-pro`, `gpt-codex`, …) map to the right vendor.
41+
// An exact `family === "anthropic"` check would silently fall through to
42+
// PROMPT_CODEX on any altimate-backend gateway path that exposes a Claude
43+
// model with the specific family — recreating the #887 misrouting class
44+
// this PR is meant to fix (see #888 J1).
45+
// Unknown vendors default to PROMPT_CODEX because the gateway is registered
46+
// as `@ai-sdk/openai-compatible`.
4147
if (model.providerID === "altimate-backend") {
42-
const family = model.family?.toLowerCase()
43-
if (family === "anthropic") return [PROMPT_ANTHROPIC]
44-
if (family === "gemini") return [PROMPT_GEMINI]
45-
return [PROMPT_CODEX]
48+
switch (familyVendor(model.family)) {
49+
case "anthropic":
50+
return [PROMPT_ANTHROPIC]
51+
case "gemini":
52+
return [PROMPT_GEMINI]
53+
default:
54+
return [PROMPT_CODEX]
55+
}
4656
}
4757
// altimate_change end
4858
if (model.api.id.includes("gpt-5")) return [PROMPT_CODEX]
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
import { describe, expect, test } from "bun:test"
2+
import { familyVendor } from "../../src/provider/family"
3+
4+
// Locks the contract for the shared family→vendor classifier introduced for
5+
// #888 J1. The model registry uses *specific* family values like
6+
// `claude-sonnet`, `claude-haiku`, `gemini-pro`, `gemini-flash`, etc., not the
7+
// coarse `anthropic`/`gemini`/`openai` literals — so exact-match logic in the
8+
// previous version silently fell through to PROMPT_CODEX on any altimate-
9+
// backend gateway path that exposed a Claude model.
10+
describe("familyVendor", () => {
11+
test.each([
12+
// Coarse vendor literals.
13+
["anthropic", "anthropic"],
14+
["openai", "openai"],
15+
["openai-compatible", "openai"],
16+
["gemini", "gemini"],
17+
["claude", "anthropic"],
18+
["gpt", "openai"],
19+
// Specific names that the model registry actually emits.
20+
["claude-sonnet", "anthropic"],
21+
["claude-haiku", "anthropic"],
22+
["claude-opus", "anthropic"],
23+
["gemini-pro", "gemini"],
24+
["gemini-flash", "gemini"],
25+
["gemini-flash-lite", "gemini"],
26+
["gpt-codex", "openai"],
27+
["gpt-codex-mini", "openai"],
28+
// Case-insensitivity.
29+
["ANTHROPIC", "anthropic"],
30+
["Claude-Sonnet", "anthropic"],
31+
["GEMINI-PRO", "gemini"],
32+
["GPT", "openai"],
33+
] as const)("classifies %p as %p", (family, expected) => {
34+
expect(familyVendor(family)).toBe(expected)
35+
})
36+
37+
test.each([undefined, "", "unknown-future-family", "kimi", "qwen", "llama"])("returns undefined for %p", (family) => {
38+
expect(familyVendor(family)).toBeUndefined()
39+
})
40+
})

‎packages/opencode/test/session/plan-layer-e2e.test.ts‎

Lines changed: 45 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -472,15 +472,10 @@ describe("plan prompt safety", () => {
472472
"utf-8",
473473
)
474474
// Must consume the trusted-parts contract.
475-
expect(promptTs).toMatch(/trustedReminderParts/)
476-
// The hoist loop must iterate the returned trusted parts.
477-
expect(promptTs).toMatch(/for \(const part of reminderResult\.trustedReminderParts\)/)
475+
expect(promptTs).toMatch(/reminderResult\.trustedReminderParts/)
478476
// The hoist must NOT have been re-implemented as a scan over `part.synthetic`,
479477
// which would re-open the prompt-injection vector via attached file content.
480478
expect(promptTs).not.toMatch(/hoistSyntheticReminders/)
481-
// The loose `if (!part.synthetic) continue` shape must not reappear in any
482-
// trust-boundary context. The remaining `synthetic` usages should only be
483-
// setters or the unrelated text-wrap skip at the queued-message reminder.
484479
})
485480

486481
test("insertReminders return shape includes trustedReminderParts", async () => {
@@ -495,6 +490,50 @@ describe("plan prompt safety", () => {
495490
/type InsertRemindersResult = \{ messages: MessageV2\.WithParts\[\]; trustedReminderParts: MessageV2\.TextPart\[\] \}/,
496491
)
497492
})
493+
494+
// #888 J2 regression guard. Persisted experimental-mode reminders must carry
495+
// `ignored: true` for non-Anthropic models — otherwise on turn 2+ they reload
496+
// from DB with `ignored: undefined`, slip into the user-role payload via
497+
// `toModelMessages` (which gates on `!part.ignored`), and re-trigger the GPT-5.x
498+
// refusal class this PR was meant to fix.
499+
test("insertReminders bakes `ignored: true` into trusted reminders for non-Anthropic models", async () => {
500+
const promptTs = await fs.readFile(
501+
path.join(__dirname, "../../src/session/prompt.ts"),
502+
"utf-8",
503+
)
504+
// The function must take `model` so it can compute the hoist decision once.
505+
expect(promptTs).toMatch(/async function insertReminders\(input:\s*\{[\s\S]*?model: Provider\.Model/)
506+
// The `nonAnthropic` decision must drive an `ignored: true` annotation on
507+
// each trusted part (and on the Session.updatePart payload for experimental
508+
// persisted rows).
509+
expect(promptTs).toMatch(/const nonAnthropic = !isAnthropicLikeModel\(input\.model\)/)
510+
// At least one persisted-write site must include the conditional `ignored: true`.
511+
const nonAnthropicIgnored = promptTs.match(/nonAnthropic \? \{ ignored: true \} : \{\}/g)
512+
expect((nonAnthropicIgnored ?? []).length).toBeGreaterThanOrEqual(4)
513+
// And the caller must no longer mutate `part.ignored = true` — that contract
514+
// is now owned by insertReminders.
515+
expect(promptTs).not.toMatch(/part\.ignored = true/)
516+
})
517+
518+
// #888 J1 regression guard. `family` is a free-form string and the model
519+
// registry uses specific values (`claude-sonnet`, `gemini-pro`, …); an exact
520+
// `family === "anthropic"` match would silently fall through to PROMPT_CODEX
521+
// on any altimate-backend gateway path exposing a Claude/Gemini model.
522+
test("family routing uses the shared familyVendor helper, not exact-match literals", async () => {
523+
const systemTs = await fs.readFile(
524+
path.join(__dirname, "../../src/session/system.ts"),
525+
"utf-8",
526+
)
527+
const promptTs = await fs.readFile(
528+
path.join(__dirname, "../../src/session/prompt.ts"),
529+
"utf-8",
530+
)
531+
expect(systemTs).toMatch(/familyVendor\(model\.family\)/)
532+
expect(promptTs).toMatch(/familyVendor\(model\.family\)/)
533+
// Exact-match `family === "anthropic"` shape must not reappear in the
534+
// altimate-backend routing block.
535+
expect(systemTs).not.toMatch(/model\.providerID === "altimate-backend"[\s\S]{0,200}family === "anthropic"/)
536+
})
498537
})
499538

500539
// ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)