One-off prompts (goal reminder, keep-alive, compaction prompt) take the cache breakpoint and break the cached prefix #62554

Activity

1. opencodejr commented on Nov 7, 2026

   

   opencodejrBot

   on Nov 7, 2026

   Contributor

   More actions

   Thanks for the detailed report. I reproduced the goal reminder part on dev at 21bf6b4, which is your commit and also the latest dev.

   The script below builds Anthropic request bodies through the normal request path with the default cache policy. It adds the goal reminder the same way the context hook in packages/core/src/plugin/goal.ts does, across consecutive agent steps after a compaction:

   • Step 1: [summary, ok, REMINDER, prompt*]. The reminder sits before the prompt (* marks the tail breakpoint).
   • Step 2: [summary, ok, prompt, tool_use, tool_result, REMINDER*]. It no longer matches step 1 from the third message, and the tail breakpoint is on the reminder.
   • Step 3: shares only the first five messages with step 2, so step 2's tail cache entry is never read.

   The hook's own comment says this placement is meant to keep the cached prefix warm, so this is a bug, not intended behaviour. It's a code-level reproduction, not a run against Anthropic: there were no credentials here. I also didn't separately check the Plan reminder, the session.generate prompts or the compaction transcript, but they go through the same tail-breakpoint logic.

   Related: #61775, #54107, #56422.

   Reproduction script (repro-62554.test.ts)

   // Reproduction for anomalyco/opencode#62554: the goal plugin's request-only reminder takes the
   // conversation-tail cache breakpoint, and moves between steps, so no later request reads that entry.
   // Copy into packages/ai/test/ of a dev checkout, then from packages/ai: bun test test/repro-62554.test.ts
   import { expect, test } from "bun:test"
   import { Effect } from "effect"
   import { ai, Message, ToolCallPart, ToolResultPart } from "../src/effect.js"
   import { Auth } from "../src/route.js"
   import { compileRequest } from "../src/route/client.js"
   import * as AnthropicMessages from "../src/protocols/anthropic-messages.js"
   
   const model = AnthropicMessages.route
     .with({ endpoint: { baseURL: "https://api.anthropic.test/v1/" }, auth: Auth.header("x-api-key", "test") })
     .model({ id: "claude-sonnet-4-5" })
   
   const REMINDER = "<system-reminder>\nA goal is active for this session. Keep working toward it.\n...</system-reminder>"
   
   // Same placement as the `context` hook in packages/core/src/plugin/goal.ts
   const withGoalReminder = (messages: Message[]) => {
     const out = messages.slice()
     const at = out.at(-1)?.role === "user" ? out.length - 1 : out.length
     out.splice(at, 0, Message.user(REMINDER))
     return out
   }
   
   const compile = (messages: Message[]) =>
     Effect.runPromise(compileRequest(ai.request({ model, system: "sys", messages })).pipe(Effect.map((p) => p.body as any)))
   
   const summary = (body: any) =>
     body.messages.map((m: any, i: number) => {
       const marked = m.content.some((c: any) => c.cache_control)
       const text = m.content.map((c: any) => c.type + ":" + JSON.stringify(c.text ?? c.content ?? c.name ?? "").slice(0, 30)).join(",")
       return `${i} ${m.role} ${marked ? "[BREAKPOINT] " : ""}${text}`
     })
   
   test("goal reminder takes the tail breakpoint and the next step diverges before it", async () => {
     const history = [Message.user("<compaction summary>"), Message.assistant("ok")]
     const prompt = Message.user("run the tests")
     const call = (id: string) => Message.assistant([ToolCallPart.make({ id, name: "bash", input: { cmd: "ls" } })])
     const result = (id: string) => Message.tool(ToolResultPart.make({ id, name: "bash", result: "out " + id }))
   
     const step1 = await compile(withGoalReminder([...history, prompt]))
     const step2 = await compile(withGoalReminder([...history, prompt, call("c1"), result("c1")]))
     const step3 = await compile(withGoalReminder([...history, prompt, call("c1"), result("c1"), call("c2"), result("c2")]))
     for (const [n, b] of [["step1", step1], ["step2", step2], ["step3", step3]] as const)
       console.log(n + "\n  " + summary(b).join("\n  "))
   
     // Where the tail breakpoint lands on each step
     const tail = (b: any) => b.messages.findLastIndex((m: any) => m.content.some((c: any) => c.cache_control))
     const isReminder = (b: any, i: number) => JSON.stringify(b.messages[i]).includes("A goal is active")
     console.log("tail breakpoint on reminder:", isReminder(step2, tail(step2)), isReminder(step3, tail(step3)))
   
     // Longest common message prefix between step N's cached prefix (up to its tail breakpoint) and step N+1
     const strip = (m: any) => JSON.stringify({ ...m, content: m.content.map(({ cache_control, ...c }: any) => c) })
     const common = (a: any, b: any) => {
       let i = 0
       while (i < a.messages.length && i < b.messages.length && strip(a.messages[i]) === strip(b.messages[i])) i++
       return i
     }
     console.log("step2 cached up to msg", tail(step2), "; step3 matches step2 only for msgs [0,", common(step2, step3), ")")
     expect(isReminder(step2, tail(step2))).toBe(true)
     expect(common(step2, step3)).toBeLessThanOrEqual(tail(step2)) // step2's tail entry can never be read by step3
   })

   Run it: bun repro-62554.test.ts
2. opencodejr

   added

   reproduced

   on Nov 7, 2026
3. hugokindel commented on Nov 12, 2026

   

   hugokindel

   on Nov 12, 2026

   More actions

   Confirming this on a real Anthropic workload (Claude Opus through a gateway, v2.2.7). One /goal session ran for about 22 hours after a compaction with the reminder active. 2,016 of its 2,510 requests wrote 200-300k tokens to the cache within seconds of the previous request, while cache reads stopped at the ~75k token tools and system prefix. Our daily Opus cache writes went from about 2% of input in late October to 48-75% over the last two days, and it drained a subscription's allowance in a few hours

   As a stopgap we moved the reminder to the head of the history (event.messages.unshift(...)), and the next requests read the whole 300k+ history from the cache again. Your request-local approach is better though, since a head reminder still invalidates everything once when it goes away. Happy to see it land
4. swf-cmd commented on Nov 12, 2026

   

   swf-cmd

   on Nov 12, 2026

   Author

   More actions

   Thanks for the numbers. That is the case from the steps above: while the reminder sits at the tail of every request, the only cache entries a later request can read are the tools and system ones, which matches your reads stopping at about 75k.

   I've brought the branch up to date with dev (9125a540b), in case you want to run it instead of the stopgap. The PR still needs one of my three open-PR slots and will show up here once it's open. If you do run the branch, cache read and write numbers from a real session would help: I've only checked where the breakpoints land in compiled requests, not billed usage.