question(llm): does the auto cache policy leave the growing transcript unanchored? #39679

Reading packages/llm/src/schema/options.ts at ff0382e, the auto cache policy documents its reasoning:

"auto" is the recommended default for agent loops — it places one breakpoint at the last tool definition, one at the last system part, and one at the latest user message. The combination of provider invalidation hierarchy (tools → system → messages) and Anthropic/Bedrock's 20-block lookback means three trailing breakpoints reliably cover the static prefix.

I think that reasoning is correct about the static prefix, and I want to ask about the part that isn't static.

The question

When a turn appends more than 20 content blocks — a wide parallel tool call, say 24 tool_result blocks — the lookback from the breakpoint at the latest user message can no longer reach the write from the previous request, because that write is now more than 20 blocks behind. The read then falls back to the nearest boundary that does hit, which is the system part. Everything between the system prefix and the tip gets re-written at the write premium.

The static prefix is protected exactly as the comment says. But the transcript between it and the tip grows, so each overrun re-charges a larger region than the last. Placing that third breakpoint at the previous user message instead of the latest one bounds the re-write to the new blocks.

What I measured

I transcribed the auto placement into a small replay rig and billed it against the same layout with the third breakpoint moved back one user turn. Byte-identical content in both arms, 12 turns, 4 of them appending 24 blocks, anthropic/claude-sonnet-5 via OpenRouter pinned to Anthropic, 2026-07-30:

That is 1.43×, on identical bytes, with no accuracy dimension. On the overrunning turns specifically, cached sat at the static prefix while written tokens grew turn over turn (2,265 → 4,334 → 6,405 → 8,474), rather than staying flat at the size of the new blocks.

An internal contrast that may be the more useful part

applyCaching() in packages/opencode/src/provider/transform.ts selects msgs.filter(m => m.role !== "system").slice(-2). Because that marks the last two non-system messages, one marker lands adjacent to the previous request's write position, so the lookback reaches it. I measured that layout at $0.075360 — within 0.2% of the anchored arm.

So the older path appears to get this right as a side effect of slice(-2), while the newer documented policy may not. If that reading is correct, the auto default could be losing something transform.ts already had.

Caveats, because these matter here

• I transcribed the policy into my own rig rather than running opencode. A transcription error would show up as a finding about opencode, and I'd rather be corrected than believed. If the lowering step does something I've missed — e.g. injecting an additional hint I didn't model — that would explain the gap.
• Wide-turn frequency is my choice and it is the dominant lever on the ratio. At one wide turn in twelve the two layouts are within 3%.
• One provider, one model, one day.

So, concretely

Is the latest user message placement deliberate — for instance to guarantee a write at the tip so the next request has something to read — and if so, is the growing-transcript case considered and accepted? Or would moving that third breakpoint back one user turn (or spending a fourth on the previous tip, since Anthropic allows four) be a change you'd take?

Happy to share the rig if it's useful, and happy to be told I've misread the lowering step.