Skip to content

reasoning_effort participates in the prefix cache key: a compaction summary sent with effort="none" reads 1.2–8% cache — and 99.9% once that single field matches #55156

Closed as not planned

Description

@echopharmsu

Description

On the Go route a single compaction costs ~970k uncached input tokens, and ~250k of that is
avoidable and comes from one field: the summarization call is sent with
reasoning_effort: "none", while every main request in the same session uses "max".

Nothing else differs: same x-opencode-session, same upstream endpoint, same request-header name
set (12 names), and the replayed prefix is byte-identical to the previous main request
(verified at 64/256/512/768/1024 KiB); that main request was itself reading 99.6% from cache.

Patching only that field (byte-preserving — the body is never re-serialized) took the next
compaction from 1.2%–8.0% cache read to 99.9% — uncached input 227,606–280,919 -> 474.
It also restores the summary's reasoning, which was disabled (reasoningTokens = 0 on all four
pre-fix compactions).

Ask: (1) client side — auxiliary calls should carry an explicit reasoning effort instead of
resolving to the provider's "off" value; (2) gateway side — please document that
reasoning_effort participates in the prefix cache key (or make it not).

Sample sizes, up front: the "after" side is a single compaction; the "before" side is four
compactions across three sessions and eleven hours. Direction unambiguous; n is small.

Plugins

None. This is not the OpenCode CLI/TUI: it is a custom (third-party) harness that sends requests directly to your Go gateway endpoint.

OpenCode version

n/a — requests go straight to https://opencode.ai/zen/go/v1 (Go route); no client version involved.

Steps to reproduce

All numbers below are per-request usage taken from the harness's own session records
(`inputTokens` / `cacheReadTokens` on `compaction/prune`, `compaction/summary`, and the following
main request), cross-checked against the provider console, which reports the same numbers to the
token. No server-side access is needed to reproduce any of it.
Expand: the three bills, 24 summarization calls, every eliminated hypothesis, and the raw A/B numbers
**Method**: per-request usage from the harness's own session records (`inputTokens` /
`cacheReadTokens` on `compaction/summary`), cross-checked against the provider's console, which
reports the same numbers to the token. No server-side access is needed to reproduce the finding;
server-side access is only needed to see how far the effect goes.

**The three bills of one compaction** (so the magnitude is clear):

    1. the pre-prune invalidation  — the first main request after `compaction/prune` re-reads the
                                     whole prefix                              → 465,963 tokens
    2. the summarization call      — as above: 1.2%–8.0% cache read, fixable
                                     by setting one field                      → 227,606 – 280,919
    3. the post-swap request       — new prefix, inherent                     → 157,295 – 222,821
    --------------------------------------------------------------------------------------------------
    ≈ 970k uncached tokens per compaction, of which ≈250k is the field described here

**Bill 1 — the pruning step invalidates the prefix (this one is expected, and it is the bigger half)**

    2026-10-13 12:53:07  compaction/prune × 11  (all in the same second; shadowedTokenCount sums to 44,280)
      shadowed seqs: 41, 43, 53, 66, 106, 141, 315, 327, 351, 452, 781
    first main request after the prune:  input 465,963 / cache_read 50,688  (9.8%)
    session's normal per-request uncached: ~2,000  →  this one is 230× normal
    (this prune alone brought the session from 525,386 back under the 550,464 threshold, so **no
     compaction followed** — i.e. pruning is doing its job; the cost is the cache invalidation)

**Bill 2 — the summarization call misses cache (22 of 24 observed)**

    00:59:58  compaction/start
    01:00:39  compaction/summary   input 306,611 / cache_read 0        (0.0%)
    01:00:39  compaction/end       output 3,543, 41 s wall
    first main request after the swap: input 157,295 / cache_read 24,576 (13.5%)

    12:36:10  compaction/start     (different session)
    12:36:37  compaction/summary   input 227,606 / cache_read 11,520   (4.8%)

    13:37:00  compaction/start     (third session, 550k-token context)
    13:37:38  compaction/summary   input 280,919 / cache_read 24,576   (8.0%)  ← the case we measured most fully
    13:37:39  next main request    input 160,501 / cache_read 24,576   (13.3%)
    13:37:54  the one after        input   1,216 / cache_read 184,960  (99.3%) ← normal service resumes

Across 24 summarization calls / 9 sessions:
    cache_read ≈ 0%     22 calls  (input 100k–550k each)
    cache_read ≈ 100%    2 calls  (input 124 and 227 tokens — only the appended instruction was new)

**What we checked and ruled out** (so you do not have to re-derive it):

    · same x-opencode-session value as the main requests                      ✓
    · same upstream endpoint (x-opencode-endpoint-id)                         ✓
    · same provider and same model as the main requests                       ✓
    · HTTP request-header *name* set identical (12 names, neither side extra) ✓
    · the summarization call's body **is a byte-identical prefix** of the previous main request's
      — verified at 64 / 256 / 512 / 768 / 1024 KiB (its body is 1,234 KiB, so the verified part
      covers essentially all of it)                                           ✓
    · the preceding main request was itself reading 548,608 of 550,616 tokens from cache (99.6%) ✓

So the prompt was right, the endpoint was right, the headers were right, and the cache was warm.
What differed was **one field of the request**:

    reasoning_effort:   main requests        → "max"
                        summarization call   → "none"

**The test** — we patched that one field (byte-preserving: only the value is replaced, the body is
never re-serialized) and re-measured:

    before (4 compactions, "none"):  cache read  2,944 / 11,520 / 24,576 / 24,576   (1.2% – 8.0%)
                                     uncached input 227,606 – 280,919
    after  (1 compaction,  "max"):   cache read  390,528 of 391,002  →  **99.9%**
                                     uncached input **474**

Everything else was held constant: same session, same key, same model, same endpoint, same prefix.
**⇒ `reasoning_effort` participates in the prefix cache key.** Changing it invalidates the prefix
even though the token sequence is the same — which is also why the pre-fix compactions kept hitting
only ~24k: that is the shared head (system prompt + tool definitions) both requests have in common.

**Two explanations we proposed ourselves and then ruled out** (stated because a plausible wrong fix
is worse than no fix):

    · "prune first, summarize second" — we hypothesised that issuing the summary *before* the pruning
      would let it replay a cached prefix. It does not help: the prefix is already byte-identical
      (measured above), so the ordering is not what diverges.
    · "the provider does not extend its cache to auxiliary calls" — it does. The auxiliary call
      reuses the shared head fine; it is the **key**, not the nature of the call, that differs.

Screenshot and/or share link

No response

Operating System

Windows 11

Terminal

Deepseek harness, not a terminal client.

Activity

  1. opencode-agent commented on Oct 13, 2026

    @opencode-agent
    Contributor

    Thanks for the careful measurements. The A/B on reasoning_effort (1.2–8% cache read with "none" against 99.9% with "max", same prefix, same session) is very clear.

    This repository's issues cover the OpenCode app itself. You're calling the OpenCode Go gateway directly from your own harness, so how its prompt cache is keyed and billed is something only the team can look into or document. Billing and account support isn't handled in GitHub issues. Please email [email protected] with the details (you can link this issue). If you've already emailed, the team will get to it, especially now that it's been seen here.

    Please keep account information, API keys, email addresses and any payment details out of this public issue and put them in the email instead.

    For reference, these OpenCode app issues are about compaction and prompt caching. They're related but not the same problem:

    The bot will close this issue in 1 day.

  2. opencode-agent commented on Oct 14, 2026

    @opencode-agent
    Contributor

    Closing this now, @echopharmsu. Questions about how the OpenCode Go gateway caches and bills requests are handled by email, not in GitHub issues. If the reasoning_effort cache-key question is still open for you, please email [email protected] and link this issue. Please keep account and payment details out of this thread and put them in the email instead. Thanks again for the detailed write-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions