Skip to content

fix(ai): agent lane drops OpenRouter cache-write tokens and under-counts context#111435

Merged
steipete merged 3 commits into
openclaw:mainfrom
Yigtwxx:fix/openrouter-usage-accounting-fidelity
Jul 21, 2026
Merged

fix(ai): agent lane drops OpenRouter cache-write tokens and under-counts context#111435
steipete merged 3 commits into
openclaw:mainfrom
Yigtwxx:fix/openrouter-usage-accounting-fidelity

Conversation

@Yigtwxx

@Yigtwxx Yigtwxx commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Related: #9016

AI-assisted (Claude Code). Every claim below is backed by a command I ran; base-vs-head comparisons were made by stashing the branch.

Scope narrowed after review. This branch previously added an optional public Usage.cost.estimatedTotal field. ClawSweeper flagged it three review cycles running: the field crossed both the plugin SDK and the closed worker transcript schema, which makes it an upgrade contract rather than a bug fix. I agree, and measurement made the case stronger than the review did — see Why the retained-estimate field was removed. The PR is now two defects across four files, with no public API or protocol change.

What Problem This Solves

Two defects in OpenAI-compatible usage accounting on the lane that serves live OpenRouter agent turns.

1. The agent lane drops OpenRouter cache-write tokens

parseTransportChunkUsage hard-coded cacheWrite: 0 and never read prompt_tokens_details.cache_write_tokens. Its sibling parseChunkUsage (packages/ai/src/providers/openai-completions.ts:1354) reads it correctly, and that sibling's own comment states the contract:

Follow documented OpenAI/OpenRouter semantics: cached_tokens is cache-read tokens (hits). OpenAI does not document or emit cache_write_tokens, but OpenRouter-compatible providers can include it as a separate write count. […] Do not subtract writes from cached_tokens, otherwise spec-compliant providers are under-reported.

So the correct behavior was already specified in-repo; the agent lane never adopted it. Write tokens were dropped from cacheWrite and wrongly counted inside input.

This is the lane OpenRouter uses. extensions/openrouter/src/provider-catalog.ts:58 declares api: "openai-completions"; src/agents/provider-transport-stream.ts:89-90 routes that to createOpenAICompletionsTransportStreamFn(), i.e. src/agents/openai-completions-transport.ts.

2. Context-overflow detection under-counts the same tokens

resolveContextInputTokens (packages/ai/src/utils/overflow.ts:91) fell back to input + cacheRead. That was accidentally exact only while input still absorbed cache-write tokens. Fixing #1 without this would have shipped a regression, so it lands here.

The completions lane never populates contextUsage (zero occurrences in that file), so it always takes this fallback. The consumers are control decisions, not display: silent-overflow (overflow.ts:158) and the Xiaomi-MiMo length-stop check (overflow.ts:168), reached from src/agents/sessions/agent-session-compaction.ts:279 and agent-session-execution.ts:28. The length-stop test uses a 1% band, which is often narrower than a single cache-write block.

The Anthropic lane already gets this right — packages/ai/src/providers/anthropic-usage.ts:96 computes input + cacheRead + cacheWrite. It populates contextUsage and so bypasses the fallback, which is why only the completions lane was exposed. The fix aligns the fallback with that sibling and is a strict improvement for every lane.

Why the retained-estimate field was removed

The earlier revision added estimatedTotal to hold the catalog estimate that a provider-billed total replaced, because applyProviderReportedUsageCost overwrites only total and leaves the four components disagreeing with it.

Two findings retired that approach:

  1. Nothing read it. The field was written in exactly one place (model-utils.ts) and forwarded through four transport layers — SDK type, worker protocol schema, transcript commit, replay normalization — with no consumer anywhere in the tree. It was a write-only surface, and a permanent one, since it crossed a published SDK and a persisted wire schema.
  2. It did not fix the invariant it was introduced for. input + output + cacheRead + cacheWrite still disagrees with a provider-billed total after the change; the field only parked the discarded estimate beside the disagreement. The desync is real, but reconciling it means deciding what cost.total means when a provider bills a number the components cannot reproduce — an owner-level decision about the public usage contract, which is exactly the call ClawSweeper said a mechanical repair should not make.

Both fixes that remain carry independent runtime evidence and touch no exported type, so they stand on their own. The desync is left for #9016 follow-up, where the field can arrive with a consumer and an owner decision instead of ahead of both.

User Impact

  • OpenRouter cache-write tokens are no longer dropped on agent turns and no longer inflate input.
  • Context-overflow and length-stop detection count the full prompt, so truncating providers that report cache writes are no longer missed.

No config, schema migration, or operator action. No public API or protocol change. Free-model zero-cost handling is untouched.

Evidence

Mock-free runtime proof — real socket, real transport, no provider account

A real node:http server on 127.0.0.1 serves an OpenRouter-shaped SSE stream whose final chunk carries prompt_tokens_details.cache_write_tokens. The real createOpenAICompletionsTransportStreamFn() consumes it over a real socket, and the real public isContextOverflow scores the resulting message. No vi.mock, no fake timers, no stubbed transport, no provider account.

Usage prompt_tokens: 100, completion_tokens: 50, cached_tokens: 20, cache_write_tokens: 10. The context window is set to 95 so the control decision is observable rather than an internal number: the correct prompt count is 100, and a cache-write mapping without the overflow fix would count 90.

I ran three configurations by checking out each file individually:

base (origin/main) transport fix only head (both fixes)
input 80 — inflated by the writes 70 70
cacheRead 20 20 20
cacheWrite 0 — 10 tokens dropped 10 10
totalTokens 150 150 150
isContextOverflow(msg, 95) true false true

The middle column is why overflow.ts belongs in this PR rather than a follow-up. Landing the cache-write mapping alone flips a real overflow to undetected: base only reaches the correct count because input wrongly absorbed the writes, so correcting the mapping without correcting the fallback silently shrinks every overflow and length-stop decision on this lane. The two changes are one fix.

Focused tests

packages/ai/src/utils/overflow.test.ts                          8 passed
src/agents/openai-transport-stream.streaming.test.ts           44 passed

Full local run after merging current main: both suites green, node scripts/run-tsgo.mjs clean.

Negative controls

Each line reverted individually; exactly the named test reddens.

Reverted Test that fails
cacheWrite: cacheWriteTokens0 maps cache_write_tokens as a separate write count
removing - cacheWriteTokens from input same test — both edits carry independent weight
+ message.usage.cacheWrite in overflow.ts counts cache-write tokens toward the context…

Static checks

oxfmt --check clean · oxlint clean · git diff --check clean · max-lines ratchet OK · both extension-import-boundary checks return [] · repo autoreview clean.

The generated plugin-SDK API baseline is no longer touched by this branch — with estimatedTotal gone, docs/.generated/plugin-sdk-api-baseline.sha256 is byte-identical to main. That also removed the merge conflict this PR had picked up.

Scope

Two defects, four files, +52/-6. The remaining #9016 work — carrying provider-reported cost and cost_details.upstream_inference_cost into session state, agent runtime metadata, and session_status, plus the breakdown-reconciliation question above — is follow-up I'd rather bring with its consuming surface than land as write-only fields.

@openclaw-barnacle openclaw-barnacle Bot added app: web-ui App: web-ui gateway Gateway runtime agents Agent runtime and tooling size: M labels Jul 19, 2026
@openclaw-barnacle openclaw-barnacle Bot added the docs Improvements or additions to documentation label Jul 19, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jul 19, 2026
@clawsweeper

clawsweeper Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 20, 2026, 3:34 PM ET / 19:34 UTC.

Summary
The branch maps OpenRouter-compatible cache_write_tokens into the OpenAI-completions agent transport and counts that bucket in fallback context-overflow calculations.

PR surface: Source +14, Tests +32. Total +46 across 4 files.

Reproducibility: yes. at source level: current main drops the separate cache-write field in the agent transport, and the PR supplies a concrete real-SSE reproduction that makes the overflow decision differ before versus after the paired fix.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: persistent cache schema: packages/ai/src/utils/overflow.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

  • No automated repair is indicated because the review found no discrete defect in the proposed patch; the remaining action is ordinary maintainer review and exact-head merge gating.

Security
Cleared: The four-file patch changes local usage parsing, arithmetic, and tests only; it adds no dependency, workflow, secret, permission, package-resolution, or execution surface.

Review details

Best possible solution:

Land the focused transport and overflow-accounting fix after the exact-head required merge gates complete, while leaving provider-billed cost-breakdown semantics to the distinct product discussion in #9016.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main drops the separate cache-write field in the agent transport, and the PR supplies a concrete real-SSE reproduction that makes the overflow decision differ before versus after the paired fix.

Is this the best way to solve the issue?

Yes. Aligning the agent transport with the existing OpenAI-completions sibling contract and correcting the dependent overflow fallback is narrower and safer than adding a separate usage or cost API.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against e2bb04328f5c.

Label changes

Label justifications:

  • P2: The patch fixes incorrect token accounting and context-overflow control decisions for OpenRouter-compatible agent turns, with bounded rather than emergency blast radius.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR includes a concrete after-fix real socket/SSE transport exercise with an observable overflow outcome, plus focused regression coverage and negative controls; redact any private endpoint or credential details if this evidence is reposted.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR includes a concrete after-fix real socket/SSE transport exercise with an observable overflow outcome, plus focused regression coverage and negative controls; redact any private endpoint or credential details if this evidence is reposted.
Evidence reviewed

PR surface:

Source +14, Tests +32. Total +46 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 20 6 +14
Tests 2 32 0 +32
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 52 6 +46

What I checked:

  • Current-main transport behavior: Current main derives input as prompt_tokens - cached_tokens, hard-codes cacheWrite: 0, and omits cache writes from totalTokens; the PR changes exactly this agent-stream accounting path. (src/agents/openai-completions-transport.ts:1889, e2bb04328f5c)
  • Sibling contract: The plugin-SDK OpenAI-completions provider already reads prompt_tokens_details.cache_write_tokens, removes it from input, adds it to cacheWrite, and includes it in totalTokens; the PR aligns the agent transport with that established contract. (packages/ai/src/providers/openai-completions.ts:1354, e2bb04328f5c)
  • Overflow control-path coverage: The PR updates the fallback used when contextUsage is unavailable so cache-write tokens participate in both silent-overflow and length-stop decisions, with a focused regression test for that fallback. (packages/ai/src/utils/overflow.ts:91, e13518506a6f)
  • After-fix behavior proof: The PR body documents an after-fix real socket/SSE transport run showing the corrected cache buckets and restoring an observable context-overflow decision; it also records focused test results and negative controls. (src/agents/openai-transport-stream.streaming.test.ts:692, e13518506a6f)
  • Re-review continuity: The prior completed ClawSweeper cycle at baf68ebb found no remaining review findings after the public usage-contract surface was removed; the current head only refreshes the branch against main. (src/agents/openai-completions-transport.ts:1889, baf68ebb7431)

Likely related people:

  • unknown: The relevant current-main transport and sibling-provider history could not be attributed to a named person from the available read-only review evidence; route through the agents/AI ownership path rather than treating the external PR author as code owner. (role: current-main area owner unresolved; confidence: low; files: src/agents/openai-completions-transport.ts, packages/ai/src/providers/openai-completions.ts, packages/ai/src/utils/overflow.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (8 earlier review cycles)
  • reviewed 2026-07-19T16:28:24.842Z sha 02f7dec :: found issues before merge. :: [P1] Obtain approval for the new public cost contract
  • reviewed 2026-07-19T17:10:41.968Z sha 02f7dec :: found issues before merge. :: [P1] Obtain approval for the new public cost contract
  • reviewed 2026-07-19T17:21:24.155Z sha 02f7dec :: found issues before merge. :: [P1] Obtain approval for the public two-total cost contract
  • reviewed 2026-07-19T17:45:52.007Z sha 02f7dec :: found issues before merge. :: [P1] Obtain approval for the public two-total cost contract
  • reviewed 2026-07-19T18:41:52.806Z sha 8d18cb5 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-19T18:47:31.265Z sha 8d18cb5 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-19T19:29:44.514Z sha 8d18cb5 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-19T19:38:22.038Z sha baf68eb :: needs maintainer review before merge. :: none

@openclaw-barnacle openclaw-barnacle Bot added size: S and removed docs Improvements or additions to documentation app: web-ui App: web-ui gateway Gateway runtime size: M labels Jul 19, 2026
@Yigtwxx Yigtwxx changed the title fix(ai): provider-billed totals desync the cost breakdown and drop cache writes fix(ai): agent lane drops OpenRouter cache-write tokens and under-counts context Jul 19, 2026
@Yigtwxx

Yigtwxx commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

Narrowed the scope to take the review's second option.

The blocking P1 was right, and measuring it made the case stronger than the review did. estimatedTotal was written in exactly one place and forwarded through four layers — SDK type, worker protocol schema, transcript commit, replay normalization — with no reader anywhere in the tree. It also did not repair the invariant it existed for: input + output + cacheRead + cacheWrite still disagrees with a provider-billed total; the field only parked the discarded estimate beside the disagreement. A permanent public contract that nothing consumes, and that does not close the gap it was added for, is not worth an owner decision — so I removed it rather than asking for one.

That leaves two defects with independent runtime evidence and no exported type or protocol change: 16 files → 4, +307/-9+52/-6. docs/.generated/plugin-sdk-api-baseline.sha256 is now byte-identical to main, which also cleared the merge conflict this branch had picked up.

I re-ran the mock-free proof for the narrowed scope — real node:http socket, real completions transport, real public isContextOverflow. The context window is 95 so the control decision is observable rather than an internal number (correct prompt count is 100; a cache-write mapping without the overflow fix counts 90). Three configurations, each file checked out individually:

base (origin/main) transport fix only head (both fixes)
input 80 70 70
cacheRead 20 20 20
cacheWrite 0 10 10
isContextOverflow(msg, 95) true false true

The middle column answers "why is overflow.ts in this PR". Landing the cache-write mapping alone flips a real overflow to undetected: base only reaches the correct prompt count because input wrongly absorbs the writes, so correcting the mapping without correcting the fallback silently shrinks every overflow and length-stop decision on this lane. The two changes are one fix.

Local after merging current main: both suites green (52 tests), node scripts/run-tsgo.mjs clean.

The breakdown-desync question is real, but it is an owner-level call about what cost.total means when a provider bills a number the components cannot reproduce. I would rather bring it to #9016 with a consumer and that decision than land a write-only field ahead of both.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 19, 2026
@steipete steipete self-assigned this Jul 21, 2026
Yigtwxx and others added 3 commits July 20, 2026 22:20
…che writes

`applyProviderReportedUsageCost` overwrote only `cost.total`, leaving the
per-component estimates in place, so `input + output + cacheRead + cacheWrite`
stopped summing to the total and the replaced estimate was unrecoverable. The
repo rebuilds that sum in four other places; this helper was the only writer
breaking it. Retain the replaced estimate in a new optional `cost.estimatedTotal`
instead of inverting the override: free OpenRouter models report `cost: 0`, and
`shouldPreserveRecordedZeroCost` depends on a provider-billed zero staying
authoritative, so holding the estimate in `total` would write phantom spend.

`parseTransportChunkUsage` also hard-coded `cacheWrite: 0` and never read
`prompt_tokens_details.cache_write_tokens`, while its plugin-sdk sibling reads it
and documents the contract. Writes were dropped from `cacheWrite` and wrongly
counted inside `input`. OpenRouter routes through this lane.

Fixing that shifts tokens out of `input`, which `resolveContextInputTokens` used
as a proxy for prompt size, so context-overflow and length-stop detection are
aligned with the Anthropic lane's `input + cacheRead + cacheWrite`.

Related: openclaw#9016
refactor(ai): drop the retained estimatedTotal cost surface

ClawSweeper blocked the branch three review cycles running on the same P1: the
new `Usage.cost.estimatedTotal` field crossed both the plugin SDK and the closed
worker transcript schema, making it an upgrade contract that a bug-fix PR cannot
decide on its own.

Measurement settles it: the field was written in one place and forwarded through
four transport layers, but nothing ever read it. It also did not repair the
invariant it was introduced for -- `input + output + cacheRead + cacheWrite` still
disagrees with a provider-billed `total`; the field only parked the discarded
estimate beside it.

Removing it drops eight files, retires the compatibility merge risk, and leaves
the two fixes that carry real runtime evidence: OpenRouter `cache_write_tokens`
mapping and the context-overflow accounting that depends on it.
@
@steipete
steipete force-pushed the fix/openrouter-usage-accounting-fidelity branch from e135185 to b1176ac Compare July 21, 2026 05:25
@clawsweeper

clawsweeper Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix(ai): agent lane drops OpenRouter cache-write tokens and under-counts context This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

@steipete
steipete merged commit dc28032 into openclaw:main Jul 21, 2026
117 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 21, 2026
…nts context (openclaw#111435)

* fix(ai): provider-billed totals desync the cost breakdown and drop cache writes

`applyProviderReportedUsageCost` overwrote only `cost.total`, leaving the
per-component estimates in place, so `input + output + cacheRead + cacheWrite`
stopped summing to the total and the replaced estimate was unrecoverable. The
repo rebuilds that sum in four other places; this helper was the only writer
breaking it. Retain the replaced estimate in a new optional `cost.estimatedTotal`
instead of inverting the override: free OpenRouter models report `cost: 0`, and
`shouldPreserveRecordedZeroCost` depends on a provider-billed zero staying
authoritative, so holding the estimate in `total` would write phantom spend.

`parseTransportChunkUsage` also hard-coded `cacheWrite: 0` and never read
`prompt_tokens_details.cache_write_tokens`, while its plugin-sdk sibling reads it
and documents the contract. Writes were dropped from `cacheWrite` and wrongly
counted inside `input`. OpenRouter routes through this lane.

Fixing that shifts tokens out of `input`, which `resolveContextInputTokens` used
as a proxy for prompt size, so context-overflow and length-stop detection are
aligned with the Anthropic lane's `input + cacheRead + cacheWrite`.

Related: openclaw#9016

* @
refactor(ai): drop the retained estimatedTotal cost surface

ClawSweeper blocked the branch three review cycles running on the same P1: the
new `Usage.cost.estimatedTotal` field crossed both the plugin SDK and the closed
worker transcript schema, making it an upgrade contract that a bug-fix PR cannot
decide on its own.

Measurement settles it: the field was written in one place and forwarded through
four transport layers, but nothing ever read it. It also did not repair the
invariant it was introduced for -- `input + output + cacheRead + cacheWrite` still
disagrees with a provider-billed `total`; the field only parked the discarded
estimate beside it.

Removing it drops eight files, retires the compatibility merge risk, and leaves
the two fixes that carry real runtime evidence: OpenRouter `cache_write_tokens`
mapping and the context-overflow accounting that depends on it.
@

* refactor(agents): tighten OpenRouter usage contract

---------

Co-authored-by: Peter Steinberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants