Skip to content

Gateway writes streamed usage.cost.total for OpenRouter calls — up to 4× under actual billed amount on tier-priced models #68066

Description

@chrispatil

Summary

Some OpenRouter-hosted models use tiered pricing — the per-token rate changes once a request crosses a context-size threshold (for example, a long-context tier that kicks in above ~128k input tokens). For these models, the streaming completions API returns a base-tier cost in the final chunk, but OpenRouter's authoritative post-hoc record (/api/v1/generation?id=<responseId>) reports the tiered-tier total_cost, which can be substantially higher.

The OpenClaw gateway currently writes the streamed message.usage.cost.total directly into the assistant JSONL record. For tiered-pricing models, that streamed value is therefore a lower bound, not the billed amount — confirmed as a 4× undercount on long-context qwen/qwen3.6-plus calls.

Evidence

Six random qwen/qwen3.6-plus messages from a single deployment, comparing the cost written into the JSONL against the total_cost returned by GET https://openrouter.ai/api/v1/generation?id=<responseId>:

Input tokens Gateway usage.cost.total OpenRouter total_cost Ratio
19,305 $0.00628 $0.00628 1.00×
19,221 $0.00641 $0.00637 0.99×
19,033 $0.00630 $0.00630 1.00×
410,496 $0.13342 $0.53366 4.00×
19,296 $0.00637 $0.00633 0.99×
19,221 $0.00637 $0.00635 1.00×

Below the tier threshold: parity to the penny. Above it: the streamed value is exactly 1/4 of what OpenRouter actually bills.

Full /generation response for the 410k-token call (trimmed):

{
  "tokens_prompt": 485590,
  "native_tokens_prompt": 507438,
  "native_tokens_reasoning": 0,
  "native_tokens_cached": 0,
  "total_cost": 0.6596889,
  "provider_name": "Alibaba",
  "model": "qwen/qwen3.6-plus-04-02"
}

The corresponding JSONL record's message.usage.cost.total was 0.16492710 — exactly 1/4 of the authoritative total_cost.

Over ~30 days of aggregated traffic on one deployment, the cumulative gap is roughly 50%: gateway-recorded OpenRouter spend totaled $105.05 while OpenRouter actually billed $180.19 (+$75.14). The gap is concentrated in the small number of long-context calls, not spread across all traffic. This is most visible when a primary model (e.g. GPT-5.4) exceeds its context window and the gateway falls back to a long-context OpenRouter model.

Suggested fix

Two options in the gateway's OpenRouter response handler:

  1. Post-hoc fetch. After the stream completes, call GET https://openrouter.ai/api/v1/generation?id=<responseId> and overwrite message.usage.cost.total with the returned total_cost. Adds ~50–500ms per call; can be done asynchronously after the response is already flushed to the agent.
  2. Record both. Keep the streamed value in message.usage.cost.total and add the authoritative value in a new field (e.g. message.usage.cost.total_billed), leaving the choice of which to trust to downstream consumers.

Option 1 is the cleaner fix for correctness. Option 2 preserves the current schema but requires downstream awareness.

Tiered pricing is not unique to qwen — any future OpenRouter-hosted model with context-size or reasoning-token tiers will hit the same failure mode, so the fix generalizes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:data-lossCan lose, corrupt, or silently drop user/session/config data.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.staleMarked as stale due to inactivity

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions