Repository navigation
Python: [Bug]: AnthropicClient reports input_token_count (and gen_ai.usage.input_tokens) without cached prompt tokens #9808
Description
Description
With Anthropic prompt caching, AnthropicClient sets UsageDetails["input_token_count"] to Anthropic's raw usage.input_tokens. Anthropic documents that field as only the tokens after the last cache breakpoint; the total input is input_tokens + cache_creation_input_tokens + cache_read_input_tokens.
Everywhere else in Agent Framework input_token_count is the total input, with cache_read_input_token_count as a subset of it. The OpenAI chat-completions and Responses clients map prompt_tokens / input_tokens (which include cached tokens), and Gemini's prompt_token_count does too. input_token_count is exported as gen_ai.usage.input_tokens, and the OTel GenAI semconv says that value "SHOULD include all types of input tokens, including cached tokens" and that gen_ai.usage.cache_read.input_tokens "SHOULD be included in gen_ai.usage.input_tokens".
So with a cache hit, a 9,312-token Claude prompt shows up as 12 input tokens on the span, in the gen_ai.client.token.usage histogram and in response.usage_details, while an OpenAI call with the same cache split shows 9,312. Traces and dashboards that compare providers, or estimate context size from input_token_count, undercount Claude by the size of the cached prefix. The longer the cached system prompt, the bigger the gap.
Observed Behavior
input_token_count == 12 and gen_ai.usage.input_tokens == 12 (cache_read 9000, cache_creation 300), in both non-streaming and streaming.
Expected Behavior
input_token_count == 9312 (12 + 300 + 9000), matching the other connectors and the semconv. The cache fields are unchanged.
Steps to Reproduce / Minimal Reproduction
Offline, with no API key or network: a mocked anthropic_client returns a BetaMessage whose usage is BetaUsage(input_tokens=12, output_tokens=40, cache_read_input_tokens=9000, cache_creation_input_tokens=300).
transport = MagicMock(); transport.base_url = "https://example.invalid"
transport.beta.messages.create = AsyncMock(return_value=message_with_cached_usage)
client = AnthropicClient(api_key="dummy", model="claude-sonnet-4-5", anthropic_client=transport)
resp = await client.get_response([Message(role="user", contents=["hello"])])
print(resp.usage_details["input_token_count"]) # 12, expected 9312Full script (it also covers streaming and OTel spans): . Output on main c2a8f91:
[non-streaming] usage_details.input_token_count = 12 cache_read=9000 cache_creation=300
[streaming] usage_details.input_token_count = 12 cache_read=9000 output=40
[span 'chat claude-sonnet-4-5'] gen_ai.usage.input_tokens=12 gen_ai.usage.cache_read.input_tokens=9000
Package Versions
agent-framework-core 1.23.0, agent-framework-anthropic 1.0.0b261022 (main c2a8f91), anthropic 1.17.0
Python Version / OS
3.12.15 / Linux
Regression
No. _parse_usage_from_anthropic has always mapped the raw field.
Additional Context / proposed fix
In _parse_usage_from_anthropic, set input_token_count = input_tokens + (cache_creation_input_tokens or 0) + (cache_read_input_tokens or 0). The cache fields and the anthropic.* raw keys stay as they are, so the uncached count can still be recovered. Streaming keeps working, because _incremental_usage diffs cumulative snapshots per key (#7143). The change also covers the Foundry, Vertex and Bedrock Anthropic clients, which share RawAnthropicClient. This is a behavior change for anyone reading input_token_count as "uncached input", so I'd like a maintainer's OK on the direction first. agent-framework-anthropic is still beta in PACKAGE_STATUS.md.
Out of scope for this issue, but the same question applies to two other connectors: BedrockChatClient (Converse inputTokens excludes cacheRead/WriteInputTokens according to AWS's prompt-caching docs) and ClaudeAgent (it uses the Claude Agent SDK's Anthropic-style input_tokens). I'm happy to follow up there if you want them handled the same way.
I have a fix with tests ready and would like to take this.
Metadata
Metadata
Labels
Type
Projects
- Status
Activity
github-actions commentedon Oct 28, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
Repro:
python/packages/anthropic/agent_framework_anthropic/_chat_client.py::RawAnthropicClient._parse_usage_from_anthropicat lines 1574-1586 reports only rawusage.input_tokenswhen cache creation/read counts are present, affecting non-streaming and streaming callers. Minimal repro: parseBetaUsage(input_tokens=12, output_tokens=40, cache_creation_input_tokens=300, cache_read_input_tokens=9000)and assertinput_token_count == 9312; current output is12.python/packages/anthropic/tests/test_anthropic_client.py::test_parse_usage_includes_cache_tokens_in_total_inputagent-framework-anthropic 1.0.0b261022agent-framework-anthropic 1.0.0b261022lalith0407 commentedon Oct 29, 2026
The same undercount shows up in two more connectors. Could this issue cover them too?
BedrockChatClient._parse_usagemaps ConverseinputTokenstoinput_token_count. With prompt caching, Bedrock reports cache reads and writes separately incacheReadInputTokens/cacheWriteInputTokens, soinputTokensis only the uncached part (AWS prompt caching docs).totalTokensdoes include the cache, so the usage on main doesn't add up: input 12 + output 40, but total 9352.ClaudeAgenttakesinput_token_countfrom the Claude Agent SDK'susage["input_tokens"], which is Anthropic's uncached-only field, and setstotal_token_count = input + output, so both numbers come out low.Offline repro on main 9e971db (stub Bedrock runtime, mocked
ClaudeSDKClient, no keys), with a prompt of 12 uncached + 300 cache-write + 9000 cache-read tokens:Expected
input_token_count == 9312for both, and ClaudeAgenttotal_token_count == 9352.I have a small fix with tests for both, using the same approach as the Anthropic fix. Two existing unit tests assert the old value, so this has the same behavior-change note as above. Happy to include it in the #9808 PR or open a separate one, whichever you prefer.
Python: include Anthropic cache tokens in input_token_count (microsof…