Repository navigation
Commit fdbc2cc
Python: fix(anthropic): report token usage as CompletionUsage (#14618)
### Motivation and Context
The model diagnostics record token usage only when `metadata["usage"]`
is a `CompletionUsage`
(`utils/telemetry/model_diagnostics/decorators.py`). The other chat
connectors build one, but `AnthropicChatCompletion` stores the raw
`anthropic.types.Usage` (non-streaming) or a plain dict like
`{"input_tokens": ..., "output_tokens": ...}` (streaming). As a result,
spans for Anthropic calls never get `gen_ai.usage.input_tokens` /
`gen_ai.usage.output_tokens`.
Prompt caching also isn't counted: Anthropic reports
`cache_read_input_tokens` and `cache_creation_input_tokens` separately
from `input_tokens`, and the streaming path drops them.
### Description
- A new `_get_completion_usage` helper turns Anthropic usage into a
`CompletionUsage`:
- `prompt_tokens` = `input_tokens` + `cache_read_input_tokens` +
`cache_creation_input_tokens`;
- `prompt_tokens_details.cached_tokens` = `cache_read_input_tokens`
(only when present);
- `completion_tokens` = `output_tokens`.
- Non-streaming: `metadata["usage"]` is now that `CompletionUsage`.
- Streaming: `message_start` sets the prompt side. The final
`message_delta` chunk carries the full usage: its own counts when it
reports `input_tokens`, otherwise the `message_start` prompt counts plus
the delta's `output_tokens`. Since the chunk metadata is merged with
`|`, this is the usage of the combined message.
- Fields are read with `getattr` because older `anthropic` releases (the
requirement is `~= 0.32`) lack some of them.
**Behavior change:** `metadata["usage"]` for Anthropic responses is now
a `CompletionUsage`, the same type as the other connectors. Code that
read `.input_tokens` / `.output_tokens` from it should use
`.prompt_tokens` / `.completion_tokens`. The raw Anthropic usage is
still available on `inner_content`.
### Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄 (see the behavior change above)
Tests, in
`tests/unit/connectors/ai/anthropic/services/test_anthropic_chat_completion.py`,
with model diagnostics enabled, 10 input + 100 cache-read + 20
cache-write + 7 output tokens:
- `test_usage_is_reported_as_completion_usage` (non-streaming):
`metadata["usage"]` is a `CompletionUsage` (130 prompt, 100 cached, 7
completion), and the span gets `gen_ai.usage.input_tokens=130`,
`gen_ai.usage.output_tokens=7`;
- `test_streaming_usage_is_reported_as_completion_usage`: the same
checks on the combined streamed message.
Both fail on `main`. `tests/unit/connectors/ai/anthropic` +
`tests/unit/utils/model_diagnostics`: 65 passed
(`test_connector_decorated.py` was not run here because it imports the
Mistral connector). `ruff` 0.15.5 check/format is clean. `mypy` on the
changed module reports the same 3 pre-existing errors as `main`, all in
other modules.
This change was drafted with Claude Code; the tests above were run
before opening the PR.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <[email protected]>1 parent 80161f5 commit fdbc2cc
2 files changed
Lines changed: 148 additions & 9 deletions
File tree
- python
- semantic_kernel/connectors/ai/anthropic/services
- tests/unit/connectors/ai/anthropic/services
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| 24 | + | |
23 | 25 | | |
| 26 | + | |
24 | 27 | | |
25 | 28 | | |
26 | 29 | | |
| |||
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| 38 | + | |
35 | 39 | | |
36 | 40 | | |
37 | 41 | | |
| |||
309 | 313 | | |
310 | 314 | | |
311 | 315 | | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
317 | 322 | | |
318 | 323 | | |
319 | 324 | | |
| |||
338 | 343 | | |
339 | 344 | | |
340 | 345 | | |
341 | | - | |
| 346 | + | |
342 | 347 | | |
343 | 348 | | |
344 | 349 | | |
| |||
357 | 362 | | |
358 | 363 | | |
359 | 364 | | |
360 | | - | |
| 365 | + | |
361 | 366 | | |
362 | 367 | | |
363 | | - | |
| 368 | + | |
364 | 369 | | |
365 | 370 | | |
366 | 371 | | |
| |||
391 | 396 | | |
392 | 397 | | |
393 | 398 | | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
2 | 3 | | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
6 | | - | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
7 | 10 | | |
8 | 11 | | |
9 | 12 | | |
10 | 13 | | |
11 | 14 | | |
12 | 15 | | |
| 16 | + | |
13 | 17 | | |
14 | 18 | | |
15 | 19 | | |
| |||
27 | 31 | | |
28 | 32 | | |
29 | 33 | | |
| 34 | + | |
| 35 | + | |
30 | 36 | | |
31 | 37 | | |
32 | 38 | | |
| |||
547 | 553 | | |
548 | 554 | | |
549 | 555 | | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
0 commit comments