Skip to content

Commit 75637f5

Browse files
nik13claude
andcommitted
fix(ag2): alias cache tokens to the keys Future AGI reads
AG2's cache_read_input_tokens / cache_creation_input_tokens were aliased to the dotted semconv names gen_ai.usage.cache_read.input_tokens / cache_creation.input_tokens. Future AGI's attribute registry reads gen_ai.usage.cache_read_tokens / cache_write_tokens and never the dotted names, so cache counts never reached it. Alias to those keys instead (the originals stay). A record_usage span that repeats a chat span's usage moves them to ag2.usage.* with the token counts, so each call carries them once. Checked live: an AnthropicConfig agent on CLIProxy (AG2's default prompt caching) stored cache_read_tokens 9747 and cache_write_tokens 9749 on its chat spans in ClickHouse, and ag2.usage.* on the record_usage spans. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent d60a10b commit 75637f5

7 files changed

Lines changed: 87 additions & 26 deletions

File tree

‎python/frameworks/ag2/CHANGELOG.md‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,14 @@
55
- New package for AG2 1.x (PyPI `ag2`, import `ag2`). `setup()` attaches AG2's
66
own `TelemetryMiddleware` with `capture_content=False` and registers the
77
Future AGI exporter; `AG2SpanProcessor` sets `gen_ai.span.kind` from
8-
`gen_ai.operation.name` and aliases AG2's cache/thinking usage keys onto the
9-
GenAI semconv names (originals kept). Does not wrap `Agent` and never
10-
imports `autogen`. Refs: TH-8236.
8+
`gen_ai.operation.name` and aliases AG2's cache/thinking usage keys
9+
(originals kept). Does not wrap `Agent` and never imports `autogen`.
10+
Refs: TH-8236.
11+
- Cache tokens are aliased to `gen_ai.usage.cache_read_tokens` /
12+
`gen_ai.usage.cache_write_tokens`, the keys Future AGI reads (were the dotted
13+
`gen_ai.usage.cache_read.input_tokens` / `cache_creation.input_tokens`, which
14+
it does not read). A `record_usage` span that repeats a chat span's usage
15+
moves them to `ag2.usage.*` with the token counts.
1116
- With `capture_content=True`, the captured chat messages, tool arguments and
1217
result, and human input are also copied to `input.value` / `output.value`,
1318
the only keys Future AGI fills a span's input and output from. Content a

‎python/frameworks/ag2/README.md‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,14 +31,21 @@ This package does not wrap `Agent`. It:
3131
`opentelemetry.instrumentation.ag2` scope only:
3232
- sets `gen_ai.span.kind` from `gen_ai.operation.name`
3333
(`chat` -> `LLM`, `execute_tool` -> `TOOL`, `invoke_agent` -> `AGENT`),
34-
- copies AG2's usage keys onto the GenAI semconv names and keeps the originals:
34+
- copies AG2's cache and thinking usage keys onto the keys Future AGI reads
35+
and keeps the originals:
3536

3637
| AG2 key | Alias |
3738
|---|---|
38-
| `gen_ai.usage.cache_creation_input_tokens` | `gen_ai.usage.cache_creation.input_tokens` |
39-
| `gen_ai.usage.cache_read_input_tokens` | `gen_ai.usage.cache_read.input_tokens` |
39+
| `gen_ai.usage.cache_creation_input_tokens` | `gen_ai.usage.cache_write_tokens` |
40+
| `gen_ai.usage.cache_read_input_tokens` | `gen_ai.usage.cache_read_tokens` |
4041
| `gen_ai.usage.thinking_tokens` | `gen_ai.usage.reasoning.output_tokens` |
4142

43+
Future AGI reads the cache counts from `gen_ai.usage.cache_read_tokens` /
44+
`gen_ai.usage.cache_write_tokens`, not from the dotted OpenTelemetry
45+
spellings (`gen_ai.usage.cache_read.input_tokens`). On a `record_usage`
46+
span that repeats a chat span's usage (see below), they move to
47+
`ag2.usage.*` with the token counts.
48+
4249
- applies `TraceConfig` (`FI_HIDE_INPUTS`, `FI_HIDE_OUTPUTS`, ...) as a
4350
second content gate.
4451

@@ -158,7 +165,7 @@ not copied to `input.value` / `output.value` either.
158165
| Span kind | `AGENT`, `LLM`, `TOOL` from `gen_ai.operation.name` | `await_human_input` and `record_usage` spans get no kind (see below) |
159166
| Model | `gen_ai.request.model`, `gen_ai.response.model` pass through | When AG2 did not set them |
160167
| Provider | `gen_ai.provider.name` from `provider_name` or the model response | When neither is known |
161-
| Tokens | `gen_ai.usage.input_tokens` / `output_tokens` pass through; cache and thinking tokens are aliased and kept | When the usage field is zero/absent. Cost is not emitted |
168+
| Tokens | `gen_ai.usage.input_tokens` / `output_tokens` pass through; cache tokens are aliased to `gen_ai.usage.cache_read_tokens` / `cache_write_tokens` and thinking tokens to `gen_ai.usage.reasoning.output_tokens`, originals kept | When the usage field is zero/absent. Cost is not emitted |
162169
| Session | Not emitted by AG2. `session.id` from traceAI's `using_session` / `using_attributes` context, which wins over a static value; or set it yourself: `setup(..., span_attributes={"session.id": "..."})` | When neither is set |
163170
| User, metadata, tags | `user.id`, `metadata`, `tag.tags` (and the other traceAI context keys) from `using_user` / `using_metadata` / `using_tags` / `using_attributes`; a key AG2 or `span_attributes` already set is not overridden | When the context does not set them |
164171
| Tools | `gen_ai.tool.name`, `gen_ai.tool.call.id`, `gen_ai.tool.type`; arguments/result only with `capture_content=True` | |

‎python/frameworks/ag2/tests/test_contract_harness.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -94,9 +94,9 @@ def test_one_tool_run_reaches_collector_path_with_normalized_gen_ai_spans(fi_env
9494
"gen_ai.usage.input_tokens": 11,
9595
"gen_ai.usage.output_tokens": 7,
9696
"gen_ai.usage.cache_creation_input_tokens": 2,
97-
"gen_ai.usage.cache_creation.input_tokens": 2,
97+
"gen_ai.usage.cache_write_tokens": 2,
9898
"gen_ai.usage.cache_read_input_tokens": 3,
99-
"gen_ai.usage.cache_read.input_tokens": 3,
99+
"gen_ai.usage.cache_read_tokens": 3,
100100
"gen_ai.usage.thinking_tokens": 5,
101101
"gen_ai.usage.reasoning.output_tokens": 5,
102102
}
@@ -155,6 +155,6 @@ def test_short_script_flushes_spans_before_exit():
155155
assert by_name["invoke_agent weather_bot"]["gen_ai.span.kind"] == "AGENT"
156156
assert by_name["execute_tool get_weather"]["gen_ai.span.kind"] == "TOOL"
157157
assert by_name["chat fake-model"]["gen_ai.span.kind"] == "LLM"
158-
assert by_name["chat fake-model"]["gen_ai.usage.cache_read.input_tokens"] == 3
158+
assert by_name["chat fake-model"]["gen_ai.usage.cache_read_tokens"] == 3
159159
for name, attributes in by_name.items():
160160
assert_no_content(attributes, name)

‎python/frameworks/ag2/tests/test_processor.py‎

Lines changed: 45 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -105,21 +105,21 @@ def test_usage_aliases_copy_and_keep_originals(pipeline):
105105
got = attrs(span)
106106
for key, value in source.items():
107107
assert got[key] == value, key # originals kept, input/output unchanged
108-
assert got["gen_ai.usage.cache_creation.input_tokens"] == 2
109-
assert got["gen_ai.usage.cache_read.input_tokens"] == 3
108+
assert got["gen_ai.usage.cache_write_tokens"] == 2
109+
assert got["gen_ai.usage.cache_read_tokens"] == 3
110110
assert got["gen_ai.usage.reasoning.output_tokens"] == 5
111111
assert set(got) == set(source) | {
112112
"gen_ai.span.kind",
113-
"gen_ai.usage.cache_creation.input_tokens",
114-
"gen_ai.usage.cache_read.input_tokens",
113+
"gen_ai.usage.cache_write_tokens",
114+
"gen_ai.usage.cache_read_tokens",
115115
"gen_ai.usage.reasoning.output_tokens",
116116
}
117117

118118

119119
def test_alias_table_matches_spec():
120120
assert dict(USAGE_KEY_ALIASES) == {
121-
"gen_ai.usage.cache_creation_input_tokens": "gen_ai.usage.cache_creation.input_tokens",
122-
"gen_ai.usage.cache_read_input_tokens": "gen_ai.usage.cache_read.input_tokens",
121+
"gen_ai.usage.cache_creation_input_tokens": "gen_ai.usage.cache_write_tokens",
122+
"gen_ai.usage.cache_read_input_tokens": "gen_ai.usage.cache_read_tokens",
123123
"gen_ai.usage.thinking_tokens": "gen_ai.usage.reasoning.output_tokens",
124124
}
125125

@@ -153,10 +153,48 @@ def test_aliases_apply_to_record_usage_spans(pipeline):
153153
)
154154
(span,) = exporter.get_finished_spans()
155155
got = attrs(span)
156-
assert got["gen_ai.usage.cache_read.input_tokens"] == 3
156+
assert got["gen_ai.usage.cache_read_tokens"] == 3
157157
assert "gen_ai.span.kind" not in got
158158

159159

160+
def test_cache_tokens_use_the_keys_future_agi_reads():
161+
# Future AGI's attribute registry reads gen_ai.usage.cache_read_tokens /
162+
# cache_write_tokens (tracer/utils/semantic_conventions.py CACHE_READ_TOKENS,
163+
# CACHE_WRITE_TOKENS); the dotted semconv spellings are not read.
164+
got = normalize_attributes(
165+
{
166+
"gen_ai.operation.name": "chat",
167+
"gen_ai.usage.cache_creation_input_tokens": 2,
168+
"gen_ai.usage.cache_read_input_tokens": 3,
169+
}
170+
)
171+
assert (got["gen_ai.usage.cache_write_tokens"], got["gen_ai.usage.cache_read_tokens"]) == (2, 3)
172+
assert "gen_ai.usage.cache_read.input_tokens" not in got
173+
assert "gen_ai.usage.cache_creation.input_tokens" not in got
174+
175+
176+
def test_duplicate_usage_span_does_not_repeat_cache_tokens():
177+
# record_usage model_call repeats the chat span's cache counts too.
178+
got = normalize_attributes(
179+
{
180+
"ag2.span.type": "usage",
181+
"ag2.usage.kind": "model_call",
182+
"gen_ai.usage.cache_creation_input_tokens": 2,
183+
"gen_ai.usage.cache_read_input_tokens": 3,
184+
}
185+
)
186+
assert "gen_ai.usage.cache_read_tokens" not in got
187+
assert "gen_ai.usage.cache_write_tokens" not in got
188+
assert (got["ag2.usage.cache_write_tokens"], got["ag2.usage.cache_read_tokens"]) == (2, 3)
189+
190+
191+
def test_out_of_band_usage_span_keeps_cache_tokens():
192+
got = normalize_attributes(
193+
{"ag2.span.type": "usage", "ag2.usage.kind": "aggregation", "gen_ai.usage.cache_read_input_tokens": 3}
194+
)
195+
assert got["gen_ai.usage.cache_read_tokens"] == 3
196+
197+
160198
def test_model_call_usage_span_does_not_double_count_trace_tokens(pipeline):
161199
"""record_usage model_call repeats the chat span's tokens (telemetry.py:437-440 vs 501-504).
162200

‎python/frameworks/ag2/tests/test_setup.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -239,9 +239,9 @@ def test_one_tool_run_span_shape():
239239
assert a["gen_ai.usage.input_tokens"] == 11
240240
assert a["gen_ai.usage.output_tokens"] == 7
241241
assert a["gen_ai.usage.cache_creation_input_tokens"] == 2
242-
assert a["gen_ai.usage.cache_creation.input_tokens"] == 2
242+
assert a["gen_ai.usage.cache_write_tokens"] == 2
243243
assert a["gen_ai.usage.cache_read_input_tokens"] == 3
244-
assert a["gen_ai.usage.cache_read.input_tokens"] == 3
244+
assert a["gen_ai.usage.cache_read_tokens"] == 3
245245
assert a["gen_ai.usage.thinking_tokens"] == 5
246246
assert a["gen_ai.usage.reasoning.output_tokens"] == 5
247247

‎python/frameworks/ag2/traceai_ag2/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
* registers the Future AGI exporter through ``fi_instrumentation.register()``
99
when you do not pass a provider,
1010
* installs :class:`AG2SpanProcessor`, which sets ``gen_ai.span.kind`` and
11-
aliases AG2's usage keys onto the GenAI semconv names.
11+
aliases AG2's cache and thinking usage keys onto the keys Future AGI reads.
1212
1313
This is not Microsoft AutoGen (``autogen-agentchat``, see ``traceai-autogen``)
1414
and not AG2 Classic (``autogen`` 0.14.x). It never imports ``autogen``.

‎python/frameworks/ag2/traceai_ag2/_processor.py‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@
77
* sets ``gen_ai.span.kind`` from ``gen_ai.operation.name`` for the operation
88
strings AG2 emits that the Microsoft Agent Framework table also maps
99
(``chat``, ``execute_tool``, ``invoke_agent``);
10-
* copies three non-semconv usage keys onto their dotted GenAI names, keeping
11-
the originals;
10+
* copies AG2's cache usage keys onto the keys Future AGI reads
11+
(``gen_ai.usage.cache_read_tokens`` / ``cache_write_tokens``) and its thinking
12+
tokens onto ``gen_ai.usage.reasoning.output_tokens``, keeping the originals;
1213
* on ``record_usage`` spans whose tokens a chat span in the same trace already
1314
carries (every ``model_call``; a ``subtask`` rollup whose sub-agent is itself
1415
instrumented in that trace), moves ``gen_ai.usage.input_tokens`` /
@@ -73,12 +74,18 @@
7374
"invoke_agent": FiSpanKindValues.AGENT.value,
7475
}
7576

76-
# AG2 spelling -> GenAI semconv spelling. AG2 sets the left-hand keys at
77+
# AG2 spelling -> the spelling Future AGI reads. AG2 sets the left-hand keys at
7778
# telemetry.py lines 441-446 (record_usage span) and 505-510 (chat span) at
78-
# 1.1.2, only when the usage field is non-zero. Originals are kept.
79+
# 1.1.2, only when the usage field is non-zero. Originals are kept. The cache
80+
# targets are the keys Future AGI's attribute registry reads
81+
# (tracer/utils/semantic_conventions.py CACHE_READ_TOKENS / CACHE_WRITE_TOKENS,
82+
# also fi_instrumentation SpanAttributes.GEN_AI_USAGE_CACHE_*_TOKENS); it does
83+
# not read the dotted semconv spellings (gen_ai.usage.cache_read.input_tokens).
84+
_CACHE_READ_TOKENS = SpanAttributes.GEN_AI_USAGE_CACHE_READ_TOKENS # gen_ai.usage.cache_read_tokens
85+
_CACHE_WRITE_TOKENS = SpanAttributes.GEN_AI_USAGE_CACHE_WRITE_TOKENS # gen_ai.usage.cache_write_tokens
7986
USAGE_KEY_ALIASES: Mapping[str, str] = {
80-
"gen_ai.usage.cache_creation_input_tokens": "gen_ai.usage.cache_creation.input_tokens",
81-
"gen_ai.usage.cache_read_input_tokens": "gen_ai.usage.cache_read.input_tokens",
87+
"gen_ai.usage.cache_creation_input_tokens": _CACHE_WRITE_TOKENS,
88+
"gen_ai.usage.cache_read_input_tokens": _CACHE_READ_TOKENS,
8289
"gen_ai.usage.thinking_tokens": "gen_ai.usage.reasoning.output_tokens",
8390
}
8491

@@ -194,6 +201,10 @@ def _copy_content_to_io(mapped: Dict[str, Any], hidden: AbstractSet[str]) -> Non
194201
"gen_ai.usage.input_tokens",
195202
"gen_ai.usage.output_tokens",
196203
"gen_ai.usage.total_tokens",
204+
# Not summed today, but a duplicate usage span repeats the chat span's
205+
# cache counts too; keep Future AGI's cache keys on one span per call.
206+
_CACHE_READ_TOKENS,
207+
_CACHE_WRITE_TOKENS,
197208
)
198209
# Usage kinds AG2 emits (``ag2/_telemetry_consts.py`` ATTR_USAGE_KIND at
199210
# 1.1.2) and whether a chat span in the same trace already carries the tokens:

0 commit comments

Comments
 (0)