Skip to content

Commit 96ef5b3

Browse files
committed
fix(ag2-classic): keep promoted token keys on LLM spans only
Conversation spans carry the chat-wide aggregate_usage sum as gen_ai.usage.input_tokens/output_tokens (autogen 0.14.1 opentelemetry/instrumentators/agent_instrumentators/chat.py 81-82, 146-147) on top of the per-call tokens on the LLM spans (llm_wrapper.py 123-125). fi-collector promotes those keys on any span (adapter.go) and Observe sums total_tokens over a trace, so totals doubled. Non-LLM spans now keep the values under ag2.usage.*. Tests first: new processor test plus a trace-sum assertion in the two-agent setup test. 59 passed on autogen 0.14.1 and 0.14.0. approved: Nikhil 2026-10-03 blanket Refs: TH-8237
1 parent 8bfc3de commit 96ef5b3

4 files changed

Lines changed: 37 additions & 3 deletions

File tree

‎python/frameworks/ag2-classic/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ are covered by mapping unit tests only.
104104
|---|---|---|
105105
| Span kind | `gen_ai.span.kind` from upstream `ag2.span.type`: `llm` → `LLM`; `agent` → `AGENT`; `tool`, `code_execution` → `TOOL`; `conversation`, `multi_conversation`, `speaker_selection`, `human_input` → `CHAIN`; `handoff` → `CHAIN` (declared upstream as TODO, never emitted at 0.14.x). `ag2.span.type` and `gen_ai.operation.name` are kept. | The `a2a-execution` span from `instrument_a2a_server` has no `ag2.span.type` and gets no kind. |
106106
| Model, provider | `gen_ai.request.model`, `gen_ai.provider.name` on LLM, agent and conversation spans; `gen_ai.response.model` on LLM spans. Upstream keys, passed through. | Agents without an LLM config carry no model or provider. |
107-
| Tokens | `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` on LLM spans (from the response `usage`), plus `gen_ai.usage.total_tokens` added here. | A response without `usage` gives no token keys. Conversation spans also carry upstream's aggregate input/output tokens for the whole chat; they are passed through unchanged and get no `total_tokens`, so do not sum them with the LLM spans. |
107+
| Tokens | `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` on LLM spans (from the response `usage`), plus `gen_ai.usage.total_tokens` added here. | A response without `usage` gives no token keys. Conversation spans carry upstream's aggregate input/output tokens for the whole chat. Future AGI sums tokens over every span in a trace, so those move to `ag2.usage.input_tokens` / `ag2.usage.output_tokens`; each LLM call counts once. |
108108
| Cost | `gen_ai.cost.total` on LLM spans, copied from upstream `gen_ai.usage.cost` (AG2's own price table). | Conversation spans keep only upstream `gen_ai.usage.cost` (aggregate). |
109109
| Session | `session.id` from upstream `gen_ai.conversation.id` (`ChatResult.chat_id`) on the outermost `conversation` span. | Nested chats (for example the internal chat behind group-chat speaker selection) keep `gen_ai.conversation.id` but get no `session.id`. Child spans end before the chat id exists, so only the conversation span carries it. A `GroupChatManager` run emits no own conversation span at 0.14.x. |
110110
| User | Not emitted upstream. | Always. |

‎python/frameworks/ag2-classic/tests/test_processor.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,24 @@ def test_conversation_aggregate_usage_is_not_relabelled():
100100
assert "gen_ai.cost.total" not in mapped
101101

102102

103+
def test_conversation_aggregate_tokens_do_not_double_count_the_trace():
104+
"""chat.py:81-82 puts the chat-wide sum on the conversation span.
105+
106+
fi-collector promotes gen_ai.usage.* on any span and Observe sums
107+
total_tokens over a trace, so only LLM spans keep the promoted keys.
108+
"""
109+
attrs = {
110+
"ag2.span.type": "conversation",
111+
"gen_ai.usage.input_tokens": 22,
112+
"gen_ai.usage.output_tokens": 14,
113+
}
114+
mapped = map_ag2_attributes(attrs)
115+
assert "gen_ai.usage.input_tokens" not in mapped
116+
assert "gen_ai.usage.output_tokens" not in mapped
117+
assert mapped["ag2.usage.input_tokens"] == 22
118+
assert mapped["ag2.usage.output_tokens"] == 14
119+
120+
103121
# Session ----------------------------------------------------------------------
104122

105123

‎python/frameworks/ag2-classic/tests/test_setup.py‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -226,8 +226,16 @@ def test_two_agent_chat_kinds_usage_session_and_no_content(pipeline, fake):
226226
conversation = spans["conversation user"][0].attributes
227227
assert conversation["gen_ai.span.kind"] == "CHAIN"
228228
assert conversation["session.id"] == str(result.chat_id)
229-
# Upstream aggregate on the conversation span (chat.py:73-82) is passed through.
230-
assert conversation["gen_ai.usage.input_tokens"] == 2 * PROMPT_TOKENS
229+
# Upstream aggregate on the conversation span (chat.py:73-82) is kept under
230+
# ag2.usage.* so the trace total counts each LLM call once.
231+
assert "gen_ai.usage.input_tokens" not in conversation
232+
assert conversation["ag2.usage.input_tokens"] == 2 * PROMPT_TOKENS
233+
promoted = sum(
234+
s.attributes.get("gen_ai.usage.input_tokens", 0)
235+
for group in spans.values()
236+
for s in group
237+
)
238+
assert promoted == 2 * PROMPT_TOKENS
231239

232240
# LLM spans are children of the agent span, all in one trace.
233241
trace_ids = {s.context.trace_id for group in spans.values() for s in group}

‎python/frameworks/ag2-classic/traceai_ag2_classic/_processor.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -210,6 +210,14 @@ def map_ag2_attributes(
210210
):
211211
mapped[SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS] = inp + out
212212
_bundle_request_parameters(mapped)
213+
else:
214+
# fi-collector promotes these keys into the token columns on any span
215+
# (adapter.go inputTokenKeys/outputTokenKeys/totalTokenKeys) and Observe
216+
# sums total_tokens over a trace. Only LLM spans carry per-call usage;
217+
# the conversation span's chat-wide sum (chat.py:81-82) would double it.
218+
for key in (INPUT_TOKENS, OUTPUT_TOKENS, SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS):
219+
if key in mapped:
220+
mapped.setdefault("ag2.usage." + key.rsplit(".", 1)[1], mapped.pop(key))
213221

214222
if capture_content:
215223
_surface_content(mapped, str(span_type))

0 commit comments

Comments
 (0)