Skip to content

Commit 4a114d1

Browse files
committed
fix(examples): move every TanStack AI usage key off the chat root
applyRootUsage (@tanstack/ai 0.64.0 otel.ts) sums every gen_ai.usage.* key onto the root: cost, cache read/creation and reasoning tokens as well as input/output/total. The recipe moved only the three token keys, so cache, reasoning and cost were still counted twice per trace. When the model-call spans carry usage, every gen_ai.usage.<suffix> key on the root now moves to tanstack.ai.root_usage.<suffix> (full suffix, so cache_read.input_tokens and reasoning.output_tokens do not collide) and every gen_ai.cost.<suffix> key to tanstack.ai.root_usage.cost.<suffix>. Test first: the contract test asserts no gen_ai.usage.*/gen_ai.cost.* key on the root and trace sums equal to the per-call sums for input, output, total, cache_read and reasoning tokens; it failed on the root's cache_read and reasoning keys. New tests/span_kinds.test.mjs (node --test, run from pytest) drives futureAgiSpanKinds() with fake spans: no model call reported usage (root keeps it; kills reviewer mutation M2), one model call (no kind, nothing promoted on the root, trace sum equals the call), two calls with cost and cache keys (failed before this change). approved: Nikhil 2026-10-03 blanket Refs: TH-8238
1 parent 73d9943 commit 4a114d1

4 files changed

Lines changed: 185 additions & 35 deletions

File tree

‎typescript/examples/tanstack-ai/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ const middleware = otelMiddleware({
5757

5858
| Span | Name | Kind set by the recipe | Key attributes |
5959
|---|---|---|---|
60-
| chat() call | `chat <model>` | `AGENT` when it ran more than one model call, else none | `gen_ai.request.model`, `tanstack.ai.iterations`, summed usage as `tanstack.ai.root_usage.*` (Future AGI sums `gen_ai.usage.*` over the trace, so the root's copy would double it; kept on the root only when no model call reported usage) |
60+
| chat() call | `chat <model>` | `AGENT` when it ran more than one model call, else none | `gen_ai.request.model`, `tanstack.ai.iterations`, and TanStack's sum of every model call's usage, renamed: each `gen_ai.usage.<suffix>` becomes `tanstack.ai.root_usage.<suffix>` and each `gen_ai.cost.<suffix>` becomes `tanstack.ai.root_usage.cost.<suffix>`. Future AGI sums `gen_ai.usage.*` over the trace, so the root's copy would double tokens, cache, reasoning and cost. The root keeps `gen_ai.usage.*` only when no model call reported usage. |
6161
| model call | `chat <model> #<n>` | `LLM` | `gen_ai.operation.name=chat`, `gen_ai.request.model`, `gen_ai.response.model`, `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `tanstack.ai.iteration` |
6262
| tool call | `execute_tool <name>` | `TOOL` | `gen_ai.tool.name`, `gen_ai.tool.call.id`, `gen_ai.tool.type` |
6363

‎typescript/examples/tanstack-ai/src/tracing.mjs‎

Lines changed: 22 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -68,29 +68,34 @@ export async function shutdownTraces(tracerProvider) {
6868
* root when chat() starts, before the iteration count is known. A
6969
* single-iteration root and media "generation" spans get no kind.
7070
*
71-
* Usage: TanStack sums every model call's usage onto the root
72-
* (otel.ts applyRootUsage, set just before onSpanEnd). Future AGI promotes
73-
* gen_ai.usage.* into its token columns on any span and sums them over the
74-
* whole trace, so leaving the root's copy would count each call twice. When
75-
* the model-call spans carry usage, the root's sum moves to
76-
* tanstack.ai.root_usage.*. If no model-call span reported usage, the root
77-
* keeps it, because then it is the only copy.
71+
* Usage: TanStack sums every model call's gen_ai.usage.* keys onto the root
72+
* (otel.ts applyRootUsage, set just before onSpanEnd): input, output and
73+
* total tokens, cost, cache read/creation and reasoning tokens. Future AGI
74+
* promotes gen_ai.usage.* (and gen_ai.cost.*) into its token and cost
75+
* columns on any span and sums them over the whole trace, so leaving the
76+
* root's copy would count each call twice. When the model-call spans carry
77+
* usage, every gen_ai.usage.<suffix> key on the root moves to
78+
* tanstack.ai.root_usage.<suffix>, and every gen_ai.cost.<suffix> key to
79+
* tanstack.ai.root_usage.cost.<suffix>. If no model-call span reported
80+
* usage, the root keeps it, because then it is the only copy.
7881
*/
79-
const PROMOTED_USAGE_KEYS = [
80-
"gen_ai.usage.input_tokens",
81-
"gen_ai.usage.output_tokens",
82-
"gen_ai.usage.total_tokens",
83-
];
82+
const PROMOTED_PREFIXES = ["gen_ai.usage.", "gen_ai.cost."];
83+
84+
function rootUsageKey(key) {
85+
return key.startsWith("gen_ai.usage.")
86+
? `tanstack.ai.root_usage.${key.slice("gen_ai.usage.".length)}`
87+
: `tanstack.ai.root_usage.${key.slice("gen_ai.".length)}`;
88+
}
8489

8590
function moveRootUsage(span) {
8691
// The SDK span's attributes object is what the exporter reads at end().
8792
const attributes = span?.attributes;
8893
if (!attributes) return;
89-
for (const key of PROMOTED_USAGE_KEYS) {
90-
if (key in attributes) {
91-
span.setAttribute(`tanstack.ai.root_usage.${key.split(".").pop()}`, attributes[key]);
92-
delete attributes[key];
93-
}
94+
for (const key of Object.keys(attributes)) {
95+
if (!PROMOTED_PREFIXES.some((prefix) => key.startsWith(prefix))) continue;
96+
const value = attributes[key];
97+
span.setAttribute(rootUsageKey(key), value);
98+
delete attributes[key];
9499
}
95100
}
96101

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
// Unit tests for futureAgiSpanKinds(), run with `node --test` (the contract
2+
// test runs this file). Fake spans stand in for the SDK span, and run() calls
3+
// the hooks in the order otelMiddleware does at @tanstack/ai 0.64.0:
4+
// attributeEnricher for the root at start, then per model call
5+
// attributeEnricher, the usage attributes and onSpanEnd, then applyRootUsage
6+
// on the root (the sum over calls, or the finish usage when no call reported
7+
// any) just before the root's onSpanEnd.
8+
import assert from "node:assert/strict";
9+
import { test } from "node:test";
10+
import { GEN_AI_SPAN_KIND, futureAgiSpanKinds } from "../src/tracing.mjs";
11+
12+
class FakeSpan {
13+
constructor() {
14+
this.attributes = {};
15+
}
16+
setAttribute(key, value) {
17+
this.attributes[key] = value;
18+
return this;
19+
}
20+
setAttributes(attributes) {
21+
for (const [key, value] of Object.entries(attributes ?? {})) {
22+
this.setAttribute(key, value);
23+
}
24+
return this;
25+
}
26+
}
27+
28+
function usage(input, output, extra = {}) {
29+
return {
30+
"gen_ai.usage.input_tokens": input,
31+
"gen_ai.usage.output_tokens": output,
32+
"gen_ai.usage.total_tokens": input + output,
33+
...extra,
34+
};
35+
}
36+
37+
function run(hooks, calls, { finishUsage = {}, rootExtra = {}, ctx = {} } = {}) {
38+
const runCtx = { threadId: "thread-1700000000000-abc1234", ...ctx };
39+
const root = new FakeSpan().setAttributes(
40+
hooks.attributeEnricher({ kind: "chat", ctx: runCtx }),
41+
);
42+
const modelCalls = calls.map((callUsage, iteration) => {
43+
const info = { kind: "iteration", ctx: runCtx, iteration };
44+
const span = new FakeSpan().setAttributes(hooks.attributeEnricher(info));
45+
span.setAttributes(callUsage ?? {});
46+
hooks.onSpanEnd(info, span);
47+
return span;
48+
});
49+
const summed = {};
50+
for (const callUsage of calls) {
51+
for (const [key, value] of Object.entries(callUsage ?? {})) {
52+
summed[key] = (summed[key] ?? 0) + value;
53+
}
54+
}
55+
root.setAttributes(Object.keys(summed).length > 0 ? summed : finishUsage);
56+
root.setAttributes(rootExtra);
57+
hooks.onSpanEnd({ kind: "chat", ctx: runCtx }, root);
58+
return { root, modelCalls, spans: [root, ...modelCalls] };
59+
}
60+
61+
function traceSum(spans, key) {
62+
return spans.reduce((total, span) => total + (span.attributes[key] ?? 0), 0);
63+
}
64+
65+
function promotedKeys(attributes) {
66+
return Object.keys(attributes).filter(
67+
(key) => key.startsWith("gen_ai.usage.") || key.startsWith("gen_ai.cost."),
68+
);
69+
}
70+
71+
function rootUsageKeys(attributes) {
72+
return Object.keys(attributes).filter((key) =>
73+
key.startsWith("tanstack.ai.root_usage."),
74+
);
75+
}
76+
77+
test("no model call reported usage: the root keeps its usage", () => {
78+
const finishUsage = usage(30, 12);
79+
const { root, spans } = run(futureAgiSpanKinds(), [null, null], { finishUsage });
80+
for (const [key, value] of Object.entries(finishUsage)) {
81+
assert.equal(root.attributes[key], value, key);
82+
}
83+
assert.deepEqual(rootUsageKeys(root.attributes), []);
84+
assert.equal(traceSum(spans, "gen_ai.usage.input_tokens"), 30);
85+
assert.equal(traceSum(spans, "gen_ai.usage.output_tokens"), 12);
86+
assert.equal(traceSum(spans, "gen_ai.usage.total_tokens"), 42);
87+
});
88+
89+
test("one model call: no kind and no promoted usage on the root", () => {
90+
const call = usage(11, 7);
91+
const { root, modelCalls, spans } = run(futureAgiSpanKinds(), [call]);
92+
assert.equal(root.attributes[GEN_AI_SPAN_KIND], undefined);
93+
assert.equal(modelCalls[0].attributes[GEN_AI_SPAN_KIND], "LLM");
94+
assert.deepEqual(promotedKeys(root.attributes), []);
95+
assert.equal(root.attributes["tanstack.ai.root_usage.input_tokens"], 11);
96+
for (const [key, value] of Object.entries(call)) {
97+
assert.equal(traceSum(spans, key), value, key);
98+
}
99+
});
100+
101+
test("two model calls: every promoted key leaves the root and each counts once", () => {
102+
const first = usage(11, 7, { "gen_ai.usage.cost": 0.25 });
103+
const second = usage(23, 5, {
104+
"gen_ai.usage.cost": 0.5,
105+
"gen_ai.usage.cache_read.input_tokens": 3,
106+
"gen_ai.usage.cache_creation.input_tokens": 4,
107+
"gen_ai.usage.reasoning.output_tokens": 2,
108+
});
109+
const { root, spans } = run(futureAgiSpanKinds(), [first, second], {
110+
rootExtra: { "gen_ai.cost.total": 0.75 },
111+
});
112+
assert.equal(root.attributes[GEN_AI_SPAN_KIND], "AGENT");
113+
assert.deepEqual(promotedKeys(root.attributes), []);
114+
115+
const expected = {
116+
input_tokens: 34,
117+
output_tokens: 12,
118+
total_tokens: 46,
119+
cost: 0.75,
120+
"cache_read.input_tokens": 3,
121+
"cache_creation.input_tokens": 4,
122+
"reasoning.output_tokens": 2,
123+
};
124+
for (const [suffix, value] of Object.entries(expected)) {
125+
assert.equal(root.attributes[`tanstack.ai.root_usage.${suffix}`], value, suffix);
126+
assert.equal(traceSum(spans, `gen_ai.usage.${suffix}`), value, suffix);
127+
}
128+
// gen_ai.cost.<x> keeps its own namespace, so it cannot collide with
129+
// gen_ai.usage.cost.
130+
assert.equal(root.attributes["tanstack.ai.root_usage.cost.total"], 0.75);
131+
assert.equal(traceSum(spans, "gen_ai.cost.total"), 0);
132+
});

‎typescript/examples/tanstack-ai/tests/test_tanstack_ai_contract.py‎

Lines changed: 30 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -174,26 +174,30 @@ def test_spans_carry_model_usage_and_span_kinds(default_run):
174174
assert second["gen_ai.usage.cache_read.input_tokens"] == 3
175175
assert second["gen_ai.usage.reasoning.output_tokens"] == 2
176176

177-
# Root: the whole chat() call. No operation name. TanStack sums usage over
178-
# calls onto the root; Future AGI sums promoted gen_ai.usage.* over every
179-
# span in a trace, so the recipe keeps the root's sum under
180-
# tanstack.ai.root_usage.* and each model call counts once.
177+
# Root: the whole chat() call. No operation name. TanStack sums every
178+
# gen_ai.usage.* key over calls onto the root; Future AGI sums promoted
179+
# gen_ai.usage.* over every span in a trace, so the recipe keeps the
180+
# root's sum under tanstack.ai.root_usage.<same suffix> and each model
181+
# call counts once.
181182
assert root["gen_ai.request.model"] == MODEL
182183
assert "gen_ai.operation.name" not in root
183184
assert root["tanstack.ai.iterations"] == 2
184-
assert "gen_ai.usage.input_tokens" not in root
185-
assert "gen_ai.usage.output_tokens" not in root
186-
assert "gen_ai.usage.total_tokens" not in root
187-
assert root["tanstack.ai.root_usage.input_tokens"] == (
188-
TOOL_CALL_USAGE["prompt_tokens"] + ANSWER_USAGE["prompt_tokens"]
189-
)
190-
assert root["tanstack.ai.root_usage.output_tokens"] == (
191-
TOOL_CALL_USAGE["completion_tokens"] + ANSWER_USAGE["completion_tokens"]
192-
)
193-
promoted_input = sum(
194-
_attributes(span).get("gen_ai.usage.input_tokens", 0) for span in spans.values()
195-
)
196-
assert promoted_input == TOOL_CALL_USAGE["prompt_tokens"] + ANSWER_USAGE["prompt_tokens"]
185+
assert not [k for k in root if k.startswith(("gen_ai.usage.", "gen_ai.cost."))], root
186+
per_call = {
187+
"input_tokens": (TOOL_CALL_USAGE["prompt_tokens"], ANSWER_USAGE["prompt_tokens"]),
188+
"output_tokens": (
189+
TOOL_CALL_USAGE["completion_tokens"], ANSWER_USAGE["completion_tokens"]
190+
),
191+
"total_tokens": (TOOL_CALL_USAGE["total_tokens"], ANSWER_USAGE["total_tokens"]),
192+
"cache_read.input_tokens": (0, 3),
193+
"reasoning.output_tokens": (0, 2),
194+
}
195+
for suffix, calls in per_call.items():
196+
assert root["tanstack.ai.root_usage." + suffix] == sum(calls), suffix
197+
promoted = sum(
198+
_attributes(span).get("gen_ai.usage." + suffix, 0) for span in spans.values()
199+
)
200+
assert promoted == sum(calls), suffix
197201
assert root["gen_ai.response.finish_reasons"] == ["stop"]
198202
assert root["gen_ai.span.kind"] == "AGENT"
199203

@@ -212,6 +216,15 @@ def test_spans_carry_model_usage_and_span_kinds(default_run):
212216
assert not _is_error(span.get("status", {})), span["name"]
213217

214218

219+
def test_span_kind_unit_tests():
220+
"""tests/span_kinds.test.mjs: futureAgiSpanKinds() against fake spans."""
221+
env = {name: os.environ[name] for name in ("PATH", "HOME", "SYSTEMROOT") if name in os.environ}
222+
result = run([NODE, "--test", str(TESTS / "span_kinds.test.mjs")], env, None, 60)
223+
output = result.stdout.decode() + result.stderr.decode()
224+
assert not result.timed_out, output
225+
assert result.returncode == 0, output
226+
227+
215228
def test_no_prompt_or_response_content_by_default(default_run):
216229
assert default_run.spans
217230
exported = json.dumps(default_run.spans)

0 commit comments

Comments
 (0)