Skip to content

Commit 48e2af0

Browse files
committed
feat(examples): send the caller's TanStack AI thread id as session.id
The architecture says the recipe shows the traceAI session helper, but @traceai/fi-core setSession() only sets an OpenTelemetry context value. otelMiddleware never reads it, and register()'s provider hands out the plain SDK tracer, so no session reached the spans. futureAgiOtelMiddleware(tracer, { threadIdAsSession: true }) now adds session.id = ctx.threadId (ChatMiddlewareContext.threadId, @tanstack/ai 0.64.0 middleware/types.ts:207) to every span from attributeEnricher. It is opt-in because chat() generates thread-<ms>-<random> when the caller passes no threadId, which would make each request a session. chatRoute and answer accept { threadId }; the CLI takes it as argv[3]. The README documents this as a deviation from the spec's helper. Test first: the contract run now passes a thread id and asserts session.id on all four spans (KeyError before), and the failed-call run (no thread id) asserts none. Unit tests cover the enricher with and without threadIdAsSession, including tool spans. approved: Nikhil 2026-10-03 blanket Refs: TH-8238
1 parent 056bb44 commit 48e2af0

5 files changed

Lines changed: 110 additions & 20 deletions

File tree

‎typescript/examples/tanstack-ai/README.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,28 @@ model-call spans carry `gen_ai.operation.name`. The root's kind is set in
7878
`onSpanEnd`, because `attributeEnricher` runs for the root before the number
7979
of model calls is known.
8080

81+
## Session
82+
83+
Pass the caller's conversation id as TanStack's `threadId`, and build the
84+
middleware with `threadIdAsSession: true`. Every span then carries
85+
`session.id` = that thread id, so Future AGI groups the conversation's traces:
86+
87+
```js
88+
const middleware = futureAgiOtelMiddleware(tracer, { threadIdAsSession: true });
89+
chat({ adapter, messages, threadId: conversationId, middleware: [middleware] });
90+
```
91+
92+
`src/chat.mjs` takes it as a second argument:
93+
`node src/chat.mjs "What is the weather in Paris?" conversation-42`.
94+
95+
Deviation from the traceAI context helper: `setSession()` from
96+
`@traceai/fi-core` only sets an OpenTelemetry context value. Neither
97+
`otelMiddleware` nor the plain SDK tracer that `register()`'s provider hands
98+
out reads it, so it would not reach these spans. The
99+
recipe uses `threadId` instead. Leave `threadIdAsSession` off when the caller
100+
has no conversation id: `chat()` then generates a new `thread-<ms>-<random>`
101+
id per call, and each request would become its own session.
102+
81103
## Privacy
82104

83105
`captureContent` defaults to `false`, so no prompt, completion, system prompt,

‎typescript/examples/tanstack-ai/src/chat.mjs‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
// A Node "route" that answers one question with TanStack AI and traces it to
2-
// Future AGI. Run: node src/chat.mjs "What is the weather in Paris?"
2+
// Future AGI. Run: node src/chat.mjs "What is the weather in Paris?" [thread-id]
3+
// A thread id (the caller's conversation id) becomes the Future AGI session.
34
//
45
// Server side only. FI_API_KEY and FI_SECRET_KEY must never enter a browser
56
// bundle.
@@ -33,13 +34,14 @@ const getWeather = toolDefinition({
3334
* onError hook, so the chat and model-call spans are never ended or exported
3435
* and a failed request leaves no trace.
3536
*/
36-
export async function answer(question, middleware) {
37+
export async function answer(question, middleware, { threadId } = {}) {
3738
const stream = chat({
3839
adapter: openaiChatCompletions(process.env.OPENAI_MODEL ?? "gpt-4o-mini"),
3940
systemPrompts: ["You are a concise weather assistant."],
4041
messages: [{ role: "user", content: question }],
4142
tools: [getWeather],
4243
middleware: [middleware],
44+
...(threadId ? { threadId } : {}),
4345
});
4446
let text = "";
4547
let runError = null;
@@ -56,23 +58,29 @@ export async function answer(question, middleware) {
5658
return text;
5759
}
5860

59-
/** The route: one traced chat() call, flushed in finally. */
60-
export async function chatRoute(question, tracerProvider) {
61+
/**
62+
* The route: one traced chat() call, flushed in finally. `threadId` is the
63+
* caller's conversation id; when given, it is chat()'s threadId and every
64+
* span's session.id.
65+
*/
66+
export async function chatRoute(question, tracerProvider, { threadId } = {}) {
6167
const middleware = futureAgiOtelMiddleware(
6268
tracerProvider.getTracer("tanstack-ai"),
69+
{ threadIdAsSession: Boolean(threadId) },
6370
);
6471
try {
65-
return await answer(question, middleware);
72+
return await answer(question, middleware, { threadId });
6673
} finally {
6774
await flushTraces(tracerProvider);
6875
}
6976
}
7077

7178
async function main() {
7279
const question = process.argv[2] ?? "What is the weather in Paris?";
80+
const threadId = process.argv[3];
7381
const tracerProvider = registerFutureAgiTracing();
7482
try {
75-
console.log(await chatRoute(question, tracerProvider));
83+
console.log(await chatRoute(question, tracerProvider, { threadId }));
7684
} catch (error) {
7785
// The route's error response. The spans are already exported.
7886
console.error(`chat failed: ${error?.message ?? String(error)}`);

‎typescript/examples/tanstack-ai/src/tracing.mjs‎

Lines changed: 33 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -106,19 +106,39 @@ function moveRootUsage(span) {
106106
}
107107
}
108108

109-
export function futureAgiSpanKinds() {
109+
export const SESSION_ID = "session.id";
110+
111+
/**
112+
* Span kinds, usage and session for the spans otelMiddleware emits.
113+
*
114+
* With `threadIdAsSession: true`, every span also gets
115+
* session.id = ctx.threadId, the `threadId` the caller passed to chat().
116+
* Set it only when the caller passed one: chat() otherwise generates a
117+
* fresh thread-<ms>-<random> id per call, which would make every request its
118+
* own session. @traceai/fi-core's setSession() only sets a context value;
119+
* neither otelMiddleware nor the plain SDK tracer register()'s provider
120+
* hands out reads it, so the recipe uses TanStack's own threadId instead.
121+
*/
122+
export function futureAgiSpanKinds({ threadIdAsSession = false } = {}) {
110123
const iterationsByRun = new WeakMap();
111124
const usageOnIterations = new WeakSet();
125+
const kindAttributes = (info) => {
126+
if (info.kind === "iteration") {
127+
iterationsByRun.set(info.ctx, info.iteration + 1);
128+
return { [GEN_AI_SPAN_KIND]: "LLM" };
129+
}
130+
if (info.kind === "tool") {
131+
return { [GEN_AI_SPAN_KIND]: "TOOL" };
132+
}
133+
return {};
134+
};
112135
return {
113136
attributeEnricher(info) {
114-
if (info.kind === "iteration") {
115-
iterationsByRun.set(info.ctx, info.iteration + 1);
116-
return { [GEN_AI_SPAN_KIND]: "LLM" };
117-
}
118-
if (info.kind === "tool") {
119-
return { [GEN_AI_SPAN_KIND]: "TOOL" };
137+
const attributes = kindAttributes(info);
138+
if (threadIdAsSession && info.ctx?.threadId) {
139+
attributes[SESSION_ID] = info.ctx.threadId;
120140
}
121-
return {};
141+
return attributes;
122142
},
123143
onSpanEnd(info, span) {
124144
if (info.kind === "iteration" && span?.attributes?.["gen_ai.usage.input_tokens"] !== undefined) {
@@ -140,10 +160,13 @@ export function futureAgiSpanKinds() {
140160
* The one middleware to pass to chat(). captureContent is left unset, so it
141161
* stays at its default of false and no prompt, completion, or tool argument
142162
* text lands on a span. Do not pass captureContent: true.
163+
*
164+
* Pass `{ threadIdAsSession: true }` when this chat() call gets the caller's
165+
* `threadId` (see futureAgiSpanKinds).
143166
*/
144-
export function futureAgiOtelMiddleware(tracer) {
167+
export function futureAgiOtelMiddleware(tracer, { threadIdAsSession = false } = {}) {
145168
return otelMiddleware({
146169
tracer,
147-
...futureAgiSpanKinds(),
170+
...futureAgiSpanKinds({ threadIdAsSession }),
148171
});
149172
}

‎typescript/examples/tanstack-ai/tests/span_kinds.test.mjs‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,3 +223,30 @@ test("a tight attribute count limit never leaves usage counted twice", () => {
223223
assert.equal(traceSum([root, ...modelCalls], "gen_ai.usage.input_tokens"), 34);
224224
assert.equal(traceSum([root, ...modelCalls], "gen_ai.usage.output_tokens"), 12);
225225
});
226+
227+
test("threadIdAsSession: every span carries the caller's thread id as session.id", () => {
228+
const hooks = futureAgiSpanKinds({ threadIdAsSession: true });
229+
const ctx = { threadId: "conversation-42" };
230+
const { spans } = run(hooks, [usage(11, 7), usage(23, 5)], { ctx });
231+
assert.equal(spans.length, 3);
232+
for (const span of spans) {
233+
assert.equal(span.attributes["session.id"], "conversation-42");
234+
}
235+
const tool = hooks.attributeEnricher({
236+
kind: "tool",
237+
ctx,
238+
toolCallId: "call_1",
239+
toolName: "get_weather",
240+
iteration: 0,
241+
});
242+
assert.deepEqual(tool, { [GEN_AI_SPAN_KIND]: "TOOL", "session.id": "conversation-42" });
243+
});
244+
245+
test("no session.id by default, although TanStack always sets a threadId", () => {
246+
// chat() generates thread-<ms>-<random> when the caller passes none, so
247+
// ctx.threadId alone does not say the caller has a conversation.
248+
const { spans } = run(futureAgiSpanKinds(), [usage(11, 7)]);
249+
for (const span of spans) {
250+
assert.equal(span.attributes["session.id"], undefined);
251+
}
252+
});

‎typescript/examples/tanstack-ai/tests/test_tanstack_ai_contract.py‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@
5656
CITY = "CITY-MARKER-9a2e"
5757
ANSWER = "ANSWER-MARKER-3d7b it is sunny"
5858
SYSTEM_PROMPT = "You are a concise weather assistant." # fixed in src/chat.mjs
59+
THREAD_ID = "conversation-th8238-7c41" # the caller's conversation id
5960

6061
ROOT = "chat {0}".format(MODEL)
6162
ITERATION_0 = "chat {0} #0".format(MODEL)
@@ -81,8 +82,9 @@ def _env(fi_base_url: str, openai_base_url: str) -> dict[str, str]:
8182
return env
8283

8384

84-
def _node(script: Path, fi_base_url: str, fake: FakeOpenAI) -> Any:
85-
result = run([NODE, str(script), PROMPT], _env(fi_base_url, fake.base_url), None, 60)
85+
def _node(script: Path, fi_base_url: str, fake: FakeOpenAI, *args: str) -> Any:
86+
argv = [NODE, str(script), PROMPT, *args]
87+
result = run(argv, _env(fi_base_url, fake.base_url), None, 60)
8688
assert not result.timed_out, result.stderr.decode()
8789
return result
8890

@@ -117,11 +119,12 @@ def _is_error(status: dict[str, Any]) -> bool:
117119

118120
@pytest.fixture(scope="module")
119121
def default_run():
120-
"""One run of the example exactly as shipped (captureContent unset)."""
122+
"""One run of the example exactly as shipped (captureContent unset),
123+
called with a thread id."""
121124
with Receiver() as receiver, FakeOpenAI(
122125
CITY, ANSWER
123126
) as fake:
124-
result = _node(EXAMPLE / "src" / "chat.mjs", receiver.origin, fake)
127+
result = _node(EXAMPLE / "src" / "chat.mjs", receiver.origin, fake, THREAD_ID)
125128
yield SimpleNamespace(
126129
result=result,
127130
spans=receiver.spans(),
@@ -207,6 +210,10 @@ def test_spans_carry_model_usage_and_span_kinds(default_run):
207210
assert tool["tanstack.ai.tool.outcome"] == "success"
208211
assert tool["gen_ai.span.kind"] == "TOOL"
209212

213+
# The caller's thread id is the Future AGI session, on every span.
214+
for name, span in spans.items():
215+
assert _attributes(span)["session.id"] == THREAD_ID, name
216+
210217
# One trace, nested root -> iteration -> tool, without a context manager.
211218
assert len({span["traceId"] for span in spans.values()}) == 1
212219
assert spans[ITERATION_0]["parentSpanId"] == spans[ROOT]["spanId"]
@@ -313,6 +320,9 @@ def test_failed_model_call_exports_error_spans():
313320
assert status.get("message") == "500 boom", (name, status)
314321
assert spans[ITERATION_0]["parentSpanId"] == spans[ROOT]["spanId"]
315322
assert _attributes(spans[ITERATION_0])["gen_ai.span.kind"] == "LLM"
323+
# No thread id was passed, so no session (chat() generated its own id).
324+
for name in (ROOT, ITERATION_0):
325+
assert "session.id" not in _attributes(spans[name]), name
316326

317327

318328
def test_abort_mid_stream_ends_spans_as_cancelled():

0 commit comments

Comments
 (0)