Skip to content

Commit cc1a581

Browse files
committed
fix(claude-agent-sdk): review-r2 N1-N4: clean close after a result, task_id notifications, partial baselines
N1: close()/asyncDispose/abort after a result with no open turn, tool or subagent ends the query as returned (conversation OK), the normal end of a streaming-input session. A close with work still in flight stays cancelled. N2: task_notification without tool_use_id (optional, sdk.d.ts:5997) is matched through the task_started task_id mapping. N3: an unknown usage baseline covers only the first result; later results in the same query promote their delta against it (flag stays true). N4: resume/fork with resumeSessionAt has no known baseline. README notes the /clear undercount edge. Tests first: 8 jest cases and the new real-CLI streaming_close contract test failed on 06a3a20 and pass now. approved: Nikhil 2026-10-03 blanket Refs: TH-8235
1 parent 06a3a20 commit cc1a581

7 files changed

Lines changed: 204 additions & 8 deletions

File tree

‎typescript/packages/traceai_claude_agent_sdk/README.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ await shutdown(); // flush before a short script exits
5252
### What the wrapper does and does not do
5353

5454
- Calls the original `query()` with the same params object. Options (`resume`, `forkSession`, `sessionId`, `hooks`, `mcpServers`, `agents`, `abortController`, `allowedTools`, `permissionMode`, `maxTurns`, `env`, ...) are passed through untouched. The wrapper never sets `ANTHROPIC_BASE_URL`.
55-
- Yields the same message objects, in order. Control methods on the returned `Query` (`interrupt()`, `setModel()`, ...) go to the original. `close()` and `Symbol.asyncDispose` first end the query's open spans as cancelled, then go to the original.
55+
- Yields the same message objects, in order. Control methods on the returned `Query` (`interrupt()`, `setModel()`, ...) go to the original. `close()` and `Symbol.asyncDispose` first end the query's open spans, then go to the original: as cancelled if a turn, tool or subagent was still running, or as a normal end if a result had arrived and nothing was running (the usual way to end a streaming-input session).
5656
- A tracing error never fails the agent. An exporter or collector failure is logged and does not reject the iterator.
5757
- `startup()` is not wrapped in 0.1.0.
5858

@@ -69,7 +69,7 @@ One API response that the SDK splits over several assistant messages (same `mess
6969

7070
### Background subagents
7171

72-
A foreground subagent span ends with its `tool.Agent` / `tool.Task` result. A subagent that runs in the background stays open after that tool result (which is only the "running in the background" placeholder) until its `system/task_notification`, so its later turns and tools still nest under a live span. A subagent counts as background when the tool input has `run_in_background: true`, `task_started.is_backgrounded` is true, a later `task_updated` carries `patch.is_backgrounded: true`, or the app calls `Query.backgroundTasks()` (all foreground subagents, or the one whose `tool_use_id` it was given; undone if the call rejects or returns `false`). The subagent span is ERROR when the notification status is `failed` or `stopped` (`claude_agent.subagent.status` has the value). A background subagent with no notification by the time the query ends is closed as ERROR "Subagent span not completed". The subagent span can therefore end after its parent tool span.
72+
A foreground subagent span ends with its `tool.Agent` / `tool.Task` result. A subagent that runs in the background stays open after that tool result (which is only the "running in the background" placeholder) until its `system/task_notification` (matched by `tool_use_id`, or by `task_id` through `task_started` when the notification has no `tool_use_id`), so its later turns and tools still nest under a live span. A subagent counts as background when the tool input has `run_in_background: true`, `task_started.is_backgrounded` is true, a later `task_updated` carries `patch.is_backgrounded: true`, or the app calls `Query.backgroundTasks()` (all foreground subagents, or the one whose `tool_use_id` it was given; undone if the call rejects or returns `false`). The subagent span is ERROR when the notification status is `failed` or `stopped` (`claude_agent.subagent.status` has the value). A background subagent with no notification by the time the query ends is closed as ERROR "Subagent span not completed". The subagent span can therefore end after its parent tool span.
7373

7474
| Field | Attributes | Source |
7575
|---|---|---|
@@ -92,7 +92,8 @@ The Future AGI collector promotes `gen_ai.usage.*` and `gen_ai.cost.total` on an
9292
- Tokens come from the latest result's `modelUsage`, summed over every model (main loop, subagents, compaction). Cost comes from `total_cost_usd`. Both are running totals for the session (`sdk.d.ts:5679`, `5687`). `result.usage` is not read: the SDK documents it as main-loop only and per turn in streaming-input mode (`sdk.d.ts:5683`). This replaces the spec's "result message usage" mapping.
9393
- Streaming input (one `query()`, several user turns) yields one result per turn. The conversation span keeps the latest running totals, so tokens and cost cover every turn. It stays one conversation span per `query()`; the Python `ClaudeSDKClient` path makes one per user turn.
9494
- A resumed (`resume`), continued (`continue`) or forked (`forkSession`) session's first result already carries the earlier turns. The wrapper keeps an in-process map of session id to the last totals it saw (at most 1000 sessions) and writes only the difference. A fork starts from its parent's totals. A drop in the running total within one query (a `/clear`) counts the spend on both sides of it.
95-
- When there is no baseline in this process (resume after a restart, `continue` + `forkSession`, or a first result below the saved totals), no `gen_ai.usage.*` / `gen_ai.cost.total` / `claude_agent.cost.total_usd` is written and `claude_agent.usage.baseline_unknown=true`.
95+
- When there is no baseline in this process (resume after a restart, `continue` + `forkSession`, `resume` with `resumeSessionAt`, or a first result below the saved totals), the first result writes no `gen_ai.usage.*` / `gen_ai.cost.total` / `claude_agent.cost.total_usd` and the span has `claude_agent.usage.baseline_unknown=true`. Later results in the same `query()` (streaming input) use that first result's totals as their baseline, so the promoted keys then hold the spend after the first result and the flag stays `true`.
96+
- Known undercount: a `/clear` followed by more spend than the session had before it is not seen as a drop, so the pre-clear share of that query is missed. Nothing here can count spend twice.
9697
- Every conversation span with a result also carries the running totals on unpromoted keys: `claude_agent.cumulative.cost_usd`, `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_creation_tokens`.
9798
- A result with no cost and no `modelUsage`, or a zeroed crash result, writes none of these keys and leaves the session's saved totals unchanged.
9899

@@ -130,7 +131,7 @@ With 0.3.289 the CLI then calls `POST {ANTHROPIC_BASE_URL}/v1/messages?beta=true
130131
- **No traces.** `register()` needs `projectName` or `FI_PROJECT_NAME`.
131132
- **Zero spans from a short script.** `await shutdown()` before exit.
132133
- **Duplicate model spans.** Do not also run a provider instrumentor on the same calls.
133-
- **A span left open after abort.** Every span for the query is ended with status ERROR and `claude_agent.cancelled=true` when its `AbortController` aborts, when you call `close()` on the returned `Query` (the SDK's abort path), or when `await using` disposes it (`Symbol.asyncDispose`). Calling `close()` after the stream has completed changes nothing. Anything else is a bug here.
134+
- **A span left open after abort.** Every span for the query is ended with status ERROR and `claude_agent.cancelled=true` when its `AbortController` aborts, when you call `close()` on the returned `Query` (the SDK's abort path), or when `await using` disposes it (`Symbol.asyncDispose`). Calling `close()` after the stream has completed changes nothing, and `close()` after a result with nothing still running ends the conversation OK. Anything else is a bug here.
134135

135136
## Tests
136137

‎typescript/packages/traceai_claude_agent_sdk/contract/run_real_sdk.mjs‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@
2222
// starts a session, PHASE=second resumes RESUME_SESSION_ID.
2323
// streaming one query() with an AsyncIterable prompt of two user turns;
2424
// the second turn is sent after the first result.
25+
// streaming_close one streaming-input query(); after its result the app
26+
// calls close() while the prompt iterable is still open (the
27+
// usual way to end a streaming-input session).
2528
// WORKDIR (optional): use and keep this directory instead of a temp dir.
2629
// Prints {"requests": [...], "messages": [...], "queries": [[...], ...]} on stdout.
2730
import http from "node:http";
@@ -221,6 +224,26 @@ const SCENARIOS = {
221224
}
222225
return [out];
223226
},
227+
async streaming_close() {
228+
let release;
229+
const held = new Promise((resolve) => (release = resolve));
230+
const userTurn = (content) => ({ type: "user", message: { role: "user", content }, parent_tool_use_id: null });
231+
async function* prompt() {
232+
yield userTurn("First question.");
233+
await held; // more input could still come: the stream stays open
234+
}
235+
const q = tracedQuery({ prompt: prompt(), options: options() });
236+
const out = [];
237+
for (;;) {
238+
const { value, done } = await q.next();
239+
if (done) break;
240+
out.push(value);
241+
if (value.type === "result") break;
242+
}
243+
q.close();
244+
release();
245+
return [out];
246+
},
224247
};
225248

226249
function sessionOf(messages) {

‎typescript/packages/traceai_claude_agent_sdk/contract/test_harness_contract.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -478,6 +478,27 @@ def test_real_sdk_streaming_input_counts_tokens_once(built_package: Path) -> Non
478478
_assert_only_conversations_carry_usage(spans)
479479

480480

481+
@requires_real_sdk
482+
def test_real_sdk_close_after_result_ends_streaming_session_ok(built_package: Path) -> None:
483+
"""N1: close() after a result, with nothing still running, is the normal end of a
484+
streaming-input session (the prompt iterable stays open). The conversation must end
485+
OK, not cancelled, and still carry the session's spend once."""
486+
output, spans, exported = _run_real_sdk("streaming_close", "th8235-real-sdk-streaming-close")
487+
(messages,) = output["queries"]
488+
results = _results(messages)
489+
assert len(results) == 1 and messages[-1]["type"] == "result"
490+
491+
names = sorted(span["name"] for span in spans)
492+
assert names == sorted(["claude_agent.conversation", "claude_agent.assistant_turn"]), names
493+
for span in spans:
494+
assert "claude_agent.cancelled" not in _attrs(span), span["name"]
495+
conversation = _one(spans, "claude_agent.conversation")
496+
assert _status_code(conversation) == 1
497+
_assert_promoted_sum_equals([conversation], _cumulative(results[0]))
498+
_assert_only_conversations_carry_usage(spans)
499+
_assert_exported_with_both_keys(exported, "th8235-real-sdk-streaming-close")
500+
501+
481502
@requires_real_sdk
482503
def test_real_sdk_query_through_anthropic_base_url_mock(built_package: Path) -> None:
483504
"""The real SDK query() and bundled CLI, wrapped, against a loopback Messages API mock.

‎typescript/packages/traceai_claude_agent_sdk/src/__tests__/usage.test.ts‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,50 @@ describe("R1: resumed, continued and forked sessions are counted once", () => {
268268
expect(conversation.attributes["claude_agent.usage.baseline_unknown"]).toBe(true);
269269
});
270270

271+
it("N3: an unknown baseline covers only the first result; later results in the query promote their delta", async () => {
272+
const id = newSessionId(); // resumed, but this process never saw the session (e.g. after a restart)
273+
const journey: SDKMessage[] = [
274+
init({ session_id: id }),
275+
assistant("msg_u1", [text("one")], null, { session_id: id }),
276+
resultSuccess("one", { session_id: id, total_cost_usd: 0.001, modelUsage: { [MODEL]: modelUsage(100, 40, 0.001) } }),
277+
assistant("msg_u2", [text("two")], null, { session_id: id }),
278+
resultSuccess("two", { session_id: id, total_cost_usd: 0.0014, modelUsage: { [MODEL]: modelUsage(130, 52, 0.0014) } }),
279+
];
280+
const { conversation } = await runQuery(journey, { resume: id });
281+
expect(conversation.attributes["gen_ai.cost.total"]).toBeCloseTo(0.0004, 12);
282+
expect(conversation.attributes["claude_agent.cost.total_usd"]).toBeCloseTo(0.0004, 12);
283+
expect(conversation.attributes["gen_ai.usage.input_tokens"]).toBe(30);
284+
expect(conversation.attributes["gen_ai.usage.output_tokens"]).toBe(12);
285+
expect(conversation.attributes["gen_ai.usage.total_tokens"]).toBe(42);
286+
// Still flagged: the spend before the first result is not in the promoted keys.
287+
expect(conversation.attributes["claude_agent.usage.baseline_unknown"]).toBe(true);
288+
expect(conversation.attributes["claude_agent.cumulative.cost_usd"]).toBeCloseTo(0.0014, 12);
289+
// The next resume in this process starts from the latest totals.
290+
const next = await runQuery(sessionQuery(id, { cost: 0.002, input: 160, output: 64 }), { resume: id });
291+
expect(next.conversation.attributes["gen_ai.cost.total"]).toBeCloseTo(0.0006, 12);
292+
expect(next.conversation.attributes["claude_agent.usage.baseline_unknown"]).toBe(false);
293+
});
294+
295+
it.each([
296+
["resume", false],
297+
["fork", true],
298+
] as const)(
299+
"N4: %s with resumeSessionAt has no known baseline (the saved totals are for a later message)",
300+
async (_mode, forkSession) => {
301+
const id = newSessionId();
302+
await runQuery(sessionQuery(id, { cost: 0.001, input: 100, output: 40 }));
303+
const target = forkSession ? newSessionId() : id;
304+
const { conversation } = await runQuery(sessionQuery(target, { cost: 0.0015, input: 150, output: 60 }), {
305+
resume: id,
306+
resumeSessionAt: "msg-uuid-earlier",
307+
...(forkSession ? { forkSession: true } : {}),
308+
});
309+
expect(promoted(conversation)).toEqual({});
310+
expect(conversation.attributes["claude_agent.cost.total_usd"]).toBeUndefined();
311+
expect(conversation.attributes["claude_agent.usage.baseline_unknown"]).toBe(true);
312+
},
313+
);
314+
271315
it("a mid-query /clear (running total drops) counts the spend before and after it", async () => {
272316
const id = newSessionId();
273317
const journey: SDKMessage[] = [

‎typescript/packages/traceai_claude_agent_sdk/src/__tests__/wrapper.test.ts‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import type { SDKMessage } from "@anthropic-ai/claude-agent-sdk";
12
import { SpanStatusCode, context, trace } from "@opentelemetry/api";
23
import {
34
BasicTracerProvider,
@@ -423,6 +424,20 @@ describe("wrapQuery", () => {
423424
expect(subagent.status.code).toBe(SpanStatusCode.ERROR);
424425
},
425426
);
427+
428+
it("N2: matches a task_notification without tool_use_id (optional, sdk.d.ts:5997) by its task_id", async () => {
429+
const journey = backgroundedSubagentJourney().map((m) => {
430+
if (m.type === "system" && m.subtype === "task_notification") {
431+
const { tool_use_id: _dropped, ...rest } = m as typeof m & { tool_use_id?: string };
432+
return rest as SDKMessage;
433+
}
434+
return m;
435+
});
436+
const { spans } = await run(journey);
437+
const subagent = one(spans, "claude_agent.subagent.code-reviewer");
438+
expect(subagent.attributes["claude_agent.subagent.status"]).toBe("completed");
439+
expect(subagent.status.code).toBe(SpanStatusCode.OK);
440+
});
426441
});
427442

428443
describe("J3: MCP tool error and error result", () => {
@@ -618,6 +633,65 @@ describe("wrapQuery", () => {
618633
}
619634
});
620635

636+
/** Read until the stream has yielded a result message (done is still false). */
637+
async function readThroughResult(q: AsyncGenerator<SDKMessage, void>): Promise<void> {
638+
for (;;) {
639+
const step = await q.next();
640+
if (step.done) throw new Error("stream ended before a result");
641+
if (step.value.type === "result") return;
642+
}
643+
}
644+
645+
it.each([
646+
["close", "simpleToolJourney"],
647+
["asyncDispose", "simpleToolJourney"],
648+
["close", "streamingInputJourney"],
649+
["asyncDispose", "streamingInputJourney"],
650+
] as const)(
651+
"N1: %s() after a result with nothing in flight ends the conversation OK, not cancelled (%s)",
652+
async (method, journeyName) => {
653+
const { provider, exporter } = memoryProvider();
654+
const journey = journeyName === "simpleToolJourney" ? simpleToolJourney() : streamingInputJourney();
655+
const fake = makeFakeQuery(journey);
656+
const q = wrapQuery(fake.query, { tracerProvider: provider })({ prompt: PROMPT });
657+
await readThroughResult(q);
658+
expect(exporter.getFinishedSpans().some((s) => s.name === "claude_agent.conversation")).toBe(false);
659+
660+
if (method === "close") {
661+
q.close();
662+
} else {
663+
await (q as unknown as Record<symbol, () => Promise<void>>)[ASYNC_DISPOSE]();
664+
}
665+
666+
const spans = exporter.getFinishedSpans();
667+
const conversation = one(spans, "claude_agent.conversation");
668+
expect(conversation.status.code).toBe(SpanStatusCode.OK);
669+
expect(conversation.attributes["claude_agent.cancelled"]).toBeUndefined();
670+
for (const span of spans) {
671+
expect(span.ended).toBe(true);
672+
expect(span.attributes["claude_agent.cancelled"]).toBeUndefined();
673+
}
674+
},
675+
);
676+
677+
it("N1: close() after a result while a background subagent still runs stays cancelled", async () => {
678+
const { provider, exporter } = memoryProvider();
679+
// The result arrives before the background subagent's task_notification.
680+
const journey = backgroundedSubagentJourney().filter(
681+
(m) => !(m.type === "system" && m.subtype === "task_notification"),
682+
);
683+
const fake = makeFakeQuery(journey);
684+
const q = wrapQuery(fake.query, { tracerProvider: provider })({ prompt: PROMPT });
685+
await readThroughResult(q);
686+
q.close();
687+
const spans = exporter.getFinishedSpans();
688+
const subagent = one(spans, "claude_agent.subagent.code-reviewer");
689+
expect(subagent.attributes["claude_agent.cancelled"]).toBe(true);
690+
const conversation = one(spans, "claude_agent.conversation");
691+
expect(conversation.status.code).toBe(SpanStatusCode.ERROR);
692+
expect(conversation.attributes["claude_agent.cancelled"]).toBe(true);
693+
});
694+
621695
it("ends open spans when the consumer breaks out early", async () => {
622696
const { provider, exporter } = memoryProvider();
623697
const fake = makeFakeQuery(simpleToolJourney());

0 commit comments

Comments
 (0)