feat(claude-agent-sdk): add support for @anthropic-ai/claude-agent-sdk#9202
Merged
Conversation
…agent-sdk
Adds automatic instrumentation for the Claude Agent SDK, providing
full visibility into agentic sessions via APM tracing and LLM Obs.
Span hierarchy aligned with trajectory-spec APPENDIX-DD-LLMOBS-MAPPING:
agent (session)
└── agent (turn)
├── tool ({tool_name})
└── agent (subagent-{agent_type})
Key design decisions:
- Uses SDK's first-class hooks API (SessionStart, SessionEnd, Stop,
PreToolUse, PostToolUse, SubagentStart, SubagentStop, etc.)
- Turn spans are `agent` kind (not workflow) per spec
- Dynamic span names: session, turn, {toolName}, subagent-{type}
- Model name split from provider prefix (anthropic/claude-sonnet-4-6)
- Turn output from Stop hook's last_assistant_message
- Captures cwd, transcript_path, agent_type, is_interrupt, start_trigger
- Pure ESM package handled via import-in-the-middle with esmFirst: true
- User hooks preserved via mergeHooks (user matchers before tracer matchers)
Known gap: No LLM-level spans - the Agent SDK bundles its own Anthropic
client internally, so the existing @anthropic-ai/sdk shimmer doesn't fire.
This matches dd-trace-py's approach (agent + tool spans only).
Tests: 41 APM tracing + 13 LLM Obs tests.
Root cause: OTEL env vars caused dd-trace to use OTLP exporter instead of the agent exporter. The mock test agent only handles /v0.4/traces. - APM test: clear OTEL_* env vars before tracer init - LLM Obs test: rewrite to test span event structure directly - Instrumentation: finishSession closes pending child spans first - VCR proxy: use canonical body for stable cassette hashing
Add await before origIterator.return/throw calls to satisfy require-await lint rule. Bump minimum SDK version from 0.2.0 to 0.2.1 in rewriter versionRange and test withVersions filter — 0.2.0 uses require() in ESM scope which breaks on Node 22.
…solution
The LLM Obs test was missing the withVersions wrapper that sets NODE_PATH
to the versioned SDK's node_modules directory. Without it, the bare
require('@anthropic-ai/claude-agent-sdk') fails with "Cannot find module".
This matches the pattern used by the APM test.
…o turn spans Turn spans are now root spans carrying all session metadata (model, start_trigger, project_dir, permission_mode, agent_type, transcript_path). Session-level span removed — the turn is the primary unit of work.
This reverts commit d4be02b.
…g/dd-trace-js into feat/claude-agent-sdk-integration
mr-lee
marked this pull request as ready for review
July 7, 2026 18:13
mr-lee
requested review from
crysmags and
khanayan123
and removed request for
a team
July 7, 2026 18:13
sabrenner
changed the title
@anthropic-ai/claude-agent-sdk
sabrenner
previously approved these changes
Jul 7, 2026
sabrenner
left a comment
sabrenner
left a comment
Collaborator
There was a problem hiding this comment.
discussed offline, but this PR is in a good state, especially seeing as how unconventional of an integration this is. we will follow up with changes as needed!
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c5cbd1020
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| addHook(hook, exports => { | ||
| if (!querySubscribed) { | ||
| querySubscribed = true | ||
| queryChannel.subscribe({ |
There was a problem hiding this comment.
When the claude-agent-sdk plugin is disabled, this unconditional subscription still makes the orchestrion query channel look subscribed, so every query() call has tracer hooks injected and its async iterator wrapped/fully scanned even though the downstream APM/LLMObs channels have no subscribers. That means tracer.use('claude-agent-sdk', false) or the disabled env var still changes SDK hook behavior and adds stream-processing overhead; gate the start/end work on downstream subscribers or register it only while the plugin is enabled.
Useful? React with 👍 / 👎.
Comment on lines
+385
to
+386
| if (resultChunk?.type === 'user') { | ||
| toolCtx.output = resultChunk.message?.content |
There was a problem hiding this comment.
For parallel tool calls, Claude can emit one user chunk containing multiple tool_result blocks. In that case each processTool() call reaches the same user chunk, but this assigns the entire message.content array to every tool span, so each tool's APM/LLMObs output can include sibling tools' results. Filter the content to the block whose tool_use_id matches the current id before storing toolCtx.output.
Useful? React with 👍 / 👎.
Comment on lines
+76
to
+77
| static id = 'claude_agent_sdk_step_llmobs' | ||
| static system = 'claude-agent-sdk' |
There was a problem hiding this comment.
This class and the LLM/tool LLMObs classes below define system but not static integration, while LLMObsPlugin.start() passes this.constructor.integration to both telemetry and registerLLMObsSpan(). As a result, all step/llm/tool span events from this integration are registered without the integration: claude-agent-sdk tag and are counted as non-instrumented/N/A in LLMObs telemetry; add the same integration value used by the query plugin to these child classes.
Useful? React with 👍 / 👎.
| processChunks(chunks, ctx) | ||
|
|
||
| ctx.streamResolved = true | ||
| queryChannel.asyncEnd.publish(ctx) |
There was a problem hiding this comment.
For any query whose stream runs longer than the initial query() call, ctx.finishTime is still the orchestrion end time captured when the async iterable was returned. Publishing asyncEnd here makes QueryTracingPlugin finish the root span at that stale timestamp, so query spans can end before all step/LLM/tool children and report near-zero duration for long agent runs. Set ctx.finishTime when the iterator completes before publishing asyncEnd.
Useful? React with 👍 / 👎.
Comment on lines
+456
to
+457
| iterator.next = shimmer.wrapCallback(iterator.next, next => function () { | ||
| return next.apply(this, arguments).then(result => { |
There was a problem hiding this comment.
If a consumer breaks out of for await or calls return() on the async iterator after receiving enough messages, no later next() call returns { done: true }, so this wrapper never runs processChunks() or publishes asyncEnd; the query span stays open and collected child spans are dropped. Wrap iterator.return/finalization as well as next.
Useful? React with 👍 / 👎.
Comment on lines
+66
to
+67
| end (ctx) { | ||
| ctx.currentStore?.span?.finish(ctx.finishTime) |
There was a problem hiding this comment.
When the SDK fires PostToolUseFailure, instrumentation stores ctx.error on the tool context, but only the normal end event is published; this end handler just finishes the span, so denied or failed tool calls are reported as successful. In the failure path, set the error tag before finishing or publish the tracing channel error event.
Useful? React with 👍 / 👎.
| Object.assign(getTool(sessionCtx, id), { | ||
| id, | ||
| name: input.tool_name || sessionCtx.tools.get(id)?.name, | ||
| output: input.tool_response, |
There was a problem hiding this comment.
If a user's PreToolUse hook returns updatedInput, the pre-tool tracer hook can only have captured the original request, while PostToolUseHookInput.tool_input contains the input that actually ran. This assignment leaves the stale tool.input in place and processStep() prefers that over the stream value, so tool spans report the wrong arguments whenever users rewrite tool inputs; overwrite input here from input.tool_input as well.
Useful? React with 👍 / 👎.
| } | ||
| } | ||
|
|
||
| if (tool?.hookFinishTime) return chunks.length |
There was a problem hiding this comment.
When a failed or interrupted tool has no matching tool_result chunk but the agent continues, toolResultIndex stays undefined and this return advances the caller to the end of the stream. That drops any later assistant messages from step/LLM tracing after the failed tool; end the failed tool without consuming the remainder of chunks.
Useful? React with 👍 / 👎.
Comment on lines
+241
to
+242
| if (Array.isArray(raw)) { | ||
| output = raw.map(b => b.text ?? JSON.stringify(b)).join('\n') |
There was a problem hiding this comment.
ctx.output is the stream user chunk's message.content, so normal entries are tool_result blocks with text under content, not top-level text. This falls through to JSON.stringify(b) for ordinary tool results, causing LLMObs tool outputs to contain the protocol envelope and IDs instead of the actual tool result; unwrap string/array content before joining.
Useful? React with 👍 / 👎.
sabrenner
previously approved these changes
Jul 7, 2026
sabrenner
left a comment
sabrenner
left a comment
Collaborator
There was a problem hiding this comment.
just leaving a note for a follow up. might not be valid but just wanted to write it down
| if (lastChunk?.type === 'result') ctx.output = lastChunk.result | ||
|
|
||
| try { | ||
| processChunks(chunks, ctx) |
Collaborator
There was a problem hiding this comment.
one note (we can handle in a follow-up): i'd rather just let errors hit the normal error handling we have for our instrumentation harness, rather than try-catching our own processing function and then re-throwing the error. but it's def not blocking
sabrenner
approved these changes
Jul 7, 2026
dd-octo-sts Bot
added a commit
that referenced
this pull request
Jul 7, 2026
…dk` (#9202) * feat(claude-agent-sdk): add instrumentation for @anthropic-ai/claude-agent-sdk Adds automatic instrumentation for the Claude Agent SDK, providing full visibility into agentic sessions via APM tracing and LLM Obs. Span hierarchy aligned with trajectory-spec APPENDIX-DD-LLMOBS-MAPPING: agent (session) └── agent (turn) ├── tool ({tool_name}) └── agent (subagent-{agent_type}) Key design decisions: - Uses SDK's first-class hooks API (SessionStart, SessionEnd, Stop, PreToolUse, PostToolUse, SubagentStart, SubagentStop, etc.) - Turn spans are `agent` kind (not workflow) per spec - Dynamic span names: session, turn, {toolName}, subagent-{type} - Model name split from provider prefix (anthropic/claude-sonnet-4-6) - Turn output from Stop hook's last_assistant_message - Captures cwd, transcript_path, agent_type, is_interrupt, start_trigger - Pure ESM package handled via import-in-the-middle with esmFirst: true - User hooks preserved via mergeHooks (user matchers before tracer matchers) Known gap: No LLM-level spans - the Agent SDK bundles its own Anthropic client internally, so the existing @anthropic-ai/sdk shimmer doesn't fire. This matches dd-trace-py's approach (agent + tool spans only). Tests: 41 APM tracing + 13 LLM Obs tests. * fix(claude-agent-sdk): resolve trace delivery and test failures Root cause: OTEL env vars caused dd-trace to use OTLP exporter instead of the agent exporter. The mock test agent only handles /v0.4/traces. - APM test: clear OTEL_* env vars before tracer init - LLM Obs test: rewrite to test span event structure directly - Instrumentation: finishSession closes pending child spans first - VCR proxy: use canonical body for stable cassette hashing * fix: resolve lint errors and skip broken SDK 0.2.0 in CI Add await before origIterator.return/throw calls to satisfy require-await lint rule. Bump minimum SDK version from 0.2.0 to 0.2.1 in rewriter versionRange and test withVersions filter — 0.2.0 uses require() in ESM scope which breaks on Node 22. * fix(claude-agent-sdk): add withVersions to LLM Obs test for module resolution The LLM Obs test was missing the withVersions wrapper that sets NODE_PATH to the versioned SDK's node_modules directory. Without it, the bare require('@anthropic-ai/claude-agent-sdk') fails with "Cannot find module". This matches the pattern used by the APM test. * chore: clean up stale comments in claude-agent-sdk instrumentation * feat(claude-agent-sdk): parameterize turn span names and add VCR to LLM Obs test * refactor(claude-agent-sdk): remove session span, propagate metadata to turn spans Turn spans are now root spans carrying all session metadata (model, start_trigger, project_dir, permission_mode, agent_type, transcript_path). Session-level span removed — the turn is the primary unit of work. * refactor(claude-agent-sdk): adopt orchestrion-only instrumentation path * fix(claude-agent-sdk): suppress stdin write-after-end error in tests * test(claude-agent-sdk): run SDK CLI through Node in tests * chore: update supported-integrations * test(claude-agent-sdk): import SDK through ESM hooks * test(claude-agent-sdk): cover instrumentation edge hooks * ci(benchmarks): restore Node 26 sirun run * Revert "ci(benchmarks): restore Node 26 sirun run" This reverts commit d4be02b. * fixups * fix apm tests * fix llmobs tests * fixup * fix merge conflict * fix llmobs job definition * run `npm run generate:config:types` * Apply suggestion from @sabrenner * fix test partially * change underlying implementation * remove benchmark - to add back later * bump supported range * update instrumentation * bump tested version * llmobs tests * rest of plugin tests * esm integration tests * find claude binary installed with package for CI * better normalizers for CI linux runners * finalized test changes (maybe??) * finalized cassettes and regex normalizations * re-do cassettes * new cassettes * fmt * temp debugging * more debug logs * debug log * undo debug, add rewriter handler * address codex review comments * refactor(claude-agent-sdk): index lifecycle from hooks * test(claude-agent-sdk): expect subagent wrapper before children * test(claude-agent-sdk): cover hook-index lifecycle * ci: refresh claude agent sdk checks * benchmark(claude-agent-sdk): model delayed stream scans * benchmark(claude-agent-sdk): lengthen delayed stream fixture * benchmark(claude-agent-sdk): stabilize delayed fixture guard * ci: ignore sirun benchmarks in patch coverage * perf(claude-agent-sdk): add compact lifecycle fast path Avoid eagerly building the full stream lifecycle index when the tool result is adjacent to the tool use. The hook-first path now scans a small local window first and only falls back to the per-trace index when the local lookup misses. Local timing driver over benchmark/sirun/plugin-claude-agent-sdk, 15 samples, OPERATIONS=25000: compact stream-scan: 44.80 ms median (44.07 min, 47.09 max) compact hook-indexed: 52.62 ms median (51.10 min, 56.09 max), 1.17x stream scan delayed-noisy stream-scan: 369.73 ms median (327.76 min, 401.03 max) delayed-noisy hook-indexed: 160.64 ms median (156.64 min, 165.25 max), 56.6% faster than stream scan * update claude agent sdk tested version with new cassettes * update cassettes again for ci runner compat * reduce timeout * remove rewriter patch for testing * Apply suggestions from code review Co-authored-by: Sam Brenner <[email protected]> * docs(claude-agent-sdk): align plugin docs ordering * fix(claude-agent-sdk): address review feedback * fix(claude-agent-sdk): trim agent tool output metadata * fix(claude-agent-sdk): normalize agent tool output footer * fix(claude-agent-sdk): normalize structured tool results * fix(claude-agent-sdk): match agent output footer --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com> Co-authored-by: Sam Brenner <[email protected]> Co-authored-by: Sam Brenner <[email protected]>
Merged
dd-octo-sts Bot
added a commit
that referenced
this pull request
Jul 7, 2026
…dk` (#9202) * feat(claude-agent-sdk): add instrumentation for @anthropic-ai/claude-agent-sdk Adds automatic instrumentation for the Claude Agent SDK, providing full visibility into agentic sessions via APM tracing and LLM Obs. Span hierarchy aligned with trajectory-spec APPENDIX-DD-LLMOBS-MAPPING: agent (session) └── agent (turn) ├── tool ({tool_name}) └── agent (subagent-{agent_type}) Key design decisions: - Uses SDK's first-class hooks API (SessionStart, SessionEnd, Stop, PreToolUse, PostToolUse, SubagentStart, SubagentStop, etc.) - Turn spans are `agent` kind (not workflow) per spec - Dynamic span names: session, turn, {toolName}, subagent-{type} - Model name split from provider prefix (anthropic/claude-sonnet-4-6) - Turn output from Stop hook's last_assistant_message - Captures cwd, transcript_path, agent_type, is_interrupt, start_trigger - Pure ESM package handled via import-in-the-middle with esmFirst: true - User hooks preserved via mergeHooks (user matchers before tracer matchers) Known gap: No LLM-level spans - the Agent SDK bundles its own Anthropic client internally, so the existing @anthropic-ai/sdk shimmer doesn't fire. This matches dd-trace-py's approach (agent + tool spans only). Tests: 41 APM tracing + 13 LLM Obs tests. * fix(claude-agent-sdk): resolve trace delivery and test failures Root cause: OTEL env vars caused dd-trace to use OTLP exporter instead of the agent exporter. The mock test agent only handles /v0.4/traces. - APM test: clear OTEL_* env vars before tracer init - LLM Obs test: rewrite to test span event structure directly - Instrumentation: finishSession closes pending child spans first - VCR proxy: use canonical body for stable cassette hashing * fix: resolve lint errors and skip broken SDK 0.2.0 in CI Add await before origIterator.return/throw calls to satisfy require-await lint rule. Bump minimum SDK version from 0.2.0 to 0.2.1 in rewriter versionRange and test withVersions filter — 0.2.0 uses require() in ESM scope which breaks on Node 22. * fix(claude-agent-sdk): add withVersions to LLM Obs test for module resolution The LLM Obs test was missing the withVersions wrapper that sets NODE_PATH to the versioned SDK's node_modules directory. Without it, the bare require('@anthropic-ai/claude-agent-sdk') fails with "Cannot find module". This matches the pattern used by the APM test. * chore: clean up stale comments in claude-agent-sdk instrumentation * feat(claude-agent-sdk): parameterize turn span names and add VCR to LLM Obs test * refactor(claude-agent-sdk): remove session span, propagate metadata to turn spans Turn spans are now root spans carrying all session metadata (model, start_trigger, project_dir, permission_mode, agent_type, transcript_path). Session-level span removed — the turn is the primary unit of work. * refactor(claude-agent-sdk): adopt orchestrion-only instrumentation path * fix(claude-agent-sdk): suppress stdin write-after-end error in tests * test(claude-agent-sdk): run SDK CLI through Node in tests * chore: update supported-integrations * test(claude-agent-sdk): import SDK through ESM hooks * test(claude-agent-sdk): cover instrumentation edge hooks * ci(benchmarks): restore Node 26 sirun run * Revert "ci(benchmarks): restore Node 26 sirun run" This reverts commit d4be02b. * fixups * fix apm tests * fix llmobs tests * fixup * fix merge conflict * fix llmobs job definition * run `npm run generate:config:types` * Apply suggestion from @sabrenner * fix test partially * change underlying implementation * remove benchmark - to add back later * bump supported range * update instrumentation * bump tested version * llmobs tests * rest of plugin tests * esm integration tests * find claude binary installed with package for CI * better normalizers for CI linux runners * finalized test changes (maybe??) * finalized cassettes and regex normalizations * re-do cassettes * new cassettes * fmt * temp debugging * more debug logs * debug log * undo debug, add rewriter handler * address codex review comments * refactor(claude-agent-sdk): index lifecycle from hooks * test(claude-agent-sdk): expect subagent wrapper before children * test(claude-agent-sdk): cover hook-index lifecycle * ci: refresh claude agent sdk checks * benchmark(claude-agent-sdk): model delayed stream scans * benchmark(claude-agent-sdk): lengthen delayed stream fixture * benchmark(claude-agent-sdk): stabilize delayed fixture guard * ci: ignore sirun benchmarks in patch coverage * perf(claude-agent-sdk): add compact lifecycle fast path Avoid eagerly building the full stream lifecycle index when the tool result is adjacent to the tool use. The hook-first path now scans a small local window first and only falls back to the per-trace index when the local lookup misses. Local timing driver over benchmark/sirun/plugin-claude-agent-sdk, 15 samples, OPERATIONS=25000: compact stream-scan: 44.80 ms median (44.07 min, 47.09 max) compact hook-indexed: 52.62 ms median (51.10 min, 56.09 max), 1.17x stream scan delayed-noisy stream-scan: 369.73 ms median (327.76 min, 401.03 max) delayed-noisy hook-indexed: 160.64 ms median (156.64 min, 165.25 max), 56.6% faster than stream scan * update claude agent sdk tested version with new cassettes * update cassettes again for ci runner compat * reduce timeout * remove rewriter patch for testing * Apply suggestions from code review Co-authored-by: Sam Brenner <[email protected]> * docs(claude-agent-sdk): align plugin docs ordering * fix(claude-agent-sdk): address review feedback * fix(claude-agent-sdk): trim agent tool output metadata * fix(claude-agent-sdk): normalize agent tool output footer * fix(claude-agent-sdk): normalize structured tool results * fix(claude-agent-sdk): match agent output footer --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com> Co-authored-by: Sam Brenner <[email protected]> Co-authored-by: Sam Brenner <[email protected]>
Merged
juan-fernandez
pushed a commit
that referenced
this pull request
Jul 8, 2026
…dk` (#9202) * feat(claude-agent-sdk): add instrumentation for @anthropic-ai/claude-agent-sdk Adds automatic instrumentation for the Claude Agent SDK, providing full visibility into agentic sessions via APM tracing and LLM Obs. Span hierarchy aligned with trajectory-spec APPENDIX-DD-LLMOBS-MAPPING: agent (session) └── agent (turn) ├── tool ({tool_name}) └── agent (subagent-{agent_type}) Key design decisions: - Uses SDK's first-class hooks API (SessionStart, SessionEnd, Stop, PreToolUse, PostToolUse, SubagentStart, SubagentStop, etc.) - Turn spans are `agent` kind (not workflow) per spec - Dynamic span names: session, turn, {toolName}, subagent-{type} - Model name split from provider prefix (anthropic/claude-sonnet-4-6) - Turn output from Stop hook's last_assistant_message - Captures cwd, transcript_path, agent_type, is_interrupt, start_trigger - Pure ESM package handled via import-in-the-middle with esmFirst: true - User hooks preserved via mergeHooks (user matchers before tracer matchers) Known gap: No LLM-level spans - the Agent SDK bundles its own Anthropic client internally, so the existing @anthropic-ai/sdk shimmer doesn't fire. This matches dd-trace-py's approach (agent + tool spans only). Tests: 41 APM tracing + 13 LLM Obs tests. * fix(claude-agent-sdk): resolve trace delivery and test failures Root cause: OTEL env vars caused dd-trace to use OTLP exporter instead of the agent exporter. The mock test agent only handles /v0.4/traces. - APM test: clear OTEL_* env vars before tracer init - LLM Obs test: rewrite to test span event structure directly - Instrumentation: finishSession closes pending child spans first - VCR proxy: use canonical body for stable cassette hashing * fix: resolve lint errors and skip broken SDK 0.2.0 in CI Add await before origIterator.return/throw calls to satisfy require-await lint rule. Bump minimum SDK version from 0.2.0 to 0.2.1 in rewriter versionRange and test withVersions filter — 0.2.0 uses require() in ESM scope which breaks on Node 22. * fix(claude-agent-sdk): add withVersions to LLM Obs test for module resolution The LLM Obs test was missing the withVersions wrapper that sets NODE_PATH to the versioned SDK's node_modules directory. Without it, the bare require('@anthropic-ai/claude-agent-sdk') fails with "Cannot find module". This matches the pattern used by the APM test. * chore: clean up stale comments in claude-agent-sdk instrumentation * feat(claude-agent-sdk): parameterize turn span names and add VCR to LLM Obs test * refactor(claude-agent-sdk): remove session span, propagate metadata to turn spans Turn spans are now root spans carrying all session metadata (model, start_trigger, project_dir, permission_mode, agent_type, transcript_path). Session-level span removed — the turn is the primary unit of work. * refactor(claude-agent-sdk): adopt orchestrion-only instrumentation path * fix(claude-agent-sdk): suppress stdin write-after-end error in tests * test(claude-agent-sdk): run SDK CLI through Node in tests * chore: update supported-integrations * test(claude-agent-sdk): import SDK through ESM hooks * test(claude-agent-sdk): cover instrumentation edge hooks * ci(benchmarks): restore Node 26 sirun run * Revert "ci(benchmarks): restore Node 26 sirun run" This reverts commit d4be02b. * fixups * fix apm tests * fix llmobs tests * fixup * fix merge conflict * fix llmobs job definition * run `npm run generate:config:types` * Apply suggestion from @sabrenner * fix test partially * change underlying implementation * remove benchmark - to add back later * bump supported range * update instrumentation * bump tested version * llmobs tests * rest of plugin tests * esm integration tests * find claude binary installed with package for CI * better normalizers for CI linux runners * finalized test changes (maybe??) * finalized cassettes and regex normalizations * re-do cassettes * new cassettes * fmt * temp debugging * more debug logs * debug log * undo debug, add rewriter handler * address codex review comments * refactor(claude-agent-sdk): index lifecycle from hooks * test(claude-agent-sdk): expect subagent wrapper before children * test(claude-agent-sdk): cover hook-index lifecycle * ci: refresh claude agent sdk checks * benchmark(claude-agent-sdk): model delayed stream scans * benchmark(claude-agent-sdk): lengthen delayed stream fixture * benchmark(claude-agent-sdk): stabilize delayed fixture guard * ci: ignore sirun benchmarks in patch coverage * perf(claude-agent-sdk): add compact lifecycle fast path Avoid eagerly building the full stream lifecycle index when the tool result is adjacent to the tool use. The hook-first path now scans a small local window first and only falls back to the per-trace index when the local lookup misses. Local timing driver over benchmark/sirun/plugin-claude-agent-sdk, 15 samples, OPERATIONS=25000: compact stream-scan: 44.80 ms median (44.07 min, 47.09 max) compact hook-indexed: 52.62 ms median (51.10 min, 56.09 max), 1.17x stream scan delayed-noisy stream-scan: 369.73 ms median (327.76 min, 401.03 max) delayed-noisy hook-indexed: 160.64 ms median (156.64 min, 165.25 max), 56.6% faster than stream scan * update claude agent sdk tested version with new cassettes * update cassettes again for ci runner compat * reduce timeout * remove rewriter patch for testing * Apply suggestions from code review Co-authored-by: Sam Brenner <[email protected]> * docs(claude-agent-sdk): align plugin docs ordering * fix(claude-agent-sdk): address review feedback * fix(claude-agent-sdk): trim agent tool output metadata * fix(claude-agent-sdk): normalize agent tool output footer * fix(claude-agent-sdk): normalize structured tool results * fix(claude-agent-sdk): match agent output footer --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com> Co-authored-by: Sam Brenner <[email protected]> Co-authored-by: Sam Brenner <[email protected]>
juan-fernandez
pushed a commit
that referenced
this pull request
Jul 8, 2026
…dk` (#9202) * feat(claude-agent-sdk): add instrumentation for @anthropic-ai/claude-agent-sdk Adds automatic instrumentation for the Claude Agent SDK, providing full visibility into agentic sessions via APM tracing and LLM Obs. Span hierarchy aligned with trajectory-spec APPENDIX-DD-LLMOBS-MAPPING: agent (session) └── agent (turn) ├── tool ({tool_name}) └── agent (subagent-{agent_type}) Key design decisions: - Uses SDK's first-class hooks API (SessionStart, SessionEnd, Stop, PreToolUse, PostToolUse, SubagentStart, SubagentStop, etc.) - Turn spans are `agent` kind (not workflow) per spec - Dynamic span names: session, turn, {toolName}, subagent-{type} - Model name split from provider prefix (anthropic/claude-sonnet-4-6) - Turn output from Stop hook's last_assistant_message - Captures cwd, transcript_path, agent_type, is_interrupt, start_trigger - Pure ESM package handled via import-in-the-middle with esmFirst: true - User hooks preserved via mergeHooks (user matchers before tracer matchers) Known gap: No LLM-level spans - the Agent SDK bundles its own Anthropic client internally, so the existing @anthropic-ai/sdk shimmer doesn't fire. This matches dd-trace-py's approach (agent + tool spans only). Tests: 41 APM tracing + 13 LLM Obs tests. * fix(claude-agent-sdk): resolve trace delivery and test failures Root cause: OTEL env vars caused dd-trace to use OTLP exporter instead of the agent exporter. The mock test agent only handles /v0.4/traces. - APM test: clear OTEL_* env vars before tracer init - LLM Obs test: rewrite to test span event structure directly - Instrumentation: finishSession closes pending child spans first - VCR proxy: use canonical body for stable cassette hashing * fix: resolve lint errors and skip broken SDK 0.2.0 in CI Add await before origIterator.return/throw calls to satisfy require-await lint rule. Bump minimum SDK version from 0.2.0 to 0.2.1 in rewriter versionRange and test withVersions filter — 0.2.0 uses require() in ESM scope which breaks on Node 22. * fix(claude-agent-sdk): add withVersions to LLM Obs test for module resolution The LLM Obs test was missing the withVersions wrapper that sets NODE_PATH to the versioned SDK's node_modules directory. Without it, the bare require('@anthropic-ai/claude-agent-sdk') fails with "Cannot find module". This matches the pattern used by the APM test. * chore: clean up stale comments in claude-agent-sdk instrumentation * feat(claude-agent-sdk): parameterize turn span names and add VCR to LLM Obs test * refactor(claude-agent-sdk): remove session span, propagate metadata to turn spans Turn spans are now root spans carrying all session metadata (model, start_trigger, project_dir, permission_mode, agent_type, transcript_path). Session-level span removed — the turn is the primary unit of work. * refactor(claude-agent-sdk): adopt orchestrion-only instrumentation path * fix(claude-agent-sdk): suppress stdin write-after-end error in tests * test(claude-agent-sdk): run SDK CLI through Node in tests * chore: update supported-integrations * test(claude-agent-sdk): import SDK through ESM hooks * test(claude-agent-sdk): cover instrumentation edge hooks * ci(benchmarks): restore Node 26 sirun run * Revert "ci(benchmarks): restore Node 26 sirun run" This reverts commit d4be02b. * fixups * fix apm tests * fix llmobs tests * fixup * fix merge conflict * fix llmobs job definition * run `npm run generate:config:types` * Apply suggestion from @sabrenner * fix test partially * change underlying implementation * remove benchmark - to add back later * bump supported range * update instrumentation * bump tested version * llmobs tests * rest of plugin tests * esm integration tests * find claude binary installed with package for CI * better normalizers for CI linux runners * finalized test changes (maybe??) * finalized cassettes and regex normalizations * re-do cassettes * new cassettes * fmt * temp debugging * more debug logs * debug log * undo debug, add rewriter handler * address codex review comments * refactor(claude-agent-sdk): index lifecycle from hooks * test(claude-agent-sdk): expect subagent wrapper before children * test(claude-agent-sdk): cover hook-index lifecycle * ci: refresh claude agent sdk checks * benchmark(claude-agent-sdk): model delayed stream scans * benchmark(claude-agent-sdk): lengthen delayed stream fixture * benchmark(claude-agent-sdk): stabilize delayed fixture guard * ci: ignore sirun benchmarks in patch coverage * perf(claude-agent-sdk): add compact lifecycle fast path Avoid eagerly building the full stream lifecycle index when the tool result is adjacent to the tool use. The hook-first path now scans a small local window first and only falls back to the per-trace index when the local lookup misses. Local timing driver over benchmark/sirun/plugin-claude-agent-sdk, 15 samples, OPERATIONS=25000: compact stream-scan: 44.80 ms median (44.07 min, 47.09 max) compact hook-indexed: 52.62 ms median (51.10 min, 56.09 max), 1.17x stream scan delayed-noisy stream-scan: 369.73 ms median (327.76 min, 401.03 max) delayed-noisy hook-indexed: 160.64 ms median (156.64 min, 165.25 max), 56.6% faster than stream scan * update claude agent sdk tested version with new cassettes * update cassettes again for ci runner compat * reduce timeout * remove rewriter patch for testing * Apply suggestions from code review Co-authored-by: Sam Brenner <[email protected]> * docs(claude-agent-sdk): align plugin docs ordering * fix(claude-agent-sdk): address review feedback * fix(claude-agent-sdk): trim agent tool output metadata * fix(claude-agent-sdk): normalize agent tool output footer * fix(claude-agent-sdk): normalize structured tool results * fix(claude-agent-sdk): match agent output footer --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com> Co-authored-by: Sam Brenner <[email protected]> Co-authored-by: Sam Brenner <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for the
@anthropic-ai/claude-agent-sdkpackage by tracing agent, subagent, step, llm, and tool calls. Submits both Agent Observability and APM traces.Implementation Notes
Why
SDK hooks are cleaner instrumentation wedges for lifecycle facts: they fire at the semantic boundary where the SDK knows session, prompt, tool, permission, and subagent state. The event stream is still useful, but it is better treated as payload/enrichment data rather than as the source of execution-context lifecycle.
This avoids trying to manufacture tracing-channel parentage around individual stream events while still keeping the tracing-channel model centered on query/LLM/tool execution contexts.
Why hooks for tracing
The intended split is: use SDK hooks for lifecycle and tracing semantics, and use the event stream for transcript/payload enrichment.
Hooks give the tracer higher-fidelity boundaries for session start/end, pre/post tool use, tool failures, interrupts, and subagent start/stop. Those are the points where the SDK has already decided what is executing, so the integration can attach parentage and tracing-channel context to the semantic operation instead of inferring it later from stream ordering.
The event stream is still necessary and valuable for assistant messages, LLM usage, tool result content, and nested chunks. It is just a weaker source of lifecycle truth: delayed lifecycle chunks, interleaved subagent output, and unrelated assistant chunks make parentage reconstruction more fragile and more expensive as traces grow.
So this PR keeps the hybrid approach: hooks establish execution lifecycle and parentage; the stream fills in message, token, and result details. That gives cleaner instrumentation semantics without giving up stream-derived payload fidelity.
Benchmark results
Local direct Node runs of
benchmark/sirun/plugin-claude-agent-sdk/index.json macOS arm64, Nodev26.4.0. Each sample starts a fresh Node process, so the short compact case includes startup overhead. The hook-indexed variant first checks a small local window for nearby lifecycle data and only builds the full stream index when that lookup misses; when the index is needed, it is rebuilt inside each measured operation so the benchmark still charges the per-trace indexing cost.compactstream-scancompacthook-indexeddelayed-noisystream-scandelayed-noisyhook-indexedWhat this shows:
So the performance claim is now stronger but still scoped: hook-indexed lifecycle capture is not strictly faster for tiny/compact streams, but it is close enough in the compact best case and scales much better for larger delayed/noisy traces while giving a cleaner semantic boundary for parentage and lifecycle facts.
Relationship to existing PRs
feat/claude-agent-sdk-integration)Validation
./node_modules/.bin/eslint packages/datadog-instrumentations/src/claude-agent-sdk.js benchmark/sirun/plugin-claude-agent-sdk/index.jsnode -c packages/datadog-instrumentations/src/claude-agent-sdk.jsnode -c benchmark/sirun/plugin-claude-agent-sdk/index.jsPLUGINS=claude-agent-sdk SPEC=hook-index npm run test:pluginsbenchmark/sirun/plugin-claude-agent-sdk/index.js, 15 samples,OPERATIONS=25000; results above@anthropic-ai/claude-agent-sdk#9202 headcc94900b6c5ff3182a94c863b18b72d723901e18: all 654 reported CI contexts passed or skipped; 0 failures, 0 pending.all-green,dd-gitlab/default-pipeline,dd-gitlab/finished, andcodecov/patchare green.