feat(claude-agent-sdk): add hook-indexed lifecycle instrumentation#9201
Closed
mr-lee wants to merge 55 commits into
Closed
feat(claude-agent-sdk): add hook-indexed lifecycle instrumentation#9201mr-lee wants to merge 55 commits into
mr-lee wants to merge 55 commits into
Conversation
…agent-sdk
Adds automatic instrumentation for the Claude Agent SDK, providing
full visibility into agentic sessions via APM tracing and LLM Obs.
Span hierarchy aligned with trajectory-spec APPENDIX-DD-LLMOBS-MAPPING:
agent (session)
└── agent (turn)
├── tool ({tool_name})
└── agent (subagent-{agent_type})
Key design decisions:
- Uses SDK's first-class hooks API (SessionStart, SessionEnd, Stop,
PreToolUse, PostToolUse, SubagentStart, SubagentStop, etc.)
- Turn spans are `agent` kind (not workflow) per spec
- Dynamic span names: session, turn, {toolName}, subagent-{type}
- Model name split from provider prefix (anthropic/claude-sonnet-4-6)
- Turn output from Stop hook's last_assistant_message
- Captures cwd, transcript_path, agent_type, is_interrupt, start_trigger
- Pure ESM package handled via import-in-the-middle with esmFirst: true
- User hooks preserved via mergeHooks (user matchers before tracer matchers)
Known gap: No LLM-level spans - the Agent SDK bundles its own Anthropic
client internally, so the existing @anthropic-ai/sdk shimmer doesn't fire.
This matches dd-trace-py's approach (agent + tool spans only).
Tests: 41 APM tracing + 13 LLM Obs tests.
Root cause: OTEL env vars caused dd-trace to use OTLP exporter instead of the agent exporter. The mock test agent only handles /v0.4/traces. - APM test: clear OTEL_* env vars before tracer init - LLM Obs test: rewrite to test span event structure directly - Instrumentation: finishSession closes pending child spans first - VCR proxy: use canonical body for stable cassette hashing
Add await before origIterator.return/throw calls to satisfy require-await lint rule. Bump minimum SDK version from 0.2.0 to 0.2.1 in rewriter versionRange and test withVersions filter — 0.2.0 uses require() in ESM scope which breaks on Node 22.
…solution
The LLM Obs test was missing the withVersions wrapper that sets NODE_PATH
to the versioned SDK's node_modules directory. Without it, the bare
require('@anthropic-ai/claude-agent-sdk') fails with "Cannot find module".
This matches the pattern used by the APM test.
…o turn spans Turn spans are now root spans carrying all session metadata (model, start_trigger, project_dir, permission_mode, agent_type, transcript_path). Session-level span removed — the turn is the primary unit of work.
This reverts commit d4be02b.
…g/dd-trace-js into feat/claude-agent-sdk-integration
…g/dd-trace-js into feat/claude-agent-sdk-integration
|
Contributor
Author
|
Closing this fork duplicate to avoid split CI/check-run state. Canonical upstream PR is #9202. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
SDK hooks are cleaner instrumentation wedges for lifecycle facts: they fire at the semantic boundary where the SDK knows session, prompt, tool, permission, and subagent state. The event stream is still useful, but it is better treated as payload/enrichment data rather than as the source of execution-context lifecycle.
This avoids trying to manufacture tracing-channel parentage around individual stream events while still keeping the tracing-channel model centered on query/LLM/tool execution contexts.
Relationship to existing PRs
feat/claude-agent-sdk-integration)Validation
./node_modules/.bin/eslint packages/datadog-instrumentations/src/claude-agent-sdk.js packages/datadog-plugin-claude-agent-sdk/test/index.spec.js packages/dd-trace/test/llmobs/plugins/claude-agent-sdk/index.spec.js benchmark/sirun/plugin-claude-agent-sdk/index.jsnode -c packages/datadog-instrumentations/src/claude-agent-sdk.js && node -c packages/datadog-plugin-claude-agent-sdk/test/index.spec.js && node -c packages/dd-trace/test/llmobs/plugins/claude-agent-sdk/index.spec.js && node -c benchmark/sirun/plugin-claude-agent-sdk/index.jsVARIANT=stream-scan OPERATIONS=25000 node benchmark/sirun/plugin-claude-agent-sdk/index.jsVARIANT=hook-indexed OPERATIONS=25000 node benchmark/sirun/plugin-claude-agent-sdk/index.js0.2.113passed after the subagent ordering fix; SDK0.3.198was inconclusive locally because the Docker VM hitENOSPCbefore the relevant assertions09aad76cd:claude-agent-sdkpassed; broadpackage-size-reportandPerformance and correctness testsfailed before checkout inDataDog/dd-octo-sts-actionwith missingid-tokenenvironment/permission metadata, not from this code path