test(ai): add preserve OpenTelemetry span receiver test#9309
Conversation
Overall package sizeSelf size: 7.52 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.2 | 124.41 kB | 440.65 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 1cb6374 | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-07-22 19:00:09 Comparing candidate commit 1cb6374 in PR branch Found 1 performance improvements and 0 performance regressions! Performance is the same for 2311 metrics, 46 unstable metrics.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9309 +/- ##
==========================================
- Coverage 98.44% 96.79% -1.66%
==========================================
Files 943 941 -2
Lines 127310 127270 -40
Branches 10824 10453 -371
==========================================
- Hits 125335 123190 -2145
- Misses 1975 4080 +2105 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
wconti27
added
the
apm-integration-toolkit
| /** | ||
| * Builds a per-invocation delegating wrapper around an OTel span. | ||
| * | ||
| * Every method forwards to the underlying `span` with `this === span` (via | ||
| * `.apply(span, arguments)`), so any private-field access inside the span | ||
| * implementation lands on the real instance. This is why we cannot use | ||
| * `Object.create(span)` or `new Proxy(span, …)` here: both fail a private-field | ||
| * brand check (e.g. dd-trace's own OTel-bridge span stores status in a private | ||
| * `#statusCode` field, so a prototype clone's `setStatus()` throws | ||
| * "Cannot read private member #statusCode"). | ||
| * | ||
| * A fresh wrapper per `startActiveSpan` callback also preserves the behavior the | ||
| * shared AI SDK `noopSpan` needs: the publishing methods close over this call's | ||
| * `ctx` without mutating the reused singleton. | ||
| * | ||
| * Only `end`, `setAttributes`, and `recordException` publish to the diagnostic | ||
| * channels before delegating; the rest are straight pass-throughs. |
There was a problem hiding this comment.
nit: verbose comment can be cleaned up, not a blocker
There was a problem hiding this comment.
Cleaned this up in 035715d. The comment now keeps just the two constraints that are easy to miss: preserving the original method receiver and avoiding mutation of the shared no-op span.
dd-trace-js/packages/datadog-instrumentations/src/ai.js
Lines 119 to 126 in 035715d
| }) | ||
|
|
||
| return originalCb.call(this, freshSpan) | ||
| return originalCb.call(this, createDelegatingSpan(span, ctx)) |
There was a problem hiding this comment.
This code makes me feel like we should have an Otel to DD reusable helper (could be a followup), considering more and more libs are using the OTel API
There was a problem hiding this comment.
Agreed this could be worth extracting once another integration needs the same behavior. I kept it local for now because the diagnostic-channel publications and per-call ctx are Vercel AI-specific; with only one consumer, a shared helper would make that integration-specific contract look more general than it is.
wconti27
left a comment
wconti27
left a comment
There was a problem hiding this comment.
Left some nit comments, otherwise LGTM
BridgeAR
left a comment
BridgeAR
left a comment
There was a problem hiding this comment.
The code change itself seems good, just the tests need to be reworked to be more realistic
| }) | ||
|
|
||
| module.exports = { wrapModelWithLifecycle } | ||
| module.exports = { wrapModelWithLifecycle, createDelegatingSpan } |
There was a problem hiding this comment.
Please do not export createDelegatingSpan. That is only used in tests and that means we actually do not test things as a user would be able to reach the code. Thus, the tests are not really well testing the surface.
There was a problem hiding this comment.
@BridgeAR removed the test-only export and the direct createDelegatingSpan test suite. The regression now reaches the behavior only through ai.generateText with the real dd-trace OTel provider, and verifies both the original model error and the emitted error span:
dd-trace-js/packages/datadog-plugin-ai/test/index.spec.js
Lines 74 to 107 in 7ab09df
| const otelTracer = new TracerProvider().getTracer('ai') | ||
| const checkTraces = agent.assertSomeTraces(traces => { | ||
| assertObjectContains(traces[0][0], { | ||
| name: 'ai.generateText.doGenerate', |
There was a problem hiding this comment.
I think that name depends on the version, no?
There was a problem hiding this comment.
@BridgeAR yea, it does. I removed the name assertion and now find the span by the original error metadata, then verify it is marked as an error. This passes against both 6.0.0 and master’s resolved 6.0.227 fixture:
dd-trace-js/packages/datadog-plugin-ai/test/index.spec.js
Lines 84 to 103 in c60e8dc
BridgeAR
changed the title
|
i think this change merged last week might address this issue? #9152 |
5b3ab17 to
1cb6374
Compare
4 participants
What does this PR do?
Builds on #9152's fix for #9151 by keeping the delegating OTel span wrapper private and making its forwarding contract exact. Every Span method runs against the real underlying span with the caller's original arguments, while chainable methods return the per-invocation wrapper and
end,setAttributes, andrecordExceptioncontinue publishing the existing diagnostic-channel events.The wrapper remains fresh per invocation, so the AI SDK's shared no-op span is not mutated and each publication keeps the correct invocation context.
Why is this stronger than the implementation on
master?The implementation on
masterfixes the private-field receiver bug, but manually restates each method signature. This version usesapply(span, arguments)so optional arguments, return behavior, and future compatible Span signature changes are forwarded without drift. The helper stays local to the instrumentation and is not exported for tests.Test coverage
TracerProvider, asserting that the original model error is preserved and the emitted span records that error.Verification
git diff --check: passing.