[SDTEST-3830] CI Visibility instrumentation telemetry: metrics + logs#279
Merged
Conversation
anmarchenko
approved these changes
Jun 15, 2026
… test
DDTracer read Bundle.main directly, which is the bare xctest runner for
host-less test bundles, so it reported a wrong/unknown application version.
Add Bundle.productUnderTest, a resolver that picks the bundle representing
the product being tested and reports its version, with priority:
1. DD_VERSION override (quick-exit: skips all discovery when set)
2. Bundle.main when it is an .app (app-/UI-hosted tests)
3. a dynamic framework named after XCODE_SCHEME_NAME, co-located with the
.xctest bundle (sibling on macOS/simulator/device, or embedded), found
among loaded frameworks or probed on disk (static frameworks ship in the
products folder but are never loaded into memory)
4. the .xctest bundle version (static/SPM-merged products)
5. "<unknown>"
The selection policy is a pure function over value-type descriptors so it is
fully unit-tested. Adds DD_VERSION as a configuration key.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…tracer on shutdown Replace the Mach-O LC_BUILD_VERSION compiler-version scanner (CompilerVersion.c/h) with RuntimeInfo: an Xcode/SPM enum exposing runtimeName, runtimeVersion and swiftVersion. It maps the XCTest bundle's DTXcode to a Swift version (Xcode 26+), falling back to a single `swiftc --version` spawn only when needed (avoids the subprocess on macOS for both Xcode and SPM). XCODE_SCHEME_NAME selects the case. Thread the runtimeName/runtimeVersion through APIServiceConfig and TestOptimizationApiService so TelemetryApplication reports them instead of hardcoding "swift". Make DDTestMonitor.tracer lazily recreatable: SessionManager.endSession now shuts the tracer down (which tears down the global OpenTelemetry provider), and DDTracer.deinit restores the default providers. The next session rebuilds a tracer that re-registers a live SDK provider, so spans stay SpanSdk instead of crashing the `as! SpanSdk` casts. The backing store is guarded by Synced to keep the lazy init/reset race-free. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…refine device/runtime info - shutdownTracer(): clear the tracer slot under the lock but run shutdown() outside it. shutdown() flushes/logs/uploads and stdout/stderr capture routes back through DDTestMonitor.tracer, so holding the non-reentrant lock during shutdown deadlocked. - URLSessionInstrumentation: check URLSessionDelegate conformance via the ObjC runtime (InstrumentationUtils.classConforms, walking the superclass chain with class_conformsToProtocol) instead of a Swift `is` cast. The cast messages every class from objc_getClassList() and aborted on pathological internal classes (__NSGenericDeallocHandler, __NSAtom, __NSMessageBuilder). - EnvironmentTests: update to the refactored PlatformUtils API (getPlatformVersion) and assert runtime tags against getRuntimeInfo(). - Device/Platform/RuntimeInfo refactor wiring (getDeviceInfo/getKernelInfo). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…umentation self-contained Move the tracer off the global lazy accessor and onto the DDTestMonitor instance: it's created in init and torn down with the monitor (stop() flushes and shuts it down; DDTracer.deinit restores the default OpenTelemetry providers). Removed the static `tracer` accessor entirely — callers use `DDTestMonitor.instance?.tracer`, so no tracer is created when no monitor is installed (this is what caused the accidental backend connection), and there's no lock to deadlock on during shutdown. Session-scoped span creators (DDSession/DDModule/DDSuite/DDTest) get the tracer from SessionConfig.tracer, captured from the monitor when the session bootstraps. Ambient callers (stdout/stderr capture, crash, XCUI, instrumentation control) use the optional instance accessor and no-op without a monitor. DDNetworkInstrumentation is now self-contained: injectHeaders / recordPayload / disableNetworkCallStack / enableNetworkCallStackSymbolicated are its own properties passed at init, and the URLSessionInstrumentationConfiguration callbacks capture self weakly to break a retain cycle (config -> closures -> self -> urlSessionInstrumentation -> config). DDCrashes.install takes the tracer and returns CrashInformation? instead of reaching into DDTestMonitor.instance; the monitor stores the result. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Restore the upload logging path: DataUploadWorker logs each batch's success/retry/drop with a failure description (DataUploadStatus.failureDescription), threaded through FeatureStoreAndUpload and the feature exporters. Remove the now-redundant per-request logs in the API services and the orphaned clientLibraryVersionHeader call. Drop the obsolete DeviceTests. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Move telemetry metrics and logs off OpenTelemetry into hand-written stores, mirroring the existing MetricStore rationale (the DD intake wants raw samples / minimal log values, not OTel-shaped data). Metrics: drop the Typed* prefix — Counter<Tags>/Distribution<Tags> are now the only instruments, each paired with a per-metric MetricTags type and a constrained extension exposing the typed add/record. Removes the parallel untyped Counter/ Distribution/Factory and the one-off wrapper structs. Logs: add a LogStore + telemetry.logs.error/warn/debug handler on Telemetry, deduplicating identical logs into one entry with a count and draining on the shared flush timer as the `logs` request type. Tags reuse the metric rendering. Delete the now-unused OTel TelemetryLogExporter bridge and its test. Also send DD-Client-Library-Version on the telemetry request directly, and document both stores in ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…mode Capture the uncompressed request body before deflate and log it alongside the response. Render payloads as UTF-8 text (JSON in nearly all cases), falling back to a `<binary N bytes>` marker when the bytes aren't valid UTF-8. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…ields Add request and response headers to the HTTPClient debug log. Capture just the url, headers, and uncompressed body into a small LogFields value up front (only when debug is on), so the completion handler no longer retains the full URLRequest for the request's duration. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
TelemetryMetric.Series.Point.timestamp is now Int64 unix seconds (rounded), matching the intake's expected format, with a `time:` convenience init that rounds a TimeInterval. MetricStore stamps points with the rounded clock value. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Replace DD-API-KEY and DD-APPLICATION-KEY values with **** in HTTPClient debug header logs (names sourced from HTTPHeader.Field, matched case-insensitively). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Flip debugTelemetry to false for release; backend debug processing was only needed while validating the telemetry pipeline. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The SDK can't instrument its own test run, so report unit-test results by uploading the JUnit reports xcbeautify produces. Give each scheme/platform a distinct report filename so the two unit-test schemes no longer overwrite one junit.xml, then upload build/reports via datadog-ci in the unit-tests workflow (runs on failure too; skips gracefully when DD_API_KEY is absent, e.g. fork PRs). Use `npx --yes @datadog/datadog-ci` instead of a global install, here and in the codeQL SARIF upload. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
ypopovych
force-pushed
the
SDTEST-3830-instrumentation-telemetry
branch
from
June 15, 2026 10:27
c2999a9 to
4a2ed0b
Compare
The function-name regex used `\S+` for Swift symbols, truncating a backtick- quoted name with spaces (e.g. `failing parameterized test`(_:)) at the first space, so the source-line parser missed the function. Match backtick-quoted segments as a unit, and split the module on the last dot before the first backtick so a dot inside a quoted name isn't taken as the module separator. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Brings the CI Visibility instrumentation-telemetry subsystem to a releasable state: SDK self-metrics and self-logs are collected through a custom, OpenTelemetry-free pipeline, plus the supporting tracer-lifecycle, runtime/device-info, and debug-logging fixes found while testing it.
Why
The DD telemetry intake wants distributions as raw sample arrays and logs as minimal value types — neither maps cleanly onto OTel's histogram/log SDK. Bridging through OTel was lossy (percentile distortion from bucket reconstruction) and complex. Hand-written stores export a verbatim copy and let the whole OTel metrics/logs SDK be dropped.
See
Sources/DatadogSDKTesting/Telemetry/ARCHITECTURE.mdfor the full design.Highlights
Telemetry metrics
Telemetry.metrics— a typed, discoverable tree of CI Visibility metrics over a customMetricStore(per-interval count deltas + raw distribution samples), drained on a short timer and force-flushed on a full buffer.Counter<Tags>/Distribution<Tags>with per-metricMetricTagstypes (dropped the parallel untyped variants and the OTel meter path).Telemetry logs
Telemetry.logs.error / warn / debugover a customLogStore, deduplicating identical logs into one entry with an occurrencecount, draining on the same timer as thelogsrequest type. Removed the OTelTelemetryLogExporterbridge.Lifecycle & wiring
Telemetryowns the app-lifecycle protocol (app-startedon init, heartbeat timer,app-closingon shutdown) and is created with the tracer; observers thread network/storage facts back up via neutral protocols.RuntimeInfo; correct product-version resolution for the app/framework under test; separate app vs. library versions.Upload & debug logging
DataUploadWorker.HTTPClientdebug mode now logs the request body, response data, and request/response headers — bodies rendered as UTF-8 text with a<binary N bytes>fallback, andDD-API-KEY/DD-APPLICATION-KEYredacted to****.Testing
EventsExporterandDatadogSDKTestingbuild from source; telemetry exporter tests pass (metrics, distributions, logs, mixed batches, app-lifecycle).🤖 Generated with Claude Code