chore(native): bump libdatadog v35 → v37 by mabdinur · Pull Request #18821 · DataDog/dd-trace-py

mabdinur · 2026-06-30T17:58:26Z

Description

Bump all libdatadog dependencies from v35.0.0 to v37.0.0.
Rename datadog-remote-config → libdd-remote-config (package renamed in v36).
SharedRuntime is now a trait in v37; ForkSafeRuntime is the concrete fork-safe implementation. Thread ForkSafeRuntime through SharedRuntimePy, TraceExporterBuilderPy, and TraceExporterPy.
Rewrite src/native/http_client/ for the v37 libdd-http-client API: HttpClient::new() no longer accepts a runtime argument; block_on moved to the BlockingRuntime trait (imported separately from SharedRuntime).
Update tracer_flare.rs to the new libdd-remote-config config::agent_config / config::agent_task parse API.
Handle AssignmentReason::Default added to datadog-ffe in v37.
Re-enable stats-obfuscation feature on libdd-data-pipeline (still gated behind an explicit feature flag in v37; not folded into defaults).

Testing

n/a

Risks

n/a

Additional Notes

n/a

- Rename datadog-remote-config → libdd-remote-config (package renamed in v36). - Remove libdd-http-client and stats-obfuscation feature (folded into defaults in v36). - Remove bytes/url deps that were only needed by http-client. - SharedRuntime is now a trait in v37; ForkSafeRuntime is the concrete type. Thread ForkSafeRuntime through SharedRuntimePy, TraceExporterBuilderPy, and TraceExporterPy. - Update tracer_flare.rs to the new libdd-remote-config config::agent_config/agent_task API. - Handle AssignmentReason::Default added to datadog-ffe in v37. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

datadog-prod-us1-4 · 2026-06-30T17:58:50Z

Tests

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 10 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | baseline:build

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64, 1]

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741491-d2b8243-musllinux_1_2_x86_64, 1]

View all 10 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 3 jobs - 3 passed on retry

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 64fe2e3 | Docs | Datadog PR Page | Give us feedback!}

libdatadog removed the libdd-http-client crate in v36; it no longer ships in v37. Remove the native http_client module and all its dependents: - Delete src/native/http_client/ (client, response, errors, mod) - Remove mod/register_http_client from lib.rs - Remove enable_client_side_stats_obfuscation from data_pipeline/mod.rs (stats-obfuscation feature merged into libdd-data-pipeline defaults) - Remove HTTPClient, HttpResponse, HttpClientError and subclasses from _native.pyi and ddtrace/internal/native/__init__.py - Delete ddtrace/internal/http_client.py and its tests The HTTPClient had no production callers outside its own wrapper module. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

cit-pr-commenter-54b7da · 2026-06-30T18:05:50Z

Codeowners resolved as

src/native/Cargo.lock                                                   @DataDog/apm-core-python
src/native/Cargo.toml                                                   @DataDog/apm-core-python
src/native/crashtracker/crashtracker_runtime_stacks.rs                  @DataDog/profiling-python @DataDog/apm-core-python
src/native/data_pipeline/mod.rs                                         @DataDog/apm-core-python
src/native/ffe.rs                                                       @DataDog/apm-core-python
src/native/http_client/client.rs                                        @DataDog/apm-core-python
src/native/shared_runtime/mod.rs                                        @DataDog/apm-core-python
src/native/tracer_flare.rs                                              @DataDog/apm-core-python
tests/crashtracker/test_crashtracker.py                                 @DataDog/profiling-python

stats-obfuscation is now on by default in libdatadog v37; the enable_client_side_stats_obfuscation() builder method no longer exists. Remove the call from TraceExporterWriter. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

…bfuscation feature The method still exists in libdatadog v37 but requires the stats-obfuscation feature on libdd-data-pipeline. Re-add the feature flag and restore the PyO3 wrapper, Python stub, and writer call that were incorrectly removed. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

pr-commenter · 2026-06-30T18:40:06Z

Benchmarks

Benchmark execution time: 2026-07-08 19:09:07

Comparing candidate commit b441a95 in PR branch munir/bump-libdatadog-v37 with baseline commit c12bb9d in branch main.

Found 0 performance improvements and 3 performance regressions! Performance is the same for 613 metrics, 10 unstable metrics.

scenario:iastaspectsospath-ospathbasename_aspect

🟥 execution_time [+92.440µs; +101.790µs] or [+21.280%; +23.432%]

scenario:span-start

🟥 execution_time [+1.727ms; +1.869ms] or [+11.172%; +12.096%]

scenario:tracer-small

🟥 execution_time [+28.276µs; +30.844µs] or [+8.337%; +9.094%]

libdd-http-client still exists in v37 — it was redesigned, not removed. Restore the http_client Rust module rewritten for the v37 API: - HttpClient::new() no longer takes a runtime; BlockingRuntime trait (not SharedRuntime) provides block_on — update imports accordingly - Arc<ForkSafeRuntime> replaces Arc<SharedRuntime> (SharedRuntime is now a trait in v37; ForkSafeRuntime is the concrete type) - Re-add libdd-http-client, bytes, url deps to Cargo.toml - Restore mod http_client and register_http_client in lib.rs - Restore ddtrace/internal/http_client.py and its test suite - Restore HTTPClient, HttpResponse, HttpClientError and subclasses in _native.pyi and ddtrace/internal/native/__init__.py - Add missing Iterable import to _native.pyi Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

The merged libdatadog FFE remaps per-flag config parse failures (bad regex, variant type mismatch) from ConfigurationParseError to the new FlagConfigurationInvalid variant. dd-trace-py's From<EvaluationError> match didn't handle it, so it fell through to ErrorCode::General and the feature_flag.evaluations metric reported error.type=general instead of parse_error, failing the FEATURE_FLAGGING_AND_EXPERIMENTATION system tests. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

Co-authored-by: Munir Abdinur <[email protected]>

Restore the bytes 1.10+ comment and the original ordering of url and libdd-http-client that were cosmetically changed during the v37 bump. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

With collect_all_threads enabled (default in v37), the receiver uses PTRACE_SEIZE + PTRACE_INTERRUPT on the crashing thread, which interrupts poll() in wait_for_pollhup with EINTR. When crashing from a dynamically- loaded C++ extension on Python 3.14, repeated EINTR prevents the 5-second timeout from expiring, causing an indefinite hang. Consistent with the existing pattern of Python 3.14 crashtracker skips ("Stack v2 not supported on 3.14") in 9d9bd80. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

…-v37

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

cit-pr-commenter-54b7da · 2026-07-06T15:05:13Z

Circular import analysis

⚠️ Existing circular imports

There are 4974 circular imports that already exist on the base branch and have not been changed by this PR.

Show existing cycles (showing 5 of 4974 shortest)

ddtrace.internal.datastreams -> ddtrace.internal.datastreams.botocore -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.aiokafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.google_cloud_pubsub -> ddtrace.internal.datastreams
ddtrace.internal.core -> ddtrace._trace.span -> ddtrace.internal.core
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kafka -> ddtrace.internal.datastreams

To see all cycles, download the cycles-base.json and cycles-pr.json artifacts from this CI job and run:
uv run --script scripts/import-analysis/cycles.py compare cycles-base.json cycles-pr.json

…d from ptrace The receiver in v37 began ptracing the crashing thread (crashing_tid) as part of collect_all_threads. That thread is blocked in poll() inside its signal handler on the alternate signal stack; the resulting PTRACE_INTERRUPT issues EINTR in a loop, causing the receiver to hang indefinitely on CPython 3.14. Pin all libdatadog deps to commit 504ea9d7 on the fix branch, which restores the v35 behaviour of excluding crashing_tid from stream_thread_contexts. With the fix in place the 3.14 skip is no longer needed. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

gyuheon0h · 2026-07-07T15:08:27Z

Im not sure if claude's diagnosis on 52fa5a9 is correct. Do you mind if I try to fix this on your branch sometime later today before you merge?

mabdinur · 2026-07-07T16:04:40Z

Im not sure if claude's diagnosis on 52fa5a9 is correct. Do you mind if I try to fix this on your branch sometime later today before you merge?

@gyuheon0h Yes, that was a mistake. I was looking into a timeout in a crash tracking test: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-py/-/pipelines/123032120. It seems like v37 introduced a regression. Here's a potential fix: https://github.com/DataDog/libdatadog/pull/2207/changes.

I will revert the build changes in this PR.

…ng thread from ptrace" This reverts commit 52fa5a9.

Co-authored-by: Munir Abdinur <[email protected]>

mabdinur · 2026-07-10T14:01:07Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-07-10T14:01:12Z

View all feedbacks in Devflow UI.

2026-07-10 14:01:11 UTC ℹ️ Start processing command /merge

2026-07-10 14:01:18 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 1h (p90).

2026-07-10 16:01:52 UTC ❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 7934734439606769018 took longer than expected. The current limit for the base branch 'main' is 120 minutes.

mabdinur · 2026-07-10T17:57:19Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-07-10T17:57:23Z

View all feedbacks in Devflow UI.

2026-07-10 17:57:23 UTC ℹ️ Start processing command /merge

2026-07-10 17:57:31 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2026-07-10 20:00:39 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in main is approximately 1h (p90).

2026-07-10 20:38:09 UTC ℹ️ MergeQueue: This merge request was merged

mabdinur · 2026-07-10T19:56:52Z

/merge -f --reason PR is taking forever to merge

gh-worker-devflow-routing-ef8351 · 2026-07-10T19:56:57Z

View all feedbacks in Devflow UI.

2026-07-10 19:56:56 UTC ℹ️ Start processing command /merge -f --reason PR is taking forever to merge

2026-07-10 19:56:59 UTC ❌ MergeQueue

PR already in the queue with status waiting

## Description - Bump all libdatadog dependencies from v35.0.0 to v37.0.0. - Rename datadog-remote-config → libdd-remote-config (package renamed in v36). - SharedRuntime is now a trait in v37; ForkSafeRuntime is the concrete fork-safe implementation. Thread ForkSafeRuntime through SharedRuntimePy, TraceExporterBuilderPy, and TraceExporterPy. - Rewrite src/native/http_client/ for the v37 libdd-http-client API: HttpClient::new() no longer accepts a runtime argument; block_on moved to the BlockingRuntime trait (imported separately from SharedRuntime). - Update tracer_flare.rs to the new libdd-remote-config config::agent_config / config::agent_task parse API. - Handle AssignmentReason::Default added to datadog-ffe in v37. - Re-enable stats-obfuscation feature on libdd-data-pipeline (still gated behind an explicit feature flag in v37; not folded into defaults). ## Testing n/a ## Risks n/a ## Additional Notes n/a Co-authored-by: gyuheon0h <[email protected]> Co-authored-by: bm1549 <[email protected]> Co-authored-by: munir.abdinur <[email protected]>

…guration (#18495) ## Description This PR adds ``DD_TRACE_OTEL_SEMANTICS_ENABLED`` to opt in to OpenTelemetry trace semantics for spans. The result is that using OpenTelemetry Tracing instrumentations allows DD SDKs to produce OTLP spans with nearly identical span fields. When enabled, this updates the dd-trace-py behavior in the following ways: - General tracing behaviors - Stops the library from adding the `version`, `language`, and `env` tags as span attributes - OpenTelemetry Tracing APIs - `OtelSpan.set_attribute` no longer remaps span tags such as `http.response.status_code` or `service.name` so they will maintain their intended key and value type - `OtelSpan.record_exception` no longer sets span attributes `error.message`, `error.type`, or `error.stack`. These exist on the `exception` span event and are updated in this PR so that the `exception.stacktrace` and `exception.type` attributes on the span event match what is emitted by the OpenTelemetry SDK - `OtelSpan.set_status` updates the `error.message` span attribute each time a new status is registered, including recording a newer status that lacks a message - OTLP traces export - Calls `enable_otel_trace_semantics()` on the libdatadog native trace exporter to remove Datadog-specific span attributes from being emitted on OTLP spans (see the related libdatadog PR for specific attribute changes) Also makes the following fixes: - When OTLP traces export is enabled, native span events are also enabled because OTLP spans have a first-class `events` field. Relies on DataDog/libdatadog#2091 which implements the OTLP trace exporter changes and on #18821 which updates the libdatadog dependency. ## Testing Unit tests are included for the updated OTel test span behavior and the native span events configuration ## Risks Besides the native span events change, the remaining changes are opt-in so the risk is low. ## Additional Notes Currently the libdatadog dependency is set to a development branch which contains the updated `TraceExporterBuilder::enable_otel_trace_semantics()` API Co-authored-by: gyuheon0h <[email protected]> Co-authored-by: mabdinur <[email protected]> Co-authored-by: bm1549 <[email protected]> Co-authored-by: zach.montoya <[email protected]>

mabdinur and others added 2 commits June 30, 2026 14:10

mabdinur force-pushed the munir/bump-libdatadog-v37 branch from 3d0adcb to 4fcb1a9 Compare June 30, 2026 19:10

mabdinur marked this pull request as ready for review June 30, 2026 19:15

mabdinur requested review from a team as code owners June 30, 2026 19:15

mabdinur requested review from brettlangdon, dubloom and rithikanarayan June 30, 2026 19:15

brettlangdon reviewed Jun 30, 2026

View reviewed changes

Comment thread releasenotes/notes/bump-libdatadog-v37-88a1d6d2f0f7f696.yaml Outdated

Comment thread src/native/lib.rs

mabdinur force-pushed the munir/bump-libdatadog-v37 branch from 4fcb1a9 to 89139de Compare June 30, 2026 19:21

mabdinur requested a review from brettlangdon June 30, 2026 19:22

mabdinur added the changelog/no-changelog A changelog entry is not required for this PR. label Jun 30, 2026

bm1549 and others added 2 commits June 30, 2026 16:45

Merge branch 'main' into munir/bump-libdatadog-v37

0b12d69

mabdinur mentioned this pull request Jul 1, 2026

fix(native): bump libdatadog to v36 #18713

Closed

dubloom reviewed Jul 2, 2026

View reviewed changes

Comment thread src/native/http_client/client.rs

Comment thread src/native/shared_runtime/mod.rs

Comment thread src/native/Cargo.toml

mabdinur commented Jul 2, 2026

View reviewed changes

Comment thread src/native/Cargo.toml

mabdinur commented Jul 2, 2026

View reviewed changes

Comment thread src/native/http_client/client.rs

mabdinur commented Jul 2, 2026

View reviewed changes

Comment thread src/native/shared_runtime/mod.rs

Apply suggestions from code review

e68bfd0

Co-authored-by: Munir Abdinur <[email protected]>

mabdinur requested a review from dubloom July 2, 2026 14:06

chore(native): restore Cargo.toml comment and dependency ordering

07006b5

Restore the bytes 1.10+ comment and the original ordering of url and libdd-http-client that were cosmetically changed during the v37 bump. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

VianneyRuhlmann approved these changes Jul 2, 2026

View reviewed changes

Comment thread src/native/ffe.rs Outdated

mabdinur requested a review from a team as a code owner July 2, 2026 17:58

mabdinur requested a review from taegyunkim July 2, 2026 17:58

mabdinur and others added 2 commits July 6, 2026 11:01

Merge remote-tracking branch 'origin/main' into munir/bump-libdatadog…

9a336c1

…-v37

style: fix ruff line-length on skipif decorator in crashtracker test

17fb480

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

mabdinur commented Jul 6, 2026

View reviewed changes

Comment thread tests/crashtracker/test_crashtracker.py Outdated

mabdinur and others added 2 commits July 6, 2026 15:28

Apply suggestion from @mabdinur

c000991

gh-worker-ownership-write-b05516 Bot removed the request for review from a team July 7, 2026 15:03

Revert "fix(crashtracker): pin libdatadog to fix that excludes crashi…

0b16da7

…ng thread from ptrace" This reverts commit 52fa5a9.

mabdinur commented Jul 7, 2026

View reviewed changes

Comment thread tests/crashtracker/test_crashtracker.py Outdated

mabdinur and others added 2 commits July 7, 2026 12:07

Apply suggestions from code review

f884eba

Co-authored-by: Munir Abdinur <[email protected]>

gil guard

b441a95

gyuheon0h approved these changes Jul 8, 2026

View reviewed changes

brettlangdon approved these changes Jul 10, 2026

View reviewed changes

Comment thread src/native/crashtracker/crashtracker_runtime_stacks.rs

Comment thread tests/crashtracker/test_crashtracker.py

Merge branch 'main' into munir/bump-libdatadog-v37

64fe2e3

zacharycmontoya mentioned this pull request Jul 10, 2026

feat(otel): enable improved OTel trace compatibility via opt-in configuration #18495

Merged

gh-worker-dd-mergequeue-cf854d Bot merged commit da57a6a into main Jul 10, 2026
1273 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the munir/bump-libdatadog-v37 branch July 10, 2026 20:38

Uh oh!

Conversation

mabdinur commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Risks

Additional Notes

Uh oh!

datadog-prod-us1-4 Bot commented Jun 30, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codeowners resolved as

Uh oh!

pr-commenter Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

scenario:iastaspectsospath-ospathbasename_aspect

scenario:span-start

scenario:tracer-small

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Circular import analysis

⚠️ Existing circular imports

Uh oh!

Uh oh!

gyuheon0h commented Jul 7, 2026

Uh oh!

mabdinur commented Jul 7, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mabdinur commented Jul 10, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mabdinur commented Jul 10, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mabdinur commented Jul 10, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mabdinur commented Jun 30, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented Jun 30, 2026 •

edited by datadog-official Bot

Loading

cit-pr-commenter-54b7da Bot commented Jun 30, 2026 •

edited

Loading

pr-commenter Bot commented Jun 30, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 6, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 10, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 10, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 10, 2026 •

edited

Loading