…st mapper

main landed feat(otlp)!: attribute-level OTel compatibility (#2091), which adds
`otel_trace_semantics_enabled`: when set, the OTLP payload omits DD-specific
per-span attributes (service.name/operation.name/resource.name/span.type) and
the error.*/span.kind meta tags (that info lives in the OTLP Status and Span.kind
fields). Ported that gating into the prost mapper:

- map_traces_to_otlp / map_span / collect_span_attributes thread the flag; the
  promoted tags and the error.*/span.kind meta tags are skipped when enabled, and
  the dropped-count accounting mirrors what was emitted (promoted/excluded
  compat tags), matching #2091's behavior.
- OtlpTraceConfig gains otel_trace_semantics_enabled; the builder wires it; the
  send path passes config.otel_trace_semantics_enabled. Trace protocol stays
  protobuf-capable (config.protocol.encode), metrics stay JSON.
- Ported #2091's test as a prost-native assertion; existing OTLP/metrics tests
  unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

…Status

span_status only read meta["error.msg"], but the #2091 OTel-semantics gating
drops both error.msg and error.message meta tags. For the common case
(error.message — used by every SDK except .NET), that silently lost the error
text: empty Status.message and, under OTel-semantics, no error.message attribute
either. Fall back to error.message (mirroring main's json_types mapper) so the
message is always promoted to Status. Add an OTel-semantics regression test.

Found by Codex adversarial review of the #2091 integration.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

# What does this PR do?

Adds OTLP HTTP/protobuf as a trace-export encoding alongside HTTP/JSON, selectable via the OTel-standard `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` (`http/json` default, `http/protobuf`). The generated `prost` OTLP types are the single intermediate representation: the mapper builds them directly from native spans, protobuf is `prost::encode_to_vec`, and JSON is a serde serializer over the same types.

# Motivation

libdatadog's OTLP trace export only spoke HTTP/JSON. SDKs that honor `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` need `http/protobuf` to match the OTel default and to talk to collectors that expect protobuf.

# Additional Notes

- Prost is the single IR: no parallel hand-rolled JSON model, no string round-trip. `encode_otlp_protobuf` is `encode_to_vec`; `encode_otlp_json` is a serde serializer (`json_serializer.rs`) emitting OTLP-spec JSON (hex ids, base64 bytes, int64-as-string, lowerCamelCase, proto3 defaults omitted).
- `OtlpProtocol` is `{HttpJson, HttpProtobuf}`; `grpc` is rejected at the parse boundary (`FromStr`) rather than carried as an unsupported variant. `content_type()`/`encode()` live on the type.
- Integrates with the OTLP metrics exporter already on main: the shared low-level sender takes the content-type per request, so traces use the configured protocol while metrics stay JSON.
- Carries through `otel_trace_semantics_enabled` (OTel attribute compatibility, #2091): when set, the prost mapper omits the DD-specific span attributes and promotes the error message to the OTLP `Status`.
- Span-link W3C trace flags are carried through to OTLP `Link.flags`.
- Benchmarks for the encoder hot paths, plus allocation tuning (pre-sized Vecs, allocation-free id/timestamp/int serialization).

# How to test the change?

- `cargo test -p libdd-trace-utils -p libdd-data-pipeline -p libdd-data-pipeline-ffi`, `cargo test --doc`, clippy, fmt, and `cargo ffi-test` pass. A parity test asserts the JSON and protobuf encodings carry the same span from the one prost IR; a protobuf round-trip test asserts the encoding is lossless.
- End-to-end through dd-trace-py (DataDog/dd-trace-py#18609): emitted protobuf-only OTLP traces through a local Agent to the backend. Wire was `application/x-protobuf` (HTTP 200), spans ingested with correct service/resource and a preserved 128-bit trace id.
- Benchmarks: `cargo bench -p libdd-trace-utils --bench main -- otlp/`.

BREAKING CHANGE: removes the previously public `libdd_trace_utils::otlp_encoder::json_types` module (the hand-rolled OTLP JSON model). OTLP encoding now builds prost-generated types as the single IR. libdatadog consumers pin by version, so they pick this up on the next release.


Co-authored-by: brian.marks <[email protected]>

## chore: release v37.0.0

Bumps the workspace version from `36.0.0` to `37.0.0` in `Cargo.toml` and regenerates `Cargo.lock`.

## Post-merge steps

1. Trigger the `create_release` job on GitLab — builds artifacts and creates a draft GitHub release.
2. Ask someone from `libdatadog-core` or `libdatadog-release` to publish the draft release.
3. Trigger the `release-proposal-dispatch` GitHub Actions workflow for per-crate crates.io publishing:
   crates: libdd-data-pipeline,libdd-trace-stats,libdd-trace-utils,libdd-ddsketch

## Notable changes since v36.0.0

- `feat(data-pipeline)`: export client-computed span stats as OTLP trace metrics (#2067)
- `feat(otel-thread-ctx)`: add self check capability (#2095)
- `fix(trace-stats)`: add `grpc_method` to aggregation key (#2151)
- `feat(data-pipeline)`: add stdout log trace exporter (#2074)
- `feat(remote-config)`: use the proto file from the agent (#2165)
- `feat(sidecar)`: expose `default_service_name` for `svc.*` process tags (#2053)
- feat(otlp)!: Export OTLP spans with attribute-level OTel compatibility (#2091)

Co-authored-by: munir.abdinur <[email protected]>

…ibdd-data-pipeline, libdd-li... (#2201)

# Release proposal for libdd-capabilities-impl, libdd-common,
libdd-data-pipeline, libdd-library-config, libdd-remote-config,
libdd-sampling, libdd-telemetry, libdd-tinybytes, libdd-trace-utils and
their dependencies

This PR contains version bumps based on public API changes and commits
since last release.

## libdd-capabilities
**Next version:** `2.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-capabilities-v2.1.0`

### Commits

- feat(data-pipeline)!: add stdout log trace exporter (#2074)

## libdd-common
**Next version:** `5.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-common-v5.1.0`

### Commits

- refactor(clippy): prefer core and alloc imports (#2196)
- fix: update rustls-webpki to 0.103.13 (#2187)
- fix: update anyhow for unsoundness (#2186)
- feat(machine id): Add helpers in ddcommon to fetch the machine UUID l…
(#2163)

## libdd-ddsketch
**Next version:** `1.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-ddsketch-v1.1.0`

### Commits

- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- test(ddsketch): add microbenchmarks for add/encode/collapse (#2125)

## libdd-trace-protobuf
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-protobuf-v4.0.0`

### Commits

- chore!: update protobufs to be in sync with datadog-agent (#2180)
- feat(stats)!: add whole key cardinality limit (#2158)
- feat(remote-config)!: use the proto file from the agent (#2165)
- feat(data-pipeline): OTLP HTTP/protobuf trace export (#2115)

## libdd-capabilities-impl
**Next version:** `3.0.0`
**Semver bump:** `major`
**Tag:** `libdd-capabilities-impl-v3.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.1.0 → ^5.1.0

### Commits

- feat(data-pipeline)!: add stdout log trace exporter (#2074)

## libdd-library-config
**Next version:** `3.0.0`
**Semver bump:** `major`
**Tag:** `libdd-library-config-v3.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0

### Commits

- refactor(clippy): prefer core and alloc imports (#2196)
- feat(library-config)!: caller-supplied threadlocal schema and extra
process-context attributes (#2162)
- fix(otel-thread-ctx): put the threadlocal attributes at the right
place in the context (#2167)

## libdd-remote-config
**Next version:** `2.0.0`
**Semver bump:** `major`
**Tag:** `libdd-remote-config-v2.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0

### Commits

- refactor(libdd-remote-config)!: hide Target inner properties so they
are not leaked (#2182)
- feat(remote-config)!: use the proto file from the agent (#2165)
- refactor(rc): reexport Endpoint and Tag common types (#2147)

## libdd-trace-normalization
**Next version:** `3.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-normalization-v3.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^3.0.1 → ^4.0.0

### Commits

- feat(data-pipeline)!: CSS Trace Filters (#1985)

## libdd-shared-runtime
**Next version:** `2.0.0`
**Semver bump:** `major`
**Tag:** `libdd-shared-runtime-v2.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.1.0 → ^5.1.0

### Commits

- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)
- feat(shared-runtime)!: use weak waker in trigger [APMSP-3371] (#2050)

## libdd-trace-utils
**Next version:** `9.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-utils-v9.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0

### Commits

- ci(miri): skip slow miri tests (#2188)
- chore!: update protobufs to be in sync with datadog-agent (#2180)
- feat(data-pipeline): add agentless export (#2081)
- feat(data-pipeline)!: add stdout log trace exporter (#2074)
- feat(data-pipeline): OTLP HTTP/protobuf trace export (#2115)
- feat(otlp)!: Export OTLP spans with attribute-level OTel compatibility
(#2091)
- test(trace-utils): add V05 msgpack decode microbenchmark (#2127)
- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- test(trace-utils): add VecMap microbenchmarks (#2126)
- chore(stats)!: submit p0 telemetry in stats (#2130)
- refactor(change-buffer)!: replace slot index with span_id, fix segment
isolation (#2105)
- feat(data-pipeline)!: CSS Trace Filters (#1985)
- feat(trace-exporter): add v1 span and its encoder (#2039)
- fix(trace-utils): mark decoded span maps as deduped (#2110)
- feat(trace-utils)!: change buffer implementation (#2055)
- feat(native-spans)!: change buffer foundation (#2046)
- refactor(span)!: use VecMap for `meta`, `metrics` and `meta_struct`
for v04 spans (#2043)
- test: fix timeouts on heavily contended scenarios (#2093)

## libdd-telemetry
**Next version:** `6.0.0`
**Semver bump:** `major`
**Tag:** `libdd-telemetry-v6.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-shared-runtime`: ^1.0.0 → ^2.0.0

### Commits

- ci(miri): skip slow miri tests (#2188)
- refactor(libdd-telemetry)!: avoid leaking libdd-common types in the
public API (#2152)
- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)

## libdd-trace-obfuscation
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-obfuscation-v5.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0

### Commits

- refactor(clippy): prefer core and alloc imports (#2196)
- ci(miri): skip slow miri tests (#2188)
- fix: update anyhow for unsoundness (#2186)

## libdd-trace-stats
**Next version:** `6.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-stats-v6.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-shared-runtime`: ^1.0.0 → ^2.0.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0

### Commits

- chore!: update protobufs to be in sync with datadog-agent (#2180)
- feat(stats)!: send telemetry for cardinality limits (#2159)
- feat(stats)!: add whole key cardinality limit (#2158)
- fix(trace-stats)!: add grpc_method to aggregation key (#2151)
- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)
- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- refactor(span)!: use VecMap for `meta`, `metrics` and `meta_struct`
for v04 spans (#2043)

## libdd-data-pipeline
**Next version:** `7.0.0`
**Semver bump:** `major`
**Tag:** `libdd-data-pipeline-v7.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-shared-runtime`: ^1.0.0 → ^2.0.0
- `libdd-telemetry`: ^5.0.1 → ^6.0.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0
- `libdd-trace-stats`: ^5.0.0 → ^6.0.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0

### Commits

- feat(trace_exporter): enable telemetry in stats exporter (#2160)
- refactor(libdd-telemetry)!: avoid leaking libdd-common types in the
public API (#2152)
- feat(stats): emit canonical gRPC status name for OTLP
rpc.response.status_code (#2183)
- feat(data-pipeline): add agentless export (#2081)
- feat(stats)!: send telemetry for cardinality limits (#2159)
- feat(stats)!: add whole key cardinality limit (#2158)
- fix(trace-stats)!: add grpc_method to aggregation key (#2151)
- feat(data-pipeline)!: add stdout log trace exporter (#2074)
- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)
- feat(data-pipeline): OTLP HTTP/protobuf trace export (#2115)
- feat(otlp)!: Export OTLP spans with attribute-level OTel compatibility
(#2091)
- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- chore(stats)!: submit p0 telemetry in stats (#2130)
- feat(data-pipeline)!: CSS Trace Filters (#1985)
- feat(shared-runtime)!: use weak waker in trigger [APMSP-3371] (#2050)
- refactor(span)!: use VecMap for `meta`, `metrics` and `meta_struct`
for v04 spans (#2043)
- feat(stats)!: add endpoint gating to client-side stats [APMSP-3361]
(#2040)

## libdd-dogstatsd-client
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-dogstatsd-client-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.1.0 → ^5.1.0


## libdd-sampling
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-sampling-v5.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0


[APMSP-3371]:
https://datadoghq.atlassian.net/browse/APMSP-3371?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: iunanua <[email protected]>

#2091)

# What does this PR do?

This PR adds a configuration to the Trace Exporter to export spans with purely OpenTelemetry semantics. When enabled, this configuration updates the DD Span => OTLP Span mapping to exclude the following span tags from the output:
- `service.name`
- `operation.name`
- `resource.name`
- `span.type`
- `error.msg`
- `error.message`
- `span.kind`

This change also fixes the DD Span error message lookup to account for both the `error.msg` and `error.message` tag keys, since they are used interchangeably across Datadog SDKs.

# Motivation

We want to enable OpenTelemetry SDK users to migrate to the Datadog SDK in their language ecosystem, so this change allows them to export OTLP spans from the Datadog SDK without the shape of their data changing. This is most notable when using the OpenTelemetry Tracing API and OpenTelemetry Instrumentation Libraries to generate traces -- with this new semantic mode, the OTLP span should reach the backend without its semantics changing.

# Additional Notes

This may result in a breaking change since it affects the public function `libdd_trace_utils::otlp_encoder::map_traces_to_otlp`.

# How to test the change?

This is tested through unit tests in `libdd-trace-utils/src/otlp_encoder/mapper.rs` that validate the Datadog-specific attributes are excluded from the OTLP span.

Co-authored-by: zach.montoya <[email protected]>
Signed-off-by: Taegyun Kim <[email protected]>

# What does this PR do?

Adds OTLP HTTP/protobuf as a trace-export encoding alongside HTTP/JSON, selectable via the OTel-standard `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` (`http/json` default, `http/protobuf`). The generated `prost` OTLP types are the single intermediate representation: the mapper builds them directly from native spans, protobuf is `prost::encode_to_vec`, and JSON is a serde serializer over the same types.

# Motivation

libdatadog's OTLP trace export only spoke HTTP/JSON. SDKs that honor `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` need `http/protobuf` to match the OTel default and to talk to collectors that expect protobuf.

# Additional Notes

- Prost is the single IR: no parallel hand-rolled JSON model, no string round-trip. `encode_otlp_protobuf` is `encode_to_vec`; `encode_otlp_json` is a serde serializer (`json_serializer.rs`) emitting OTLP-spec JSON (hex ids, base64 bytes, int64-as-string, lowerCamelCase, proto3 defaults omitted).
- `OtlpProtocol` is `{HttpJson, HttpProtobuf}`; `grpc` is rejected at the parse boundary (`FromStr`) rather than carried as an unsupported variant. `content_type()`/`encode()` live on the type.
- Integrates with the OTLP metrics exporter already on main: the shared low-level sender takes the content-type per request, so traces use the configured protocol while metrics stay JSON.
- Carries through `otel_trace_semantics_enabled` (OTel attribute compatibility, #2091): when set, the prost mapper omits the DD-specific span attributes and promotes the error message to the OTLP `Status`.
- Span-link W3C trace flags are carried through to OTLP `Link.flags`.
- Benchmarks for the encoder hot paths, plus allocation tuning (pre-sized Vecs, allocation-free id/timestamp/int serialization).

# How to test the change?

- `cargo test -p libdd-trace-utils -p libdd-data-pipeline -p libdd-data-pipeline-ffi`, `cargo test --doc`, clippy, fmt, and `cargo ffi-test` pass. A parity test asserts the JSON and protobuf encodings carry the same span from the one prost IR; a protobuf round-trip test asserts the encoding is lossless.
- End-to-end through dd-trace-py (DataDog/dd-trace-py#18609): emitted protobuf-only OTLP traces through a local Agent to the backend. Wire was `application/x-protobuf` (HTTP 200), spans ingested with correct service/resource and a preserved 128-bit trace id.
- Benchmarks: `cargo bench -p libdd-trace-utils --bench main -- otlp/`.

BREAKING CHANGE: removes the previously public `libdd_trace_utils::otlp_encoder::json_types` module (the hand-rolled OTLP JSON model). OTLP encoding now builds prost-generated types as the single IR. libdatadog consumers pin by version, so they pick this up on the next release.

Co-authored-by: brian.marks <[email protected]>
Signed-off-by: Taegyun Kim <[email protected]>

## chore: release v37.0.0

Bumps the workspace version from `36.0.0` to `37.0.0` in `Cargo.toml` and regenerates `Cargo.lock`.

## Post-merge steps

1. Trigger the `create_release` job on GitLab — builds artifacts and creates a draft GitHub release.
2. Ask someone from `libdatadog-core` or `libdatadog-release` to publish the draft release.
3. Trigger the `release-proposal-dispatch` GitHub Actions workflow for per-crate crates.io publishing:
   crates: libdd-data-pipeline,libdd-trace-stats,libdd-trace-utils,libdd-ddsketch

## Notable changes since v36.0.0

- `feat(data-pipeline)`: export client-computed span stats as OTLP trace metrics (#2067)
- `feat(otel-thread-ctx)`: add self check capability (#2095)
- `fix(trace-stats)`: add `grpc_method` to aggregation key (#2151)
- `feat(data-pipeline)`: add stdout log trace exporter (#2074)
- `feat(remote-config)`: use the proto file from the agent (#2165)
- `feat(sidecar)`: expose `default_service_name` for `svc.*` process tags (#2053)
- feat(otlp)!: Export OTLP spans with attribute-level OTel compatibility (#2091)

Co-authored-by: munir.abdinur <[email protected]>
Signed-off-by: Taegyun Kim <[email protected]>

…guration (#18495)

## Description

This PR adds ``DD_TRACE_OTEL_SEMANTICS_ENABLED`` to opt in to OpenTelemetry trace semantics for spans. The result is that using OpenTelemetry Tracing instrumentations allows DD SDKs to produce OTLP spans with nearly identical span fields.

When enabled, this updates the dd-trace-py behavior in the following ways:
- General tracing behaviors
  - Stops the library from adding the `version`, `language`, and `env` tags as span attributes
- OpenTelemetry Tracing APIs
  - `OtelSpan.set_attribute` no longer remaps span tags such as `http.response.status_code` or `service.name` so they will maintain their intended key and value type
  - `OtelSpan.record_exception` no longer sets span attributes `error.message`, `error.type`, or `error.stack`. These exist on the `exception` span event and are updated in this PR so that the `exception.stacktrace` and `exception.type` attributes on the span event match what is emitted by the OpenTelemetry SDK
  - `OtelSpan.set_status` updates the `error.message` span attribute each time a new status is registered, including recording a newer status that lacks a message

- OTLP traces export
  - Calls `enable_otel_trace_semantics()` on the libdatadog native trace exporter to remove Datadog-specific span attributes from being emitted on OTLP spans (see the related libdatadog PR for specific attribute changes)


Also makes the following fixes:
- When OTLP traces export is enabled, native span events are also enabled because OTLP spans have a first-class `events` field.

Relies on DataDog/libdatadog#2091 which implements the OTLP trace exporter changes and on #18821 which updates the libdatadog dependency.

## Testing

Unit tests are included for the updated OTel test span behavior and the native span events configuration

## Risks

Besides the native span events change, the remaining changes are opt-in so the risk is low.

## Additional Notes

Currently the libdatadog dependency is set to a development branch which contains the updated `TraceExporterBuilder::enable_otel_trace_semantics()` API 

Co-authored-by: gyuheon0h <[email protected]>
Co-authored-by: mabdinur <[email protected]>
Co-authored-by: bm1549 <[email protected]>
Co-authored-by: zach.montoya <[email protected]>