Skip to content

fix(tracer): skip stats concentrator in OTLP export mode#4609

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 78 commits into
mainfrom
mtoff/stats-disabled-otlp
Apr 20, 2026
Merged

fix(tracer): skip stats concentrator in OTLP export mode#4609
gh-worker-dd-mergequeue-cf854d[bot] merged 78 commits into
mainfrom
mtoff/stats-disabled-otlp

Conversation

@mtoffl01

@mtoffl01 mtoffl01 commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Introduces a statsConcentrator interface with a noopConcentrator implementation so the tracer skips stats computation in OTLP export mode.

  • tracer.stats is now a statsConcentrator interface instead of a concrete *concentrator. In OTLP mode it's assigned a noopConcentrator; otherwise a real concentrator.
  • The non-blocking channel send to concentrator.In was moved into a new trySendSpan method since interface fields can't expose struct fields — this also encapsulates the channel as an internal detail.

Motivation:

In OTLP export mode, Datadog-format stats shouldn't be computed or sent. This approach uses a no-op behind an interface so the disabled case is type-safe and can't be broken by forgetting a guard.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • New code is free of linting errors. You can check this by running make lint locally.
  • New code doesn't break existing tests. You can check this by running make test locally.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • All generated files are up to date. You can check this by running make generate locally.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. Make sure all nested modules are up to date by running make fix-modules locally.

Unsure? Have a question? Request a review!

mtoffl01 and others added 30 commits March 23, 2026 09:30
… service name differs from global service name
…me (#4576)

### What does this PR do?

Moves the `strconv.ParseInt` call for `_dd.p.dm` (decision maker) out of the v1 payload flush path and into the write path.

A `dm uint32` field is added to the `trace` struct. It is kept in sync with `propagatingTags[keyDecisionMaker]` at all mutation sites (`setPropagatingTagLocked`, `unsetPropagatingTagLocked`, `replacePropagatingTags`). A shared `unsetPropagatingTagLocked` helper is introduced so the delete-and-clear logic is not duplicated between `unsetPropagatingTag` and `setSamplingPriorityLockedWithForce`.

`payloadV1.push` now reads the pre-computed value via `trace.decisionMaker()` instead of fetching the string from the propagating tags map and parsing it on every flush.

### Motivation

`payloadV1.push` is called once per finished trace chunk. On every call it was acquiring a read lock, doing a map lookup, and then running `strconv.ParseInt` — all to read a value that was set at most once or twice during the lifetime of the trace. Moving the parse to write-time eliminates the per-flush map lookup and parse, replacing them with a single scalar read behind an `RLock`.

### Reviewer's Checklist

- [x] Changed code has unit tests for its functionality at or near 100% coverage.
- [x] New code is free of linting errors. You can check this by running `make lint` locally.
- [x] New code doesn't break existing tests. You can check this by running `make test` locally.
- [x] Add an appropriate team label so this PR gets put in the right place for the release notes.
- [x] All generated files are up to date. You can check this by running `make generate` locally.


Co-authored-by: dario.castane <[email protected]>
…ED` once, use a global `bool` (#4548)

Stop checking environment variable `DD_TRACE_128_BIT_TRACEID_GENERATION_ENABLED` on span start. Checks it once, use a global `bool`.

It'd be better to load it on `newConfig` and make the value available, but this one happens in `newSpanContext`, which modification will cause a lot of changes in multiple files, including tests.

Complete #1905 work using the current benchmarking platform.

- [x] Changed code has unit tests for its functionality at or near 100% coverage.
- [x] There is a benchmark for any new code, or changes to existing code.
- [x] New code is free of linting errors. You can check this by running `make lint` locally.
- [x] New code doesn't break existing tests. You can check this by running `make test` locally.
- [x] Add an appropriate team label so this PR gets put in the right place for the release notes.

Unsure? Have a question? Request a review!

Co-authored-by: kakkoyun <[email protected]>
Co-authored-by: kemal.akkoyun <[email protected]>
Co-authored-by: Benjamin De Bernardi <[email protected]>
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Apr 15, 2026
### What does this PR do?

Implements the OTLP trace export pipeline (See: [RFC](https://docs.google.com/document/d/1AsUrJxjJavLvSG33kUAJLYzGU8IYgrnf1SMbFDsuGmo/edit?tab=t.0#heading=h.184fd4da9raa)) for dd-trace-go: when `OTEL_TRACES_EXPORTER=otlp` is set, the tracer converts Datadog spans into OTLP protobuf format and sends them directly to an OpenTelemetry Collector instead of the Datadog agent.

This adds three new components:

1. span_to_otlp.go — converts internal Datadog spans to OTLP [TracesData](https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/trace/v1/trace.proto#L28-L38) protobuf, including resource attributes, span kind/status, events, links, 128-bit trace IDs, and sampling filtering (only sampled spans are exported).
2. otlp_writer.go — a traceWriter implementation that buffers converted OTLP spans, marshals them to protobuf, and flushes with retry logic.
3. otlp_transport.go — a lightweight HTTP transport for sending protobuf payloads to an OTLP endpoint, separate from the existing Datadog-protocol transport.

The PR also renames the existing transport interface and config.transport field to ddTransport to make it clear that it handles Datadog-specific traffic (msgpack traces, stats), and ensures ddTransport always points at the Datadog agent even when OTLP mode is active.

### Motivation
We are adding native OTLP export support to dd-trace-go so users can send traces directly to any OTLP-compatible collector. A prior PR added the configuration layer that resolves `OTEL_TRACES_EXPORTER`, `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`, and `OTEL_EXPORTER_OTLP_TRACES_HEADERS`. This PR builds the runtime pipeline on top of that configuration.

### IMPORTANT NOTE!
The existing `agentTraceWriter` and `ddTransport` was too tightly coupled to Datadog conventions — msgpack encoding, agent sampling rate feedback, stats collection — to extend it for the otlp path, so a separate writer and transport were introduced instead. The `otlpTraceWriter` owns its own `otlpTransport` directly (rather than reaching through `config.ddTransport`), keeping OTLP concerns isolated from the Datadog agent path. A follow-up PR may remove `ddTransport` from the config struct entirely, instead passing the transport directly to each writer and consumer at construction time. This would make all writer implementations structurally consistent — each owning its transport as a field — and eliminate the implicit coupling through config.

An additional PR to disable stats collection in otlp mode is in progress [here](#4609).

### Reviewer's Checklist
<!--
* Authors can use this list as a reference to ensure that there are no problems
  during the review but the signing off is to be done by the reviewer(s).
-->

- [ ] Changed code has unit tests for its functionality at or near 100% coverage.
- [ ] [System-Tests](https://github.com/DataDog/system-tests/) covering this feature have been added and enabled with the va.b.c-dev version tag.
- [ ] There is a benchmark for any new code, or changes to existing code.
- [ ] If this interacts with the agent in a new way, a system test has been added.
- [ ] New code is free of linting errors. You can check this by running `make lint` locally.
- [ ] New code doesn't break existing tests. You can check this by running `make test` locally.
- [ ] Add an appropriate team label so this PR gets put in the right place for the release notes.
- [ ] All generated files are up to date. You can check this by running `make generate` locally.
- [ ] Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. Make sure all nested modules are up to date by running `make fix-modules` locally.

Unsure? Have a question? Request a review!


Co-authored-by: kakkoyun <[email protected]>
Co-authored-by: moezein0 <[email protected]>
Co-authored-by: darccio <[email protected]>
Co-authored-by: kemal.akkoyun <[email protected]>
Base automatically changed from mtoff/otlp-export-traces to main April 15, 2026 12:56
@mtoffl01
mtoffl01 marked this pull request as ready for review April 15, 2026 15:50
@mtoffl01
mtoffl01 requested review from a team as code owners April 15, 2026 15:50

@genesor genesor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@darccio

darccio commented Apr 20, 2026

Copy link
Copy Markdown
Member

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Apr 20, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-04-20 08:20:57 UTC ℹ️ Start processing command /merge


2026-04-20 08:21:02 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 15m (p90).


2026-04-20 08:34:00 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit d6dc3b9 into main Apr 20, 2026
231 of 233 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the mtoff/stats-disabled-otlp branch April 20, 2026 08:33
darccio added a commit that referenced this pull request Apr 24, 2026
### What does this PR do?

Introduces a `statsConcentrator` interface with a `noopConcentrator` implementation so the tracer skips stats computation in OTLP export mode.

- `tracer.stats` is now a `statsConcentrator` interface instead of a concrete `*concentrator.` In OTLP mode it's assigned a `noopConcentrator`; otherwise a real concentrator.
- The non-blocking channel send to `concentrator.In` was moved into a new `trySendSpan` method since interface fields can't expose struct fields — this also encapsulates the channel as an internal detail.

### Motivation:

In OTLP export mode, Datadog-format stats shouldn't be computed or sent. This approach uses a no-op behind an interface so the disabled case is type-safe and can't be broken by forgetting a guard.

### Reviewer's Checklist
<!--
* Authors can use this list as a reference to ensure that there are no problems
  during the review but the signing off is to be done by the reviewer(s).
-->

- [ ] Changed code has unit tests for its functionality at or near 100% coverage.
- [ ] [System-Tests](https://github.com/DataDog/system-tests/) covering this feature have been added and enabled with the va.b.c-dev version tag.
- [ ] There is a benchmark for any new code, or changes to existing code.
- [ ] If this interacts with the agent in a new way, a system test has been added.
- [ ] New code is free of linting errors. You can check this by running `make lint` locally.
- [ ] New code doesn't break existing tests. You can check this by running `make test` locally.
- [ ] Add an appropriate team label so this PR gets put in the right place for the release notes.
- [ ] All generated files are up to date. You can check this by running `make generate` locally.
- [ ] Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. Make sure all nested modules are up to date by running `make fix-modules` locally.

Unsure? Have a question? Request a review!


Co-authored-by: kakkoyun <[email protected]>
Co-authored-by: moezein0 <[email protected]>
Co-authored-by: darccio <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants