Refactor TracesTransportType and fix OTLP traces endpoint bug when APM UDS socket is set#8614
Closed
zacharycmontoya wants to merge 7 commits into
Closed
Refactor TracesTransportType and fix OTLP traces endpoint bug when APM UDS socket is set#8614zacharycmontoya wants to merge 7 commits into
TracesTransportType and fix OTLP traces endpoint bug when APM UDS socket is set#8614zacharycmontoya wants to merge 7 commits into
Conversation
BenchmarksBenchmark execution time: 2026-05-15 00:49:51 Comparing candidate commit b676d57 in PR branch Some scenarios are present only in baseline or only in candidate runs. If you didn't create or remove some scenarios in your branch, this maybe a sign of crashed benchmarks 💥💥💥 Scenarios present only in baseline:
Found 3 performance improvements and 5 performance regressions! Performance is the same for 46 metrics, 18 unstable metrics, 89 known flaky benchmarks, 37 flaky benchmarks without significant changes.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8614) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (73ms) : 70, 76
master - mean (73ms) : 70, 75
section Bailout
This PR (8614) - mean (79ms) : 75, 83
master - mean (79ms) : 75, 83
section CallTarget+Inlining+NGEN
This PR (8614) - mean (1,103ms) : 1063, 1143
master - mean (1,104ms) : 1056, 1153
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (116ms) : 109, 123
master - mean (113ms) : 109, 117
section Bailout
This PR (8614) - mean (117ms) : 112, 122
master - mean (118ms) : 113, 124
section CallTarget+Inlining+NGEN
This PR (8614) - mean (786ms) : 764, 807
master - mean (787ms) : 759, 814
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (103ms) : 97, 109
master - mean (102ms) : 98, 106
section Bailout
This PR (8614) - mean (101ms) : 99, 104
master - mean (106ms) : 100, 111
section CallTarget+Inlining+NGEN
This PR (8614) - mean (944ms) : 900, 988
master - mean (946ms) : 907, 986
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (99ms) : 96, 102
master - mean (99ms) : 95, 103
section Bailout
This PR (8614) - mean (104ms) : 98, 109
master - mean (103ms) : 98, 108
section CallTarget+Inlining+NGEN
This PR (8614) - mean (821ms) : 778, 864
master - mean (822ms) : 788, 856
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (200ms) : 192, 208
master - mean (202ms) : 194, 210
section Bailout
This PR (8614) - mean (204ms) : 201, 207
master - mean (206ms) : 198, 213
section CallTarget+Inlining+NGEN
This PR (8614) - mean (1,202ms) : 1157, 1248
master - mean (1,205ms) : 1159, 1251
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (293ms) : 286, 300
master - mean (288ms) : 281, 295
section Bailout
This PR (8614) - mean (290ms) : 283, 297
master - mean (290ms) : 283, 296
section CallTarget+Inlining+NGEN
This PR (8614) - mean (964ms) : 943, 985
master - mean (967ms) : 941, 992
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (283ms) : 275, 290
master - mean (281ms) : 275, 287
section Bailout
This PR (8614) - mean (283ms) : 277, 290
master - mean (280ms) : 275, 285
section CallTarget+Inlining+NGEN
This PR (8614) - mean (1,168ms) : 1123, 1212
master - mean (1,163ms) : 1117, 1208
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8614) - mean (279ms) : 272, 287
master - mean (281ms) : 274, 289
section Bailout
This PR (8614) - mean (280ms) : 274, 286
master - mean (282ms) : 275, 288
section CallTarget+Inlining+NGEN
This PR (8614) - mean (1,047ms) : 1007, 1087
master - mean (1,041ms) : 997, 1085
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
- Rename enum TracesTransportType to AgentTransportType (still used by ExporterSettings.TracesTransport) - Add new AgentTransport property (type=AgentTransportType) to the ExporterSettings to separate logical traces endpoint vs agent endpoint - Change some usage of ExporterSettings.TracesTransport to ExporterSettings.AgentTransport
…ing an IApiRequestFactory
zacharycmontoya
force-pushed
the
zach.montoya/otlp-when-uds-enabled
branch
from
May 12, 2026 22:36
146136f to
629b45b
Compare
zacharycmontoya
added
type:refactor
area:opentelemetry
…ces HTTP client FOR ALL TRANSPORTS
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ed58050984
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… as a regression test
- Do not set OTLP trace headers twice. - Reset the traces transport back to the Datadog Agent transport if the OTLP protocol is unsupported
…lpTraces as a regression test" This reverts commit 4243173.
5 tasks
andrewlock
approved these changes
May 19, 2026
andrewlock
left a comment
andrewlock
left a comment
Member
There was a problem hiding this comment.
Thanks, I have some questions about the naming options, and if the proposed split is the best one, but I accept it probably is, and this stuff is just a bit confusing 😅
| internal AgentTransportType AgentTransport { get; } | ||
|
|
||
| /// <summary> | ||
| /// Gets the transport used to send traces to the Agent. |
Member
There was a problem hiding this comment.
I wonder if we should tweak this documentation, to try to note the difference between AgentTransport and TracesTransport. The two have always been synonymous, and I don't think it's at all obvious why they aren't?
Suggested change
| /// Gets the transport used to send traces to the Agent. | |
| /// Gets the transport used to send traces to the Agent. This | |
| /// may be different to <see cref="AgentTransport" /> when | |
| /// OTLP export is enabled. |
| /// <summary> | ||
| /// Gets the transport used to send requests to the Agent. | ||
| /// </summary> | ||
| internal AgentTransportType AgentTransport { get; } |
Member
There was a problem hiding this comment.
We now have MetricsTransport, TracesTransport, and AgentTransport... but the first two are both "agent transports" conceptually 🤔
I wonder if we should rename this to be TraceAgentTransport, or DefaultAgentTransport, or just DefaultTransport, or Something 🤔 Because I forsee a lot of confusion between this and TracesTransport tbh. I mean, I've read the PR description, I understand why we need it, and I still know that I'm going to be super confused 😂
Member
There was a problem hiding this comment.
Thinking about this some more - does it actually make sense to explicitly separate the two properties based on OTLP instead? 🤔 My concern right now is that it's not entirely clear to me which of the properties you're supposed to use at any particular time.. But if we had OtlpTraceTransport (for example), then it would be more obvious?
I really don't know if that makes sense tbh, just spitballing 😅
Contributor
Author
There was a problem hiding this comment.
So I've thought about it some more, and I think we could actually isolate the OtlpTraceTransport into its own setting, that way we have the following distinct settings:
Agent settings
TraceAgentTransportDogStatsDTransport
OTLP settings
OtlpTracesTransport-- We already have one setting for each of the configurable parameters (endpoint, protocol, timeout headers) so maybe unnecessary.OtlpMetricsTransport-- We already have one setting for each of the configurable parameters (endpoint, protocol, timeout headers) so maybe unnecessary.
WDYT? Is this separation clearer? If so, I can minimize the scope of the fix to just the OtlpTracesTransport
Contributor
Author
There was a problem hiding this comment.
With that in mind, here's a proposed set of changes which is the bare minimum set of changes (and it looks very similar to Duncan's original PR) but it uses our transports as you'd expect: https://github.com/DataDog/dd-trace-dotnet/compare/zach.montoya/otlp-when-uds-enabled-feedback?expand=1
Contributor
Author
There was a problem hiding this comment.
I just created a new PR here which removes the proposed refactorings altogether as they're not necessary to solve the underlying issue #8668
| } | ||
|
|
||
| #endif | ||
| var transport = uri.OriginalString.StartsWith(UnixDomainSocketPrefix, StringComparison.OrdinalIgnoreCase) |
Member
There was a problem hiding this comment.
nit: seeing as the uri.OriginalString.StartsWith is tested in all paths, maybe we move the #if !NETCOREAPP3_1_OR_GREATER inside the if, put this in the #else branch?
return new(AgentTransportType.UnixDomainSocket, uri, OtlpProtocol: otlpProtocol);and inline AgentTransportType.Default here?
|
|
||
| switch (strategy) | ||
| { | ||
| case AgentTransportType.WindowsNamedPipe: |
Member
There was a problem hiding this comment.
You mentioned in the description that these are untested, but are they supported? If so, this seems fine to me.
| src/Datadog.Trace/TracerConstants.cs | ||
| src/Datadog.Trace/TracerManager.cs | ||
| src/Datadog.Trace/TracerManagerFactory.cs | ||
| src/Datadog.Trace/Agent/AgentTransportType.cs |
Member
There was a problem hiding this comment.
nit: Could we add nullability if we're touching the file and it doesn't cause too much cascading? 😄
Member
There was a problem hiding this comment.
There should probably be a couple of additional tests for the new AgentTransport property?
| // This product includes software developed at Datadog (https://www.datadoghq.com/). Copyright 2017 Datadog, Inc. | ||
| // </copyright> | ||
|
|
||
| #if NET6_0_OR_GREATER |
Member
There was a problem hiding this comment.
AFACIT, nothing in ExporterSettings is .NET 6 only - should it be? If we only support OTLP in .NET 6+?
Contributor
Author
|
Closing in favor of #8668 to minimize the code changes for resolving the bug! |
zacharycmontoya
added a commit
that referenced
this pull request
Jun 2, 2026
… is set (#8668) ## Summary of changes Fixes a bug where setting the APM UDS socket would inadvertently set the HttpClient used by OTLP traces export, which led to trace submission issues. This is accomplished by having the OTLP traces exporter use a new code path to configure the underlying HTTP client. ## Reason for change The primary motivation was to resolve the following OTLP traces export issue: When OTLP traces export is enabled through the configuration OTEL_TRACES_EXPORTER=otlp, the endpoint defaults to TCP/IP communication with endpoint http://localhost:4317 or http://localhost:4318. However, when DD_APM_RECEIVER_SOCKET is configured and its socket file is present, the underlying HTTP client for the OTLP traces exporter is erroneously being configured for UDS communication rather than TCP/IP. ## Implementation details - Adds a new `OtlpTransportStrategy` that is a factory for OTLP traces HTTP clients. This supports TCP/IP endpoints and UDS endpoints (though UDS is not currently tested) - `ManagedApiOtlp` gets an HTTP client from `OtlpTransportStrategy.GetTraces` rather than `TracesTransportStrategy.Get` - Creates and uses a new `OtlpHeaderHelper` instance to set default headers for all OTLP traces HTTP requests ## Test coverage Adds unit tests in `ManagedApiOtlpTests`. Note: UDS is supported but currently untested. ## Other details Replaces #8614 and #8460
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Refactors the
TracesTransportTypetoAgentTransportTypeand updates its usage across the repository to better separate "agent communication" from "traces communication".Also fixes a bug related to this issue where setting the APM UDS socket affected the HttpClient used by OTLP traces export.
Reason for change
The primary motivation was to resolve the following OTLP traces export issue: When OTLP traces export is enabled through the configuration
OTEL_TRACES_EXPORTER=otlp, the endpoint defaults to TCP/IP communication with endpointhttp://localhost:4317orhttp://localhost:4318. However, whenDD_APM_RECEIVER_SOCKETis configured and its socket file is present, the underlying HTTP client is erroneously being configured for UDS communication rather than TCP/IP.Implementation details
ExporterSettings.TracesTransport)ExporterSettings.TracesTransporttoExporterSettings.AgentTransport. When OTLP traces export is enabled, override the previously calculatedExporterSettings.TracesTransportwith the calculated transport from the OTLP configurationsManagedApiOtlpto create an api factory by calling a newOtlpTransportStrategy.GetTracesmethod rather thanTracesTransportStrategy.Get. This will properly set the headers, endpoint, and timeout values for the underlying HTTP client following the OTLP configuration rather than the original DD Agent configuration.Test coverage
Existing tests are exercised and a few new unit tests are added to validate the transport and endpoint. Some tests still use the
ExporterSettings.TracesTransportproperty to make assertions about traces specifically, while other tests have been updated to assert againstExporterSettings.AgentTransportOther details
OtlpTransportStrategyreuses the logic fromAgentTransportStrategyto create HTTP clients for HTTP, Named Pipes, and UDS even though the latter two are untested. If desired, we can remove those code paths as they are currently untested.Supercedes #8460