Describe the bug
The tracing using UDS doesn't work when DD_AGENT_HOST and DD_TRACE_AGENT_URL is configured.
Then there won't be any traces submitted and the logfile logs:
2022-02-11 12:56:48.626 +00:00 [ERR] An error occurred while sending 1 traces to the agent at unix:///v0.4/traces System.NotSupportedException: The 'unix' scheme is not supported.
at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state)
at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Datadog.Trace.Agent.Transports.HttpClientRequest.PostAsync(ArraySegment`1 bytes, String contentType)
at Datadog.Trace.Agent.Api.SendTracesAsync(ArraySegment`1 traces, Int32 numberOfTraces, IApiRequest request, Boolean finalTry)
at Datadog.Trace.Agent.Api.SendTracesAsync(ArraySegment`1 traces, Int32 numberOfTraces)
{ MachineName: ".", Process: "[1 dotnet]", AppDomain: "[1 ConsoleApp]", AssemblyLoadContext: "\"Default\" System.Runtime.Loader.DefaultAssemblyLoadContext #2", TracerVersion: "2.3.0.0" }
The root cause seems to be ExporterSettings.ConfigureTraceTransport, because it reads the agentHost from DD_AGENT_HOST, then checks if it starts with unix:// and sets TracesTransportType to Default. Later on the agentUri will be read from DD_TRACE_AGENT_URL.
Which causes the TracesTransportStrategy to return a HttpClientRequestFactory, but the configuration is set to a UDS uri.
|
var agentHost = source?.GetString(ConfigurationKeys.AgentHost) ?? |
|
source?.GetString("DD_TRACE_AGENT_URL") ?? |
|
// backwards compatibility for names used in the past |
|
source?.GetString("DD_TRACE_AGENT_HOSTNAME") ?? |
|
source?.GetString("DATADOG_TRACE_AGENT_HOSTNAME"); |
|
|
|
var agentPort = source?.GetInt32(ConfigurationKeys.AgentPort) ?? |
|
// backwards compatibility for names used in the past |
|
source?.GetInt32("DATADOG_TRACE_AGENT_PORT"); |
|
|
|
TracesPipeName = source?.GetString(ConfigurationKeys.TracesPipeName); |
|
|
|
// Agent port is set to zero in places like AAS where it's needed to prevent port conflict |
|
// The agent will fail to start if it can not bind a port, so we need to override 8126 to prevent port conflict |
|
// Port 0 means it will pick some random available port |
|
var hasExplicitHostOrPortSettings = (agentPort != null && agentPort != 0) || agentHost != null; |
|
|
|
if (hasExplicitHostOrPortSettings) |
|
{ |
|
if (agentHost?.StartsWith(UnixDomainSocketPrefix) ?? false) |
|
{ |
|
traceTransport = TracesTransportType.UnixDomainSocket; |
|
TracesUnixDomainSocketPath = agentHost; |
|
} |
|
else |
|
{ |
|
// The agent host is explicitly configured, we should assume UDP for metrics |
|
forceMetricsOverUdp = true; |
|
traceTransport = TracesTransportType.Default; |
|
} |
|
} |
|
else if (!string.IsNullOrWhiteSpace(TracesPipeName)) |
|
{ |
|
traceTransport = TracesTransportType.WindowsNamedPipe; |
|
|
|
TracesPipeTimeoutMs = source?.GetInt32(ConfigurationKeys.TracesPipeTimeoutMs) |
|
#if DEBUG |
|
?? 20_000; |
|
#else |
|
?? 500; |
|
#endif |
|
} |
|
|
|
if (traceTransport == null) |
|
{ |
|
// Check for UDS |
|
var traceSocket = source?.GetString(ConfigurationKeys.TracesUnixDomainSocketPath); |
|
|
|
if (traceSocket != null) |
|
{ |
|
traceTransport = TracesTransportType.UnixDomainSocket; |
|
TracesUnixDomainSocketPath = traceSocket; |
|
} |
|
else |
|
{ |
|
// check for default file |
|
if (_fileExists(DefaultTracesUnixDomainSocket)) |
|
{ |
|
traceTransport = TracesTransportType.UnixDomainSocket; |
|
TracesUnixDomainSocketPath = DefaultTracesUnixDomainSocket; |
|
} |
|
} |
|
} |
|
|
|
// Still build the Uri no matter what the transport as we send it in the http message |
|
agentHost = agentHost ?? DefaultAgentHost; |
|
agentPort = agentPort ?? DefaultAgentPort; |
|
|
|
var agentUri = source?.GetString(ConfigurationKeys.AgentUri) ?? |
|
// default value |
|
$"http://{agentHost}:{agentPort}"; |
|
|
|
AgentUri = new Uri(agentUri); |
To Reproduce
A repository with a dockerfile to reproduce can be found here: https://github.com/hiiru/dd-net-unix-issue
Basically it prepares a linux container with a .NET app and the tracer with these ENV vars set:
DD_AGENT_HOST=localhost
DD_TRACE_AGENT_URL unix:///var/run/datadog/apm.socket
When DD_AGENT_HOST is set, the NotSupportedException gets thrown.
Otherwise the correct SocketExceptionFactory+ExtendedSocketException gets thrown (since there is no socket in the example docker).
Expected behavior
I except that the TracesTransportType will be set from DD_TRACE_AGENT_URL and the tracer uses the UDS transfer.
So that it's possible to use UDS for trace and HTTP for Dogstatsd.
Describe the bug
The tracing using UDS doesn't work when DD_AGENT_HOST and DD_TRACE_AGENT_URL is configured.
Then there won't be any traces submitted and the logfile logs:
The root cause seems to be
ExporterSettings.ConfigureTraceTransport, because it reads the agentHost from DD_AGENT_HOST, then checks if it starts with unix:// and setsTracesTransportTypeto Default. Later on the agentUri will be read from DD_TRACE_AGENT_URL.Which causes the
TracesTransportStrategyto return aHttpClientRequestFactory, but the configuration is set to a UDS uri.dd-trace-dotnet/tracer/src/Datadog.Trace/Configuration/ExporterSettings.cs
Lines 231 to 303 in 57f8232
To Reproduce
A repository with a dockerfile to reproduce can be found here: https://github.com/hiiru/dd-net-unix-issue
Basically it prepares a linux container with a .NET app and the tracer with these ENV vars set:
DD_AGENT_HOST=localhost
DD_TRACE_AGENT_URL unix:///var/run/datadog/apm.socket
When DD_AGENT_HOST is set, the NotSupportedException gets thrown.
Otherwise the correct SocketExceptionFactory+ExtendedSocketException gets thrown (since there is no socket in the example docker).
Expected behavior
I except that the
TracesTransportTypewill be set from DD_TRACE_AGENT_URL and the tracer uses the UDS transfer.So that it's possible to use UDS for trace and HTTP for Dogstatsd.