Extract ContainsSpanId to SpanCollection#8435
Merged
andrewlock merged 2 commits intoApr 20, 2026
Merged
Conversation
andrewlock
added
area:tracer
andrewlock
force-pushed
the
andrew/client-side-stats/refactor-trace-chunk-model
branch
from
April 9, 2026 22:41
d367d99 to
efd496f
Compare
andrewlock
force-pushed
the
andrew/client-side-stats/client-side-stats-1-2
branch
from
April 9, 2026 22:41
46c887a to
d2adf6e
Compare
BenchmarksBenchmark execution time: 2026-04-20 17:21:28 Comparing candidate commit ce99c4b in PR branch Found 1 performance improvements and 0 performance regressions! Performance is the same for 26 metrics, 0 unstable metrics, 87 known flaky benchmarks.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8435) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (71ms) : 69, 74
master - mean (72ms) : 69, 75
section Bailout
This PR (8435) - mean (76ms) : 73, 78
master - mean (76ms) : 74, 78
section CallTarget+Inlining+NGEN
This PR (8435) - mean (1,067ms) : 1019, 1116
master - mean (1,071ms) : 1008, 1134
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (112ms) : 109, 116
master - mean (112ms) : 109, 115
section Bailout
This PR (8435) - mean (113ms) : 110, 116
master - mean (114ms) : 112, 117
section CallTarget+Inlining+NGEN
This PR (8435) - mean (780ms) : 760, 801
master - mean (776ms) : 756, 796
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (99ms) : 96, 103
master - mean (99ms) : 95, 103
section Bailout
This PR (8435) - mean (99ms) : 96, 102
master - mean (99ms) : 96, 101
section CallTarget+Inlining+NGEN
This PR (8435) - mean (934ms) : 895, 972
master - mean (927ms) : 893, 961
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (98ms) : 94, 103
master - mean (97ms) : 94, 100
section Bailout
This PR (8435) - mean (99ms) : 96, 102
master - mean (99ms) : 96, 101
section CallTarget+Inlining+NGEN
This PR (8435) - mean (817ms) : 789, 845
master - mean (811ms) : 783, 840
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (194ms) : 189, 198
master - mean (194ms) : 191, 197
section Bailout
This PR (8435) - mean (197ms) : 194, 199
master - mean (198ms) : 195, 200
section CallTarget+Inlining+NGEN
This PR (8435) - mean (1,143ms) : 1104, 1182
master - mean (1,155ms) : 1094, 1216
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (277ms) : 272, 282
master - mean (279ms) : 275, 283
section Bailout
This PR (8435) - mean (278ms) : 274, 283
master - mean (279ms) : 276, 282
section CallTarget+Inlining+NGEN
This PR (8435) - mean (933ms) : 910, 955
master - mean (927ms) : 905, 949
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (270ms) : 266, 275
master - mean (271ms) : 267, 275
section Bailout
This PR (8435) - mean (271ms) : 267, 274
master - mean (271ms) : 268, 273
section CallTarget+Inlining+NGEN
This PR (8435) - mean (1,141ms) : 1107, 1176
master - mean (1,142ms) : 1097, 1186
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8435) - mean (269ms) : 265, 272
master - mean (269ms) : 266, 273
section Bailout
This PR (8435) - mean (269ms) : 266, 272
master - mean (269ms) : 266, 273
section CallTarget+Inlining+NGEN
This PR (8435) - mean (1,026ms) : 992, 1059
master - mean (1,024ms) : 982, 1066
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
andrewlock
marked this pull request as ready for review
April 10, 2026 08:20
andrewlock
force-pushed
the
andrew/client-side-stats/refactor-trace-chunk-model
branch
from
April 10, 2026 12:29
efd496f to
bda5853
Compare
andrewlock
force-pushed
the
andrew/client-side-stats/client-side-stats-1-2
branch
from
April 10, 2026 12:29
d2adf6e to
f8a1a26
Compare
This was referenced Apr 13, 2026
lucaspimentel
approved these changes
Apr 13, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors span lookup logic by extracting ContainsSpanId from TraceChunkModel into SpanCollection, enabling efficient span-id searching directly on the collection (for upcoming client-side-stats work) and adding unit tests for the new API.
Changes:
- Added
SpanCollection.ContainsSpanId(ulong spanId, int startIndex)implementing wrap-around search semantics. - Updated
TraceChunkModelto useSpanCollection.ContainsSpanIdinstead of its privateIndexOfhelper. - Added unit tests covering
ContainsSpanIdbehavior across default, single-span, and multi-span cases.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tracer/src/Datadog.Trace/Agent/SpanCollection.cs | Introduces ContainsSpanId on SpanCollection to centralize span-id lookup logic. |
| tracer/src/Datadog.Trace/Agent/MessagePack/TraceChunkModel.cs | Switches local-root/span-id lookup to the new SpanCollection.ContainsSpanId API and removes the old IndexOf implementation. |
| tracer/test/Datadog.Trace.Tests/Agent/SpanCollectionTests.cs | Adds unit tests validating ContainsSpanId behavior and updates CreateSpan to allow custom span IDs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
andrewlock
force-pushed
the
andrew/client-side-stats/refactor-trace-chunk-model
branch
from
April 14, 2026 10:08
bda5853 to
682df90
Compare
andrewlock
force-pushed
the
andrew/client-side-stats/client-side-stats-1-2
branch
from
April 14, 2026 10:08
f8a1a26 to
5573211
Compare
andrewlock
force-pushed
the
andrew/client-side-stats/refactor-trace-chunk-model
branch
from
April 16, 2026 07:59
682df90 to
bcf6c7a
Compare
andrewlock
force-pushed
the
andrew/client-side-stats/client-side-stats-1-2
branch
from
April 16, 2026 07:59
5573211 to
e9ee48d
Compare
Base automatically changed from
andrew/client-side-stats/client-side-stats-1-2
to
master
April 20, 2026 16:11
andrewlock
force-pushed
the
andrew/client-side-stats/refactor-trace-chunk-model
branch
from
April 20, 2026 16:12
bcf6c7a to
ce99c4b
Compare
andrewlock
enabled auto-merge (squash)
April 20, 2026 16:13
andrewlock
deleted the
andrew/client-side-stats/refactor-trace-chunk-model
branch
April 20, 2026 20:14
andrewlock
added a commit
that referenced
this pull request
Apr 21, 2026
## Summary of changes Updates the processing pipeline for client side stats ## Reason for change The bulk of the CSS 1.2.0 work was done in #8420, but the span-processing pipeline wasn't the same as the Go agent: | Step | Go Agent | .NET Tracer | | ---- | --------- | -------------------------------------- | | 1 | Normalize | Normalize + Obfuscate (`ProcessTrace`) | | 2 | Filter | Filter + Sample (`ShouldKeepTrace`) | | 3 | Obfuscate | | | 4 | Sample | | | 5 | Stats | Stats (`AddRange`) | ## Implementation details The difference was mostly due to the way we had separate `ProcessTrace()` and `ShouldKeepTrace()` methods. By merging them, we can encapsulate the pipeline, and ensure that we mirror the way the Go agent processes traces. Additionally, there are two different mechanisms for dropping spans now - p0drops and "trace filtering", so added a new metric tag `reason:trace_filter` for it. Additionally, made some small optimizations: - No need for an array of `ITraceProcessor` when it only contains one - Updated `TraceFilter` algorithm to correctly look for local trace root, starting from last span - Made `_statsAggregator` non-null in `AgentWriter` to eliminate a bunch of null checks ## Test coverage Covered by existing tests. ## Other details Part of a stack: - #8417 - #8418 - #8420 - #8435
andrewlock
added a commit
that referenced
this pull request
Apr 21, 2026
## Summary of changes Improves the performance of peer-tag hash calculation for "fast path" cases (where there's an existing bucket) ## Reason for change We have to calculate the hash of all peer tags as part of client-side-stats bucketing calculations, which involves encoding them as utf-8. Additionally, when we _send_ the buckets, we have to send the tags as utf-8. In the initial CSS 1.2.0 implementation, we encoded the tags every-time we ran a calculation, which would allocate for every span that had peer tags, generally quite expensive. In this PR, we switch to doing the encoding twice: once with a zero-allocation implementation (amortized 0 on .NET Framework) to calculate the hash, and then, _if_ we need the "real" encoded tags, then we do that encoding again. ## Implementation details - Split the peer tags work in two, once to calculate the peer tags hash, once to get the actual tags as `key:value` - In the hash-calculation stage, we can use `stackalloc` for .NET Core, and array pool implementation for .NET Framework etc - As additional optimizations, we also - Pre-encode the peer tag _keys_ to utf-8, so that we only need to do those once for the hash calculation. - Pass key details from `BuildKey` to `GetEncodedPeerTags` (i.e. is this a "base service" only tag, if so, what's the tag value, otherwise how big does the peer tag list need to be) - If client-side stats is disabled, don't bother doing all the pre-calculation (or for trace filters) There are some possible future optimizations, _not_ implemented in this PR: - Track which tags require IP quantization to avoid re-doing it. (Maybe we should allow-list the quantization anyway, so that it only applies to specific tags? Or alternatively, block list?) - Instead of doing the encoding of `tag:value`, allow writing the pre-computed `byte[]` to `MessagePackBinary` and appending the `value`. This is doable, but requires updating the `MessagePackBinary` implementation to support it, so I considered it out of scope for now ## Test coverage Mostly covered by existing tests, but added some additional unit tests that directly compare the hashing to values used in Go agent tests. Additionally did some benchmarking. The key thing is that the `ClientSpanWithPeerTags` path is zero-allocation (and it's nice that the slow-path is still lower allocation than before, even if it's slower over all) | Method | Runtime | Mean | Error | Allocated | | -------------------------------------------------- | ------------------ | ----------: | ---------: | --------: | | BuildKey_SimpleSpan_Before | .NET 10.0 | 44.34 ns | 0.906 ns | - | | BuildKey_SimpleSpan_After | .NET 10.0 | 39.17 ns | 0.445 ns | - | | BuildKey_SimpleSpan_Before | .NET 6.0 | 93.13 ns | 1.837 ns | - | | BuildKey_SimpleSpan_After | .NET 6.0 | 87.56 ns | 0.839 ns | - | | BuildKey_SimpleSpan_Before | .NET Core 3.1 | 192.53 ns | 2.572 ns | - | | BuildKey_SimpleSpan_After | .NET Core 3.1 | 217.32 ns | 4.362 ns | - | | BuildKey_SimpleSpan_Before | .NET Framework 4.8 | 131.98 ns | 2.621 ns | - | | BuildKey_SimpleSpan_After | .NET Framework 4.8 | 175.20 ns | 7.647 ns | - | | | | | | | | BuildKey_ClientSpanNoPeerTags_Before | .NET 10.0 | 160.62 ns | 2.622 ns | - | | BuildKey_ClientSpanNoPeerTags_After | .NET 10.0 | 151.27 ns | 1.995 ns | - | | BuildKey_ClientSpanNoPeerTags_Before | .NET 6.0 | 243.62 ns | 3.199 ns | - | | BuildKey_ClientSpanNoPeerTags_After | .NET 6.0 | 232.55 ns | 2.975 ns | - | | BuildKey_ClientSpanNoPeerTags_Before | .NET Core 3.1 | 550.47 ns | 9.998 ns | - | | BuildKey_ClientSpanNoPeerTags_After | .NET Core 3.1 | 680.42 ns | 10.022 ns | - | | BuildKey_ClientSpanNoPeerTags_Before | .NET Framework 4.8 | 442.88 ns | 8.854 ns | - | | BuildKey_ClientSpanNoPeerTags_After | .NET Framework 4.8 | 682.12 ns | 13.468 ns | - | | | | | | | | BuildKey_ClientSpanWithPeerTags_Before | .NET 10.0 | 800.82 ns | 15.062 ns | 840 B | | BuildKey_ClientSpanWithPeerTags_After | .NET 10.0 | 575.04 ns | 5.604 ns | - | | BuildKey_ClientSpanWithPeerTags_Before | .NET 6.0 | 1,076.07 ns | 13.124 ns | 840 B | | BuildKey_ClientSpanWithPeerTags_After | .NET 6.0 | 1,440.26 ns | 28.755 ns | - | | BuildKey_ClientSpanWithPeerTags_Before | .NET Core 3.1 | 1,531.27 ns | 13.509 ns | 840 B | | BuildKey_ClientSpanWithPeerTags_After | .NET Core 3.1 | 1,774.35 ns | 89.601 ns | - | | BuildKey_ClientSpanWithPeerTags_Before | .NET Framework 4.8 | 1,682.15 ns | 17.082 ns | 859 B | | BuildKey_ClientSpanWithPeerTags_After | .NET Framework 4.8 | 2,040.90 ns | 162.895 ns | - | | | | | | | | BuildKey_ClientSpanWithPeerTags_GetEncodedPeerTags | .NET 10.0 | 1,141.85 ns | 9.980 ns | 760 B | | BuildKey_ClientSpanWithPeerTags_GetEncodedPeerTags | .NET 6.0 | 2,720.55 ns | 53.097 ns | 760 B | | BuildKey_ClientSpanWithPeerTags_GetEncodedPeerTags | .NET Core 3.1 | 3,534.28 ns | 141.234 ns | 760 B | | BuildKey_ClientSpanWithPeerTags_GetEncodedPeerTags | .NET Framework 4.8 | 3,438.80 ns | 84.352 ns | 778 B | <details><summary>Details</summary> <p> ```csharp // <copyright file="StatsAggregatorBenchmark.cs" company="Datadog"> // Unless explicitly stated otherwise all files in this repository are licensed under the Apache 2 License. // This product includes software developed at Datadog (https://www.datadoghq.com/). Copyright 2017 Datadog, Inc. // </copyright> using System; using System.Collections.Generic; using System.Threading.Tasks; using BenchmarkDotNet.Attributes; using Datadog.Trace; using Datadog.Trace.Agent; using Datadog.Trace.Agent.DiscoveryService; using Datadog.Trace.Configuration; using Datadog.Trace.Tagging; namespace Benchmarks.Trace; /// <summary> /// StatsAggregator.BuildKey benchmarks /// </summary> [MemoryDiagnoser] [BenchmarkCategory(Constants.TracerCategory)] public class StatsAggregatorBenchmark { private static readonly List<StatsAggregator.PeerTagKey> PeerTagKeys = [ new("_dd.base_service"), new("aws.queue.name"), new("aws.queue.url"), new("aws.s3.bucket"), new("aws.stream.name"), new("bucketname"), new("db.couchbase.seed.nodes"), new("db.hostname"), new("db.instance"), new("db.system"), new("messaging.destination"), new("messaging.kafka.bootstrap.servers"), new("messaging.rabbitmq.exchange"), new("messaging.system"), new("network.destination.name"), new("peer.hostname"), new("peer.service"), new("server.address"), new("topic"), ]; private StatsAggregator _aggregator; private Span _simpleSpan; private Span _clientSpanNoPeerTags; private Span _clientSpanWithPeerTags; private StatsAggregationKey _key; private List<byte[]> _encoded; [GlobalSetup] public void GlobalSetup() { _aggregator = new StatsAggregator( new NoOpApi(), new TracerSettings(), NullDiscoveryService.Instance, isOtlp: false); var now = DateTimeOffset.UtcNow; // Simple span: no span kind, no peer tags — exercises the "internal" fast path _simpleSpan = CreateSpan(now, "web-service", "web.request", "GET /api/users", "web"); // Client span with SpanKind but no matching peer tags — iterates peer tag keys but finds nothing var clientTags = new HttpTags { HttpMethod = "GET", HttpStatusCode = "200" }; _clientSpanNoPeerTags = CreateSpan(now, "web-service", "http.request", "GET /api/orders", "http", clientTags); // Client span with several peer tags set — the most expensive path var peerTags = new HttpTags { HttpMethod = "POST", HttpStatusCode = "200" }; _clientSpanWithPeerTags = CreateSpan(now, "web-service", "http.request", "POST /api/data", "http", peerTags); _clientSpanWithPeerTags.Tags.SetTag("peer.service", "remote-service"); _clientSpanWithPeerTags.Tags.SetTag("db.instance", "i-1234"); _clientSpanWithPeerTags.Tags.SetTag("db.system", "postgres"); _clientSpanWithPeerTags.Tags.SetTag("server.address", "db.example.com"); _clientSpanWithPeerTags.Tags.SetTag("network.destination.name", "db.example.com"); } [GlobalCleanup] public void GlobalCleanup() { _aggregator.DisposeAsync().GetAwaiter().GetResult(); var value = _key; var encoded = _encoded; } /// <summary> /// BuildKey for a simple span with no span kind (internal fast-path, no peer tag iteration). /// </summary> [Benchmark] public void BuildKey_SimpleSpan() { _key = _aggregator.BuildKey(_simpleSpan, PeerTagKeys, out _); } /// <summary> /// BuildKey for a client span that has no matching peer tags (iterates all peer tag keys, finds none). /// </summary> [Benchmark] public void BuildKey_ClientSpanNoPeerTags() { _key = _aggregator.BuildKey(_clientSpanNoPeerTags, PeerTagKeys, out _); } /// <summary> /// BuildKey for a client span with several peer tags set (UTF-8 encoding + FNV hashing). /// </summary> [Benchmark] public void BuildKey_ClientSpanWithPeerTags() { _key = _aggregator.BuildKey(_clientSpanWithPeerTags, PeerTagKeys, out var results); } /// <summary> /// BuildKey for a client span with several peer tags set (UTF-8 encoding + FNV hashing). /// </summary> [Benchmark] public void BuildKey_ClientSpanWithPeerTags_GetEncodedPeerTags() { _key = _aggregator.BuildKey(_clientSpanWithPeerTags, PeerTagKeys, out var results); _encoded = StatsAggregator.GetEncodedPeerTags(_clientSpanWithPeerTags, PeerTagKeys, in results); } private static Span CreateSpan(DateTimeOffset start, string serviceName, string operationName, string resourceName, string type, ITags tags = null) { var tracer = Benchmarks.Trace.Asm.EmptyDatadogTracer.Instance; var traceContext = new TraceContext(tracer); var context = new SpanContext(null, traceContext, serviceName); var span = new Span(context, start, tags); span.OperationName = operationName; span.ResourceName = resourceName; span.Type = type; return span; } private sealed class NoOpApi : IApi { public TracesEncoding TracesEncoding => TracesEncoding.DatadogV0_4; public Task<bool> Ping() => Task.FromResult(true); public Task<bool> SendTracesAsync(ArraySegment<byte> traces, int numberOfTraces, bool statsComputationEnabled, long numberOfDroppedP0Traces, long numberOfDroppedP0Spans, bool apmTracingEnabled = true) => Task.FromResult(true); public Task<bool> SendStatsAsync(StatsBuffer stats, long bucketDuration, int tracerObfuscationVersion) => Task.FromResult(true); } } ``` </p> </details> ## Other details Part of a stack: - #8417 - #8418 - #8420 - #8435 - #8436 - #8444
andrewlock
added a commit
that referenced
this pull request
Apr 23, 2026
## Summary of changes Removes the minimum-agent version check from the client-side-stats check ## Reason for change We didn't have it before, which means people who previously had stats, suddenly won't have them. Also other agents (like bottlecap etc) don't have the version. And we haven't figured out how to handle that yet. While CSS is off by default, this seems like the best option to take. ## Implementation details Remove the code that is checking for CSS support. ## Test coverage Covered by existing (and removed a test for the behaviour) ## Other details Follows on from - #8417 - #8418 - #8420 - #8435 - #8436 - #8444 - #8445
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Extracts
ContainsSpanIdfromTraceChunkModeltoSpanCollectionReason for change
I need to find a given span in a
SpanCollectionfor client-side-stats. The algorithm used byTraceChunkModeldoes it "properly" (by stating from the last span when looking for a root span), so it seems to make sense to move this method toSpanCollection. Additionally, this essentially hides some of the internals about theSpanCollectiontype, so seems like a better place for it anyway.Implementation details
IndexOf()method fromTraceChunkModeltoSpanCollectionContains()instead, seeing as the index is never actually used AFAICT. It's easy to switch it back later if we do need it.Test coverage
Added some unit tests for the method
Other details
Part of a stack
charhelpers likeIsAsciiHexDigit#8417IpAddressObfuscationUtilfor use with client-side-stats #8418