[DSM] Initial transaction tracking implementation by kr-igor · Pull Request #7949 · DataDog/dd-trace-dotnet

kr-igor · 2025-12-12T21:50:07Z

What Does This Do

Initial transaction tracking implementation.

Support for extractors configuration
Serialization for extractors
Extractors for http / kafka
Updated / added unit tests

Motivation

Multiple customers mentioned the need to track individual message across multiple service and environments and log messages which haven't reached the end of the pre-defined pipeline.
See this document for details.
Payload format description is here.

Additional Notes

Transaction tracking is currently in closed beta, so the configuration flags will be documented after public availability.

Technical details

Adds DSM transaction tracking by introducing a DataStreamsTransactionContainer that accumulates serialized transaction records (checkpoint ID, nanosecond timestamp, and UTF-8 transaction ID bytes) within the existing
DataStreamsAggregator flush cycle, triggering an early flush when the buffer exceeds 512 KB.
Transaction extraction is driven by a configurable DataStreamsExtractorRegistry (loaded from DD_DATA_STREAMS_TRANSACTION_EXTRACTORS
JSON) that maps named extractors to header keys for four integration types:

HTTP_IN_HEADERS (ASP.NET Core incoming requests)
HTTP_OUT_HEADERS (HttpClient outgoing requests)
KAFKA_CONSUME_HEADERS, and KAFKA_PRODUCE_HEADERS.

dd-trace-dotnet-ci-bot · 2025-12-13T00:17:13Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7949) and master.

⚠️ Potential regressions detected

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	71.12 ± (71.14 - 71.48) ms	77.50 ± (77.46 - 77.82) ms	+9.0%	❌⬆️
.NET Framework 4.8 - Bailout
duration	75.81 ± (75.66 - 76.02) ms	81.86 ± (81.65 - 82.08) ms	+8.0%	❌⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1060.34 ± (1060.56 - 1066.04) ms	1132.00 ± (1132.04 - 1138.14) ms	+6.8%	❌⬆️

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Bailout
duration	194.44 ± (194.43 - 194.70) ms	211.03 ± (211.12 - 212.11) ms	+8.5%	❌⬆️

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	71.12 ± (71.14 - 71.48) ms	77.50 ± (77.46 - 77.82) ms	+9.0%	❌⬆️
.NET Framework 4.8 - Bailout
duration	75.81 ± (75.66 - 76.02) ms	81.86 ± (81.65 - 82.08) ms	+8.0%	❌⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1060.34 ± (1060.56 - 1066.04) ms	1132.00 ± (1132.04 - 1138.14) ms	+6.8%	❌⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	22.06 ± (22.02 - 22.10) ms	24.03 ± (23.97 - 24.10) ms	+8.9%	✅⬆️
process.time_to_main_ms	83.08 ± (82.88 - 83.27) ms	92.58 ± (92.34 - 92.82) ms	+11.4%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.90 ± (10.90 - 10.91) MB	10.92 ± (10.92 - 10.92) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	22.09 ± (22.04 - 22.13) ms	23.77 ± (23.70 - 23.84) ms	+7.6%	✅⬆️
process.time_to_main_ms	84.51 ± (84.31 - 84.70) ms	93.91 ± (93.69 - 94.14) ms	+11.1%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.94 ± (10.93 - 10.94) MB	10.96 ± (10.95 - 10.96) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	223.97 ± (222.65 - 225.29) ms	242.06 ± (240.98 - 243.14) ms	+8.1%	✅⬆️
process.time_to_main_ms	515.23 ± (514.03 - 516.42) ms	562.92 ± (561.46 - 564.37) ms	+9.3%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	48.37 ± (48.34 - 48.40) MB	48.43 ± (48.40 - 48.46) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.0%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	21.24 ± (21.14 - 21.33) ms	22.45 ± (22.34 - 22.57) ms	+5.7%	✅⬆️
process.time_to_main_ms	73.11 ± (72.66 - 73.55) ms	79.92 ± (79.48 - 80.37) ms	+9.3%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.60 ± (10.59 - 10.60) MB	10.64 ± (10.64 - 10.64) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	20.95 ± (20.90 - 20.99) ms	22.09 ± (22.04 - 22.14) ms	+5.4%	✅⬆️
process.time_to_main_ms	72.57 ± (72.42 - 72.71) ms	80.09 ± (79.87 - 80.31) ms	+10.4%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.71 ± (10.70 - 10.71) MB	10.75 ± (10.75 - 10.76) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	389.45 ± (387.22 - 391.68) ms	383.62 ± (381.62 - 385.61) ms	-1.5%	✅
process.time_to_main_ms	516.46 ± (515.52 - 517.40) ms	554.99 ± (553.77 - 556.22) ms	+7.5%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	49.97 ± (49.94 - 50.00) MB	49.72 ± (49.68 - 49.75) MB	-0.5%	✅
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.4%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	19.07 ± (19.04 - 19.10) ms	20.52 ± (20.47 - 20.57) ms	+7.6%	✅⬆️
process.time_to_main_ms	70.93 ± (70.78 - 71.09) ms	78.06 ± (77.88 - 78.24) ms	+10.0%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.68 ± (7.67 - 7.68) MB	7.69 ± (7.68 - 7.70) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	19.18 ± (19.15 - 19.22) ms	20.55 ± (20.49 - 20.60) ms	+7.1%	✅⬆️
process.time_to_main_ms	72.54 ± (72.37 - 72.71) ms	79.74 ± (79.54 - 79.94) ms	+9.9%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.71 ± (7.70 - 7.71) MB	7.74 ± (7.73 - 7.74) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	305.63 ± (303.62 - 307.64) ms	306.78 ± (304.19 - 309.37) ms	+0.4%	✅⬆️
process.time_to_main_ms	477.25 ± (476.43 - 478.06) ms	520.24 ± (519.26 - 521.22) ms	+9.0%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	36.98 ± (36.95 - 37.00) MB	37.17 ± (37.14 - 37.20) MB	+0.5%	✅⬆️
runtime.dotnet.threads.count	27 ± (27 - 27)	27 ± (27 - 27)	-1.0%	✅

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	192.50 ± (197.62 - 201.07) ms	204.30 ± (201.26 - 203.45) ms	+6.1%	✅⬆️
.NET Framework 4.8 - Bailout
duration	194.44 ± (194.43 - 194.70) ms	211.03 ± (211.12 - 212.11) ms	+8.5%	❌⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1136.42 ± (1136.91 - 1143.73) ms	1207.55 ± (1200.15 - 1209.53) ms	+6.3%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	185.05 ± (184.78 - 185.32) ms	201.94 ± (201.47 - 202.40) ms	+9.1%	✅⬆️
process.time_to_main_ms	79.69 ± (79.53 - 79.85) ms	87.53 ± (87.24 - 87.83) ms	+9.8%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.24 ± (16.21 - 16.27) MB	16.06 ± (16.05 - 16.08) MB	-1.1%	✅
runtime.dotnet.threads.count	20 ± (19 - 20)	20 ± (20 - 20)	+0.1%	✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms	184.69 ± (184.51 - 184.88) ms	198.72 ± (197.79 - 199.66) ms	+7.6%	✅⬆️
process.time_to_main_ms	81.00 ± (80.90 - 81.10) ms	87.60 ± (87.11 - 88.08) ms	+8.1%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	15.98 ± (15.83 - 16.13) MB	16.08 ± (16.06 - 16.10) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	20 ± (20 - 20)	21 ± (21 - 21)	+4.6%	✅⬆️
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	391.85 ± (390.62 - 393.09) ms	410.06 ± (408.49 - 411.62) ms	+4.6%	✅⬆️
process.time_to_main_ms	509.06 ± (507.91 - 510.21) ms	538.48 ± (536.21 - 540.76) ms	+5.8%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	58.65 ± (58.44 - 58.86) MB	59.32 ± (59.28 - 59.36) MB	+1.1%	✅⬆️
runtime.dotnet.threads.count	30 ± (30 - 30)	30 ± (30 - 30)	+0.1%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	189.31 ± (189.08 - 189.54) ms	200.74 ± (199.91 - 201.58) ms	+6.0%	✅⬆️
process.time_to_main_ms	69.48 ± (69.34 - 69.62) ms	73.15 ± (72.82 - 73.48) ms	+5.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	15.80 ± (15.62 - 15.98) MB	16.29 ± (16.27 - 16.31) MB	+3.1%	✅⬆️
runtime.dotnet.threads.count	18 ± (18 - 18)	19 ± (19 - 19)	+7.7%	✅⬆️
.NET 6 - Bailout
process.internal_duration_ms	188.32 ± (188.12 - 188.53) ms	200.95 ± (200.21 - 201.68) ms	+6.7%	✅⬆️
process.time_to_main_ms	70.45 ± (70.38 - 70.52) ms	74.40 ± (74.10 - 74.70) ms	+5.6%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	15.49 ± (15.32 - 15.66) MB	16.35 ± (16.33 - 16.37) MB	+5.6%	✅⬆️
runtime.dotnet.threads.count	18 ± (18 - 19)	20 ± (20 - 20)	+9.6%	✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	595.27 ± (592.34 - 598.19) ms	598.66 ± (596.38 - 600.94) ms	+0.6%	✅⬆️
process.time_to_main_ms	504.66 ± (503.90 - 505.41) ms	532.17 ± (530.32 - 534.03) ms	+5.5%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	61.53 ± (61.44 - 61.62) MB	61.60 ± (61.51 - 61.69) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	30 ± (30 - 30)	30 ± (30 - 30)	+0.5%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	187.32 ± (187.14 - 187.51) ms	199.52 ± (198.81 - 200.23) ms	+6.5%	✅⬆️
process.time_to_main_ms	68.93 ± (68.81 - 69.06) ms	73.29 ± (73.00 - 73.57) ms	+6.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.71 ± (11.61 - 11.80) MB	11.66 ± (11.65 - 11.68) MB	-0.4%	✅
runtime.dotnet.threads.count	18 ± (17 - 18)	19 ± (18 - 19)	+5.7%	✅⬆️
.NET 8 - Bailout
process.internal_duration_ms	186.36 ± (186.22 - 186.51) ms	197.27 ± (196.64 - 197.90) ms	+5.9%	✅⬆️
process.time_to_main_ms	69.84 ± (69.76 - 69.92) ms	74.10 ± (73.84 - 74.36) ms	+6.1%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.75 ± (11.66 - 11.83) MB	11.71 ± (11.70 - 11.73) MB	-0.3%	✅
runtime.dotnet.threads.count	19 ± (18 - 19)	19 ± (19 - 20)	+5.2%	✅⬆️
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	519.95 ± (517.52 - 522.37) ms	514.20 ± (510.84 - 517.57) ms	-1.1%	✅
process.time_to_main_ms	463.30 ± (462.67 - 463.93) ms	496.18 ± (494.64 - 497.72) ms	+7.1%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	50.69 ± (50.66 - 50.71) MB	50.62 ± (50.58 - 50.66) MB	-0.1%	✅
runtime.dotnet.threads.count	30 ± (30 - 30)	30 ± (30 - 30)	+0.2%	✅⬆️

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (78ms)  : 75, 80
    master - mean (71ms)  : 69, 74

    section Bailout
    This PR (7949) - mean (82ms)  : crit, 79, 85
    master - mean (76ms)  : 73, 78

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (1,135ms)  : crit, 1090, 1180
    master - mean (1,063ms)  : 1023, 1104

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (124ms)  : 120, 128
    master - mean (112ms)  : 108, 116

    section Bailout
    This PR (7949) - mean (125ms)  : crit, 122, 129
    master - mean (113ms)  : 109, 117

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (844ms)  : crit, 813, 874
    master - mean (776ms)  : 754, 798

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (110ms)  : 97, 122
    master - mean (101ms)  : 93, 109

    section Bailout
    This PR (7949) - mean (109ms)  : crit, 106, 113
    master - mean (100ms)  : 97, 102

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (966ms)  : 937, 994
    master - mean (933ms)  : 897, 969

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (107ms)  : 103, 111
    master - mean (98ms)  : 95, 100

    section Bailout
    This PR (7949) - mean (108ms)  : crit, 106, 111
    master - mean (99ms)  : 96, 102

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (858ms)  : 819, 897
    master - mean (813ms)  : 777, 850

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (202ms)  : 185, 219
    master - mean (199ms)  : 174, 225

    section Bailout
    This PR (7949) - mean (212ms)  : crit, 205, 218
    master - mean (195ms)  : 193, 196

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (1,205ms)  : 1137, 1273
    master - mean (1,140ms)  : 1092, 1189

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (299ms)  : 287, 311
    master - mean (273ms)  : 269, 277

    section Bailout
    This PR (7949) - mean (295ms)  : crit, 273, 317
    master - mean (274ms)  : 271, 277

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (984ms)  : crit, 929, 1039
    master - mean (928ms)  : 907, 949

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (282ms)  : 263, 301
    master - mean (267ms)  : 263, 271

    section Bailout
    This PR (7949) - mean (284ms)  : crit, 267, 300
    master - mean (267ms)  : 264, 269

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (1,162ms)  : 1113, 1212
    master - mean (1,131ms)  : 1088, 1174

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7949) - mean (283ms)  : 266, 299
    master - mean (266ms)  : 262, 269

    section Bailout
    This PR (7949) - mean (282ms)  : crit, 269, 295
    master - mean (265ms)  : 263, 267

    section CallTarget+Inlining+NGEN
    This PR (7949) - mean (1,046ms)  : 984, 1109
    master - mean (1,015ms)  : 980, 1049

pr-commenter · 2025-12-13T00:34:06Z

Benchmarks

Benchmark execution time: 2026-04-09 18:28:40

Comparing candidate commit 340c71a in PR branch kr-igor/dsm-tt-v0 with baseline commit 549e87e in branch master.

Found 32 performance improvements and 37 performance regressions! Performance is the same for 211 metrics, 8 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild net6.0

🟩 throughput [+9464.282op/s; +11626.503op/s] or [+7.955%; +9.773%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net472

🟥 execution_time [+308.476ms; +310.467ms] or [+153.077%; +154.065%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

🟥 execution_time [+384.475ms; +387.040ms] or [+303.759%; +305.785%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

🟥 execution_time [+399.368ms; +402.235ms] or [+353.425%; +355.962%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody net6.0

🟩 execution_time [-16.108ms; -11.939ms] or [-7.523%; -5.576%]
🟩 throughput [+9363.931op/s; +12156.151op/s] or [+6.835%; +8.873%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net6.0

🟩 execution_time [-25.875ms; -20.996ms] or [-11.539%; -9.363%]
🟩 throughput [+80243.893op/s; +103371.397op/s] or [+8.573%; +11.043%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1

🟩 throughput [+75449.716op/s; +91968.393op/s] or [+10.841%; +13.214%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody net6.0

🟩 throughput [+12703.729op/s; +15672.936op/s] or [+8.083%; +9.972%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody netcoreapp3.1

🟩 throughput [+7796.724op/s; +10436.351op/s] or [+6.211%; +8.314%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net6.0

🟩 throughput [+454869.877op/s; +478398.967op/s] or [+15.167%; +15.952%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody netcoreapp3.1

🟩 execution_time [-19.035ms; -14.672ms] or [-8.775%; -6.763%]
🟩 throughput [+218994.155op/s; +272563.732op/s] or [+8.692%; +10.819%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net472

🟥 execution_time [+300.006ms; +300.846ms] or [+149.903%; +150.323%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net6.0

🟥 execution_time [+299.053ms; +304.752ms] or [+150.813%; +153.687%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs netcoreapp3.1

🟥 execution_time [+300.296ms; +303.093ms] or [+151.266%; +152.675%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net472

🟥 execution_time [+297.252ms; +298.130ms] or [+145.998%; +146.429%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net6.0

🟥 execution_time [+294.800ms; +296.693ms] or [+144.117%; +145.042%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs netcoreapp3.1

🟥 execution_time [+301.042ms; +302.469ms] or [+150.460%; +151.174%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack net6.0

🟥 execution_time [+21.825µs; +45.422µs] or [+6.968%; +14.501%]
🟥 throughput [-423.584op/s; -225.000op/s] or [-13.204%; -7.014%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net472

🟥 execution_time [+299.836ms; +300.479ms] or [+149.649%; +149.970%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net6.0

🟩 throughput [+1043.310op/s; +1171.024op/s] or [+8.573%; +9.622%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest netcoreapp3.1

🟩 throughput [+653.205op/s; +853.859op/s] or [+6.323%; +8.266%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net472

🟥 throughput [-607.310op/s; -541.650op/s] or [-55.028%; -49.079%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

🟥 throughput [-748.268op/s; -664.810op/s] or [-49.910%; -44.343%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

🟥 execution_time [+338.858ms; +346.807ms] or [+202.676%; +207.431%]
🟥 throughput [-427.425op/s; -392.361op/s] or [-29.761%; -27.320%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net472

🟥 execution_time [+301.487ms; +302.924ms] or [+151.823%; +152.547%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net6.0

🟥 execution_time [+302.960ms; +304.494ms] or [+151.814%; +152.582%]
🟩 throughput [+43663.370op/s; +48040.369op/s] or [+6.884%; +7.574%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch netcoreapp3.1

🟥 execution_time [+300.626ms; +303.888ms] or [+151.022%; +152.660%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net472

🟥 execution_time [+300.775ms; +302.390ms] or [+151.039%; +151.850%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net6.0

🟥 execution_time [+299.736ms; +302.077ms] or [+148.206%; +149.363%]
🟩 throughput [+36994.129op/s; +42444.331op/s] or [+5.960%; +6.839%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync netcoreapp3.1

🟥 execution_time [+302.647ms; +306.006ms] or [+153.395%; +155.097%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net472

🟥 execution_time [+300.954ms; +302.731ms] or [+151.052%; +151.944%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net6.0

🟥 execution_time [+299.055ms; +301.281ms] or [+149.052%; +150.161%]
🟩 throughput [+65439.098op/s; +69517.791op/s] or [+12.994%; +13.804%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync netcoreapp3.1

🟥 execution_time [+299.648ms; +302.343ms] or [+149.072%; +150.413%]

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net472

🟥 throughput [-5989.961op/s; -5645.946op/s] or [-6.838%; -6.445%]

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net6.0

🟩 throughput [+9034.275op/s; +10075.292op/s] or [+6.137%; +6.844%]

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog net6.0

🟩 execution_time [-15.798ms; -12.113ms] or [-7.346%; -5.633%]
🟩 throughput [+25467.497op/s; +31982.581op/s] or [+6.986%; +8.774%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark net6.0

🟩 allocated_mem [-19.095KB; -19.074KB] or [-6.965%; -6.958%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark net6.0

🟥 execution_time [+5.194µs; +9.173µs] or [+12.276%; +21.682%]
🟥 throughput [-4248.705op/s; -2491.937op/s] or [-17.886%; -10.490%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark netcoreapp3.1

🟩 throughput [+2052.187op/s; +3530.032op/s] or [+12.591%; +21.658%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net472

🟥 execution_time [+300.424ms; +302.056ms] or [+151.851%; +152.676%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net6.0

🟥 execution_time [+303.325ms; +305.430ms] or [+154.391%; +155.463%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1

🟥 execution_time [+297.311ms; +299.863ms] or [+148.841%; +150.118%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net6.0

🟩 throughput [+50166.903op/s; +53124.059op/s] or [+9.496%; +10.055%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net472

🟥 execution_time [+298.309ms; +300.001ms] or [+148.680%; +149.523%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net6.0

🟥 execution_time [+302.554ms; +303.886ms] or [+151.928%; +152.597%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog netcoreapp3.1

🟥 execution_time [+303.686ms; +306.281ms] or [+154.010%; +155.326%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net472

🟥 execution_time [+300.098ms; +300.803ms] or [+149.691%; +150.042%]
🟩 throughput [+65334497.414op/s; +65574495.734op/s] or [+47.581%; +47.755%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net6.0

🟥 execution_time [+425.112ms; +429.247ms] or [+528.702%; +533.845%]
🟩 throughput [+948.494op/s; +1119.371op/s] or [+7.332%; +8.653%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1

🟥 execution_time [+299.316ms; +300.257ms] or [+149.292%; +149.761%]
🟩 throughput [+18482022.049op/s; +19428458.190op/s] or [+8.186%; +8.606%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope net6.0

🟩 throughput [+95244.192op/s; +104427.046op/s] or [+8.893%; +9.750%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope netcoreapp3.1

🟩 throughput [+48007.065op/s; +67409.877op/s] or [+5.557%; +7.803%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net6.0

🟩 throughput [+79679.189op/s; +109695.109op/s] or [+6.167%; +8.491%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan netcoreapp3.1

🟩 throughput [+89854.046op/s; +99653.068op/s] or [+8.924%; +9.897%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net6.0

🟩 throughput [+47836.351op/s; +53882.848op/s] or [+8.686%; +9.784%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes netcoreapp3.1

🟩 throughput [+23096.747op/s; +32655.359op/s] or [+5.170%; +7.309%]

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net6.0

🟩 throughput [+74404.702op/s; +91421.363op/s] or [+8.313%; +10.214%]

robcarlan-datadog · 2026-03-26T17:17:53Z

@codex review

chatgpt-codex-connector · 2026-03-26T17:18:00Z

To use Codex here, create a Codex account and connect to github.

robcarlan-datadog · 2026-03-26T17:41:31Z

+        IsInDefaultState = tracerSettings.IsDataStreamsMonitoringInDefaultState;
+        _registry = new DataStreamsExtractorRegistry(tracerSettings.DataStreamsTransactionExtractors);

+        Log.Debug(@"Data Streams extractors loaded: {AsJson}", _registry.AsJson());


Is the @ needed?

robcarlan-datadog · 2026-03-26T17:46:30Z

+    {
+        var registry = new DataStreamsExtractorRegistry("[{\"name\": \"transaction-origin\", \"type\": \"HTTP_OUT_HEADERS\", \"value\": \"transaction-id\"}]");
+        registry.AsJson().Should().Be("{\"HttpOutHeaders\":[{\"name\":\"transaction-origin\",\"type\":\"HTTP_OUT_HEADERS\",\"value\":\"transaction-id\",\"ExtractorType\":1}]}");
+    }


[nit] Could add a test for GetExtractorsByType that it can correctly store multiple values for the same type

robcarlan-datadog · 2026-03-26T17:48:01Z

+        var span = new Span(new SpanContext(traceId: 123, spanId: 456), DateTimeOffset.UtcNow);
+
+        var act = () => span.TrackTransaction(dsm, "tx-abc", "some-checkpoint");
+        act.Should().NotThrow();


Probably worth checking that writer.DataStreamsTransactions.Size() remains 0 here?

lucaspimentel · 2026-03-30T17:32:11Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b5235f8902

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-30T17:39:17Z

+            return;
+        }
+
+        var deserialized = JsonHelper.DeserializeObject<List<DataStreamsTransactionExtractor>>(extractorsJson);


Handle invalid extractor JSON without throwing

DD_DATA_STREAMS_TRANSACTION_EXTRACTORS is user-provided text, but this constructor deserializes it without a guard. JsonHelper.DeserializeObject throws on malformed JSON, and this path runs during DataStreamsManager initialization, so a bad env var can abort tracer startup instead of being ignored as invalid config.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-30T17:39:17Z

+    {
+        _idBytes = Encoding.UTF8.GetBytes(id);
+        _timestamp = timestamp;
+        _checkpointId = Cache.GetOrAdd(checkpoint, Interlocked.Increment(ref _counter));


Allocate checkpoint IDs only on first key insertion

This GetOrAdd call eagerly increments _counter even when checkpoint already exists, so the counter grows on every transaction rather than on new checkpoint names. Because checkpoint IDs are serialized to a single byte, a delayed new checkpoint can quickly wrap/collide with existing IDs, making TransactionCheckpointIds ambiguous and corrupting transaction decoding.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-30T17:39:17Z

+        buffer[offset + 8] = (byte)_timestamp;
+
+        // id size, up to 256 bytes
+        buffer[offset + 9] = (byte)_idBytes.Length;


Validate transaction ID length before encoding

The wire format stores transaction ID length in one byte, but this code writes _idBytes.Length modulo 256 and still copies the full byte array. If a header-derived transaction ID is longer than 255 bytes, the encoded length no longer matches the payload, so downstream parsing becomes misaligned for this and subsequent records.

Useful? React with 👍 / 👎.

I think this is still applicable, i.e. as a question of correctness? 🤔

True, fixed now.

chatgpt-codex-connector · 2026-03-30T17:39:17Z

+            if (_transactionBuffer.TryEnqueue(transaction))
+            {


Drain transaction queue in FlushAsync

Transactions are queued in _transactionBuffer, but FlushAsync only drains _buffer and _backlogBuffer. That means explicit flushes (and shutdown flushes when the processing loop is stopping or misses the semaphore window) can complete while queued transactions remain unsent, causing silent transaction loss.

Useful? React with 👍 / 👎.

I think this needs to be addressed

kr-igor · 2026-03-31T17:26:04Z

All comments are now addressed.

bouwkast

I think we need integration tests for this

DataStreamsMonitoringKafkaTests.cs

And it looks like since we want this added to HttpClient I think we'd get some tests there, I don't see any DSM stuff in it, unsure if we'd need a new sample or not

kr-igor · 2026-03-31T21:45:23Z

I think we need integration tests for this

DataStreamsMonitoringKafkaTests.cs

And it looks like since we want this added to HttpClient I think we'd get some tests there, I don't see any DSM stuff in it, unsure if we'd need a new sample or not

Added tests for http and kafka

HttpClient requires an explicit assembly reference on .NET Framework. Mirrors the same pattern used in Samples.HttpMessageHandler.

bouwkast

Still in progress, mainly have the test code to go through which I will pick up first thing tomorrow

bouwkast · 2026-04-01T21:03:57Z

+        IsInDefaultState = tracerSettings.IsDataStreamsMonitoringInDefaultState;
+        _registry = new DataStreamsExtractorRegistry(tracerSettings.DataStreamsTransactionExtractors);

+        Log.Debug(@"Data Streams extractors loaded: {AsJson}", _registry.AsJson());


Suggested change

Log.Debug(@"Data Streams extractors loaded: {AsJson}", _registry.AsJson());

if (Log.IsEnabled(LogEventLevel.Debug))

{

Log.Debug(@"Data Streams extractors loaded: {AsJson}", _registry.AsJson());

}

I think as written .AsJson() is called every single time this would be hit, unsure how often, but best to wrap I would think

💯 definitely should do this (also remove the @ 😄)

bouwkast · 2026-04-01T21:06:24Z

+
+    public void TrackTransaction(byte[] transactionIdBytes, string checkpointName)
+    {
+        if (!IsEnabled)


Suggested change

if (!IsEnabled)

if (!IsTransactionTrackingEnabled)

Is this supposed to be this?

Good catch, thanks!

bouwkast · 2026-04-01T21:07:08Z


+    public void TrackTransaction(string transactionId, string checkpointName)
+    {
+        if (!IsEnabled)


Suggested change

if (!IsEnabled)

if (!IsTransactionTrackingEnabled)

Is this supposed to be this?

Yeah, this looks like a bug to me?

bouwkast · 2026-04-01T21:12:46Z

+                            {
+                                foreach (var headerValue in headerValues)
+                                {
+                                    dataStreamsManager.TrackTransaction(headerValue, extractor.Name);


Just wanted to confirm: Should we be tracking a transaction if scope is null?

There is a check below on line 65 if (scope is not null)

I think the transaction stuff isn't dependent on the scope, but just wanted to make sure that is accurate

(If so it may be helpful to have a comment somewhere here stating why)

I also haven't checked when the scope could / would be null here

It's not dependent, but we need to account for active spans for linking purpose.

bouwkast · 2026-04-01T21:14:52Z

+    /// <param name="manager">The <see cref="DataStreamsManager"/> to use</param>
+    /// <param name="transactionId">The transaction identifier</param>
+    /// <param name="checkpointName">The checkpoint name at which the transaction is being tracked</param>
+    internal static void TrackTransaction(this Span span, DataStreamsManager? manager, string transactionId, string checkpointName)


I don't see any usages of this, is this still needed?

Good catch, it should be used when there's an active span - we want to add transaction ids as span attributes.

bouwkast · 2026-04-01T21:43:04Z

+
+    public byte[] GetDataAndReset()
+    {
+        // trim zeros


Could we check if _size is 0 right away and if so just return an empty []? Otherwise we always do _data = new byte[_initialByteSize]; even when we don't need to (I think)

bouwkast · 2026-04-01T21:58:38Z

+
+        foreach (var pair in Cache)
+        {
+            var keyBytes = Encoding.UTF8.GetBytes(pair.Key);


Suggested change

var keyBytes = Encoding.UTF8.GetBytes(pair.Key);

var keyBytes = Truncate(Encoding.UTF8.GetBytes(pair.Key));

Do we need to truncate this as well? I see we truncate the _idBytes = Truncate(idBytes); above

It's OK for the checkpoint name to be longer than 255 characters. This limitation is in place only for transaction ids.

bouwkast · 2026-04-01T22:16:04Z

+    {
+        var registry = new DataStreamsExtractorRegistry("[{\"name\": \"n\", \"type\": \"HTTP_OUT_HEADERS\", \"value\": \"v\"}]");
+        var extractor = registry.GetExtractorsByType(DataStreamsTransactionExtractor.Type.HttpOutHeaders)!.Single();
+        extractor.ExtractorType.Should().Be(extractor.ExtractorType);


Isn't this just validating that it is the same type as what it is?

Good catch, I used it for debugging, but forgot to cleanup

andrewlock

Haven't finished yet, but some things to be going on with

andrewlock · 2026-04-02T09:29:48Z

+        IsInDefaultState = tracerSettings.IsDataStreamsMonitoringInDefaultState;
+        _registry = new DataStreamsExtractorRegistry(tracerSettings.DataStreamsTransactionExtractors);

+        Log.Debug(@"Data Streams extractors loaded: {AsJson}", _registry.AsJson());


💯 definitely should do this (also remove the @ 😄)

andrewlock · 2026-04-02T09:54:52Z

+            return;
+        }
+
+        if (deserialized == null)


Suggested change

if (deserialized == null)

if (deserialized == null || deserialized.Length == 0)

andrewlock · 2026-04-02T09:59:33Z

+        List<DataStreamsTransactionExtractor>? deserialized;
+        try
+        {
+            deserialized = JsonHelper.DeserializeObject<List<DataStreamsTransactionExtractor>>(extractorsJson);


This seems like a very complicated and error prone format we're getting customers to need to use, have we considered a simpler public configuration API for this 😟

This format is flexible, since we don't know what other extractors we may need later. The intention was to use Remote Config in the future. For now we help customers onboard.

andrewlock · 2026-04-02T10:00:33Z

+
+    private Type? _cachedType;
+
+    public enum Type


Please don't call it Type, it's too common an API and will clash with System namespace

Suggested change

public enum Type

public enum ExtractorType

andrewlock · 2026-04-02T10:01:53Z

+        {
+            if (_cachedType is null)
+            {
+                _cachedType = TypeMap.TryGetValue(StringType, out var t) ? t : Type.Unknown;


nit: just use a switch expression and ditch the TypeMap. I'm not really sure you need the cachedType tbh:

Suggested change

_cachedType = TypeMap.TryGetValue(StringType, out var t) ? t : Type.Unknown;

_cachedType = StringType switch

{

"HTTP_OUT_HEADERS" => Type.HttpOutHeaders,

"HTTP_IN_HEADERS" => Type.HttpInHeaders,

"KAFKA_CONSUME_HEADERS" => Type.KafkaConsumeHeaders,

"KAFKA_PRODUCE_HEADERS" => Type.KafkaProduceHeaders,

_ => Type.Unknown,

};

andrewlock

Looking good, just a last few bits I think (I haven't checked the tests thoroughly though!)

andrewlock · 2026-04-09T10:51:17Z

+            if (hasTransactions)
+            {
+                var currentTs = DateTimeOffset.UtcNow.ToUnixTimeNanoseconds();
+                var bucketStartTime = currentTs - (currentTs % bucketDurationNs);


I don't know if it matters, but this seems like it has the potential to be essentially "wrong" if there are delays processing? Should this be passed into the Serialize method and tracked externally, when the data is actually recorded/snapshotted?

We carry the timestamp with each transaction, so the actual bucket is not important. It is used only as a way of splitting the data into chunks.

andrewlock · 2026-04-09T10:56:58Z

+        }
+    }
+
+    public static IReadOnlyList<DataStreamsTransactionExtractor> ParseList(string json)


nit: return the concrete type when you know it(it's better for perf)

Suggested change

public static IReadOnlyList<DataStreamsTransactionExtractor> ParseList(string json)

public static List<DataStreamsTransactionExtractor> ParseList(string json)

andrewlock · 2026-04-09T10:57:56Z

+                                                       defaultValue: new DefaultResult<IReadOnlyList<DataStreamsTransactionExtractor>>([], "[]"),
+                                                       converter: json => ParsingResult<IReadOnlyList<DataStreamsTransactionExtractor>>.Success(DataStreamsTransactionExtractor.ParseList(json)),


nit: use the concrete type when we know it (better for perf)

Suggested change

defaultValue: new DefaultResult<IReadOnlyList<DataStreamsTransactionExtractor>>([], "[]"),

converter: json => ParsingResult<IReadOnlyList<DataStreamsTransactionExtractor>>.Success(DataStreamsTransactionExtractor.ParseList(json)),

defaultValue: new DefaultResult<List<DataStreamsTransactionExtractor>>([], "[]"),

converter: json => ParsingResult<List<DataStreamsTransactionExtractor>>.Success(DataStreamsTransactionExtractor.ParseList(json)),

andrewlock · 2026-04-09T10:58:01Z

+        /// <summary>
+        /// Gets a raw value for DSM extractors
+        /// </summary>
+        internal IReadOnlyList<DataStreamsTransactionExtractor> DataStreamsTransactionExtractors { get; }


Suggested change

internal IReadOnlyList<DataStreamsTransactionExtractor> DataStreamsTransactionExtractors { get; }

internal List<DataStreamsTransactionExtractor> DataStreamsTransactionExtractors { get; }

andrewlock · 2026-04-09T11:10:45Z

+        buffer[offset + 8] = (byte)_timestamp;
+
+        // id size, up to 256 bytes
+        buffer[offset + 9] = (byte)_idBytes.Length;


I think this is still applicable, i.e. as a question of correctness? 🤔

andrewlock · 2026-04-09T11:14:26Z

+    public bool IsTransactionTrackingEnabled
    {
-        get => Volatile.Read(ref _isInDefaultState);
+        get => !_isInDefaultState && IsEnabled;
    }


nit:

Suggested change

public bool IsTransactionTrackingEnabled

{

get => Volatile.Read(ref _isInDefaultState);

get => !_isInDefaultState && IsEnabled;

}

public bool IsTransactionTrackingEnabled => !_isInDefaultState && IsEnabled;

andrewlock · 2026-04-09T11:14:47Z


+    public void TrackTransaction(string transactionId, string checkpointName)
+    {
+        if (!IsEnabled)


Yeah, this looks like a bug to me?

This comment has been minimized.

Sign in to view

kr-igor added the area:data-streams-monitoring label Mar 25, 2026

[DSM] Add transaction tracking support for HTTP and Kafka integrations

ef669de

kr-igor force-pushed the kr-igor/dsm-tt-v0 branch from de4b6c3 to ef669de Compare March 26, 2026 14:31

Merge branch 'master' into kr-igor/dsm-tt-v0

4362a5b

kr-igor added the type:new-feature label Mar 26, 2026

kr-igor marked this pull request as ready for review March 26, 2026 14:49

kr-igor requested review from a team as code owners March 26, 2026 14:49

robcarlan-datadog approved these changes Mar 26, 2026

View reviewed changes

Merge branch 'master' into kr-igor/dsm-tt-v0

b5235f8

lucaspimentel changed the title ~~Initial transaction tracking implementation~~ [DSM] Initial transaction tracking implementation Mar 30, 2026

chatgpt-codex-connector Bot reviewed Mar 30, 2026

View reviewed changes

kr-igor added 2 commits March 30, 2026 13:35

Fixes to address comments

2f0ae36

Added two more tests

6607b76

bouwkast reviewed Mar 31, 2026

View reviewed changes

Comment thread ...er/src/Datadog.Trace/DataStreamsMonitoring/TransactionTracking/DataStreamsTransactionInfo.cs

Added test application and more integration

57cdd9d

kr-igor and others added 3 commits April 1, 2026 08:48

Regenerated solution

a382aaa

Add System.Net.Http reference for net48 in HttpClient sample

d730113

HttpClient requires an explicit assembly reference on .NET Framework. Mirrors the same pattern used in Samples.HttpMessageHandler.

Merge branch 'master' into kr-igor/dsm-tt-v0

c2f4815

kr-igor requested a review from bouwkast April 1, 2026 17:52

bouwkast reviewed Apr 1, 2026

View reviewed changes

andrewlock reviewed Apr 2, 2026

View reviewed changes

kr-igor and others added 10 commits April 8, 2026 12:04

Initial refactoring (extractor type)

9efed30

Merge branch 'master' into kr-igor/dsm-tt-v0

702feb6

Moved parsing to tracer settings plus other minor fixes

9d45f57

Use IsTransactionTrackingEnabled everywhere

6a0b627

Use the default wait time to allow for transactions to flush on shutdown

eb1c792

Refactoring and allocations optimizations

229bec2

Simplified the code a bit

e482b19

Updated datadog.trace.trimming

a93ff70

Fixed flaky test

1381341

Fixed test

01ff2d1

andrewlock reviewed Apr 9, 2026

View reviewed changes

kr-igor added 2 commits April 9, 2026 09:51

More fixes

f4e4608

Added hard limit on transaction count

340c71a

andrewlock approved these changes Apr 10, 2026

View reviewed changes

kr-igor merged commit 58f22e8 into master Apr 10, 2026
139 checks passed

kr-igor deleted the kr-igor/dsm-tt-v0 branch April 10, 2026 14:08

github-actions Bot added this to the vNext-v3 milestone Apr 10, 2026

	var keyBytes = Encoding.UTF8.GetBytes(pair.Key);
	var keyBytes = Truncate(Encoding.UTF8.GetBytes(pair.Key));

	if (deserialized == null)
	if (deserialized == null \|\| deserialized.Length == 0)

-                _cachedType = TypeMap.TryGetValue(StringType, out var t) ? t : Type.Unknown;
+                _cachedType = StringType switch
+                {
+                    "HTTP_OUT_HEADERS" => Type.HttpOutHeaders,
+                    "HTTP_IN_HEADERS" => Type.HttpInHeaders,
+                    "KAFKA_CONSUME_HEADERS" => Type.KafkaConsumeHeaders,
+                    "KAFKA_PRODUCE_HEADERS" => Type.KafkaProduceHeaders,
+                    _ => Type.Unknown,
+                };

	public static IReadOnlyList<DataStreamsTransactionExtractor> ParseList(string json)
	public static List<DataStreamsTransactionExtractor> ParseList(string json)

		defaultValue: new DefaultResult<IReadOnlyList<DataStreamsTransactionExtractor>>([], "[]"),
		converter: json => ParsingResult<IReadOnlyList<DataStreamsTransactionExtractor>>.Success(DataStreamsTransactionExtractor.ParseList(json)),

	internal IReadOnlyList<DataStreamsTransactionExtractor> DataStreamsTransactionExtractors { get; }
	internal List<DataStreamsTransactionExtractor> DataStreamsTransactionExtractors { get; }

Uh oh!

Conversation

kr-igor commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Technical details

Uh oh!

This comment has been minimized.

dd-trace-dotnet-ci-bot Bot commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

⚠️ Potential regressions detected

FakeDbCommand

HttpMessageHandler

FakeDbCommand

HttpMessageHandler

Uh oh!

pr-commenter Bot commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild net6.0

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net472

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody net6.0

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net6.0

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody net6.0

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net6.0

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net472

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net6.0

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net472

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net6.0

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack net6.0

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net472

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net6.0

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest netcoreapp3.1

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net472

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net472

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net6.0

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch netcoreapp3.1

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net472

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net6.0

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync netcoreapp3.1

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net472

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net6.0

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync netcoreapp3.1

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net472

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net6.0

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog net6.0

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark net6.0

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark net6.0

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark netcoreapp3.1

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net472

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net6.0

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net6.0

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net472

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net6.0

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog netcoreapp3.1

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net472

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net6.0

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope net6.0

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope netcoreapp3.1

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net6.0

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan netcoreapp3.1

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net6.0

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes netcoreapp3.1

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net6.0

kr-igor commented Dec 12, 2025 •

edited

Loading

dd-trace-dotnet-ci-bot Bot commented Dec 13, 2025 •

edited

Loading

pr-commenter Bot commented Dec 13, 2025 •

edited

Loading