[SDTEST-2993] Switch to paginated endpoint for retrieving known tests by calvinbayer · Pull Request #8380 · DataDog/dd-trace-dotnet

calvinbayer · 2026-03-26T22:15:15Z

Summary

Implements cursor-based pagination for the known tests API endpoint (api/v2/ci/libraries/tests) as part of SDTEST-2993
The client now sends page_info in requests and follows has_next/cursor pagination, deep-merging test results across all pages
Includes a 10,000 page safety valve and all-or-nothing error handling (any page failure returns default)
Backward compatible: responses without page_info are treated as single-page (existing behavior)

Changes

tracer/src/Datadog.Trace/Ci/Net/TestOptimizationClient.GetKnownTestsAsync.cs — Core pagination loop with MergeKnownTests(), new PageInfoRequest/PageInfoResponse/KnownTestsPageResponse DTOs
tracer/test/.../CI/TestingFrameworkEvpTest.cs — Mock server supports paginated response mode via TestsJsonPages array
tracer/test/.../CI/XUnitEvpTests.cs — New efd_with_test_bypass_paginated test scenario
tracer/test/snapshots/XUnitEvpTests.EarlyFlakeDetection_efd_with_test_bypass_paginated.verified.txt — Snapshot for paginated scenario

Test plan

All 90 existing TestOptimization unit tests pass
New paginated integration test scenario added
CI pipeline validation

🤖 Generated with Claude Code

The known tests endpoint now returns results page by page instead of in a single bulk response. This reduces load on the backend storage layer and avoids timeouts for repositories with large numbers of tests. The client sends page_info:{} on the first request to signal pagination support, then follows cursor-based pagination until has_next is false. Results are merged locally across all pages. A 10,000-page safety valve prevents runaway loops. Backward compatible with non-paginated responses. Co-Authored-By: Claude Opus 4.6 <[email protected]>

- Change KnownTestsSuites value type from string[] to List<string> and use AddRange instead of allocating new arrays for merging - Extract pageResponse.Value and pageInfo.Value.Cursor to local vars - Use StringUtil.IsNullOrEmpty instead of string.IsNullOrEmpty - Rename TestsJsonPages to KnownTestsJsonPages in MockData - Return collected data instead of default when hitting max pages limit Co-Authored-By: Claude Opus 4.6 <[email protected]>

pr-commenter · 2026-03-26T23:25:23Z

Benchmarks

Benchmark execution time: 2026-04-15 18:27:12

Comparing candidate commit b7901c5 in PR branch calvin.bayer/SDTEST-2993/known-tests-pagination with baseline commit 6963bde in branch master.

Found 1 performance improvements and 1 performance regressions! Performance is the same for 25 metrics, 0 unstable metrics, 87 known flaky benchmarks.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net472

🟥 throughput [-6677.310op/s; -5875.072op/s] or [-7.623%; -6.707%]

scenario:Benchmarks.Trace.HttpClientBenchmark.SendAsync net6.0

🟩 throughput [+8782.114op/s; +9731.855op/s] or [+5.966%; +6.611%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.008%; +0.005%]
ignore execution_time [-530.263µs; +437.195µs] or [-0.263%; +0.217%]
ignore throughput [+1092.024op/s; +1525.090op/s] or [+1.295%; +1.808%]

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.007%]
ignore execution_time [-1773.174µs; +1739.254µs] or [-0.885%; +0.868%]
🟩 throughput [+9498.097op/s; +11876.120op/s] or [+7.984%; +9.982%]

scenario:Benchmarks.Trace.ActivityBenchmark.StartStopWithChild netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.003%; +0.007%]
ignore execution_time [+0.546ms; +2.685ms] or [+0.275%; +1.351%]
ignore throughput [-672.320op/s; +453.100op/s] or [-0.684%; +0.461%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net472

ignore allocated_mem [+21 bytes; +22 bytes] or [+0.662%; +0.676%]
🟥 execution_time [+308.582ms; +313.274ms] or [+153.129%; +155.458%]
ignore throughput [-5.751op/s; -0.484op/s] or [-1.035%; -0.087%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.009%; +0.002%]
🟥 execution_time [+380.665ms; +383.016ms] or [+300.748%; +302.606%]
ignore throughput [+7.530op/s; +11.220op/s] or [+0.993%; +1.479%]

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.009%; +0.002%]
🟥 execution_time [+392.978ms; +395.534ms] or [+347.770%; +350.033%]
ignore throughput [-9.346op/s; -6.174op/s] or [-1.320%; -0.872%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody net472

🟥 allocated_mem [+1.308KB; +1.308KB] or [+27.529%; +27.541%]
ignore execution_time [-539.325µs; +17.859µs] or [-0.269%; +0.009%]
ignore throughput [-3481.738op/s; -3083.311op/s] or [-2.709%; -2.399%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody net6.0

🟥 allocated_mem [+471 bytes; +472 bytes] or [+9.977%; +9.987%]
🟩 execution_time [-16.429ms; -12.242ms] or [-7.673%; -5.717%]
ignore throughput [+4955.365op/s; +7734.347op/s] or [+3.617%; +5.646%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody netcoreapp3.1

🟥 allocated_mem [+1.272KB; +1.272KB] or [+27.502%; +27.510%]
ignore execution_time [-13.155ms; -8.984ms] or [-6.264%; -4.278%]
ignore throughput [-699.224op/s; +1613.905op/s] or [-0.632%; +1.459%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net472

🟥 allocated_mem [+1.307KB; +1.307KB] or [+105.746%; +105.759%]
ignore execution_time [-1039.279µs; -465.619µs] or [-0.517%; -0.232%]
🟥 throughput [-246185.594op/s; -242313.090op/s] or [-25.137%; -24.741%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net6.0

🟥 allocated_mem [+471 bytes; +472 bytes] or [+38.558%; +38.566%]
🟩 execution_time [-26.455ms; -21.607ms] or [-11.798%; -9.636%]
🟥 throughput [-83504.093op/s; -60517.954op/s] or [-8.921%; -6.465%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1

🟥 allocated_mem [+1.272KB; +1.272KB] or [+105.292%; +105.304%]
ignore execution_time [-3.778ms; +0.607ms] or [-1.886%; +0.303%]
🟥 throughput [-130315.402op/s; -113990.358op/s] or [-18.724%; -16.378%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.007%; +0.003%]
ignore execution_time [-1161.014µs; -121.644µs] or [-0.579%; -0.061%]
ignore throughput [-197.353op/s; +617.860op/s] or [-0.133%; +0.416%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.003%]
ignore execution_time [-10.725ms; -0.596ms] or [-5.411%; -0.301%]
🟩 throughput [+11024.743op/s; +13991.895op/s] or [+7.015%; +8.903%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.007%; +0.003%]
ignore execution_time [+1.827ms; +5.848ms] or [+0.931%; +2.981%]
🟩 throughput [+7969.428op/s; +10636.533op/s] or [+6.349%; +8.473%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.008%; +0.004%]
ignore execution_time [-469.097µs; -253.317µs] or [-0.234%; -0.126%]
ignore throughput [+109159.976op/s; +116410.012op/s] or [+3.321%; +3.541%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.008%]
ignore execution_time [-2.129ms; -1.218ms] or [-1.053%; -0.602%]
🟩 throughput [+429570.277op/s; +458059.342op/s] or [+14.324%; +15.274%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.004%]
🟩 execution_time [-18.913ms; -14.536ms] or [-8.718%; -6.701%]
🟩 throughput [+201718.286op/s; +256370.324op/s] or [+8.007%; +10.176%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net472

ignore allocated_mem [+1 bytes; +4 bytes] or [+0.004%; +0.012%]
🟥 execution_time [+299.193ms; +300.109ms] or [+149.497%; +149.954%]
ignore throughput [+130.229op/s; +150.829op/s] or [+1.438%; +1.666%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs net6.0

ignore allocated_mem [-1 bytes; +2 bytes] or [-0.004%; +0.008%]
🟥 execution_time [+299.048ms; +307.113ms] or [+150.810%; +154.878%]
ignore throughput [+232.580op/s; +491.497op/s] or [+1.779%; +3.759%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeArgs netcoreapp3.1

ignore allocated_mem [-1 bytes; +2 bytes] or [-0.004%; +0.008%]
🟥 execution_time [+299.570ms; +302.122ms] or [+150.900%; +152.186%]
ignore throughput [+91.698op/s; +218.021op/s] or [+0.885%; +2.105%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net472

ignore allocated_mem [+2 bytes; +3 bytes] or [+0.137%; +0.150%]
🟥 execution_time [+297.605ms; +298.518ms] or [+146.172%; +146.620%]
ignore throughput [-1.506op/s; +9.798op/s] or [-0.040%; +0.260%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.009%]
🟥 execution_time [+295.604ms; +298.207ms] or [+144.510%; +145.782%]
ignore throughput [+58.953op/s; +107.520op/s] or [+0.856%; +1.562%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.009%]
🟥 execution_time [+300.786ms; +302.112ms] or [+150.333%; +150.995%]
ignore throughput [+39.401op/s; +55.405op/s] or [+0.782%; +1.100%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark net472

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [+4.657µs; +8.871µs] or [+0.956%; +1.821%]
ignore throughput [-36.291op/s; -19.052op/s] or [-1.768%; -0.928%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.000%; +0.010%]
ignore execution_time [+14.707µs; +41.467µs] or [+3.373%; +9.510%]
ignore throughput [-204.857op/s; -84.465op/s] or [-8.906%; -3.672%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.003%; +0.003%]
ignore execution_time [+6.794µs; +28.760µs] or [+1.456%; +6.162%]
ignore throughput [-142.533op/s; -61.754op/s] or [-6.580%; -2.851%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack net472

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [-5.627µs; -1.563µs] or [-1.519%; -0.422%]
ignore throughput [+12.356op/s; +41.583op/s] or [+0.458%; +1.540%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.003%; +0.007%]
🟥 execution_time [+24.271µs; +47.867µs] or [+7.748%; +15.281%]
🟥 throughput [-443.770op/s; -245.232op/s] or [-13.834%; -7.645%]

scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.003%; +0.003%]
ignore execution_time [-16.684µs; +5.732µs] or [-4.564%; +1.568%]
ignore throughput [-75.689op/s; +58.219op/s] or [-2.716%; +2.089%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net472

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
🟥 execution_time [+299.924ms; +300.558ms] or [+149.693%; +150.009%]
ignore throughput [+3036687.840op/s; +3599049.012op/s] or [+1.521%; +1.803%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net6.0

ignore allocated_mem [+117 bytes; +119 bytes] or [+0.654%; +0.665%]
🟥 execution_time [+412.964ms; +419.180ms] or [+448.702%; +455.456%]
🟩 throughput [+958.306op/s; +1113.903op/s] or [+7.875%; +9.153%]

scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest netcoreapp3.1

ignore allocated_mem [+49 bytes; +51 bytes] or [+0.237%; +0.248%]
unstable execution_time [+330.887ms; +359.457ms] or [+251.240%; +272.933%]
🟩 throughput [+550.044op/s; +749.190op/s] or [+5.325%; +7.253%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net472

ignore allocated_mem [+2.805KB; +2.810KB] or [+4.984%; +4.992%]
unstable execution_time [+311.887ms; +371.575ms] or [+143.402%; +170.846%]
🟥 throughput [-505.995op/s; -444.611op/s] or [-45.848%; -40.286%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

ignore allocated_mem [-1.270KB; -1.268KB] or [-2.995%; -2.990%]
unstable execution_time [+206.908ms; +340.139ms] or [+88.176%; +144.953%]
🟥 throughput [-741.261op/s; -657.811op/s] or [-49.442%; -43.876%]

scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1

ignore allocated_mem [+1.314KB; +1.317KB] or [+3.102%; +3.109%]
🟥 execution_time [+339.271ms; +347.948ms] or [+202.923%; +208.113%]
🟥 throughput [-425.836op/s; -389.837op/s] or [-29.650%; -27.144%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice net472

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [-17.138µs; +61.965µs] or [-0.862%; +3.118%]
ignore throughput [-9.649op/s; +6.632op/s] or [-1.917%; +1.318%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [-15.792µs; -6.289µs] or [-1.085%; -0.432%]
ignore throughput [+3.120op/s; +7.651op/s] or [+0.454%; +1.114%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [-176.922µs; -93.818µs] or [-6.155%; -3.264%]
ignore throughput [+11.842op/s; +29.327op/s] or [+3.404%; +8.430%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net472

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [-21.592µs; -16.687µs] or [-1.865%; -1.441%]
ignore throughput [+12.723op/s; +16.429op/s] or [+1.473%; +1.902%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [-10.463µs; +5.082µs] or [-0.970%; +0.471%]
ignore throughput [-3.302op/s; +10.221op/s] or [-0.356%; +1.102%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
ignore execution_time [-88.027µs; +7.158µs] or [-4.716%; +0.383%]
unstable throughput [+1.590op/s; +62.854op/s] or [+0.297%; +11.732%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice net472

ignore allocated_mem [-43 bytes; +21 bytes] or [-0.007%; +0.003%]
ignore execution_time [+52.052µs; +70.207µs] or [+2.033%; +2.742%]
ignore throughput [-10.316op/s; -7.674op/s] or [-2.641%; -1.965%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice net6.0

ignore allocated_mem [-38 bytes; +46 bytes] or [-0.006%; +0.007%]
ignore execution_time [-128.651µs; -81.613µs] or [-6.517%; -4.134%]
ignore throughput [+24.499op/s; +36.278op/s] or [+4.836%; +7.162%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice netcoreapp3.1

ignore allocated_mem [-42 bytes; +23 bytes] or [-0.007%; +0.004%]
ignore execution_time [-122.903µs; -82.359µs] or [-3.117%; -2.089%]
ignore throughput [+5.663op/s; +8.236op/s] or [+2.233%; +3.248%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.001%; +0.008%]
🟥 execution_time [+302.909ms; +304.070ms] or [+152.539%; +153.124%]
ignore throughput [+9323.584op/s; +11056.788op/s] or [+3.000%; +3.558%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.004%]
🟥 execution_time [+300.447ms; +302.133ms] or [+150.555%; +151.399%]
🟩 throughput [+38950.926op/s; +45124.873op/s] or [+6.141%; +7.114%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.004%]
🟥 execution_time [+301.455ms; +305.098ms] or [+151.438%; +153.268%]
ignore throughput [+16899.113op/s; +25601.584op/s] or [+3.560%; +5.393%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.008%; +0.004%]
🟥 execution_time [+301.041ms; +302.730ms] or [+151.173%; +152.021%]
ignore throughput [+495.143op/s; +2325.311op/s] or [+0.166%; +0.779%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.007%; +0.006%]
🟥 execution_time [+298.395ms; +300.388ms] or [+147.543%; +148.528%]
ignore throughput [+28036.276op/s; +34443.128op/s] or [+4.517%; +5.549%]

scenario:Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.004%]
🟥 execution_time [+303.122ms; +306.532ms] or [+153.635%; +155.364%]
ignore throughput [-1582.893op/s; +6684.738op/s] or [-0.342%; +1.444%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net472

ignore allocated_mem [+0 bytes; +1 bytes] or [+0.108%; +0.119%]
🟥 execution_time [+301.563ms; +303.169ms] or [+151.358%; +152.163%]
ignore throughput [+669.962op/s; +3484.065op/s] or [+0.174%; +0.904%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.006%]
🟥 execution_time [+299.082ms; +300.831ms] or [+149.065%; +149.937%]
🟩 throughput [+61052.859op/s; +66129.344op/s] or [+12.123%; +13.131%]

scenario:Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.006%]
🟥 execution_time [+301.295ms; +303.842ms] or [+149.892%; +151.159%]
ignore throughput [-13040.694op/s; -7646.343op/s] or [-3.087%; -1.810%]

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.003%; +0.006%]
ignore execution_time [-1240.725µs; -384.973µs] or [-0.617%; -0.191%]
ignore throughput [-848.832op/s; +235.690op/s] or [-0.341%; +0.095%]

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.003%]
🟩 execution_time [-16.418ms; -12.766ms] or [-7.634%; -5.936%]
🟩 throughput [+24440.539op/s; +31018.027op/s] or [+6.705%; +8.509%]

scenario:Benchmarks.Trace.ILoggerBenchmark.EnrichedLog netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.008%]
ignore execution_time [-0.605ms; +3.350ms] or [-0.303%; +1.681%]
ignore throughput [+4446.060op/s; +10245.353op/s] or [+1.623%; +3.740%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark net472

ignore allocated_mem [-4.459KB; -4.431KB] or [-1.623%; -1.613%]
ignore execution_time [+8.146µs; +48.408µs] or [+2.012%; +11.957%]
ignore throughput [-255.316op/s; -51.537op/s] or [-10.274%; -2.074%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark net6.0

🟩 allocated_mem [-19.397KB; -19.377KB] or [-7.076%; -7.068%]
unstable execution_time [-46.015µs; +6.503µs] or [-9.095%; +1.285%]
ignore throughput [-13.083op/s; +170.543op/s] or [-0.653%; +8.510%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark netcoreapp3.1

🟩 allocated_mem [-17.279KB; -17.263KB] or [-6.299%; -6.293%]
ignore execution_time [-37.528µs; +18.575µs] or [-6.503%; +3.219%]
ignore throughput [-47.806op/s; +104.123op/s] or [-2.731%; +5.949%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark net472

ignore allocated_mem [-2 bytes; +2 bytes] or [-0.005%; +0.006%]
ignore execution_time [-513.935ns; +976.601ns] or [-0.890%; +1.692%]
ignore throughput [-262.920op/s; +157.657op/s] or [-1.517%; +0.910%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark net6.0

ignore allocated_mem [-4 bytes; +0 bytes] or [-0.010%; -0.001%]
🟥 execution_time [+4.074µs; +7.656µs] or [+9.630%; +18.097%]
🟥 throughput [-3767.263op/s; -2073.592op/s] or [-15.859%; -8.729%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark netcoreapp3.1

ignore allocated_mem [-1 bytes; +1 bytes] or [-0.002%; +0.002%]
unstable execution_time [-14.948µs; -7.980µs] or [-23.192%; -12.380%]
🟩 throughput [+2068.373op/s; +3535.619op/s] or [+12.690%; +21.692%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net472

ignore allocated_mem [+1 bytes; +2 bytes] or [+0.039%; +0.050%]
🟥 execution_time [+301.717ms; +302.928ms] or [+152.504%; +153.117%]
ignore throughput [-123.441op/s; -102.801op/s] or [-2.063%; -1.718%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.005%]
🟥 execution_time [+302.605ms; +304.772ms] or [+154.025%; +155.128%]
ignore throughput [-28.503op/s; +46.906op/s] or [-0.353%; +0.582%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.005%]
🟥 execution_time [+298.689ms; +301.320ms] or [+149.531%; +150.848%]
ignore throughput [-170.365op/s; -103.898op/s] or [-2.170%; -1.324%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.005%]
ignore execution_time [-695.981µs; -109.097µs] or [-0.347%; -0.054%]
ignore throughput [-14591.934op/s; -13306.418op/s] or [-4.039%; -3.684%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.007%]
ignore execution_time [+46.549µs; +653.473µs] or [+0.023%; +0.327%]
🟩 throughput [+48033.595op/s; +52321.402op/s] or [+9.092%; +9.903%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.006%]
ignore execution_time [+1.272ms; +4.949ms] or [+0.645%; +2.509%]
ignore throughput [+1041.761op/s; +9474.052op/s] or [+0.247%; +2.242%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.006%]
🟥 execution_time [+300.359ms; +302.393ms] or [+149.702%; +150.716%]
ignore throughput [+855.108op/s; +1875.624op/s] or [+0.565%; +1.239%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [+0.000%; +0.009%]
🟥 execution_time [+301.561ms; +303.365ms] or [+151.429%; +152.335%]
ignore throughput [+4168.433op/s; +6278.635op/s] or [+1.813%; +2.730%]

scenario:Benchmarks.Trace.SerilogBenchmark.EnrichedLog netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.003%]
🟥 execution_time [+304.061ms; +306.450ms] or [+154.200%; +155.412%]
ignore throughput [+1149.205op/s; +3554.519op/s] or [+0.647%; +2.002%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net472

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
🟥 execution_time [+300.584ms; +301.290ms] or [+149.933%; +150.285%]
🟩 throughput [+65238884.597op/s; +65484187.173op/s] or [+47.511%; +47.690%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net6.0

ignore allocated_mem [+73 bytes; +74 bytes] or [+0.425%; +0.435%]
unstable execution_time [+345.176ms; +391.963ms] or [+429.288%; +487.476%]
🟩 throughput [+863.734op/s; +1037.700op/s] or [+6.677%; +8.022%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [+nan%; +nan%]
🟥 execution_time [+299.426ms; +300.355ms] or [+149.347%; +149.810%]
🟩 throughput [+18235828.566op/s; +19180898.745op/s] or [+8.077%; +8.496%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.006%]
ignore execution_time [+184.117µs; +919.459µs] or [+0.092%; +0.460%]
ignore throughput [-8050.754op/s; -5016.631op/s] or [-0.898%; -0.560%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.007%]
ignore execution_time [-4.841ms; -3.720ms] or [-2.371%; -1.822%]
🟩 throughput [+97163.882op/s; +108541.560op/s] or [+9.072%; +10.134%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.003%; +0.006%]
ignore execution_time [+2.154ms; +6.262ms] or [+1.090%; +3.169%]
🟩 throughput [+49683.220op/s; +69479.104op/s] or [+5.751%; +8.042%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.000%; +0.009%]
ignore execution_time [-73.803µs; +234.223µs] or [-0.037%; +0.117%]
ignore throughput [-17064.119op/s; -13981.370op/s] or [-1.562%; -1.280%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.004%]
ignore execution_time [+6.143ms; +10.234ms] or [+3.201%; +5.332%]
🟩 throughput [+94180.691op/s; +124565.352op/s] or [+7.290%; +9.642%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.001%; +0.008%]
ignore execution_time [-4.897ms; -3.294ms] or [-2.406%; -1.618%]
🟩 throughput [+90756.636op/s; +100251.147op/s] or [+9.014%; +9.957%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.008%; +0.002%]
ignore execution_time [-1053.672µs; +174.377µs] or [-0.524%; +0.087%]
ignore throughput [+13284.020op/s; +16205.826op/s] or [+2.960%; +3.611%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.004%; +0.009%]
ignore execution_time [-704.480µs; +695.101µs] or [-0.352%; +0.347%]
🟩 throughput [+57372.102op/s; +62844.199op/s] or [+10.418%; +11.411%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.006%; +0.004%]
ignore execution_time [-0.052ms; +4.040ms] or [-0.026%; +2.030%]
ignore throughput [+20368.220op/s; +30361.610op/s] or [+4.559%; +6.796%]

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net472

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.006%]
ignore execution_time [-992.014µs; +56.147µs] or [-0.494%; +0.028%]
ignore throughput [-30842.393op/s; -27629.881op/s] or [-4.514%; -4.044%]

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net6.0

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.007%]
ignore execution_time [-1604.854µs; +1885.342µs] or [-0.803%; +0.943%]
🟩 throughput [+77904.827op/s; +94962.469op/s] or [+8.704%; +10.610%]

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin netcoreapp3.1

ignore allocated_mem [+0 bytes; +0 bytes] or [-0.005%; +0.005%]
ignore execution_time [+2.006ms; +5.913ms] or [+1.019%; +3.003%]
ignore throughput [+27313.735op/s; +42200.410op/s] or [+3.814%; +5.892%]

dd-trace-dotnet-ci-bot · 2026-03-26T23:28:44Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (8380) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	72.05 ± (72.02 - 72.35) ms	71.58 ± (71.72 - 72.17) ms	-0.6%	✅
.NET Framework 4.8 - Bailout
duration	76.10 ± (75.91 - 76.27) ms	76.11 ± (76.03 - 76.39) ms	+0.0%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1059.98 ± (1063.04 - 1069.60) ms	1060.04 ± (1060.34 - 1065.92) ms	+0.0%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	22.49 ± (22.45 - 22.53) ms	22.29 ± (22.25 - 22.32) ms	-0.9%	✅
process.time_to_main_ms	84.28 ± (84.11 - 84.45) ms	83.11 ± (82.92 - 83.30) ms	-1.4%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.92 ± (10.91 - 10.92) MB	10.91 ± (10.91 - 10.92) MB	-0.0%	✅
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	22.42 ± (22.39 - 22.45) ms	22.26 ± (22.22 - 22.30) ms	-0.7%	✅
process.time_to_main_ms	85.78 ± (85.61 - 85.95) ms	84.70 ± (84.47 - 84.92) ms	-1.3%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.95 ± (10.94 - 10.95) MB	10.94 ± (10.94 - 10.94) MB	-0.1%	✅
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	227.19 ± (225.82 - 228.57) ms	227.72 ± (226.36 - 229.09) ms	+0.2%	✅⬆️
process.time_to_main_ms	520.40 ± (519.29 - 521.51) ms	520.84 ± (519.61 - 522.07) ms	+0.1%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	48.45 ± (48.41 - 48.49) MB	48.47 ± (48.44 - 48.50) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.0%	✅
.NET 6 - Baseline
process.internal_duration_ms	21.09 ± (21.06 - 21.12) ms	21.04 ± (21.01 - 21.07) ms	-0.2%	✅
process.time_to_main_ms	72.45 ± (72.30 - 72.61) ms	72.66 ± (72.50 - 72.82) ms	+0.3%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.60 ± (10.60 - 10.60) MB	10.63 ± (10.63 - 10.63) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	20.81 ± (20.78 - 20.85) ms	20.98 ± (20.94 - 21.03) ms	+0.8%	✅⬆️
process.time_to_main_ms	72.51 ± (72.36 - 72.66) ms	73.70 ± (73.52 - 73.88) ms	+1.6%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.71 ± (10.71 - 10.72) MB	10.75 ± (10.75 - 10.75) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	384.37 ± (382.61 - 386.12) ms	386.33 ± (384.38 - 388.27) ms	+0.5%	✅⬆️
process.time_to_main_ms	518.40 ± (517.57 - 519.22) ms	521.55 ± (520.46 - 522.65) ms	+0.6%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	50.08 ± (50.05 - 50.10) MB	50.01 ± (49.99 - 50.03) MB	-0.1%	✅
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	-0.4%	✅
.NET 8 - Baseline
process.internal_duration_ms	19.30 ± (19.27 - 19.33) ms	19.26 ± (19.22 - 19.29) ms	-0.2%	✅
process.time_to_main_ms	71.82 ± (71.66 - 71.97) ms	71.53 ± (71.36 - 71.70) ms	-0.4%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.68 ± (7.67 - 7.68) MB	7.66 ± (7.65 - 7.66) MB	-0.3%	✅
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	19.31 ± (19.28 - 19.34) ms	19.38 ± (19.33 - 19.43) ms	+0.3%	✅⬆️
process.time_to_main_ms	72.56 ± (72.42 - 72.71) ms	73.46 ± (73.28 - 73.63) ms	+1.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.70 ± (7.69 - 7.71) MB	7.71 ± (7.70 - 7.71) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	304.90 ± (302.54 - 307.26) ms	306.76 ± (304.45 - 309.08) ms	+0.6%	✅⬆️
process.time_to_main_ms	479.47 ± (478.70 - 480.24) ms	480.16 ± (479.44 - 480.89) ms	+0.1%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	37.07 ± (37.04 - 37.09) MB	37.12 ± (37.10 - 37.14) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	27 ± (27 - 27)	27 ± (27 - 27)	-0.5%	✅

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	211.87 ± (211.61 - 212.55) ms	210.86 ± (210.57 - 211.48) ms	-0.5%	✅
.NET Framework 4.8 - Bailout
duration	215.11 ± (214.99 - 215.82) ms	214.32 ± (214.18 - 215.07) ms	-0.4%	✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1227.96 ± (1227.14 - 1234.21) ms	1222.87 ± (1224.05 - 1231.16) ms	-0.4%	✅
.NET Core 3.1 - Baseline
process.internal_duration_ms	205.90 ± (205.44 - 206.36) ms	206.14 ± (205.68 - 206.59) ms	+0.1%	✅⬆️
process.time_to_main_ms	90.96 ± (90.68 - 91.23) ms	90.56 ± (90.29 - 90.82) ms	-0.4%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	15.90 ± (15.88 - 15.93) MB	15.93 ± (15.91 - 15.96) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (20 - 20)	-0.4%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	204.66 ± (204.22 - 205.10) ms	204.76 ± (204.36 - 205.17) ms	+0.1%	✅⬆️
process.time_to_main_ms	91.62 ± (91.32 - 91.91) ms	91.33 ± (91.08 - 91.58) ms	-0.3%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.04 ± (16.02 - 16.06) MB	16.02 ± (16.00 - 16.04) MB	-0.1%	✅
runtime.dotnet.threads.count	21 ± (21 - 21)	21 ± (20 - 21)	-2.1%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	422.44 ± (421.24 - 423.63) ms	424.19 ± (422.94 - 425.44) ms	+0.4%	✅⬆️
process.time_to_main_ms	554.71 ± (553.46 - 555.96) ms	553.92 ± (552.65 - 555.19) ms	-0.1%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	59.43 ± (59.39 - 59.46) MB	59.31 ± (59.26 - 59.36) MB	-0.2%	✅
runtime.dotnet.threads.count	30 ± (30 - 30)	30 ± (30 - 30)	-0.4%	✅
.NET 6 - Baseline
process.internal_duration_ms	210.82 ± (210.36 - 211.28) ms	210.56 ± (210.12 - 211.01) ms	-0.1%	✅
process.time_to_main_ms	78.69 ± (78.44 - 78.93) ms	78.58 ± (78.30 - 78.87) ms	-0.1%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.18 ± (16.15 - 16.20) MB	16.14 ± (16.12 - 16.17) MB	-0.2%	✅
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (19 - 20)	-0.1%	✅
.NET 6 - Bailout
process.internal_duration_ms	210.46 ± (210.05 - 210.87) ms	209.26 ± (208.80 - 209.71) ms	-0.6%	✅
process.time_to_main_ms	79.62 ± (79.39 - 79.84) ms	79.27 ± (79.01 - 79.52) ms	-0.4%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.26 ± (16.24 - 16.28) MB	16.23 ± (16.21 - 16.25) MB	-0.2%	✅
runtime.dotnet.threads.count	20 ± (20 - 21)	21 ± (20 - 21)	+0.1%	✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	600.31 ± (597.74 - 602.88) ms	599.07 ± (596.30 - 601.84) ms	-0.2%	✅
process.time_to_main_ms	550.08 ± (548.78 - 551.37) ms	551.41 ± (550.04 - 552.78) ms	+0.2%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	61.75 ± (61.64 - 61.87) MB	61.44 ± (61.32 - 61.55) MB	-0.5%	✅
runtime.dotnet.threads.count	31 ± (31 - 31)	31 ± (31 - 31)	-0.6%	✅
.NET 8 - Baseline
process.internal_duration_ms	209.09 ± (208.68 - 209.50) ms	209.48 ± (209.07 - 209.89) ms	+0.2%	✅⬆️
process.time_to_main_ms	77.78 ± (77.55 - 78.01) ms	78.02 ± (77.80 - 78.24) ms	+0.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.55 ± (11.53 - 11.57) MB	11.58 ± (11.56 - 11.61) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	+0.2%	✅⬆️
.NET 8 - Bailout
process.internal_duration_ms	208.27 ± (207.82 - 208.72) ms	208.87 ± (208.52 - 209.22) ms	+0.3%	✅⬆️
process.time_to_main_ms	78.71 ± (78.52 - 78.91) ms	78.95 ± (78.81 - 79.08) ms	+0.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.61 ± (11.59 - 11.63) MB	11.68 ± (11.66 - 11.69) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (20 - 20)	+1.1%	✅⬆️
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	529.77 ± (523.86 - 535.68) ms	550.25 ± (543.28 - 557.23) ms	+3.9%	✅⬆️
process.time_to_main_ms	512.83 ± (511.95 - 513.72) ms	512.38 ± (511.50 - 513.26) ms	-0.1%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	50.72 ± (50.65 - 50.80) MB	51.05 ± (50.96 - 51.14) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	30 ± (30 - 30)	30 ± (30 - 30)	+0.3%	✅⬆️

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (72ms)  : 69, 75
    master - mean (72ms)  : 70, 75

    section Bailout
    This PR (8380) - mean (76ms)  : 74, 79
    master - mean (76ms)  : 74, 78

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (1,063ms)  : 1022, 1104
    master - mean (1,066ms)  : 1018, 1115

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (112ms)  : 108, 116
    master - mean (114ms)  : 110, 117

    section Bailout
    This PR (8380) - mean (113ms)  : 110, 116
    master - mean (115ms)  : 112, 118

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (787ms)  : 768, 807
    master - mean (786ms)  : 767, 806

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (100ms)  : 97, 103
    master - mean (100ms)  : 97, 102

    section Bailout
    This PR (8380) - mean (101ms)  : 98, 103
    master - mean (99ms)  : 97, 101

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (935ms)  : 897, 973
    master - mean (931ms)  : 901, 962

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (98ms)  : 94, 103
    master - mean (99ms)  : 95, 102

    section Bailout
    This PR (8380) - mean (100ms)  : 98, 103
    master - mean (99ms)  : 97, 102

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (818ms)  : 779, 857
    master - mean (815ms)  : 780, 849

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (211ms)  : 207, 215
    master - mean (212ms)  : 206, 218

    section Bailout
    This PR (8380) - mean (215ms)  : 210, 219
    master - mean (215ms)  : 211, 219

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (1,228ms)  : 1174, 1281
    master - mean (1,231ms)  : 1180, 1281

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (306ms)  : 298, 314
    master - mean (306ms)  : 299, 314

    section Bailout
    This PR (8380) - mean (306ms)  : 300, 312
    master - mean (305ms)  : 297, 313

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (1,021ms)  : 995, 1046
    master - mean (1,018ms)  : 990, 1047

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (298ms)  : 289, 307
    master - mean (299ms)  : 292, 306

    section Bailout
    This PR (8380) - mean (297ms)  : 290, 305
    master - mean (299ms)  : 293, 304

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (1,183ms)  : 1151, 1214
    master - mean (1,182ms)  : 1152, 1213

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8380) - mean (298ms)  : 293, 304
    master - mean (297ms)  : 292, 303

    section Bailout
    This PR (8380) - mean (299ms)  : 293, 304
    master - mean (298ms)  : 292, 303

    section CallTarget+Inlining+NGEN
    This PR (8380) - mean (1,096ms)  : 988, 1204
    master - mean (1,080ms)  : 983, 1177

MergeKnownTests returned early without initializing the aggregate dictionary when a page had Count: 0. This left aggregate as null, causing KnownTestsFeature.Enabled to return false and preventing EFD from activating. Co-Authored-By: Claude Opus 4.6 <[email protected]>

…ation

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0792d15b95

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

andrewlock

I think the error-path handling may need some more explicit handling one way or another + potentially some testing... There's currently a single happy-path integration test, maybe we should have tests for when the response isn't what we expect etc? Otherwise just minor things, mostly not for addressing in this PR I think

andrewlock · 2026-04-08T11:10:09Z

+            if (!aggregate.TryGetValue(moduleEntry.Key, out var existingSuites) || existingSuites is null)
+            {
+                existingSuites = new KnownTestsResponse.KnownTestsSuites();
+                aggregate[moduleEntry.Key] = existingSuites;


this seems to be the point you need to create the aggregate if you haven't already?

andrewlock

Oops, meant to approve with the above comments

…ation

calvinbayer added area:ci-visibility type:enhancement Improvement to an existing feature labels Mar 26, 2026