(CI) Migrating benchmark-serverless CI job to short-lived tokens#8219
Conversation
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8219) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (75ms) : 72, 78
master - mean (77ms) : 75, 80
section Bailout
This PR (8219) - mean (79ms) : 77, 82
master - mean (83ms) : 80, 85
section CallTarget+Inlining+NGEN
This PR (8219) - mean (1,093ms) : 1046, 1140
master - mean (1,121ms) : 1067, 1175
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (118ms) : 115, 121
master - mean (121ms) : 117, 125
section Bailout
This PR (8219) - mean (120ms) : 117, 123
master - mean (123ms) : 120, 127
section CallTarget+Inlining+NGEN
This PR (8219) - mean (783ms) : 721, 845
master - mean (796ms) : 730, 863
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (104ms) : 101, 108
master - mean (106ms) : 103, 109
section Bailout
This PR (8219) - mean (105ms) : 103, 107
master - mean (109ms) : 106, 112
section CallTarget+Inlining+NGEN
This PR (8219) - mean (771ms) : 731, 811
master - mean (776ms) : 708, 844
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (104ms) : 101, 107
master - mean (106ms) : 103, 110
section Bailout
This PR (8219) - mean (104ms) : 102, 106
master - mean (107ms) : 105, 110
section CallTarget+Inlining+NGEN
This PR (8219) - mean (686ms) : 667, 706
master - mean (708ms) : 684, 732
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (194ms) : 190, 198
master - mean (194ms) : 189, 199
section Bailout
This PR (8219) - mean (197ms) : 194, 200
master - mean (198ms) : 195, 201
section CallTarget+Inlining+NGEN
This PR (8219) - mean (1,149ms) : 1097, 1200
master - mean (1,156ms) : 1099, 1212
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (280ms) : 273, 287
master - mean (280ms) : 273, 286
section Bailout
This PR (8219) - mean (279ms) : 273, 285
master - mean (279ms) : 275, 283
section CallTarget+Inlining+NGEN
This PR (8219) - mean (948ms) : 913, 983
master - mean (940ms) : 886, 994
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (274ms) : 264, 284
master - mean (272ms) : 267, 278
section Bailout
This PR (8219) - mean (272ms) : 267, 277
master - mean (273ms) : 268, 279
section CallTarget+Inlining+NGEN
This PR (8219) - mean (942ms) : 903, 981
master - mean (943ms) : 914, 973
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8219) - mean (269ms) : 263, 275
master - mean (270ms) : 264, 276
section Bailout
This PR (8219) - mean (270ms) : 265, 275
master - mean (271ms) : 266, 275
section CallTarget+Inlining+NGEN
This PR (8219) - mean (836ms) : 821, 852
master - mean (841ms) : 818, 864
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
BenchmarksBenchmark execution time: 2026-03-12 21:00:51 Comparing candidate commit 3fc445d in PR branch Found 8 performance improvements and 10 performance regressions! Performance is the same for 159 metrics, 15 unstable metrics. scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1
scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody netcoreapp3.1
scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs netcoreapp3.1
scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net472
scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net6.0
scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1
scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice net472
scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice netcoreapp3.1
scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net6.0
scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1
scenario:Benchmarks.Trace.NLogBenchmark.EnrichedLog netcoreapp3.1
scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net6.0
scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1
scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope net6.0
scenario:Benchmarks.Trace.SpanBenchmark.StartFinishTwoScopes net6.0
|
Summary of changes
Migrated from use of Gitlab personal access token to short-lived token (using authanywhere) when cloning the serverless-tools repo for the benchmark-serverless CI job.
Reason for change
Keeping up-to-date with current policies about tokens, avoiding token expiry issues.
Implementation details
Test coverage
Other details