Skip to content

Don't run benchmark trigger on tag pushes#8560

Merged
andrewlock merged 1 commit into
masterfrom
andrew/dont-run-microbenchmarks-on-tag-refs
May 5, 2026
Merged

Don't run benchmark trigger on tag pushes#8560
andrewlock merged 1 commit into
masterfrom
andrew/dont-run-microbenchmarks-on-tag-refs

Conversation

@andrewlock

@andrewlock andrewlock commented May 5, 2026

Copy link
Copy Markdown
Member

Summary of changes

Don't run the microbenchmark trigger on tag pushes

Reason for change

We don't need to run these on tag pushes (as the code hasn't changed) and we already exclude running them in microbenchmarks.yml anyway

Implementation details

Add exclude for commit tag pushes

Test coverage

We can't really test it unfortunately

Other details

Note that we use except: in other places, but rules: is apparently the recommended approach

@andrewlock
andrewlock requested a review from a team as a code owner May 5, 2026 11:03
@andrewlock andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label May 5, 2026
@andrewlock andrewlock changed the title Don't run pipeline trigger on tag pushes Don't run benchmark trigger on tag pushes May 5, 2026
bouwkast
bouwkast approved these changes May 5, 2026
View reviewed changes

@bouwkast bouwkast left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@andrewlock
andrewlock enabled auto-merge (squash) May 5, 2026 11:43
@dd-trace-dotnet-ci-bot

dd-trace-dotnet-ci-bot Bot commented May 5, 2026

Copy link
Copy Markdown

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (8560) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration73.80 ± (73.92 - 74.49) ms73.86 ± (74.13 - 74.66) ms+0.1%✅⬆️
.NET Framework 4.8 - Bailout
duration78.86 ± (78.76 - 79.30) ms79.84 ± (79.62 - 80.18) ms+1.2%✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1077.20 ± (1076.57 - 1083.04) ms1081.80 ± (1081.08 - 1090.25) ms+0.4%✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms22.83 ± (22.77 - 22.89) ms23.00 ± (22.93 - 23.07) ms+0.8%✅⬆️
process.time_to_main_ms86.98 ± (86.66 - 87.30) ms88.10 ± (87.75 - 88.46) ms+1.3%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.91 ± (10.91 - 10.91) MB10.94 ± (10.93 - 10.94) MB+0.2%✅⬆️
runtime.dotnet.threads.count12 ± (12 - 12)12 ± (12 - 12)+0.0%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms22.62 ± (22.57 - 22.67) ms22.46 ± (22.42 - 22.50) ms-0.7%✅
process.time_to_main_ms87.02 ± (86.77 - 87.27) ms86.74 ± (86.53 - 86.95) ms-0.3%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.89 ± (10.89 - 10.90) MB10.97 ± (10.96 - 10.97) MB+0.7%✅⬆️
runtime.dotnet.threads.count13 ± (13 - 13)13 ± (13 - 13)+0.0%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms214.40 ± (213.64 - 215.16) ms215.20 ± (214.41 - 215.99) ms+0.4%✅⬆️
process.time_to_main_ms529.61 ± (528.24 - 530.98) ms532.43 ± (531.11 - 533.75) ms+0.5%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed48.05 ± (48.03 - 48.08) MB48.02 ± (48.00 - 48.05) MB-0.1%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)-0.8%✅
.NET 6 - Baseline
process.internal_duration_ms21.44 ± (21.39 - 21.50) ms21.32 ± (21.28 - 21.37) ms-0.6%✅
process.time_to_main_ms73.96 ± (73.64 - 74.27) ms74.38 ± (74.17 - 74.59) ms+0.6%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.60 ± (10.60 - 10.61) MB10.64 ± (10.64 - 10.64) MB+0.3%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 6 - Bailout
process.internal_duration_ms21.14 ± (21.10 - 21.17) ms21.02 ± (20.97 - 21.07) ms-0.5%✅
process.time_to_main_ms74.02 ± (73.83 - 74.20) ms74.23 ± (74.09 - 74.37) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.72 ± (10.72 - 10.72) MB10.75 ± (10.75 - 10.76) MB+0.3%✅⬆️
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms381.08 ± (379.32 - 382.84) ms382.09 ± (380.07 - 384.11) ms+0.3%✅⬆️
process.time_to_main_ms533.89 ± (532.41 - 535.36) ms535.27 ± (534.01 - 536.53) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed49.30 ± (49.27 - 49.33) MB49.42 ± (49.39 - 49.44) MB+0.2%✅⬆️
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)-0.1%✅
.NET 8 - Baseline
process.internal_duration_ms20.01 ± (19.94 - 20.07) ms19.83 ± (19.78 - 19.89) ms-0.9%✅
process.time_to_main_ms76.21 ± (75.87 - 76.54) ms74.50 ± (74.23 - 74.78) ms-2.2%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.65 ± (7.65 - 7.66) MB7.66 ± (7.66 - 7.67) MB+0.2%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 8 - Bailout
process.internal_duration_ms19.65 ± (19.61 - 19.70) ms19.89 ± (19.84 - 19.95) ms+1.2%✅⬆️
process.time_to_main_ms75.02 ± (74.79 - 75.25) ms77.05 ± (76.79 - 77.32) ms+2.7%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.70 ± (7.69 - 7.70) MB7.72 ± (7.71 - 7.72) MB+0.2%✅⬆️
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms299.25 ± (297.02 - 301.48) ms302.82 ± (300.24 - 305.41) ms+1.2%✅⬆️
process.time_to_main_ms494.09 ± (492.83 - 495.36) ms493.02 ± (491.75 - 494.29) ms-0.2%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed36.55 ± (36.53 - 36.58) MB36.58 ± (36.55 - 36.61) MB+0.1%✅⬆️
runtime.dotnet.threads.count27 ± (27 - 27)27 ± (27 - 27)-0.1%✅

HttpMessageHandler

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration200.47 ± (200.07 - 201.02) ms197.03 ± (197.03 - 197.99) ms-1.7%✅
.NET Framework 4.8 - Bailout
duration204.08 ± (203.73 - 204.86) ms202.37 ± (201.90 - 202.89) ms-0.8%✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1185.96 ± (1190.10 - 1198.87) ms1171.70 ± (1169.02 - 1178.55) ms-1.2%✅
.NET Core 3.1 - Baseline
process.internal_duration_ms193.63 ± (193.16 - 194.10) ms191.38 ± (190.91 - 191.85) ms-1.2%✅
process.time_to_main_ms84.96 ± (84.69 - 85.23) ms82.93 ± (82.64 - 83.22) ms-2.4%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed15.98 ± (15.96 - 16.00) MB16.16 ± (16.14 - 16.18) MB+1.1%✅⬆️
runtime.dotnet.threads.count20 ± (19 - 20)20 ± (20 - 20)+0.3%✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms194.81 ± (194.34 - 195.28) ms190.98 ± (190.46 - 191.50) ms-2.0%✅
process.time_to_main_ms86.72 ± (86.44 - 86.99) ms84.68 ± (84.46 - 84.89) ms-2.4%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.08 ± (16.06 - 16.09) MB16.14 ± (16.12 - 16.17) MB+0.4%✅⬆️
runtime.dotnet.threads.count21 ± (21 - 21)21 ± (21 - 21)+0.0%✅⬆️
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms384.75 ± (383.42 - 386.09) ms384.86 ± (383.67 - 386.05) ms+0.0%✅⬆️
process.time_to_main_ms521.21 ± (519.81 - 522.61) ms521.11 ± (519.94 - 522.27) ms-0.0%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed57.34 ± (57.14 - 57.53) MB57.26 ± (57.11 - 57.41) MB-0.1%✅
runtime.dotnet.threads.count30 ± (30 - 30)30 ± (30 - 30)+0.0%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms194.68 ± (194.24 - 195.12) ms196.91 ± (196.37 - 197.44) ms+1.1%✅⬆️
process.time_to_main_ms71.89 ± (71.68 - 72.09) ms72.58 ± (72.35 - 72.82) ms+1.0%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.42 ± (16.39 - 16.46) MB16.35 ± (16.32 - 16.38) MB-0.4%✅
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)-1.2%✅
.NET 6 - Bailout
process.internal_duration_ms195.48 ± (194.97 - 195.99) ms198.86 ± (198.38 - 199.35) ms+1.7%✅⬆️
process.time_to_main_ms73.26 ± (73.11 - 73.42) ms74.73 ± (74.48 - 74.98) ms+2.0%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.41 ± (16.39 - 16.44) MB16.34 ± (16.32 - 16.37) MB-0.4%✅
runtime.dotnet.threads.count20 ± (20 - 20)20 ± (20 - 20)+0.2%✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms595.79 ± (593.21 - 598.38) ms591.15 ± (588.34 - 593.96) ms-0.8%✅
process.time_to_main_ms520.33 ± (519.34 - 521.32) ms527.71 ± (526.68 - 528.75) ms+1.4%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed61.03 ± (60.93 - 61.13) MB61.07 ± (60.97 - 61.17) MB+0.1%✅⬆️
runtime.dotnet.threads.count31 ± (31 - 31)31 ± (31 - 31)+0.4%✅⬆️
.NET 8 - Baseline
process.internal_duration_ms193.76 ± (193.28 - 194.24) ms197.23 ± (196.71 - 197.76) ms+1.8%✅⬆️
process.time_to_main_ms71.33 ± (71.15 - 71.51) ms72.79 ± (72.59 - 73.00) ms+2.0%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.71 ± (11.69 - 11.73) MB11.66 ± (11.65 - 11.68) MB-0.4%✅
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)+0.5%✅⬆️
.NET 8 - Bailout
process.internal_duration_ms193.68 ± (193.20 - 194.16) ms197.19 ± (196.73 - 197.66) ms+1.8%✅⬆️
process.time_to_main_ms72.78 ± (72.60 - 72.96) ms74.36 ± (74.14 - 74.58) ms+2.2%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.77 ± (11.75 - 11.79) MB11.72 ± (11.71 - 11.74) MB-0.4%✅
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)+0.3%✅⬆️
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms522.11 ± (518.98 - 525.23) ms520.27 ± (517.09 - 523.45) ms-0.4%✅
process.time_to_main_ms482.46 ± (481.29 - 483.63) ms487.54 ± (486.57 - 488.50) ms+1.1%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed50.33 ± (50.30 - 50.37) MB50.30 ± (50.26 - 50.34) MB-0.1%✅
runtime.dotnet.threads.count30 ± (30 - 30)30 ± (30 - 30)+0.1%✅⬆️
Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts
FakeDbCommand (.NET Framework 4.8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (74ms)  : 71, 78
    master - mean (74ms)  : 70, 79

    section Bailout
    This PR (8560) - mean (80ms)  : 76, 84
    master - mean (79ms)  : 75, 83

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (1,086ms)  : 1016, 1155
    master - mean (1,080ms)  : 1031, 1129
Loading
FakeDbCommand (.NET Core 3.1) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (119ms)  : 111, 126
    master - mean (117ms)  : 111, 123

    section Bailout
    This PR (8560) - mean (116ms)  : 111, 121
    master - mean (117ms)  : 111, 123

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (787ms)  : 755, 819
    master - mean (782ms)  : 748, 816
Loading
FakeDbCommand (.NET 6) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (102ms)  : 98, 106
    master - mean (102ms)  : 95, 109

    section Bailout
    This PR (8560) - mean (102ms)  : 99, 104
    master - mean (101ms)  : 98, 105

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (945ms)  : 915, 974
    master - mean (945ms)  : 907, 983
Loading
FakeDbCommand (.NET 8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (103ms)  : 96, 109
    master - mean (104ms)  : 98, 111

    section Bailout
    This PR (8560) - mean (105ms)  : 100, 110
    master - mean (102ms)  : 98, 107

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (828ms)  : 777, 879
    master - mean (825ms)  : 790, 861
Loading
HttpMessageHandler (.NET Framework 4.8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (198ms)  : 191, 204
    master - mean (201ms)  : 194, 207

    section Bailout
    This PR (8560) - mean (202ms)  : 196, 209
    master - mean (204ms)  : 198, 211

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (1,174ms)  : 1125, 1222
    master - mean (1,194ms)  : 1132, 1257
Loading
HttpMessageHandler (.NET Core 3.1) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (283ms)  : 275, 292
    master - mean (287ms)  : 278, 296

    section Bailout
    This PR (8560) - mean (285ms)  : 275, 294
    master - mean (291ms)  : 282, 300

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (939ms)  : 910, 968
    master - mean (943ms)  : 916, 970
Loading
HttpMessageHandler (.NET 6) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (278ms)  : 271, 285
    master - mean (275ms)  : 268, 282

    section Bailout
    This PR (8560) - mean (282ms)  : 274, 291
    master - mean (278ms)  : 267, 288

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (1,149ms)  : 1097, 1201
    master - mean (1,146ms)  : 1104, 1187
Loading
HttpMessageHandler (.NET 8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8560) - mean (280ms)  : 268, 293
    master - mean (275ms)  : 266, 284

    section Bailout
    This PR (8560) - mean (282ms)  : 271, 293
    master - mean (276ms)  : 267, 286

    section CallTarget+Inlining+NGEN
    This PR (8560) - mean (1,040ms)  : 987, 1092
    master - mean (1,037ms)  : 981, 1092
Loading

@igoragoli
igoragoli self-requested a review May 5, 2026 12:13
@andrewlock
andrewlock merged commit 0e56039 into master May 5, 2026
72 of 75 checks passed
@andrewlock
andrewlock deleted the andrew/dont-run-microbenchmarks-on-tag-refs branch May 5, 2026 12:31
@github-actions github-actions Bot added this to the vNext-v3 milestone May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bouwkast bouwkast bouwkast approved these changes
@igoragoli igoragoli igoragoli approved these changes

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

3.44.0

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewlock @bouwkast @igoragoli