Skip to content

[Perf] Windows/x64: 48 Improvements on 6/29/2023 8:27:03 AM #19475

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 199.22 ns 81.09 ns 0.41 0.01 True
Enumerate - Duration of single invocation 28.15 ns 12.45 ns 0.44 0.14 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'
Details

Payloads

Baseline
Compare

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 100)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 81.0854398276447 < 188.25762400603116.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 639.2825077013174 (T) = (0 -81.65687793261336) / Math.Sqrt((5.908875940609882 / (299)) + (0.20801261584998912 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.5894389719431751 = (198.8909622501028 - 81.65687793261336) / 198.8909622501028 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 10)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.446350194161713 < 26.699421965470638.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 57.397709660033364 (T) = (0 -12.685120233342923) / Math.Sqrt((1.4262825693768768 / (299)) + (1.1015109638774356 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.5585696386405851 = (28.73640180588899 - 12.685120233342923) / 28.73640180588899 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_CancellationToken

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateManyRegisterDispose - Duration of single invocation 44.91 ns 38.34 ns 0.85 0.18 False
CreateManyRegisterMultipleDispose - Duration of single invocation 219.55 ns 191.33 ns 0.87 0.11 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_CancellationToken*'
Details

Payloads

Baseline
Compare

System.Threading.Tests.Perf_CancellationToken.CreateManyRegisterDispose

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 38.34333888888889 < 42.27461228125001.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 13.212575170928858 (T) = (0 -38.194493826043235) / Math.Sqrt((3.9526606038060206 / (299)) + (1.2844161863593375 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.09809156600983844 = (42.34852717483278 - 38.194493826043235) / 42.34852717483278 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.Threading.Tests.Perf_CancellationToken.CreateManyRegisterMultipleDispose

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 191.3283933333333 < 208.7500396190476.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 5:12:40 PM, 4/28/2023 10:42:00 AM, 5/13/2023 1:49:08 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 40.72222661110352 (T) = (0 -193.56961519169718) / Math.Sqrt((86.01942922842348 / (299)) + (2.9754631691519506 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.12790837429453797 = (221.96018111641962 - 193.56961519169718) / 221.96018111641962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateProperties - Duration of single invocation 1.02 μs 479.60 ns 0.47 0.01 True
EnumerateProperties - Duration of single invocation 2.08 μs 1.45 μs 0.70 0.00 True
EnumerateProperties - Duration of single invocation 1.02 μs 480.88 ns 0.47 0.00 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'
Details

Payloads

Baseline
Compare

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 479.59865800139073 < 0.972198328459668.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 909.9884364825626 (T) = (0 -479.8824903537195) / Math.Sqrt((91.75726596136337 / (299)) + (0.6658729909609831 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.5291652162874901 = (1019.2163088926203 - 479.8824903537195) / 1019.2163088926203 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4453885698406819 < 1.9763320305437841.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 296.02643552095316 (T) = (0 -1448.9874526178944) / Math.Sqrt((247.7862560135117 / (299)) + (52.3820022069286 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2980825284637121 = (2064.327376616646 - 1448.9874526178944) / 2064.327376616646 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 480.87981364242205 < 0.9719399084960358.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 1030.641015264904 (T) = (0 -480.0645930103845) / Math.Sqrt((62.45529053162823 / (299)) + (0.9647077085185314 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.528775611181573 = (1018.7600735482389 - 480.0645930103845) / 1018.7600735482389 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Struct.FilteredSpanEnumerator

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Sum - Duration of single invocation 17.03 μs 7.03 μs 0.41 0.02 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Struct.FilteredSpanEnumerator*'
Details

Payloads

Baseline
Compare

Struct.FilteredSpanEnumerator.Sum

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.032960137003924 < 16.19939105112843.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 341.1317423470216 (T) = (0 -7039.320824099742) / Math.Sqrt((202648.92908929157 / (232)) + (127.58784821535383 / (15))) is greater than 1.9696939205433748 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (232) + (15) - 2, .975) and 0.5900310209587765 = (17170.374306276273 - 7039.320824099742) / 17170.374306276273 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateArray - Duration of single invocation 657.72 ns 479.14 ns 0.73 0.01 True
EnumerateArray - Duration of single invocation 660.44 ns 479.69 ns 0.73 0.01 True
EnumerateArray - Duration of single invocation 1.66 μs 1.34 μs 0.81 0.00 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
Details

Payloads

Baseline
Compare

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 479.13576201490616 < 624.1728943657387.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 260.7537607222407 (T) = (0 -479.63971231736883) / Math.Sqrt((131.67340357488578 / (299)) + (0.5321016508308727 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2727366157929472 = (659.5130770131208 - 479.63971231736883) / 659.5130770131208 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 479.6858467650631 < 625.0297142933442.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 316.96646356742156 (T) = (0 -479.695762452878) / Math.Sqrt((92.0165253883624 / (299)) + (0.2067861057199242 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2725579365870461 = (659.428133977131 - 479.695762452878) / 659.428133977131 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: Json400KB)

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3414274674321043 < 1.5648922071877085.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 202.04195198958192 (T) = (0 -1346.5090317307015) / Math.Sqrt((207.20946018729703 / (299)) + (24.23361603317301 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.18565736061272906 = (1653.4919904769374 - 1346.5090317307015) / 1653.4919904769374 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Pipelines.Tests.Perf_Pipe

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SyncReadAsync - Duration of single invocation 933.13 μs 761.38 μs 0.82 0.05 False
SyncReadAsyncWithCancellationToken - Duration of single invocation 940.48 μs 761.00 μs 0.81 0.11 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Pipelines.Tests.Perf_Pipe*'
Details

Payloads

Baseline
Compare

System.IO.Pipelines.Tests.Perf_Pipe.SyncReadAsync

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 761.3833333333332 < 903.3601610576923.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 86.75657683796594 (T) = (0 -759982.8434065935) / Math.Sqrt((1997962073.7765896 / (299)) + (13026593.667323621 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2387805103274014 = (998375.4406149836 - 759982.8434065935) / 998375.4406149836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.IO.Pipelines.Tests.Perf_Pipe.SyncReadAsyncWithCancellationToken

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 760.9981349206348 < 893.282663252801.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 74.54030336180337 (T) = (0 -762061.230489418) / Math.Sqrt((2526978071.56 / (299)) + (11969176.07003412 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.22927510878600874 = (988759.0749652229 - 762061.230489418) / 988759.0749652229 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where01ForX - Duration of single invocation 157.28 ms 119.88 ms 0.76 0.15 False
Count00ForX - Duration of single invocation 164.00 ms 76.57 ms 0.47 0.02 False
Where00ForX - Duration of single invocation 218.52 ms 144.00 ms 0.66 0.11 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
Details

Payloads

Baseline
Compare

LinqBenchmarks.Where01ForX

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.88155416666667 < 149.55379275.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 24.156864869631 (T) = (0 -124736123.27777776) / Math.Sqrt((424568548333425.9 / (299)) + (96751099905947.67 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.35203624175892445 = (192504783.93479773 - 124736123.27777776) / 192504783.93479773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

LinqBenchmarks.Count00ForX

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 76.56605875 < 155.5592123214286.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 54.99807597652045 (T) = (0 -76109601.91666667) / Math.Sqrt((462688140151816.75 / (299)) + (28762680853.764885 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4735367022160132 = (144567726.25372112 - 76109601.91666667) / 144567726.25372112 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

LinqBenchmarks.Where00ForX

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 143.99634230769232 < 208.40371544871795.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 197.03953954783645 (T) = (0 -142800025.63614163) / Math.Sqrt((36246175142230.32 / (299)) + (593603939540.9174 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3562120555613194 = (221812208.30509514 - 142800025.63614163) / 221812208.30509514 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline cfae69f62414b68e4c98dd3b294154baabf58a5a
Compare d4fcdc1240ec278980b4e63dda9769b76c1c802a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDLogicArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 420.78 ms 354.44 ms 0.84 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDLogicArray*'
Details

Payloads

Baseline
Compare

Benchstone.MDBenchI.MDLogicArray.Test

Histogram


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 354.4380785714286 < 400.23083461538465.
IsChangePoint: Marked as a change because one of 6/29/2023 3:17:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsImprovementStdDev: Marked as improvement because 459.9879912501402 (T) = (0 -353457504.94505495) / Math.Sqrt((339105178379.4533 / (299)) + (305552319626.7799 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.16025736582602082 = (420911706.2309654 - 353457504.94505495) / 420911706.2309654 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions