[Perf] Windows/arm64: 13 Regressions on 2/9/2023 7:28:55 PM #13055
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MultipleSerial - Duration of single invocation | 12.65 μs | 13.90 μs | 1.10 | 0.08 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Byte>*'Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.903131133333334 > 13.28656064346154.
IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 1/13/2023 9:23:19 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -8.47746395354439 (T) = (0 -14086.452661076604) / Math.Sqrt((201431.54204090036 / (15)) + (246586.0123967142 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.10317288185797452 = (12769.034566324784 - 14086.452661076604) / 12769.034566324784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Collections.TryAddDefaultSize<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 54.85 μs | 79.78 μs | 1.45 | 0.16 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<Int32>*'Payloads
Histogram
System.Collections.TryAddDefaultSize<Int32>.ConcurrentDictionary(Count: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.78088329081633 > 51.851510886955.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -36.02268248256153 (T) = (0 -81885.08173008244) / Math.Sqrt((7979602.18205169 / (15)) + (5263894.918123484 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.6226461805012524 = (50463.916726927666 - 81885.08173008244) / 50463.916726927666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.MathBenchmarks.Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Min - Duration of single invocation | 3.20 μs | 3.85 μs | 1.20 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'Payloads
Histogram
System.MathBenchmarks.Single.Min
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.85335250102501 > 3.363071201700767.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -72.16253197026325 (T) = (0 -3790.2917219222486) / Math.Sqrt((9.384504131651406 / (15)) + (1503.306974448837 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.1829508251283307 = (3204.0991403941616 - 3790.2917219222486) / 3204.0991403941616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in Microsoft.Extensions.DependencyInjection.GetService
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Transient - Duration of single invocation | 34.07 ns | 38.67 ns | 1.14 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'Payloads
Histogram
Microsoft.Extensions.DependencyInjection.GetService.Transient
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.674621151813895 > 36.37125237793037.
IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -5.8445217760361325 (T) = (0 -36.86058371962873) / Math.Sqrt((0.5798666611888657 / (15)) + (2.553031864730718 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.0653476189677739 = (34.59958333162965 - 36.86058371962873) / 34.59958333162965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Collections.CtorDefaultSize<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 129.41 ns | 146.60 ns | 1.13 | 0.16 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize<String>*'Payloads
Histogram
System.Collections.CtorDefaultSize<String>.ConcurrentDictionary
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 146.60192297941384 > 136.34057449634733.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -16.91327165355278 (T) = (0 -145.59783764067862) / Math.Sqrt((6.363754118412196 / (15)) + (9.069585224152616 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.11744346072528512 = (130.2954849690357 - 145.59783764067862) / 130.2954849690357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Tests.Perf_String
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Substring_Int - Duration of single invocation | 11.43 ns | 12.56 ns | 1.10 | 0.43 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'Payloads
Histogram
System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.557327310527013 > 11.955239775965355.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -5.529651190333107 (T) = (0 -12.506627675131508) / Math.Sqrt((0.32943428442269146 / (15)) + (0.21087154377118342 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.08461069007213323 = (11.530983227078226 - 12.506627675131508) / 11.530983227078226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in IfStatements.IfStatements
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Or - Duration of single invocation | 67.99 μs | 72.66 μs | 1.07 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'Payloads
Histogram
IfStatements.IfStatements.Or
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.65589669051879 > 71.42322012959868.
IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -143.00316770472034 (T) = (0 -72862.00285237939) / Math.Sqrt((1165.0564972977688 / (15)) + (24610.159540116543 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.07122547494528153 = (68017.42915617386 - 72862.00285237939) / 68017.42915617386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Collections.CreateAddAndClear<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 51.02 μs | 81.56 μs | 1.60 | 0.13 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'Payloads
Histogram
System.Collections.CreateAddAndClear<Int32>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.5626402393418 > 53.58591798155739.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -221.11291489710928 (T) = (0 -82609.4611889647) / Math.Sqrt((100966.57032097099 / (15)) + (317070.2721605625 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.6217660531301649 = (50937.96422086927 - 82609.4611889647) / 50937.96422086927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Collections.TryGetValueTrue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SortedDictionary - Duration of single invocation | 39.46 μs | 51.31 μs | 1.30 | 0.17 | False | |||||
| ConcurrentDictionary - Duration of single invocation | 5.65 μs | 6.10 μs | 1.08 | 0.11 | False |
_1.png)
_2.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'Payloads
Histogram
System.Collections.TryGetValueTrue<Int32, Int32>.SortedDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.31199669966997 > 41.368352444700854.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/13/2023 8:11:05 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.676804284881521 (T) = (0 -45275.963792007744) / Math.Sqrt((200855.31815783793 / (15)) + (15221451.44989292 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.13788360020391527 = (39789.62679828941 - 45275.963792007744) / 39789.62679828941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.TryGetValueTrue<Int32, Int32>.ConcurrentDictionary(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.09959033203125 > 5.810919315717753.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 12/8/2022 9:50:09 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.294417388317767 (T) = (0 -6197.963101804502) / Math.Sqrt((1172.3821030629772 / (15)) + (251904.92125774568 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.11939570563804007 = (5536.883043759533 - 6197.963101804502) / 5536.883043759533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Collections.CtorGivenSize<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 313.30 ns | 343.91 ns | 1.10 | 0.15 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<Int32>*'Payloads
Histogram
System.Collections.CtorGivenSize<Int32>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 343.9138301648655 > 333.1327777428095.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -7.300883956938807 (T) = (0 -348.0233671101925) / Math.Sqrt((116.67708584069996 / (15)) + (89.3220234621484 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.07716809138919987 = (323.09104762038993 - 348.0233671101925) / 323.09104762038993 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in System.Collections.CtorDefaultSize<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 116.69 ns | 132.17 ns | 1.13 | 0.25 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize<Int32>*'Payloads
Histogram
System.Collections.CtorDefaultSize<Int32>.ConcurrentDictionary
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.17113586592689 > 123.19949209542536.
IsChangePoint: Marked as a change because one of 11/4/2022 6:59:29 PM, 11/6/2022 4:58:03 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.706073118612964 (T) = (0 -130.54204983236764) / Math.Sqrt((8.370513864298212 / (15)) + (50.723413996048755 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.09336989054337658 = (119.39422418838662 - 130.54204983236764) / 119.39422418838662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Regressions in Span.IndexerBench
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteViaIndexer2 - Duration of single invocation | 829.27 ns | 884.96 ns | 1.07 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'Payloads
Histogram
Span.IndexerBench.WriteViaIndexer2(length: 1024)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 884.9602466705624 > 869.8033201657225.
IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -12.664032337961238 (T) = (0 -879.2029005585366) / Math.Sqrt((2.241603467683853 / (15)) + (349.47773767698874 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.05979659797103794 = (829.5958887221899 - 879.2029005585366) / 829.5958887221899 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository