Skip to content

[Perf] Changes at 5/30/2022 5:18:40 PM #5787

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline b8acc97aba7662204c2a6de8f6090fc115a41c9f
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IntegerFormatting - Duration of single invocation 1.96 ms 2.24 ms 1.14 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'
Details

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.IntegerFormatting


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2445433035714286 > 2.05787180859375.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/5/2022 12:13:02 AM, 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -31.794415009522343 (T) = (0 -2150774.6438138555) / Math.Sqrt((231965803.70055684 / (19)) + (1310325903.1362696 / (51))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (51) - 2, .025) and -0.1001206914351741 = (1955035.1707393483 - 2150774.6438138555) / 1955035.1707393483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.TryAddDefaultSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 18.49 μs 22.01 μs 1.19 0.02 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<String>.Dictionary(Count: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.014632248939176 > 19.606222038646766.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -76.04651819441867 (T) = (0 -21817.091876302562) / Math.Sqrt((21454.01419545054 / (20)) + (37316.649045338825 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.1737967970964892 = (18586.770666157423 - 21817.091876302562) / 18586.770666157423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 15.35 μs 17.91 μs 1.17 0.00 True
Dictionary - Duration of single invocation 13.89 μs 16.35 μs 1.18 0.01 True

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.IDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.905676313628902 > 16.014186695399744.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -60.67027082319675 (T) = (0 -17966.84472332574) / Math.Sqrt((16469.283954824245 / (18)) + (63241.25190644975 / (51))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (51) - 2, .025) and -0.18589741147856528 = (15150.420727307983 - 17966.84472332574) / 15150.420727307983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.AddGivenSize&lt;String&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.351192977190877 > 14.618129035436647.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -54.199347225226795 (T) = (0 -16177.87918644145) / Math.Sqrt((26021.973127996273 / (20)) + (18648.980065240197 / (48))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (48) - 2, .025) and -0.1597032457205982 = (13950.01630472207 - 16177.87918644145) / 13950.01630472207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 9.33 μs 10.72 μs 1.15 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.HashSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.718847039852571 > 9.811075103393149.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -52.832645303883865 (T) = (0 -10544.568412730408) / Math.Sqrt((3235.166120662894 / (20)) + (18832.796726404675 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.13053147830814618 = (9327.089616744224 - 10544.568412730408) / 9327.089616744224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Order00ManualX - Duration of single invocation 122.27 ms 129.28 ms 1.06 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'LinqBenchmarks*'
Details

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Order00ManualX


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.27875333333333 > 128.488433625.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -38.54454597589811 (T) = (0 -129277613.36457662) / Math.Sqrt((231819744706.29404 / (20)) + (1110008040841.8252 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.0575894217662071 = (122237997.7559524 - 129277613.36457662) / 122237997.7559524 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 19.36 μs 22.62 μs 1.17 0.04 True
IDictionary - Duration of single invocation 20.62 μs 24.00 μs 1.16 0.02 True

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.623560443490696 > 20.191535295976607.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -84.93380295392784 (T) = (0 -22420.502256933494) / Math.Sqrt((18579.536076080913 / (20)) + (24002.975889861234 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.16512522997957174 = (19242.99781691844 - 22420.502256933494) / 19242.99781691844 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.IDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.003170979198377 > 21.845748339052605.
IsChangePoint: Marked as a change because one of 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -76.3413935723831 (T) = (0 -24087.450989606063) / Math.Sqrt((29234.130589785927 / (19)) + (18528.67400338344 / (51))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (51) - 2, .025) and -0.16038785568230213 = (20758.103311450777 - 24087.450989606063) / 20758.103311450777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline b8acc97aba7662204c2a6de8f6090fc115a41c9f
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSpan - Duration of single invocation 19.45 ns 21.30 ns 1.09 0.14 False
ToString - Duration of single invocation 11.50 ns 13.96 ns 1.21 0.05 False
TryParse - Duration of single invocation 19.41 ns 21.27 ns 1.10 0.17 False
TryParseSpan - Duration of single invocation 20.37 ns 21.65 ns 1.06 0.11 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.297973846199817 > 20.95257439112756.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -6.932536030439059 (T) = (0 -21.5083861878528) / Math.Sqrt((0.6564661507190672 / (17)) + (0.17676872419537829 / (51))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (51) - 2, .025) and -0.07080106497489114 = (20.08625774793859 - 21.5083861878528) / 20.08625774793859 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.955535182583363 > 12.082863739364575.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -25.59922207220245 (T) = (0 -13.186056366785758) / Math.Sqrt((0.042033514740139194 / (19)) + (0.143458778258278 / (49))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (49) - 2, .025) and -0.16169058470401518 = (11.350747385238908 - 13.186056366785758) / 11.350747385238908 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParse(value: "2147483647")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.27410359821035 > 20.662531878769503.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -11.160697462911576 (T) = (0 -21.689166409317608) / Math.Sqrt((0.22448732300051827 / (19)) + (0.609357946180391 / (49))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (49) - 2, .025) and -0.08711409206618106 = (19.9511408854014 - 21.689166409317608) / 19.9511408854014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.651556514048156 > 21.318145455921517.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -16.064004845944105 (T) = (0 -21.978157532630817) / Math.Sqrt((0.07553868602637497 / (20)) + (0.4237948664590537 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.08737662968191269 = (20.212092970087124 - 21.978157532630817) / 20.212092970087124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileNameWithoutExtension - Duration of single invocation 32.69 ns 36.82 ns 1.13 0.14 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.819989165296384 > 34.261201436630564.
IsChangePoint: Marked as a change because one of 5/18/2022 10:04:15 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -9.245960335070562 (T) = (0 -34.785927806049415) / Math.Sqrt((0.48447229954819165 / (20)) + (0.9848030984694544 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.05871358680973716 = (32.856787935319886 - 34.785927806049415) / 32.856787935319886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.ContainsKeyTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 13.89 μs 15.53 μs 1.12 0.01 False
Dictionary - Duration of single invocation 12.04 μs 13.38 μs 1.11 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;String, String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<String, String>.IDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.530474950396822 > 14.644715700310952.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -62.261703004205714 (T) = (0 -15686.342640913996) / Math.Sqrt((9731.109345862587 / (20)) + (11740.54235384263 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.11889885384533469 = (14019.446518337678 - 15686.342640913996) / 14019.446518337678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsKeyTrue&lt;String, String&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.38169421199442 > 12.862822234835157.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -14.711596190993387 (T) = (0 -13197.864345229698) / Math.Sqrt((67869.52462272876 / (20)) + (35258.20875269455 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.07670791264853136 = (12257.60876296064 - 13197.864345229698) / 12257.60876296064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 29.39 μs 37.16 μs 1.26 0.28 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.Queue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.15676744186047 > 30.79484945575843.
IsChangePoint: Marked as a change because one of 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -33.69542401971071 (T) = (0 -34857.750371122194) / Math.Sqrt((3268.0609434186053 / (20)) + (1361806.300463338 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.1882715156665602 = (29334.836282403656 - 34857.750371122194) / 29334.836282403656 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline b8acc97aba7662204c2a6de8f6090fc115a41c9f
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 13.94 μs 16.64 μs 1.19 0.01 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.64486629353234 > 14.601452914211775.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -51.97652139372502 (T) = (0 -16120.685826535544) / Math.Sqrt((21362.402910936285 / (17)) + (32831.85277932331 / (51))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (51) - 2, .025) and -0.16354023342335597 = (13854.858958426757 - 16120.685826535544) / 13854.858958426757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 9380113a892ec86d147e6e7506a10eda407a82be
Compare 842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 8.94 ns 10.59 ns 1.18 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: 127)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.585584798463788 > 9.350837157895985.
IsChangePoint: Marked as a change because one of 4/15/2022 9:40:04 PM, 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -51.64417415095494 (T) = (0 -10.891277835901503) / Math.Sqrt((0.012250811866823409 / (20)) + (0.05063938126004611 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.23634923581694084 = (8.809224384487841 - 10.891277835901503) / 8.809224384487841 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions