Skip to content

[Perf] Changes at 1/28/2022 8:15:35 PM #3288

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSpan - Duration of single invocation 15.80 ns 13.57 ns 0.86 0.12 False
TryParseSpan - Duration of single invocation 13.49 ns 8.93 ns 0.66 0.32 False
ParseSpan - Duration of single invocation 16.95 ns 14.93 ns 0.88 0.13 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.TryParseSpan(value: "12345")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.57198980478645 < 15.136921797878385.
IsChangePoint: Marked as a change because one of 12/1/2021 5:00:56 AM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 46.50701030165099 (T) = (0 -13.442381428889341) / Math.Sqrt((0.38314252487873385 / (299)) + (0.049280309582014285 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1772050337667345 = (16.337461920104133 - 13.442381428889341) / 16.337461920104133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParseSpan(value: "4")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.928372373507624 < 13.349719585273247.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 21.41176964899555 (T) = (0 -9.077825039456101) / Math.Sqrt((0.8629252120821471 / (299)) + (0.9745744005674221 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.354430433990348 = (14.061730164213438 - 9.077825039456101) / 14.061730164213438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ParseSpan(value: "12345")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.931923243907843 < 16.488191704358997.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 18.45101456747729 (T) = (0 -15.06022408393209) / Math.Sqrt((0.33959652471537805 / (290)) + (0.36523189951556695 / (19))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (19) - 2, .975) and 0.14890679896884215 = (17.695152617463744 - 15.06022408393209) / 17.695152617463744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseSpan - Duration of single invocation 53.43 ns 39.28 ns 0.74 0.09 False
TryFormat - Duration of single invocation 43.51 ns 40.72 ns 0.94 0.08 False
TryFormat - Duration of single invocation 53.61 ns 48.48 ns 0.90 0.04 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.276833772199524 < 50.37622743938575.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 48.311998319847184 (T) = (0 -38.78581504872801) / Math.Sqrt((8.891154721793622 / (299)) + (0.6309443843050568 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2380989517870811 = (50.90663038159914 - 38.78581504872801) / 50.90663038159914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.7174917650661 < 40.74324344985749.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 28.696037238821496 (T) = (0 -40.45205498806371) / Math.Sqrt((0.45315285876351463 / (299)) + (0.13360276391357814 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.061547144944071874 = (43.10504760055626 - 40.45205498806371) / 43.10504760055626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.483813708571105 < 51.041635543388594.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 65.51161864393747 (T) = (0 -48.35760846966307) / Math.Sqrt((1.4799988208975405 / (299)) + (0.04305271071301033 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1032022348301541 = (53.92253454211558 - 48.35760846966307) / 53.92253454211558 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseDouble - Duration of single invocation 37.26 ns 33.83 ns 0.91 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 12345)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.82593066045887 < 35.5231194306928.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 39.91152045353655 (T) = (0 -33.783795631910245) / Math.Sqrt((0.5774628104576742 / (299)) + (0.07990026315970837 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.08470597391131726 = (36.910320256626406 - 33.783795631910245) / 36.910320256626406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 13.98 ns 9.01 ns 0.64 0.29 False
Parse - Duration of single invocation 17.33 ns 14.86 ns 0.86 0.17 False
TryParse - Duration of single invocation 17.69 ns 14.67 ns 0.83 0.14 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "0")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.012860768845778 < 13.069816306309837.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 11.421625933158522 (T) = (0 -9.57096758774044) / Math.Sqrt((1.0911324778247968 / (299)) + (2.777403935568905 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.3159692516415043 = (13.992013678783287 - 9.57096758774044) / 13.992013678783287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.Parse(value: "12345")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.860342353088381 < 16.50998045354716.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 42.50508869317527 (T) = (0 -14.893894451044655) / Math.Sqrt((0.4029757370540505 / (299)) + (0.0690240491099256 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1676421155201933 = (17.89361851285015 - 14.893894451044655) / 17.89361851285015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.TryParse(value: "12345")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.670682625935463 < 16.568873488750267.
IsChangePoint: Marked as a change because one of 12/20/2021 4:36:55 PM, 12/21/2021 5:09:01 AM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 48.42046359340322 (T) = (0 -14.624546053254868) / Math.Sqrt((0.8491265757257618 / (299)) + (0.027534850950641173 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.17819582243514712 = (17.795657959040696 - 14.624546053254868) / 17.795657959040696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 17.30 ns 14.81 ns 0.86 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.Parse(value: "-32768")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.808562436311604 < 16.507206683395175.
IsChangePoint: Marked as a change because one of 11/30/2021 9:55:21 AM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 17.02508531818762 (T) = (0 -14.996209613305112) / Math.Sqrt((0.459320359883588 / (299)) + (0.5041409469958226 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.1598106055405511 = (17.848606174031982 - 14.996209613305112) / 17.848606174031982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateUsingIndexer - Duration of single invocation 177.84 μs 166.20 μs 0.93 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: Json400KB)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 166.1993085106383 < 168.92923493953964.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 37.458129152689025 (T) = (0 -166131.46261904872) / Math.Sqrt((83083404.99510379 / (299)) + (11562.5950566131 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.10633265998903615 = (185898.54992016443 - 166131.46261904872) / 185898.54992016443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 14.25 ns 11.79 ns 0.83 0.18 False
Parse - Duration of single invocation 14.10 ns 11.50 ns 0.82 0.16 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.TryParse(value: "-128")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.78968788195822 < 13.548142905359036.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 49.69740854479627 (T) = (0 -11.762139710225062) / Math.Sqrt((0.5244717094070955 / (299)) + (0.025901429805679374 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.19087664010680838 = (14.536893004520008 - 11.762139710225062) / 14.536893004520008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_SByte.Parse(value: "127")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.496500456581057 < 13.44978908245981.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 65.79936609143762 (T) = (0 -11.525760625219105) / Math.Sqrt((0.2925815638083676 / (299)) + (0.01595331550106696 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.19577177669430135 = (14.33145504126134 - 11.525760625219105) / 14.33145504126134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringL - Duration of single invocation 113.67 ns 107.60 ns 0.95 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.ToStringL


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 107.60416441657127 < 108.44540650731831.
IsChangePoint: Marked as a change because one of 11/4/2021 5:49:01 AM, 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 36.200267174445734 (T) = (0 -107.58609567303097) / Math.Sqrt((9.524004886493891 / (299)) + (0.05802395920610269 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.05914692308565703 = (114.34951780768402 - 107.58609567303097) / 114.34951780768402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 11bf8a782d27dd0f614dd0da03b27ab025933320
Compare 8471eecd98e5493e3fbc8bcc34662310ef8762eb
Diff Diff

Improvements in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 13.77 ns 10.05 ns 0.73 0.25 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.TryParse(value: "0")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.050027531844018 < 12.965752751623338.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 29.752725387090074 (T) = (0 -9.015785079208442) / Math.Sqrt((1.007383785390198 / (299)) + (0.42111861816881907 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.34525969147026836 = (13.77001684752549 - 9.015785079208442) / 13.77001684752549 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions