Skip to content

[Perf] Changes at 1/5/2022 11:37:43 PM #2977

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 70065c1b180ebe66dcf68ce03a47e54c23135d79
Compare abb7df59eb4f8c813c07ddd6f46f94e59604577d
Diff Diff

Improvements in System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentStack - Duration of single invocation 271.02 ms 206.88 ms 0.76 0.01 True
ConcurrentBag - Duration of single invocation 375.89 ms 348.20 ms 0.93 0.00 True

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.AddRemoveFromSameThreads&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>.ConcurrentStack(Size: 2000000)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 16.67948171180381 (T) = (0 -215443465.83333334) / Math.Sqrt((10546042819398.842 / (299)) + (95493177328510.33 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.21127829658274336 = (273155239.5476019 - 215443465.83333334) / 273155239.5476019 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.Concurrent.AddRemoveFromSameThreads&lt;Int32&gt;.ConcurrentBag(Size: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 79.06356046101595 (T) = (0 -348663795.66620874) / Math.Sqrt((2438879688158.7295 / (299)) + (970959441663.8026 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07545326945332011 = (377118629.2120092 - 348663795.66620874) / 377118629.2120092 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 70065c1b180ebe66dcf68ce03a47e54c23135d79
Compare abb7df59eb4f8c813c07ddd6f46f94e59604577d
Diff Diff

Improvements in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 12.59 ns 5.72 ns 0.45 0.07 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<Int32>.Queue(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 176.103931468525 (T) = (0 -5.654123445769828) / Math.Sqrt((0.08495375148919211 / (299)) + (0.010836976491446562 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5576864764089658 = (12.7830670874935 - 5.654123445769828) / 12.7830670874935 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 70065c1b180ebe66dcf68ce03a47e54c23135d79
Compare abb7df59eb4f8c813c07ddd6f46f94e59604577d
Diff Diff

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentBag - Duration of single invocation 27.69 μs 22.26 μs 0.80 0.01 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ConcurrentBag(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 12.787278352807247 (T) = (0 -19400.039240822825) / Math.Sqrt((20194.143552133308 / (299)) + (3355232.232347668 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.2991803402211076 = (27681.92782568844 - 19400.039240822825) / 27681.92782568844 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 70065c1b180ebe66dcf68ce03a47e54c23135d79
Compare abb7df59eb4f8c813c07ddd6f46f94e59604577d
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CastToSameType - Duration of single invocation 924.66 ns 721.65 ns 0.78 0.19 False
Reverse - Duration of single invocation 1.76 μs 1.52 μs 0.87 0.12 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 11/13/2021 7:30:09 PM, 11/14/2021 9:27:23 PM, 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 17.367621436107566 (T) = (0 -735.6843592021042) / Math.Sqrt((3382.2786564589946 / (299)) + (1106.5902015446243 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.22407213142223642 = (948.134986504063 - 735.6843592021042) / 948.134986504063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/7/2021 2:09:29 PM, 12/8/2021 2:58:05 AM, 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 32.919980304971006 (T) = (0 -1524.822385350095) / Math.Sqrt((3594.218187031065 / (299)) + (353.76514803907315 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.13934672045501095 = (1771.7034508440372 - 1524.822385350095) / 1771.7034508440372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 70065c1b180ebe66dcf68ce03a47e54c23135d79
Compare abb7df59eb4f8c813c07ddd6f46f94e59604577d
Diff Diff

Improvements in System.Collections.Concurrent.Count<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 13.46 ns 7.23 ns 0.54 0.04 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<String>.Queue(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 162.30410568234697 (T) = (0 -7.422319921367619) / Math.Sqrt((0.08810113342015431 / (299)) + (0.008679943105738509 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.448190567333139 = (13.450875396413585 - 7.422319921367619) / 13.450875396413585 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions