Skip to content

[Perf] Changes at 4/25/2022 3:15:54 PM #4884

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetChars - Duration of single invocation 98.26 ns 110.39 ns 1.12 0.07 False
GetString - Duration of single invocation 90.52 ns 105.21 ns 1.16 0.09 False
GetChars - Duration of single invocation 29.83 ns 36.72 ns 1.23 0.10 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.38604202467967 > 103.14220061275307.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -73.55434422020438 (T) = (0 -110.4705984435926) / Math.Sqrt((1.3181753708610513 / (55)) + (0.007581557031792949 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.12154973336553813 = (98.49817191083739 - 110.4705984435926) / 98.49817191083739 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.20531467795311 > 95.02835537614881.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/7/2022 10:45:01 PM, 3/21/2022 1:28:14 PM, 3/25/2022 7:34:40 AM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -62.973701577965585 (T) = (0 -105.2643868896783) / Math.Sqrt((2.982663266993408 / (55)) + (0.002725657231597837 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.16343650781164562 = (90.47712202849326 - 105.2643868896783) / 90.47712202849326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.72061018190148 > 31.21869926682091.
IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -10.109131726201786 (T) = (0 -36.976784843415736) / Math.Sqrt((0.1427228925743686 / (55)) + (1.3688149389557562 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.2272866624618307 = (30.128889993185055 - 36.976784843415736) / 30.128889993185055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IFooObjIsIFoo - Duration of single invocation 501.17 μs 746.78 μs 1.49 0.38 False
CheckArrayIsNonvariantGenericInterfaceNo - Duration of single invocation 7.59 ns 12.14 ns 1.60 0.41 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'
Details

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.IFooObjIsIFoo


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 746.7817708333333 > 539.0385636268029.
IsChangePoint: Marked as a change because one of 3/8/2022 5:51:58 AM, 3/11/2022 11:18:20 AM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -3.4853443562581 (T) = (0 -947343.0085784314) / Math.Sqrt((14419596474.537815 / (55)) + (30180846071.982964 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.5969134144392011 = (593233.797156821 - 947343.0085784314) / 593233.797156821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckArrayIsNonvariantGenericInterfaceNo

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.143733030819977 > 7.730138894624011.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/3/2022 3:34:31 AM, 4/4/2022 4:14:58 AM, 4/25/2022 2:48:42 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -12.085092571856082 (T) = (0 -11.707063810166687) / Math.Sqrt((0.9530977214325811 / (52)) + (0.9227826681232261 / (6))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (6) - 2, .025) and -0.7490999033509673 = (6.693193332032101 - 11.707063810166687) / 6.693193332032101 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Select - Duration of single invocation 1.01 μs 1.11 μs 1.10 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Select(input: IList)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.112483387355955 > 1.059702745925858.
IsChangePoint: Marked as a change because one of 2/19/2022 1:52:17 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -12.495458940095526 (T) = (0 -1112.6850838105906) / Math.Sqrt((2252.3991069586846 / (55)) + (0.25533249089619414 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.07751685974488794 = (1032.6382123376047 - 1112.6850838105906) / 1032.6382123376047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a9d25b6324708127c0aada0244e7a76af390b56d
Compare 3f4ac7f4c38d4e8043175639dd3a47f921500442
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 16.92 μs 22.12 μs 1.31 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.117128418198963 > 17.76386576704546.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -424.3460759080571 (T) = (0 -22108.99384134842) / Math.Sqrt((7217.817995957334 / (55)) + (56.36996203148867 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.30733968310025006 = (16911.437881942613 - 22108.99384134842) / 16911.437881942613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions