Skip to content

[Perf] Linux/arm64: 4 Regressions on 11/1/2022 1:15:18 PM #9668

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 7d5efbb9e10b6d8beb91c90cbdefd7360869cece
Compare 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99
Diff Diff

Regressions in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AndAndAnd - Duration of single invocation 50.71 μs 53.72 μs 1.06 0.00 False
AndAnd - Duration of single invocation 52.88 μs 61.75 μs 1.17 0.00 False
And - Duration of single invocation 54.87 μs 69.67 μs 1.27 0.00 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'IfStatements.IfStatements*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

IfStatements.IfStatements.AndAndAnd


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 53.71921885738832 > 53.239329097896444.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsRegressionStdDev: Marked as regression because -390.82442546636173 (T) = (0 -53676.09471802826) / Math.Sqrt((107.75341424928423 / (12)) + (585.5989617015824 / (12))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (12) - 2, .025) and -0.0585887800584095 = (50705.33121942459 - 53676.09471802826) / 50705.33121942459 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.AndAnd

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 61.74881963582678 > 55.50106338999156.
IsChangePoint: Marked as a change because one of 10/7/2022 4:12:34 PM, 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsRegressionStdDev: Marked as regression because -261.6723363886426 (T) = (0 -61863.73800025051) / Math.Sqrt((1475.5342130135732 / (12)) + (8527.896489344652 / (8))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (8) - 2, .025) and -0.17075303041175133 = (52840.98045724739 - 61863.73800025051) / 52840.98045724739 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.And

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 69.66675746296295 > 57.883569604973594.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsRegressionStdDev: Marked as regression because -423.2446200236204 (T) = (0 -69663.6274404026) / Math.Sqrt((13326.617635331175 / (12)) + (2082.966008809837 / (26))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (26) - 2, .025) and -0.26525093392421906 = (55059.13931581815 - 69663.6274404026) / 55059.13931581815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 7d5efbb9e10b6d8beb91c90cbdefd7360869cece
Compare 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99
Diff Diff

Regressions in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GuidToString - Duration of single invocation 37.51 ns 40.52 ns 1.08 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Guid*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Guid.GuidToString


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 40.517457710811534 > 38.87710815723046.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsRegressionStdDev: Marked as regression because -10.857080525661095 (T) = (0 -40.98313485051866) / Math.Sqrt((0.572171504469558 / (12)) + (0.6504641768400117 / (7))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (7) - 2, .025) and -0.1102921571814019 = (36.91202769058446 - 40.98313485051866) / 36.91202769058446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions