[Perf] Linux/arm64: 7 Regressions on 8/16/2023 6:13:11 AM #20794
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 1eb702ce1efe3581488a37704cee47603a291069 |
| Compare | 2ed506c010b41c41c01997c81df977831f0fa217 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in PerfLabTests.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 34.60 μs | 66.73 μs | 1.93 | 0.01 | True | ||||
| 42.23 μs | 66.74 μs | 1.58 | 0.29 | False | ||||
| 34.40 μs | 66.73 μs | 1.94 | 0.04 | True | ||||
| 36.60 μs | 66.73 μs | 1.82 | 0.02 | True | ||||
| 36.30 μs | 66.74 μs | 1.84 | 0.02 | True | ||||
| 36.68 μs | 66.74 μs | 1.82 | 0.21 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'Details
Payloads
PerfLabTests.CastingPerf.CheckObjIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73232568389058 > 36.2238440422299.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -94.91188368627567 (T) = (0 -66734.08829810777) / Math.Sqrt((905714.6123888374 / (8)) + (5.706073941345815 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.9177178443006162 = (34798.70018232292 - 66734.08829810777) / 34798.70018232292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.7387235460993 > 44.32910540625.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/22/2023 10:35:01 AM, 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -251.68138676820354 (T) = (0 -66819.43071454899) / Math.Sqrt((24519.946352112434 / (8)) + (78709.24978590856 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.5860725520725225 = (42128.86139870208 - 66819.43071454899) / 42128.86139870208 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.7337411824877 > 36.11401637740386.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -334.39463039735574 (T) = (0 -66735.3113627234) / Math.Sqrt((74105.26446431954 / (8)) + (60.82104586749099 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.9319715275966103 = (34542.59569018738 - 66735.3113627234) / 34542.59569018738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckObjIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73152224544073 > 38.3692867041266.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -272.9926031056872 (T) = (0 -66734.28745182567) / Math.Sqrt((99242.9998696021 / (8)) + (2.448275140323358 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.8369790052924982 = (36328.279887553595 - 66734.28745182567) / 36328.279887553595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73695978343467 > 38.38318182327975.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -202.23312856744124 (T) = (0 -66734.15043460824) / Math.Sqrt((180713.35554949474 / (8)) + (4.545200605981834 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.8364396016799459 = (36338.87571013002 - 66734.15043460824) / 36338.87571013002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73957407801417 > 38.74307212392471.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -189.78439300211238 (T) = (0 -66739.49726234317) / Math.Sqrt((195571.75354173727 / (8)) + (37.52510022052004 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.8006500261359132 = (37064.11367763791 - 66739.49726234317) / 37064.11367763791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 1eb702ce1efe3581488a37704cee47603a291069 |
| Compare | 2ed506c010b41c41c01997c81df977831f0fa217 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in PerfLabTests.CastingPerf2.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 300.28 μs | 400.38 μs | 1.33 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'Details
Payloads
PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 400.37517852564105 > 315.3061534375.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -24137.5892398953 (T) = (0 -400380.452382322) / Math.Sqrt((50.7310081059749 / (8)) + (130.24130265022006 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.33331267048151797 = (300289.99292246054 - 400380.452382322) / 300289.99292246054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository