[Perf] Windows/arm64: 6 Regressions on 8/16/2023 6:13:11 AM #20791
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 067a3572a555f1cdef7ca29a54efb7b38cfa086f |
| Compare | 2ed506c010b41c41c01997c81df977831f0fa217 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in PerfLabTests.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 36.41 μs | 79.32 μs | 2.18 | 0.70 | False | ||||
| 46.67 μs | 74.63 μs | 1.60 | 0.70 | False | ||||
| 41.86 μs | 83.63 μs | 2.00 | 0.67 | False | ||||
| 39.31 μs | 74.70 μs | 1.90 | 0.69 | False | ||||
| 43.60 μs | 70.01 μs | 1.61 | 0.71 | False | ||||
| 38.80 μs | 81.22 μs | 2.09 | 0.72 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'Details
Payloads
PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.31732335329342 > 43.96706114825581.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -14.722018859250316 (T) = (0 -81072.3508746622) / Math.Sqrt((8958338.756797895 / (7)) + (84887836.49771088 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -1.0303260536802072 = (39930.7050843922 - 81072.3508746622) / 39930.7050843922 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.62544912060302 > 46.15421521762436.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -10.595051166471933 (T) = (0 -80064.7546468763) / Math.Sqrt((34777397.7743972 / (7)) + (97795484.96636783 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.8786082674790838 = (42619.185720030764 - 80064.7546468763) / 42619.185720030764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.63073504273503 > 42.06340885188731.
IsChangePoint: Marked as a change because one of 6/10/2023 2:06:10 PM, 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -18.829160412180457 (T) = (0 -83201.02753860132) / Math.Sqrt((9855240.942791883 / (7)) + (50139254.73130951 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -1.0801604862216092 = (39997.40793544596 - 83201.02753860132) / 39997.40793544596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.70415638297874 > 42.84878937775939.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -14.308772962258518 (T) = (0 -83572.48736400997) / Math.Sqrt((10041755.578493573 / (7)) + (98408336.73240435 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -1.0566294508964693 = (40635.6562323764 - 83572.48736400997) / 40635.6562323764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckObjIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.01218853427896 > 42.802432988617845.
IsChangePoint: Marked as a change because one of 6/13/2023 3:51:19 AM, 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -13.08225990538459 (T) = (0 -81359.57757125556) / Math.Sqrt((10784166.829992091 / (7)) + (106002934.47921033 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -1.0026325980566906 = (40626.312410077146 - 81359.57757125556) / 40626.312410077146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckObjIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.21843484848485 > 43.09016331694597.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -14.497089151562879 (T) = (0 -80462.18049592711) / Math.Sqrt((15854997.747155497 / (7)) + (62809831.921798944 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.9229569671092303 = (41842.94389951192 - 80462.18049592711) / 41842.94389951192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository