[Perf] Windows/arm64: 6 Regressions on 8/16/2023 6:13:11 AM #20784
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 067a3572a555f1cdef7ca29a54efb7b38cfa086f |
| Compare | 2ed506c010b41c41c01997c81df977831f0fa217 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in PerfLabTests.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 32.48 μs | 63.69 μs | 1.96 | 0.26 | True | ||||
| 35.30 μs | 63.71 μs | 1.80 | 0.01 | True | ||||
| 35.34 μs | 63.71 μs | 1.80 | 0.00 | True | ||||
| 40.13 μs | 63.69 μs | 1.59 | 0.23 | False | ||||
| 31.97 μs | 63.67 μs | 1.99 | 0.01 | True | ||||
| 31.87 μs | 63.66 μs | 2.00 | 0.01 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'Details
Payloads
PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.68804200542005 > 34.20576640788101.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -48.934868700664126 (T) = (0 -63682.298776765245) / Math.Sqrt((2602975.2491294513 / (7)) + (274.01115022933493 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.8818052444307493 = (33841.06775408063 - 63682.298776765245) / 33841.06775408063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.707539868667915 > 37.10626058126411.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -2522.3101379541795 (T) = (0 -63690.03638198497) / Math.Sqrt((529.1761801057397 / (7)) + (659.8570775708547 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.8023439830574745 = (35337.3368128885 - 63690.03638198497) / 35337.3368128885 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckObjIsInterfaceYes
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.70962711069419 > 37.09242580107989.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -2404.17280561599 (T) = (0 -63694.12137951948) / Math.Sqrt((171.07006183854378 / (7)) + (1492.563403663905 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.8031530565788664 = (35323.7464491044 - 63694.12137951948) / 35323.7464491044 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.691539634146345 > 42.11625757021003.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -18.579019768128223 (T) = (0 -63699.20235637586) / Math.Sqrt((14203235.976271512 / (7)) + (633.0833075041014 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.7107733111676371 = (37234.1571735766 - 63699.20235637586) / 37234.1571735766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckObjIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.66931972789116 > 33.86438246381292.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -360.58004351010146 (T) = (0 -63690.199973943956) / Math.Sqrt((52992.7415445079 / (7)) + (1008.5298714309989 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.9814527105843268 = (32143.18445942716 - 63690.199973943956) / 32143.18445942716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.66394035334583 > 33.61058687627291.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 3:20:56 PM falls between 8/15/2023 2:45:30 AM and 8/23/2023 3:20:56 PM.
IsRegressionStdDev: Marked as regression because -514.9940260610089 (T) = (0 -63688.811468491636) / Math.Sqrt((25969.96927498185 / (7)) + (992.9159840112666 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.990311581676999 = (31999.41760617634 - 63688.811468491636) / 31999.41760617634 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository