[Perf] Changes at 4/23/2022 1:52:07 AM #4992
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 0e18cfda13a8136103c83442e1a649cdecc237b0 |
| Compare | e99fb185aa10ef177d19a51fd77b7a4b75db5395 |
| Diff | Diff |
Improvements in System.Collections.Tests.Perf_BitArray
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| BitArrayCopyToIntArray - Duration of single invocation | 26.11 ns | 22.37 ns | 0.86 | 0.23 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'Details
Payloads
Histogram
System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.368399422580744 < 24.721568001956623.
IsChangePoint: Marked as a change because one of 1/6/2022 4:51:33 PM, 4/14/2022 10:59:17 PM, 4/22/2022 9:08:35 PM, 4/27/2022 8:46:50 AM falls between 4/19/2022 6:32:34 PM and 4/27/2022 8:46:50 AM.
IsImprovementStdDev: Marked as improvement because 13.268899266984123 (T) = (0 -23.04031576304076) / Math.Sqrt((0.9673489986203294 / (299)) + (0.20621560424795057 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.06981112868120978 = (24.769502703655206 - 23.04031576304076) / 24.769502703655206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 0e18cfda13a8136103c83442e1a649cdecc237b0 |
| Compare | e99fb185aa10ef177d19a51fd77b7a4b75db5395 |
| Diff | Diff |
Improvements in System.Numerics.Tests.Perf_Matrix4x4
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CreateFromScalars - Duration of single invocation | 10.29 ns | 9.08 ns | 0.88 | 0.05 | False | |||||
| CreatePerspectiveFieldOfViewBenchmark - Duration of single invocation | 32.69 ns | 24.48 ns | 0.75 | 0.47 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'Details
Payloads
Histogram
System.Numerics.Tests.Perf_Matrix4x4.CreateFromScalars
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.083188162687334 < 10.03041164571561.
IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 4/22/2022 9:08:35 PM, 4/27/2022 8:46:50 AM falls between 4/19/2022 6:32:34 PM and 4/27/2022 8:46:50 AM.
IsImprovementStdDev: Marked as improvement because 36.23144472025495 (T) = (0 -9.10247600303815) / Math.Sqrt((0.2987840247865429 / (299)) + (0.008924407395457886 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.1371342514683693 = (10.54912194455297 - 9.10247600303815) / 10.54912194455297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveFieldOfViewBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.479377449834796 < 28.840928638092482.
IsChangePoint: Marked as a change because one of 4/22/2022 9:08:35 PM, 4/27/2022 8:46:50 AM falls between 4/19/2022 6:32:34 PM and 4/27/2022 8:46:50 AM.
IsImprovementStdDev: Marked as improvement because 23.435567785467693 (T) = (0 -25.00420759455295) / Math.Sqrt((5.053433342707608 / (299)) + (0.095525380371036 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.12508910560041703 = (28.57914760761135 - 25.00420759455295) / 28.57914760761135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository