[Perf] Changes at 11/22/2021 8:11:44 PM #2533
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | f45dfdc2244166da0248c4a0fabad19a8bfcea1a |
| Compare | 48fe63793040eaee87bdb26ee7d19636527efa3b |
| Diff | Diff |
Improvements in System.Collections.ContainsTrue<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 46.91 μs | 16.74 μs | 0.36 | 0.00 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*'Details
Payloads
Histogram
System.Collections.ContainsTrue<Int32>.Queue(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.73654873164219 < 44.43147146212639.
IsChangePoint: Marked as a change because one of 11/22/2021 6:07:11 PM, 11/23/2021 6:42:08 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 6:42:08 AM.
IsImprovementStdDev: Marked as improvement because 2245.099020022153 (T) = (0 -16748.67306743795) / Math.Sqrt((9627.88071977286 / (299)) + (293.9990369766548 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.64214384951133 = (46802.80901855905 - 16748.67306743795) / 46802.80901855905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | f45dfdc2244166da0248c4a0fabad19a8bfcea1a |
| Compare | 48fe63793040eaee87bdb26ee7d19636527efa3b |
| Diff | Diff |
Improvements in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Array - Duration of single invocation | 44.36 μs | 16.58 μs | 0.37 | 0.09 | False | |||||
| ImmutableArray - Duration of single invocation | 49.76 μs | 17.55 μs | 0.35 | 0.16 | False | |||||
| Span - Duration of single invocation | 46.64 μs | 14.55 μs | 0.31 | 0.00 | False |
_1.png)
_2.png)
_3.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse<Int32>*'Details
Payloads
Histogram
System.Collections.ContainsFalse<Int32>.Array(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.581308054670913 < 45.01631357366545.
IsChangePoint: Marked as a change because one of 11/22/2021 6:07:11 PM, 11/23/2021 6:42:08 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 6:42:08 AM.
IsImprovementStdDev: Marked as improvement because 225.89633812281212 (T) = (0 -16582.481307293037) / Math.Sqrt((5202367.79363821 / (299)) + (2.7530434306416 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.642470751540489 = (46380.76850703014 - 16582.481307293037) / 46380.76850703014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<Int32>.ImmutableArray(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.547605240549828 < 45.09511961126459.
IsChangePoint: Marked as a change because one of 11/22/2021 6:07:11 PM, 11/23/2021 6:42:08 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 6:42:08 AM.
IsImprovementStdDev: Marked as improvement because 247.06234928376725 (T) = (0 -17561.855423952715) / Math.Sqrt((4610609.417617138 / (299)) + (406.13545403199686 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.6374726043643806 = (48442.83669420763 - 17561.855423952715) / 48442.83669420763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<Int32>.Span(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.551328370746353 < 44.23846940104166.
IsChangePoint: Marked as a change because one of 11/22/2021 6:07:11 PM, 11/23/2021 6:42:08 AM falls between 11/14/2021 12:07:47 PM and 11/23/2021 6:42:08 AM.
IsImprovementStdDev: Marked as improvement because 6177.767479985773 (T) = (0 -14548.389999873783) / Math.Sqrt((5460.715937762775 / (299)) + (17.268046769535285 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.6877216720979572 = (46587.89515626394 - 14548.389999873783) / 46587.89515626394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository