-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 8006e6a89bc02e410331e6323e3f6321b224b327 |
| Compare | e4163ea55ebb3673c29e1c2a850a6a790029d278 |
| Diff | Diff |
Improvements in System.Collections.ContainsTrue<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Stack - Duration of single invocation | 514.46 μs | 461.12 μs | 0.90 | 0.03 | False | |||||
| ICollection - Duration of single invocation | 494.01 μs | 455.23 μs | 0.92 | 0.04 | False | |||||
| Span - Duration of single invocation | 473.20 μs | 424.89 μs | 0.90 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<String>*'Details
Payloads
Histogram
System.Collections.ContainsTrue<String>.Stack(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 461.12389705882356 < 488.2838155241936.
IsChangePoint: Marked as a change because one of 2/25/2022 7:19:51 AM, 3/3/2022 7:35:51 PM, 4/26/2022 12:44:27 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 67.9937603115472 (T) = (0 -462611.0872621107) / Math.Sqrt((174228088.07872295 / (299)) + (935941.6289839866 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.10299045612258091 = (515725.9367190511 - 462611.0872621107) / 515725.9367190511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsTrue<String>.ICollection(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 455.22952380952376 < 469.0407380022321.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/3/2022 7:35:51 PM, 3/31/2022 1:45:25 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 30.73076162823169 (T) = (0 -456325.51539034926) / Math.Sqrt((591438002.8495884 / (299)) + (624645.9023218165 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.08682370455498944 = (499712.3969013803 - 456325.51539034926) / 499712.3969013803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsTrue<String>.Span(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 424.8935360360361 < 449.5544779040404.
IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 27.197387271793236 (T) = (0 -424371.5657012943) / Math.Sqrt((499090848.58157533 / (299)) + (212733.06170352397 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.07658447455680696 = (459567.2847254948 - 424371.5657012943) / 459567.2847254948 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository