[Perf] Changes at 5/30/2022 5:18:40 PM #5672
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 28.08 ms | 24.38 ms | 0.87 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'Details
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]).){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: None)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.37570666666667 < 26.430872249999997.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/9/2022 5:23:46 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 18.817253804289535 (T) = (0 -24249168.333333332) / Math.Sqrt((1833601388575.159 / (299)) + (14303611858.333332 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07494561364808305 = (26213775.850480925 - 24249168.333333332) / 26213775.850480925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 56.15 μs | 42.40 μs | 0.76 | 0.05 | False | |||||
| Count - Duration of single invocation | 50.62 μs | 40.68 μs | 0.80 | 0.12 | False | |||||
| Count - Duration of single invocation | 31.52 μs | 23.24 μs | 0.74 | 0.04 | False | |||||
| Count - Duration of single invocation | 46.35 μs | 39.95 μs | 0.86 | 0.12 | False | |||||
| Count - Duration of single invocation | 221.55 μs | 195.77 μs | 0.88 | 0.03 | False | |||||
| Count - Duration of single invocation | 51.18 μs | 42.15 μs | 0.82 | 0.11 | False | |||||
| Count - Duration of single invocation | 81.44 μs | 72.20 μs | 0.89 | 0.02 | False | |||||
| Count - Duration of single invocation | 175.18 μs | 145.39 μs | 0.83 | 0.03 | False | |||||
| Count - Duration of single invocation | 31.56 μs | 23.28 μs | 0.74 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'Details
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.402892107046064 < 53.31890840972796.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/16/2022 6:32:29 PM, 5/19/2022 10:26:24 PM, 5/23/2022 3:43:39 PM, 5/27/2022 11:47:05 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 20.875531363697885 (T) = (0 -42279.7758410858) / Math.Sqrt((21233891.275719523 / (299)) + (11443.327133299428 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11899372545482265 = (47990.32318233248 - 42279.7758410858) / 47990.32318233248 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.67533745659722 < 48.19826194711943.
IsChangePoint: Marked as a change because one of 4/11/2022 9:20:21 PM, 5/9/2022 11:56:12 AM, 5/25/2022 1:42:32 AM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 6.8435099865643245 (T) = (0 -40680.40003999706) / Math.Sqrt((33206423.69449426 / (299)) + (121225.39025493863 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.05386316038449578 = (42996.317590306455 - 40680.40003999706) / 42996.317590306455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.2408936531522 < 29.953385280157924.
IsChangePoint: Marked as a change because one of 4/17/2022 3:25:40 PM, 5/9/2022 11:56:12 AM, 5/25/2022 1:42:32 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 39.58165654113728 (T) = (0 -23283.322910631483) / Math.Sqrt((32642345.94679207 / (299)) + (3140.13784623673 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.360771520552199 = (36424.10133344628 - 23283.322910631483) / 36424.10133344628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.945300511508954 < 43.84228694852941.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 9:09:01 PM, 5/26/2022 4:44:18 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 34.383888932299264 (T) = (0 -40240.372304939905) / Math.Sqrt((26001179.721443553 / (299)) + (80096.89260319696 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22364376305917816 = (51832.35529027797 - 40240.372304939905) / 51832.35529027797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 195.77159895833336 < 210.8332546948357.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 25.993946880527474 (T) = (0 -193635.95832354113) / Math.Sqrt((247654614.71245816 / (299)) + (33893976.33464377 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.18728000998575697 = (238256.6698281257 - 193635.95832354113) / 238256.6698281257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.15449927431059 < 48.66368115842736.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/25/2022 1:42:32 AM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 6.763996556578288 (T) = (0 -41622.431404039955) / Math.Sqrt((31584665.66140795 / (299)) + (365978.2709005494 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.052465059614038766 = (43927.06762569179 - 41622.431404039955) / 43927.06762569179 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.20115317414094 < 77.44729031061391.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 34.64479015709217 (T) = (0 -71396.053909684) / Math.Sqrt((15506029.323107725 / (299)) + (497939.39217008994 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.12263958704715974 = (81375.96916345217 - 71396.053909684) / 81375.96916345217 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 145.39339524085895 < 167.70618455246247.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 39.1160291512815 (T) = (0 -148697.50260306118) / Math.Sqrt((222310979.77369645 / (299)) + (4810478.227005047 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.21185656557524157 = (188668.07247032502 - 148697.50260306118) / 188668.07247032502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.27752378295068 < 29.983051557239055.
IsChangePoint: Marked as a change because one of 4/20/2022 8:23:28 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 35.18658723554766 (T) = (0 -23271.795785211278) / Math.Sqrt((47040826.3401236 / (299)) + (68.68454310328774 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.37490797677234095 = (37229.39170627629 - 23271.795785211278) / 37229.39170627629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Collections.IterateForEach<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 2.88 μs | 2.59 μs | 0.90 | 0.06 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<String>*'Details
Payloads
Histogram
System.Collections.IterateForEach<String>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5909595618770758 < 2.7327823581151667.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 51.104049037762636 (T) = (0 -2591.0419960927616) / Math.Sqrt((18745.118805918188 / (299)) + (0.15373383962729054 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13512080821480615 = (2995.84268034545 - 2591.0419960927616) / 2995.84268034545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableArray - Duration of single invocation | 22.82 μs | 17.62 μs | 0.77 | 0.03 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse<Int32>*'Details
Payloads
Histogram
System.Collections.ContainsFalse<Int32>.ImmutableArray(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.615562590594298 < 21.680327943134742.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/26/2022 5:29:00 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 9.028211013761558 (T) = (0 -17594.84554875268) / Math.Sqrt((5190322.632800906 / (299)) + (1955.7841220415633 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06442607259411239 = (18806.472725825937 - 17594.84554875268) / 18806.472725825937 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Tests.Perf_String
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Replace_Char - Duration of single invocation | 124.66 ns | 114.82 ns | 0.92 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'Details
Payloads
Histogram
System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+')
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 114.82267293138024 < 118.39259528654543.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/26/2022 5:29:00 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 22.880622050580712 (T) = (0 -114.78891650917093) / Math.Sqrt((24.558389239986646 / (299)) + (0.00985013563719774 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05504943038960431 = (121.47610700578608 - 114.78891650917093) / 121.47610700578608 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 13.52 ms | 10.10 ms | 0.75 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'Details
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: Compiled)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.099220208333334 < 12.84188573478836.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 524.2547355546584 (T) = (0 -10090938.020833334) / Math.Sqrt((404943209.82758385 / (299)) + (121456670.58376989 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2515259493586258 = (13482014.522996914 - 10090938.020833334) / 13482014.522996914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Collections.Tests.Add_Remove_SteadyState<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentBag - Duration of single invocation | 42.38 ns | 38.65 ns | 0.91 | 0.07 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState<String>*'Details
Payloads
Histogram
System.Collections.Tests.Add_Remove_SteadyState<String>.ConcurrentBag(Count: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.64830306149864 < 40.179422118888624.
IsChangePoint: Marked as a change because one of 5/17/2022 1:27:07 PM, 5/18/2022 2:51:29 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 12.874439888599465 (T) = (0 -38.91326258362812) / Math.Sqrt((3.165723185474683 / (299)) + (0.21209383341246557 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08619664858071466 = (42.58384752385673 - 38.91326258362812) / 42.58384752385673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
| Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
| Diff | Diff |
Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IDictionary - Duration of single invocation | 3.83 μs | 3.57 μs | 0.93 | 0.01 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue<Int32, Int32>*'Details
Payloads
Histogram
System.Collections.ContainsKeyTrue<Int32, Int32>.IDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.5746621563794467 < 3.6565519534118613.
IsChangePoint: Marked as a change because one of 3/21/2022 5:19:15 PM, 5/11/2022 9:09:01 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsImprovementStdDev: Marked as improvement because 75.65656053861461 (T) = (0 -3575.34065217814) / Math.Sqrt((7398.343697560011 / (299)) + (12.681685018359772 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1022504022845196 = (3982.558902032787 - 3575.34065217814) / 3982.558902032787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository