[Perf] Changes at 6/22/2022 1:08:18 PM #6499
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 46f53e387e0396542f7dcefa2bb7627ef3a2bf1d |
| Compare | c9007c8cd39a5b086a74dd5580b34311041a68ff |
| Diff | Diff |
Improvements in System.IO.Tests.StreamReaderReadLineTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ReadLine - Duration of single invocation | 50.52 μs | 38.92 μs | 0.77 | 0.60 | False | |||||
| ReadLine - Duration of single invocation | 61.02 μs | 48.23 μs | 0.79 | 0.57 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'Details
Payloads
Histogram
System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024])
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.91729864690722 < 48.842652012707894.
IsChangePoint: Marked as a change because one of 6/22/2022 6:55:40 AM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 11.87582005367857 (T) = (0 -43849.19417170592) / Math.Sqrt((12651368.62172013 / (179)) + (7670984.5805535205 / (28))) is greater than 1.9716034992643237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (179) + (28) - 2, .975) and 0.1371841845863117 = (50821.03664347165 - 43849.19417170592) / 50821.03664347165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.22994725177304 < 55.73871678483726.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 16.33768564372985 (T) = (0 -48590.51845309752) / Math.Sqrt((15883088.398826573 / (181)) + (8338015.427599799 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.17473863620109664 = (58878.945004067675 - 48590.51845309752) / 58878.945004067675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 46f53e387e0396542f7dcefa2bb7627ef3a2bf1d |
| Compare | c9007c8cd39a5b086a74dd5580b34311041a68ff |
| Diff | Diff |
Improvements in System.Memory.Span<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Reverse - Duration of single invocation | 398.39 ns | 218.00 ns | 0.55 | 0.66 | False | |||||
| ToArray - Duration of single invocation | 436.40 ns | 218.21 ns | 0.50 | 0.57 | False |
_1.png)
_2.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Int32>*'Details
Payloads
Histogram
System.Memory.Span<Int32>.Reverse(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 218.0016591861942 < 375.88986241102674.
IsChangePoint: Marked as a change because one of 6/22/2022 6:55:40 AM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 37.24968640579447 (T) = (0 -235.1535064544273) / Math.Sqrt((1220.3511942235607 / (128)) + (336.0132123063595 / (29))) is greater than 1.975387131054855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (128) + (29) - 2, .975) and 0.4212926395237296 = (406.34269151319984 - 235.1535064544273) / 406.34269151319984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Int32>.ToArray(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 218.21311619605686 < 420.1725655656822.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 28.096137191574527 (T) = (0 -257.12697971817323) / Math.Sqrt((708.9863175658105 / (181)) + (1016.9647942994405 / (26))) is greater than 1.9716034992643237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (26) - 2, .975) and 0.4175165470890588 = (441.43224744529635 - 257.12697971817323) / 441.43224744529635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 46f53e387e0396542f7dcefa2bb7627ef3a2bf1d |
| Compare | c9007c8cd39a5b086a74dd5580b34311041a68ff |
| Diff | Diff |
Improvements in System.Tests.Perf_Array
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Reverse - Duration of single invocation | 199.10 ns | 124.73 ns | 0.63 | 0.59 | False | |||||
| ArrayCreate2D - Duration of single invocation | 2.40 μs | 1.23 μs | 0.51 | 0.38 | False | |||||
| ArrayCreate1D - Duration of single invocation | 2.16 μs | 1.24 μs | 0.57 | 0.48 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'Details
Payloads
Histogram
System.Tests.Perf_Array.Reverse
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.73121020627677 < 187.4522391303949.
IsChangePoint: Marked as a change because one of 6/22/2022 6:55:40 AM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 34.261680010876155 (T) = (0 -124.55390615888669) / Math.Sqrt((320.5737314977224 / (179)) + (104.8939215409171 / (29))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (179) + (29) - 2, .975) and 0.3901274896554681 = (204.22941524044614 - 124.55390615888669) / 204.22941524044614 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Array.ArrayCreate2D
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2329860297743802 < 2.445592296373041.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 50.586183512906786 (T) = (0 -1149.6791197906186) / Math.Sqrt((15086.976303877345 / (179)) + (21942.875312309396 / (27))) is greater than 1.9716608894935834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (179) + (27) - 2, .975) and 0.5685553078621269 = (2664.7195822337885 - 1149.6791197906186) / 2664.7195822337885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Array.ArrayCreate1D
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2396065686964226 < 2.396873006997163.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 49.275875548452646 (T) = (0 -1068.1914838209389) / Math.Sqrt((18197.169299796988 / (180)) + (21455.131811135976 / (27))) is greater than 1.9716034992643237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (180) + (27) - 2, .975) and 0.5799411427135432 = (2542.9566959291415 - 1068.1914838209389) / 2542.9566959291415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository