[Perf] Linux/x64: 5 Improvements on 2/18/2023 3:00:06 AM #13151
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | d32b27a0cc22cef896d64782f5e31316c1572d09 |
| Compare | 2bd2ed9c428bf3355483827e53c51f6007ba15a7 |
| Diff | Diff |
Improvements in System.Tests.Perf_Random
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Next_int_int_unseeded - Duration of single invocation | 8.74 ns | 2.68 ns | 0.31 | 0.06 | False | |||||
| Next_int_unseeded - Duration of single invocation | 8.94 ns | 3.16 ns | 0.35 | 0.08 | False | |||||
| Next_long_unseeded - Duration of single invocation | 6.14 ns | 3.65 ns | 0.59 | 0.11 | False | |||||
| Next_long_long_unseeded - Duration of single invocation | 8.66 ns | 3.31 ns | 0.38 | 0.10 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'Details
Payloads
Histogram
System.Tests.Perf_Random.Next_int_int_unseeded
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.684102694630524 < 8.31526748700254.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 123.68945443325097 (T) = (0 -2.744993128710164) / Math.Sqrt((0.037875249648536564 / (299)) + (0.03558675083910287 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.6860044354160377 = (8.742139820819517 - 2.744993128710164) / 8.742139820819517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_int_unseeded
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.156403500003078 < 8.510628888664273.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 78.7646087941399 (T) = (0 -3.2720784155940197) / Math.Sqrt((0.06051763393583248 / (299)) + (0.09126240363292969 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.6491193732785454 = (9.32533222528569 - 3.2720784155940197) / 9.32533222528569 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_long_unseeded
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.646351120995738 < 5.822788787761268.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 2/17/2023 9:53:52 PM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 87.51260856132048 (T) = (0 -3.6788560921852884) / Math.Sqrt((0.031393440169441435 / (299)) + (0.01082841159593838 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.3994412820191693 = (6.1257225680682135 - 3.6788560921852884) / 6.1257225680682135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_long_long_unseeded
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.311967936303603 < 8.228986787891039.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 148.6116407682498 (T) = (0 -3.3265659501323084) / Math.Sqrt((0.04966078780916741 / (299)) + (0.01694475699596297 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.6099363313625226 = (8.528264018415918 - 3.3265659501323084) / 8.528264018415918 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | d32b27a0cc22cef896d64782f5e31316c1572d09 |
| Compare | 2bd2ed9c428bf3355483827e53c51f6007ba15a7 |
| Diff | Diff |
Improvements in System.Memory.Span<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfAnyThreeValues - Duration of single invocation | 4.52 ns | 3.48 ns | 0.77 | 0.06 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'Details
Payloads
Histogram
System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 4)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4833065738175923 < 4.295061129658877.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 2/17/2023 9:53:52 PM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 31.50325896862367 (T) = (0 -3.5485931476313612) / Math.Sqrt((0.060037255043768105 / (299)) + (0.009207753619446708 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.19829894179074764 = (4.426329629098658 - 3.5485931476313612) / 4.426329629098658 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository