-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | affc0fc7a79812b92445eb474c87533e32b35127 |
| Compare | 86412566424353b2d9b32d629fe7efb66346ac38 |
| Diff | Diff |
Regressions in System.Tests.Perf_Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToStringWithFormat - Duration of single invocation | 218.55 ns | 234.14 ns | 1.07 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Single*'Details
Payloads
Histogram
System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "F50")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 234.1446869837515 > 230.4713008400628.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/24/2022 2:39:26 AM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsRegressionStdDev: Marked as regression because -11.933212438244922 (T) = (0 -232.26003690893282) / Math.Sqrt((15.735418770881013 / (17)) + (10.960431002755968 / (46))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (46) - 2, .025) and -0.058681740518831856 = (219.3860798950857 - 232.26003690893282) / 219.3860798950857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | affc0fc7a79812b92445eb474c87533e32b35127 |
| Compare | 86412566424353b2d9b32d629fe7efb66346ac38 |
| Diff | Diff |
Regressions in System.Tests.Perf_Double
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToStringWithFormat - Duration of single invocation | 226.19 ns | 244.63 ns | 1.08 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'Details
Payloads
Histogram
System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "F50")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 244.62571816903343 > 237.16361016675182.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/24/2022 2:39:26 AM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsRegressionStdDev: Marked as regression because -15.193521699282348 (T) = (0 -249.00495672680145) / Math.Sqrt((16.942290263584578 / (17)) + (32.025812827859845 / (46))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (46) - 2, .025) and -0.08623411523858518 = (229.23691424671276 - 249.00495672680145) / 229.23691424671276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | affc0fc7a79812b92445eb474c87533e32b35127 |
| Compare | 86412566424353b2d9b32d629fe7efb66346ac38 |
| Diff | Diff |
Regressions in System.Net.Tests.Perf_WebUtility
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Decode_NoDecodingRequired - Duration of single invocation | 62.09 ns | 74.63 ns | 1.20 | 0.03 | False | |||||
| Decode_DecodingRequired - Duration of single invocation | 96.85 ns | 110.53 ns | 1.14 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'Details
Payloads
Histogram
System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.62563026608555 > 64.47159203333908.
IsChangePoint: Marked as a change because one of 8/24/2022 2:39:26 AM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsRegressionStdDev: Marked as regression because -81.17770889130645 (T) = (0 -74.5405181097264) / Math.Sqrt((0.33999685947579533 / (17)) + (0.3264314994569256 / (47))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (47) - 2, .025) and -0.21767967119876525 = (61.21521108777626 - 74.5405181097264) / 61.21521108777626 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.53292997551272 > 101.58797140367841.
IsChangePoint: Marked as a change because one of 8/24/2022 2:39:26 AM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsRegressionStdDev: Marked as regression because -43.27633389439559 (T) = (0 -110.45661554543427) / Math.Sqrt((0.8805203234443126 / (17)) + (2.3892855263197803 / (46))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (46) - 2, .025) and -0.14441323069648826 = (96.51812176114963 - 110.45661554543427) / 96.51812176114963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | affc0fc7a79812b92445eb474c87533e32b35127 |
| Compare | 86412566424353b2d9b32d629fe7efb66346ac38 |
| Diff | Diff |
Regressions in System.Collections.AddGivenSize<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Stack - Duration of single invocation | 1.57 μs | 1.73 μs | 1.10 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize<String>*'Details
Payloads
Histogram
System.Collections.AddGivenSize<String>.Stack(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7310674038827107 > 1.6456527502399736.
IsChangePoint: Marked as a change because one of 8/5/2022 8:01:14 PM, 8/9/2022 6:41:10 AM, 8/24/2022 2:39:26 AM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsRegressionStdDev: Marked as regression because -19.3358534849793 (T) = (0 -1752.1004995086198) / Math.Sqrt((813.1735585008805 / (17)) + (1566.7266962826457 / (45))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (45) - 2, .025) and -0.11151714224393733 = (1576.3144201010423 - 1752.1004995086198) / 1576.3144201010423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository




_1.png)