[Perf] Alpine/x64: 8 Regressions on 10/26/2022 10:56:11 AM #9322
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Compare | e6bac1e15f3ea795c84cbd009eb8b77da460c9e1 |
| Diff | Diff |
Regressions in System.IO.Tests.Perf_Path
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetExtension - Duration of single invocation | 12.83 ns | 14.62 ns | 1.14 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'Related Issues
Regressions
Improvements
- [Perf] Linux/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
Details
Payloads
Histogram
Edge Detector Info
System.IO.Tests.Perf_Path.GetExtension
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 14.618362161537402 > 13.51391885533909.
IsChangePoint: Marked as a change because one of 8/5/2022 8:01:14 PM, 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -40.71908810337327 (T) = (0 -14.537088545899524) / Math.Sqrt((0.023143151419269578 / (25)) + (0.015969021218730107 / (25))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (25) - 2, .025) and -0.12459568007487869 = (12.926502211827458 - 14.537088545899524) / 12.926502211827458 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Compare | e6bac1e15f3ea795c84cbd009eb8b77da460c9e1 |
| Diff | Diff |
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsMatch - Duration of single invocation | 90.00 ns | 96.39 ns | 1.07 | 0.01 | False | |||||
| IsMatch - Duration of single invocation | 240.83 ns | 258.62 ns | 1.07 | 0.01 | False | |||||
| IsMatch - Duration of single invocation | 245.70 ns | 261.30 ns | 1.06 | 0.00 | False | |||||
| IsMatch - Duration of single invocation | 88.77 ns | 95.00 ns | 1.07 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'Related Issues
Regressions
Improvements
- [Perf] Linux/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
Details
Payloads
Histogram
Edge Detector Info
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: None)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 96.38584848152081 > 93.52871773643504.
IsChangePoint: Marked as a change because one of 7/26/2022 1:50:03 PM, 8/5/2022 10:47:59 PM, 10/11/2022 5:13:36 PM, 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -15.367783963688721 (T) = (0 -97.4942853551629) / Math.Sqrt((2.682128835109346 / (25)) + (2.470784290387129 / (25))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (25) - 2, .025) and -0.07707899604683424 = (90.51730254975986 - 97.4942853551629) / 90.51730254975986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: None)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 258.62320939121906 > 252.90870271162692.
IsChangePoint: Marked as a change because one of 7/26/2022 1:50:03 PM, 10/2/2022 11:57:04 PM, 10/6/2022 2:13:23 PM, 10/27/2022 12:34:03 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -18.856739556854656 (T) = (0 -258.40337507954274) / Math.Sqrt((17.364632582646813 / (29)) + (0.11146642084988642 / (21))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (21) - 2, .025) and -0.06012802950848066 = (243.7473285178105 - 258.40337507954274) / 243.7473285178105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: None)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 261.2972937008836 > 257.9715613638238.
IsChangePoint: Marked as a change because one of 7/26/2022 1:50:03 PM, 10/11/2022 8:42:28 PM, 10/27/2022 12:34:03 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -16.19668920519554 (T) = (0 -261.07342296015327) / Math.Sqrt((16.42443890265657 / (29)) + (1.3208362640431432 / (21))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (21) - 2, .025) and -0.05175999626887752 = (248.22528322650814 - 261.07342296015327) / 248.22528322650814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 95.00033554012693 > 93.23022935812091.
IsChangePoint: Marked as a change because one of 7/26/2022 1:50:03 PM, 8/5/2022 10:47:59 PM, 10/11/2022 5:13:36 PM, 10/25/2022 2:34:53 PM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -18.938610290039477 (T) = (0 -95.48129220964833) / Math.Sqrt((0.555857887481692 / (21)) + (2.135499623978433 / (29))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (29) - 2, .025) and -0.06695925582740556 = (89.48916435951858 - 95.48129220964833) / 89.48916435951858 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Compare | e6bac1e15f3ea795c84cbd009eb8b77da460c9e1 |
| Diff | Diff |
Regressions in System.Collections.Tests.Perf_SortedSet
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Max - Duration of single invocation | 15.04 ns | 28.95 ns | 1.92 | 0.40 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_SortedSet*'Related Issues
Regressions
Improvements
- [Perf] Linux/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
Details
Payloads
Histogram
Edge Detector Info
System.Collections.Tests.Perf_SortedSet.Max
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 28.950840763397863 > 16.285647193718273.
IsChangePoint: Marked as a change because one of 9/21/2022 2:58:31 PM, 9/27/2022 11:22:50 AM, 10/6/2022 2:13:23 PM, 10/12/2022 1:08:53 AM, 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -5.294256188348273 (T) = (0 -24.42562089715217) / Math.Sqrt((10.013183997216911 / (25)) + (47.88231697493328 / (25))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (25) - 2, .025) and -0.4921952153086213 = (16.36891785107378 - 24.42562089715217) / 16.36891785107378 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Compare | e6bac1e15f3ea795c84cbd009eb8b77da460c9e1 |
| Diff | Diff |
Regressions in System.Perf_Convert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToBase64CharArray - Duration of single invocation | 1.06 μs | 1.13 μs | 1.07 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'Related Issues
Regressions
Improvements
- [Perf] Linux/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
Details
Payloads
Histogram
Edge Detector Info
System.Perf_Convert.ToBase64CharArray(binaryDataSize: 1024, formattingOptions: InsertLineBreaks)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.1332735603010904 > 1.11458120412403.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -235.04682022048706 (T) = (0 -1136.5164174779734) / Math.Sqrt((0.23473308506444315 / (25)) + (2.3024593382833713 / (25))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (25) - 2, .025) and -0.07053179484053798 = (1061.6372376378267 - 1136.5164174779734) / 1061.6372376378267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | a8ea8e67dc20689aa2e4c738a2f84ea3483ec872 |
| Compare | e6bac1e15f3ea795c84cbd009eb8b77da460c9e1 |
| Diff | Diff |
Regressions in System.Globalization.Tests.StringEquality
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compare_Same_Upper - Duration of single invocation | 1.40 μs | 1.50 μs | 1.07 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'Related Issues
Regressions
- [Perf] Windows/x64: 13 Regressions on 10/26/2022 10:56:11 AM
- [Perf] Linux/x64: 4 Regressions on 10/26/2022 10:56:11 AM
Improvements
- [Perf] Linux/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
- [Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM
Details
Payloads
Histogram
Edge Detector Info
System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.498558046439554 > 1.472598281218323.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/1/2022 8:26:50 AM, 8/4/2022 6:25:19 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 10/26/2022 7:11:30 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -36.94482984196033 (T) = (0 -1565.2965664740377) / Math.Sqrt((67.37301004284778 / (25)) + (438.27317283109477 / (25))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (25) - 2, .025) and -0.11875300714750907 = (1399.144008081894 - 1565.2965664740377) / 1399.144008081894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository