[Perf] Windows/arm64: 21 Improvements on 4/17/2023 5:51:51 AM #16767
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LastIndexOfValue - Duration of single invocation | 15.30 ns | 13.11 ns | 0.86 | 0.05 | False | |||||
| Reverse - Duration of single invocation | 24.85 ns | 19.16 ns | 0.77 | 0.08 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Byte>*'Details
Payloads
Histogram
System.Memory.Span<Byte>.LastIndexOfValue(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.106045031185195 < 14.434494636314339.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/21/2023 9:40:41 PM, 3/10/2023 4:14:40 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 18.14178722592229 (T) = (0 -13.077622552112521) / Math.Sqrt((0.40368462050043763 / (299)) + (0.004842937693764584 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.05685344025021748 = (13.865949482530079 - 13.077622552112521) / 13.865949482530079 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Memory.Span<Byte>.Reverse(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.159741669239967 < 23.582846461478795.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/21/2023 9:40:41 PM, 2/1/2023 6:25:50 PM, 2/7/2023 3:56:56 PM, 3/10/2023 4:14:40 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 14.895679814039555 (T) = (0 -18.920715187608874) / Math.Sqrt((7.038292860386043 / (299)) + (0.10269901825075743 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1282972992567454 = (21.705468127466148 - 18.920715187608874) / 21.705468127466148 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Tests.Perf_Doubles
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteDoubles - Duration of single invocation | 23.86 ms | 22.07 ms | 0.92 | 0.00 | False | |||||
| WriteDoubles - Duration of single invocation | 24.45 ms | 22.84 ms | 0.93 | 0.00 | False | |||||
| WriteDoubles - Duration of single invocation | 24.25 ms | 22.48 ms | 0.93 | 0.00 | False | |||||
| WriteDoubles - Duration of single invocation | 23.53 ms | 21.54 ms | 0.92 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'Details
Payloads
Histogram
System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: False)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.068369736842108 < 22.62907743543956.
IsChangePoint: Marked as a change because one of 1/4/2023 10:05:35 PM, 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 15.447494648117182 (T) = (0 -22175922.954073217) / Math.Sqrt((124506669016.25471 / (299)) + (165452712447.7997 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.0871844117269466 = (24293979.24286944 - 22175922.954073217) / 24293979.24286944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.837775 < 23.34536478095238.
IsChangePoint: Marked as a change because one of 1/4/2023 10:05:35 PM, 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 27.705560939361607 (T) = (0 -22760002.324989825) / Math.Sqrt((152228266704.08298 / (299)) + (58016548204.51749 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09216439087832673 = (25070620.82199003 - 22760002.324989825) / 25070620.82199003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.477898333333332 < 23.099012952380953.
IsChangePoint: Marked as a change because one of 1/5/2023 10:03:03 PM, 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 27.188633593439572 (T) = (0 -22523062.010582015) / Math.Sqrt((171894708198.80118 / (299)) + (65595359560.08139 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.096693389953371 = (24934016.600874167 - 22523062.010582015) / 24934016.600874167 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.541084615384616 < 22.49863850874125.
IsChangePoint: Marked as a change because one of 1/4/2023 10:05:35 PM, 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 13.750716309164607 (T) = (0 -21837350.090048842) / Math.Sqrt((140572835891.61118 / (299)) + (246243372389.702 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09506158572985655 = (24131310.756280843 - 21837350.090048842) / 24131310.756280843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Text.Tests.Utf8FormatterTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| FormatterInt64 - Duration of single invocation | 51.52 ns | 37.83 ns | 0.73 | 0.01 | False | |||||
| FormatterDouble - Duration of single invocation | 224.67 ns | 196.57 ns | 0.87 | 0.00 | False | |||||
| FormatterUInt64 - Duration of single invocation | 104.03 ns | 31.75 ns | 0.31 | 0.01 | False | |||||
| FormatterUInt32 - Duration of single invocation | 52.26 ns | 14.51 ns | 0.28 | 0.04 | False | |||||
| FormatterDouble - Duration of single invocation | 134.28 ns | 115.31 ns | 0.86 | 0.00 | True | |||||
| FormatterDouble - Duration of single invocation | 220.66 ns | 198.03 ns | 0.90 | 0.00 | False | |||||
| FormatterInt64 - Duration of single invocation | 100.12 ns | 38.87 ns | 0.39 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'Details
Payloads
Histogram
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 37.834873173877 < 49.041072120759786.
IsChangePoint: Marked as a change because one of 1/15/2023 4:47:52 PM, 1/21/2023 1:51:00 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 132.1583515861013 (T) = (0 -37.857703477618664) / Math.Sqrt((2.8282933149409697 / (299)) + (0.022581769033995325 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.27636149375643865 = (52.31576698998459 - 37.857703477618664) / 52.31576698998459 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 196.5712868499286 < 212.07289337464928.
IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 66.93267901684162 (T) = (0 -197.2688933199196) / Math.Sqrt((4.362439852566719 / (299)) + (1.3359574967586738 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.12049070865150759 = (224.2942687023357 - 197.2688933199196) / 224.2942687023357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.752789751305983 < 98.71291779140186.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 22.35412679300909 (T) = (0 -32.08048842390872) / Math.Sqrt((153.39665227830733 / (299)) + (0.035963858130695275 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.33379599457767767 = (48.15415122515234 - 32.08048842390872) / 48.15415122515234 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.514841538306781 < 49.62072321234691.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 13.966593124741689 (T) = (0 -14.01057401647873) / Math.Sqrt((49.65246144946404 / (299)) + (0.18279029404714575 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.3008717812087682 = (20.040063667723956 - 14.01057401647873) / 20.040063667723956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.3125069617254 < 127.85251750185118.
IsChangePoint: Marked as a change because one of 1/17/2023 4:06:07 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 59.94570518498132 (T) = (0 -116.00652648616341) / Math.Sqrt((1.3729382673038624 / (299)) + (0.7754340827047551 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.13470019578192913 = (134.06512508227462 - 116.00652648616341) / 134.06512508227462 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 198.02941695177418 < 209.46744919320582.
IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 66.59144880807541 (T) = (0 -197.82432381804094) / Math.Sqrt((4.470707986727382 / (299)) + (1.0624835869435656 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.10934160528755973 = (222.11021081984063 - 197.82432381804094) / 222.11021081984063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 38.87079245560978 < 94.50689816738003.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 11.904322023787975 (T) = (0 -38.55428464994623) / Math.Sqrt((137.39837930270465 / (299)) + (0.13333472544546063 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.17536378152673568 = (46.75308188782422 - 38.55428464994623) / 46.75308188782422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SortedDictionary - Duration of single invocation | 42.39 μs | 37.78 μs | 0.89 | 0.16 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue<Int32, Int32>*'Details
Payloads
Histogram
System.Collections.ContainsKeyTrue<Int32, Int32>.SortedDictionary(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 37.78178700828157 < 40.24326720157657.
IsChangePoint: Marked as a change because one of 1/13/2023 8:11:05 PM, 2/9/2023 1:41:04 PM, 2/20/2023 2:00:07 PM, 2/23/2023 8:54:44 PM, 3/15/2023 9:07:40 PM, 3/16/2023 1:49:47 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 25.596719896439502 (T) = (0 -37755.224394124925) / Math.Sqrt((6972614.517358877 / (299)) + (23607.69910718655 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09844801195950043 = (41878.03354101071 - 37755.224394124925) / 41878.03354101071 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Globalization.Tests.Perf_DateTimeCultureInfo
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToString - Duration of single invocation | 231.11 ns | 177.71 ns | 0.77 | 0.01 | False | |||||
| ToString - Duration of single invocation | 232.43 ns | 180.85 ns | 0.78 | 0.01 | False | |||||
| ToString - Duration of single invocation | 233.80 ns | 215.54 ns | 0.92 | 0.01 | False | |||||
| ToString - Duration of single invocation | 202.71 ns | 164.41 ns | 0.81 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'Details
Payloads
Histogram
System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 177.70798480046406 < 219.7738427837317.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 12.147612058245379 (T) = (0 -188.38574859034526) / Math.Sqrt((40.897545440149706 / (299)) + (16.41069767037484 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.08280452528148607 = (205.3932381733137 - 188.38574859034526) / 205.3932381733137 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 180.8507790239053 < 222.1836201854619.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 15.636590661529105 (T) = (0 -190.49517113856504) / Math.Sqrt((34.651036240681876 / (299)) + (13.630306782194388 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09486664159229982 = (210.46088885032574 - 190.49517113856504) / 210.46088885032574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: )
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 215.5427411282022 < 222.02135826274622.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 5.907742030058384 (T) = (0 -192.94236729275886) / Math.Sqrt((32.33452734454438 / (299)) + (72.22680063396781 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.08031004828132968 = (209.79066579144185 - 192.94236729275886) / 209.79066579144185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: da)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 164.40887153705296 < 193.0567023730026.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 2.589893250311206 (T) = (0 -187.23201325237628) / Math.Sqrt((6.031374295944687 / (299)) + (214.30928829722163 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.06325655473384624 = (199.87544529780902 - 187.23201325237628) / 199.87544529780902 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Array
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Reverse - Duration of single invocation | 51.14 ns | 31.64 ns | 0.62 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'Details
Payloads
Histogram
System.Tests.Perf_Array.Reverse
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.6435857167785 < 48.67405143540416.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/21/2023 9:40:41 PM, 2/1/2023 6:25:50 PM, 2/7/2023 3:56:56 PM, 3/10/2023 4:14:40 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 19.61451100927661 (T) = (0 -31.283717040397775) / Math.Sqrt((91.73285126679376 / (299)) + (0.2752522180347882 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2669608276464544 = (42.676733004535265 - 31.283717040397775) / 42.676733004535265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Reverse - Duration of single invocation | 101.21 ns | 63.17 ns | 0.62 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'Details
Payloads
Histogram
System.Memory.Span<Int32>.Reverse(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 63.16610253404097 < 95.93318524833111.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/21/2023 9:40:41 PM, 2/1/2023 6:25:50 PM, 2/7/2023 3:56:56 PM, 3/10/2023 4:14:40 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 19.77364989345919 (T) = (0 -62.66269625036412) / Math.Sqrt((348.5106225431585 / (299)) + (0.33392165660546647 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2570924590609254 = (84.3479071045034 - 62.66269625036412) / 84.3479071045034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository