[Perf] Windows/arm64: 4 Improvements on 4/18/2023 2:18:01 AM #16768
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Compare | 90d0d5ce4e4fdf17209d6a4e4a96d62865e2b88f |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_UInt32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 23.73 ns | 22.33 ns | 0.94 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'Payloads
Histogram
System.Tests.Perf_UInt32.Parse(value: "4294967295")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.32712449463229 < 22.761826169570142.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 38.8696377843093 (T) = (0 -22.294988142553947) / Math.Sqrt((0.12557578205612047 / (299)) + (0.007826717476639983 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06751046627897797 = (23.90910282240665 - 22.294988142553947) / 23.90910282240665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Compare | 90d0d5ce4e4fdf17209d6a4e4a96d62865e2b88f |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Int64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ParseSpan - Duration of single invocation | 39.18 ns | 36.50 ns | 0.93 | 0.06 | False | |||||
| ParseSpan - Duration of single invocation | 37.81 ns | 35.79 ns | 0.95 | 0.08 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'Payloads
Histogram
System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.50075993300685 < 37.16245398165269.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 12.262460039468015 (T) = (0 -36.206018193319125) / Math.Sqrt((0.5563983432987323 / (299)) + (0.16922795921980244 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05546865106448962 = (38.33225676852697 - 36.206018193319125) / 38.33225676852697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.79266511772601 < 36.12298953018167.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/17/2023 8:43:19 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 10.431128215273015 (T) = (0 -35.503603922297096) / Math.Sqrt((0.743903278203336 / (299)) + (0.21085424536623507 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05392069562475338 = (37.52709076089797 - 35.503603922297096) / 37.52709076089797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Compare | 90d0d5ce4e4fdf17209d6a4e4a96d62865e2b88f |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchmark.GetChildKeysTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AddChainedConfigurationEmpty - Duration of single invocation | 37.34 ms | 23.35 ms | 0.63 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'Payloads
Histogram
Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.349639285714286 < 35.49501068005952.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/17/2023 8:43:19 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM.
IsImprovementStdDev: Marked as improvement because 100.1442066436225 (T) = (0 -23251270.83180708) / Math.Sqrt((5587746836467.395 / (299)) + (3777519528.2921476 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.37446352295668683 = (37170127.858422436 - 23251270.83180708) / 37170127.858422436 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository