[Perf] Windows/arm64: 4 Improvements on 4/17/2023 5:51:51 AM #16777
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Text.Tests.Utf8FormatterTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| FormatterUInt64 - Duration of single invocation | 129.07 ns | 49.58 ns | 0.38 | 0.70 | False | |||||
| FormatterInt64 - Duration of single invocation | 118.07 ns | 37.57 ns | 0.32 | 0.66 | False | |||||
| FormatterUInt32 - Duration of single invocation | 62.63 ns | 18.44 ns | 0.29 | 0.76 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'Details
Payloads
Histogram
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.5792093845004 < 126.75775036360568.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 5:25:38 AM falls between 4/11/2023 2:19:19 PM and 4/20/2023 5:25:38 AM.
IsImprovementStdDev: Marked as improvement because 10.693716960095285 (T) = (0 -44.537031487611564) / Math.Sqrt((281.1188093266782 / (299)) + (34.02574620490186 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.333497550299978 = (66.82200719240672 - 44.537031487611564) / 66.82200719240672 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 37.568818051038704 < 127.78952377971042.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 5:25:38 AM falls between 4/11/2023 2:19:19 PM and 4/20/2023 5:25:38 AM.
IsImprovementStdDev: Marked as improvement because 5.648263627602751 (T) = (0 -50.702700983594426) / Math.Sqrt((223.12913558358062 / (299)) + (52.51532908555545 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.21434414806811566 = (64.53550986595376 - 50.702700983594426) / 64.53550986595376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.44019838833461 < 62.98066750627956.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 5:25:38 AM falls between 4/11/2023 2:19:19 PM and 4/20/2023 5:25:38 AM.
IsImprovementStdDev: Marked as improvement because 9.046873648421979 (T) = (0 -19.309489166357217) / Math.Sqrt((90.07558160232622 / (299)) + (4.727943177438268 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.29188708887165227 = (27.268940959695776 - 19.309489166357217) / 27.268940959695776 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.25094 |
| Queue | AmpereWindows |
| Baseline | 0b081118dedcbc01d445f3f56222fde9d86f3f44 |
| Compare | e3f55cbfd83162bc85d73d69c765e7f44c478e6d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Tests.Perf_Doubles
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteDoubles - Duration of single invocation | 39.11 ms | 34.65 ms | 0.89 | 0.51 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'Details
Payloads
Histogram
System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.6490615 < 36.2488623875.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/20/2023 5:25:38 AM falls between 4/11/2023 2:19:19 PM and 4/20/2023 5:25:38 AM.
IsImprovementStdDev: Marked as improvement because 10.654421977826283 (T) = (0 -32780057.666666668) / Math.Sqrt((6383545390487.762 / (299)) + (2209169705381.5 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.14375463793767596 = (38283486.391930595 - 32780057.666666668) / 38283486.391930595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository