[Perf] Windows/arm64: 19 Improvements on 2/9/2023 7:28:55 PM #13063
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.TryAddGiventSize<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 41.52 μs | 34.96 μs | 0.84 | 0.16 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize<Int32>*'Payloads
Histogram
System.Collections.TryAddGiventSize<Int32>.ConcurrentDictionary(Count: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.96349449936224 < 39.48504155270847.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 27.377697348375097 (T) = (0 -35617.539368219725) / Math.Sqrt((3014382.6369886324 / (299)) + (919013.1545380339 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1467165425768259 = (41741.74368243459 - 35617.539368219725) / 41741.74368243459 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.CtorFromCollection<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 69.27 μs | 31.48 μs | 0.45 | 0.18 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<Int32>*'Payloads
Histogram
System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.479802499999998 < 66.14646659418108.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 150.6766615030749 (T) = (0 -31931.197273986283) / Math.Sqrt((5420041.398808392 / (299)) + (1155440.9051632679 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5523318278299633 = (71327.82551683825 - 31931.197273986283) / 71327.82551683825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in Microsoft.Extensions.Logging.LoggingOverhead
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TwoArguments - Duration of single invocation | 110.59 ns | 86.35 ns | 0.78 | 0.19 | False | |||||
| TwoArguments_FilteredByLevel - Duration of single invocation | 103.80 ns | 68.64 ns | 0.66 | 0.27 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'Payloads
Histogram
Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.35172592526857 < 99.46941944151548.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 21.550134531977147 (T) = (0 -88.466670358792) / Math.Sqrt((18.9971227735695 / (299)) + (13.857068125601138 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.16583138423953467 = (106.05370267753581 - 88.466670358792) / 106.05370267753581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_FilteredByLevel
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.63716166563664 < 83.3336593499619.
IsChangePoint: Marked as a change because one of 2/9/2023 11:32:54 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 9.550781675241074 (T) = (0 -74.20243217670605) / Math.Sqrt((28.460809520768013 / (299)) + (44.95537290723373 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.16140003376204692 = (88.48370518017775 - 74.20243217670605) / 88.48370518017775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.AddGivenSize<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 60.05 μs | 55.10 μs | 0.92 | 0.18 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<String>*'Payloads
Histogram
System.Collections.AddGivenSize<String>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.10378596668349 < 57.26615738959399.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 8.686003240661082 (T) = (0 -56480.905920743084) / Math.Sqrt((15182083.500301652 / (299)) + (8590393.821352066 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09105006649654669 = (62138.63254606696 - 56480.905920743084) / 62138.63254606696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.ContainsKeyTrue<String, String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 26.38 μs | 14.39 μs | 0.55 | 0.01 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue<String, String>*'Payloads
Histogram
System.Collections.ContainsKeyTrue<String, String>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.3871 < 25.01417949268609.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 158.52160128487427 (T) = (0 -14681.137971506034) / Math.Sqrt((91958.6467696967 / (299)) + (112668.23471344383 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4379151125174377 = (26119.0761368077 - 14681.137971506034) / 26119.0761368077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.IterateForEach<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 18.85 μs | 13.84 μs | 0.73 | 0.01 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<Int32>*'Payloads
Histogram
System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.844351295828066 < 17.911037254786294.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 78.50367330026171 (T) = (0 -13860.46638430562) / Math.Sqrt((1308699.3577010725 / (299)) + (1561.8453656542254 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.274103362196141 = (19094.269986205378 - 13860.46638430562) / 19094.269986205378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.TryGetValueFalse<String, String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 20.22 μs | 12.25 μs | 0.61 | 0.02 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*'Payloads
Histogram
System.Collections.TryGetValueFalse<String, String>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.25042426545086 < 18.865379368441705.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 158.46451877836208 (T) = (0 -12382.811226966156) / Math.Sqrt((110568.04098951812 / (299)) + (45304.284797591565 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3823281246707545 = (20047.555541306763 - 12382.811226966156) / 20047.555541306763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AddThenRemove_AbsoluteExpiration - Duration of single invocation | 44.17 μs | 32.84 μs | 0.74 | 0.52 | False | |||||
| AddThenRemove_RelativeExpiration - Duration of single invocation | 37.63 μs | 33.77 μs | 0.90 | 0.56 | False | |||||
| AddThenRemove_NoExpiration - Duration of single invocation | 36.76 μs | 31.89 μs | 0.87 | 0.50 | False | |||||
| AddThenRemove_ExpirationTokens - Duration of single invocation | 48.44 μs | 43.56 μs | 0.90 | 0.54 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests*'Payloads
Histogram
Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_AbsoluteExpiration
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.841057360444175 < 36.01210385095117.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 31.458788699637083 (T) = (0 -32837.71851652928) / Math.Sqrt((9520316.408360003 / (299)) + (89972.03967734665 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1533623935702281 = (38786.038166913335 - 32837.71851652928) / 38786.038166913335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_RelativeExpiration
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.7652136981402 < 35.917788476089584.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 15.863039891856147 (T) = (0 -33500.67549795785) / Math.Sqrt((9266622.0512892 / (299)) + (1644597.9498408474 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13163970908030562 = (38579.2347350162 - 33500.67549795785) / 38579.2347350162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_NoExpiration
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.889005102040816 < 35.799795559308066.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 30.901817905300017 (T) = (0 -32042.215894033823) / Math.Sqrt((7967845.904051591 / (299)) + (72106.24514998714 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.14268795606700616 = (37375.20791967107 - 32042.215894033823) / 37375.20791967107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_ExpirationTokens
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.56229481695185 < 46.28182069341408.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 12.066714644984584 (T) = (0 -44365.233438311036) / Math.Sqrt((12132312.783632081 / (299)) + (2421864.4160975534 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09410529390759645 = (48973.94050317551 - 44365.233438311036) / 48973.94050317551 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.Concurrent.IsEmpty<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 153.31 ns | 138.31 ns | 0.90 | 0.07 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty<String>*'Payloads
Histogram
System.Collections.Concurrent.IsEmpty<String>.Dictionary(Size: 0)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.3119462383334 < 146.8009660592413.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 85.91241949051647 (T) = (0 -137.99935471202716) / Math.Sqrt((8.587021355817003 / (299)) + (0.27546921263832264 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11157703998307529 = (155.33069373780955 - 137.99935471202716) / 155.33069373780955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SystemTextJson_SourceGen_ - Duration of single invocation | 392.88 μs | 347.77 μs | 0.89 | 0.11 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>*'Payloads
Histogram
MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 347.7688527838321 < 363.86839041337004.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 25.181047296810387 (T) = (0 -349266.8791765024) / Math.Sqrt((75163852.52230155 / (259)) + (43227009.24742889 / (23))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (259) + (23) - 2, .975) and 0.09600154533064204 = (386357.82768483664 - 349266.8791765024) / 386357.82768483664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 3ff80e90e828bac0370c1930c9950c9650ae61b9 |
| Compare | 46cb4eddbb1f4bebdea970964d7449bacde8df1a |
| Diff | Diff |
Improvements in System.Collections.Concurrent.IsEmpty<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 154.19 ns | 137.05 ns | 0.89 | 0.07 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty<Int32>*'Payloads
Histogram
System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 0)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 137.0545100819594 < 146.513757360152.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsImprovementStdDev: Marked as improvement because 70.48075947963531 (T) = (0 -137.65776710800355) / Math.Sqrt((12.425327979920477 / (299)) + (0.5215968651441077 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11485926589066708 = (155.52076839681408 - 137.65776710800355) / 155.52076839681408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository