Skip to content

[Perf] Changes at 6/11/2022 3:15:10 AM #6181

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline fae7ee8e7e3aa7f86836318a10ed676641e813ad
Compare 21b5b2e8b7c8c0e66c7a041a61cce8ff4d53ea1c
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultipleSerial - Duration of single invocation 1.65 μs 1.21 μs 0.73 0.59 False
SingleParallel - Duration of single invocation 116.02 μs 19.16 μs 0.17 0.71 False
MultipleSerial - Duration of single invocation 2.31 μs 1.97 μs 0.85 0.55 False
ProducerConsumer - Duration of single invocation 3.92 μs 3.20 μs 0.81 0.72 False
SingleSerial - Duration of single invocation 804.14 ns 599.79 ns 0.75 0.67 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2088769200000002 < 1.5893879905000001.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 20.69224157955675 (T) = (0 -1289.9206946529248) / Math.Sqrt((18415.905689558003 / (131)) + (7039.5525494249505 / (19))) is greater than 1.9761224936134032 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (19) - 2, .975) and 0.2661368389184131 = (1757.7128313019632 - 1289.9206946529248) / 1757.7128313019632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.162677263157892 < 104.31427435611111.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 11.816721018555958 (T) = (0 -17042.730805000625) / Math.Sqrt((2629410502.4694376 / (118)) + (6431475.174740647 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.7667519842756179 = (73066.99159721553 - 17042.730805000625) / 73066.99159721553 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9742098000000001 < 2.1604554675.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 16.95157786657432 (T) = (0 -1798.5585228254847) / Math.Sqrt((22986.846110184997 / (132)) + (9651.645689477846 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.19753474557147097 = (2241.2914614057872 - 1798.5585228254847) / 2241.2914614057872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.ProducerConsumer(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1959843399999994 < 3.7175679699999997.
IsChangePoint: Marked as a change because one of 6/3/2022 1:48:25 AM, 6/7/2022 11:31:29 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 5.672567136481508 (T) = (0 -2529.455993139204) / Math.Sqrt((282139.72549881256 / (119)) + (109610.18117087365 / (31))) is greater than 1.9761224936134032 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (119) + (31) - 2, .975) and 0.1470167273806424 = (2965.422739617978 - 2529.455993139204) / 2965.422739617978 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 599.794985 < 759.3754984999999.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 4.275727100039144 (T) = (0 -660.6798063429661) / Math.Sqrt((4769.433926843408 / (132)) + (7745.368062473782 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.11997707625887662 = (750.7529503144153 - 660.6798063429661) / 750.7529503144153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.25094
Baseline fae7ee8e7e3aa7f86836318a10ed676641e813ad
Compare 21b5b2e8b7c8c0e66c7a041a61cce8ff4d53ea1c
Diff Diff

Improvements in System.Threading.Tests.Perf_Interlocked

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Add_long - Duration of single invocation 10.70 ns 4.64 ns 0.43 0.73 False
Add_int - Duration of single invocation 10.58 ns 2.38 ns 0.22 0.76 False
Exchange_int - Duration of single invocation 11.83 ns 2.66 ns 0.22 0.69 False
Increment_int - Duration of single invocation 11.80 ns 5.98 ns 0.51 0.76 False
Exchange_long - Duration of single invocation 10.32 ns 4.58 ns 0.44 0.80 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Interlocked*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Interlocked.Add_long


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.641691251428749 < 9.36723488421777.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 21.241951086302546 (T) = (0 -5.738468228766232) / Math.Sqrt((1.8231346888718012 / (132)) + (0.7079363650491806 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.45549826197749105 = (10.538934640699 - 5.738468228766232) / 10.538934640699 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_Interlocked.Add_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3784293267481385 < 10.826274613683646.
IsChangePoint: Marked as a change because one of 6/10/2022 5:01:18 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 13.287350826090744 (T) = (0 -5.308242479364312) / Math.Sqrt((2.0496002943510288 / (130)) + (2.8618281423706433 / (19))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (130) + (19) - 2, .975) and 0.505209151169919 = (10.728255164612486 - 5.308242479364312) / 10.728255164612486 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_Interlocked.Exchange_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6579259811668874 < 11.151661545779572.
IsChangePoint: Marked as a change because one of 6/10/2022 5:01:18 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 21.65238823469365 (T) = (0 -5.395443424637532) / Math.Sqrt((1.8930687530826569 / (131)) + (0.9995897304698655 / (20))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (20) - 2, .975) and 0.5046188847316679 = (10.891500015528434 - 5.395443424637532) / 10.891500015528434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_Interlocked.Increment_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.978483662150146 < 11.502357677909341.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 18.659285072849425 (T) = (0 -5.364804067011612) / Math.Sqrt((1.8070535770127347 / (132)) + (1.427440153838906 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.5089746215412269 = (10.92571647488083 - 5.364804067011612) / 10.92571647488083 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_Interlocked.Exchange_long

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.580245616080989 < 10.073633430066318.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 18.439353254683184 (T) = (0 -5.1754159757046265) / Math.Sqrt((1.7975519066926897 / (81)) + (1.2503829012579408 / (19))) is greater than 1.9844674545082206 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (81) + (19) - 2, .975) and 0.5138383222011478 = (10.645462635263362 - 5.1754159757046265) / 10.645462635263362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline fae7ee8e7e3aa7f86836318a10ed676641e813ad
Compare 21b5b2e8b7c8c0e66c7a041a61cce8ff4d53ea1c
Diff Diff

Improvements in System.Threading.Tests.Perf_SemaphoreSlim

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReleaseWaitAsync_WithCancellationTokenAndTimeout - Duration of single invocation 3.90 μs 3.05 μs 0.78 0.36 False
ReleaseWaitAsync_WithTimeout - Duration of single invocation 3.62 μs 2.79 μs 0.77 0.37 False
ReleaseWaitAsync_WithCancellationToken - Duration of single invocation 3.36 μs 2.67 μs 0.79 0.38 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_SemaphoreSlim*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWaitAsync_WithCancellationTokenAndTimeout


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0541807259354914 < 3.572255502051482.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 21.828621097737024 (T) = (0 -3094.768125070218) / Math.Sqrt((26481.281037278062 / (132)) + (22445.960924753712 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.20773886433425623 = (3906.247556204632 - 3094.768125070218) / 3906.247556204632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWaitAsync_WithTimeout

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.791998931674943 < 3.2944168040954125.
IsChangePoint: Marked as a change because one of 6/11/2022 3:15:10 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 14.620896987146681 (T) = (0 -2872.9980014834323) / Math.Sqrt((33012.233016335675 / (133)) + (32018.24620631879 / (18))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (18) - 2, .975) and 0.18641082527894054 = (3531.2638008838367 - 2872.9980014834323) / 3531.2638008838367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWaitAsync_WithCancellationToken

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.670943258951298 < 3.2971070359332186.
IsChangePoint: Marked as a change because one of 6/11/2022 3:15:10 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 39.79281097264622 (T) = (0 -2621.5427262110034) / Math.Sqrt((32049.161391063677 / (133)) + (4873.156748318587 / (18))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (18) - 2, .975) and 0.255600779814822 = (3521.6892429829036 - 2621.5427262110034) / 3521.6892429829036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline fae7ee8e7e3aa7f86836318a10ed676641e813ad
Compare 21b5b2e8b7c8c0e66c7a041a61cce8ff4d53ea1c
Diff Diff

Improvements in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Yield - Duration of single invocation 1.03 μs 471.78 ns 0.46 0.86 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.Tests.Perf_AsyncMethods.Yield


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 471.784084 < 1.200061438147059.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 19.755743310903494 (T) = (0 -455.09642340283904) / Math.Sqrt((125430.13523338201 / (132)) + (4128.06549996032 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.5973036352507621 = (1130.122998965315 - 455.09642340283904) / 1130.122998965315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline fae7ee8e7e3aa7f86836318a10ed676641e813ad
Compare 21b5b2e8b7c8c0e66c7a041a61cce8ff4d53ea1c
Diff Diff

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateAndAwait_FromYieldingAsyncMethod - Duration of single invocation 2.76 μs 1.76 μs 0.64 0.62 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromYieldingAsyncMethod


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7557875310075055 < 2.3599269326358066.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 12.719107245224997 (T) = (0 -1712.4566425880823) / Math.Sqrt((52915.26037374973 / (132)) + (38800.787228289344 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.2685319233087901 = (2341.1228694140236 - 1712.4566425880823) / 2341.1228694140236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline fae7ee8e7e3aa7f86836318a10ed676641e813ad
Compare 21b5b2e8b7c8c0e66c7a041a61cce8ff4d53ea1c
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultipleSerial - Duration of single invocation 2.39 μs 2.04 μs 0.86 0.69 False
SingleSerial - Duration of single invocation 902.84 ns 760.47 ns 0.84 0.69 False
MultipleSerial - Duration of single invocation 1.81 μs 1.11 μs 0.61 0.67 False
SingleSerial - Duration of single invocation 915.21 ns 665.16 ns 0.73 0.67 False
SingleParallel - Duration of single invocation 126.37 μs 14.94 μs 0.12 0.64 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.04128068 < 2.222135557.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 12.969491327639771 (T) = (0 -1959.0685443571772) / Math.Sqrt((37561.70075525859 / (132)) + (20558.781168402307 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.19661536519368739 = (2438.518810892478 - 1959.0685443571772) / 2438.518810892478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 760.4748769230769 < 823.3627579999999.
IsChangePoint: Marked as a change because one of 6/11/2022 3:15:10 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 6.114028128651311 (T) = (0 -779.6730119640981) / Math.Sqrt((5816.837774796849 / (133)) + (3063.4954204266564 / (18))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (18) - 2, .975) and 0.10289491322500206 = (869.0988641776001 - 779.6730119640981) / 869.0988641776001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1054470749999998 < 1.8302768494999997.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 15.608500982986238 (T) = (0 -1359.3183822635926) / Math.Sqrt((19342.380699854086 / (132)) + (13674.47069714456 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.252591051423559 = (1818.7076631242244 - 1359.3183822635926) / 1818.7076631242244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 665.1576666666667 < 765.448805.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 8.44190040128098 (T) = (0 -740.5308276801005) / Math.Sqrt((7440.414620032086 / (132)) + (1899.2338186391503 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.12475122748126974 = (846.0803955760512 - 740.5308276801005) / 846.0803955760512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.93592746 < 116.55986645388887.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 11.86454814916985 (T) = (0 -16154.500536067178) / Math.Sqrt((2997369031.3971334 / (118)) + (3444563.7281827508 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.7876485531979914 = (76074.36059114423 - 16154.500536067178) / 76074.36059114423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline fae7ee8e7e3aa7f86836318a10ed676641e813ad
Compare 21b5b2e8b7c8c0e66c7a041a61cce8ff4d53ea1c
Diff Diff

Improvements in System.Threading.Tests.Perf_SpinLock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnterExit - Duration of single invocation 37.63 ns 22.04 ns 0.59 0.70 False
TryEnterExit - Duration of single invocation 35.88 ns 19.45 ns 0.54 0.71 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_SpinLock*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_SpinLock.EnterExit


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.04162518953475 < 34.635148009651786.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 26.905188904321516 (T) = (0 -20.88872708704917) / Math.Sqrt((14.1754976268178 / (132)) + (2.776138660311398 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.3933908842553616 = (34.43523439539376 - 20.88872708704917) / 34.43523439539376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_SpinLock.TryEnterExit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.450927277049765 < 34.4558404644414.
IsChangePoint: Marked as a change because one of 6/10/2022 9:37:39 PM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 24.581236212645305 (T) = (0 -21.02585357054767) / Math.Sqrt((12.026628079773038 / (132)) + (3.546477699583777 / (19))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (19) - 2, .975) and 0.38124793068905294 = (33.98106384349134 - 21.02585357054767) / 33.98106384349134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions