Skip to content

[Perf] Changes at 3/30/2022 8:03:41 PM #4427

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2f21e533978677667d698f86064bf96932bed4e4
Compare c0db07b3f3f093b5f4cd27e1f5e8aa54adad049d
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 23.35 μs 21.15 μs 0.91 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfStrings)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/6/2021 6:58:53 PM, 2/3/2022 8:57:20 PM, 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 16.268961976217884 (T) = (0 -21144.745951450263) / Math.Sqrt((382999.1493529534 / (299)) + (160039.45482607657 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.07370508266605101 = (22827.228732193435 - 21144.745951450263) / 22827.228732193435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 2f21e533978677667d698f86064bf96932bed4e4
Compare c0db07b3f3f093b5f4cd27e1f5e8aa54adad049d
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadReturnBytes - Duration of single invocation 215.42 ns 190.08 ns 0.88 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: HelloWorld)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 23.147331555551425 (T) = (0 -193.83802346955937) / Math.Sqrt((64.38656334035733 / (299)) + (8.882872847524093 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.09303375320529843 = (213.72132001008856 - 193.83802346955937) / 213.72132001008856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2f21e533978677667d698f86064bf96932bed4e4
Compare c0db07b3f3f093b5f4cd27e1f5e8aa54adad049d
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Indexer5 - Duration of single invocation 916.02 ns 792.03 ns 0.86 0.00 True
Fixed2 - Duration of single invocation 1.27 μs 981.07 ns 0.77 0.02 False
CoveredIndex1 - Duration of single invocation 1.81 μs 1.58 μs 0.87 0.02 True
Fixed1 - Duration of single invocation 1.27 μs 936.81 ns 0.74 0.02 False
Indexer6 - Duration of single invocation 919.34 ns 789.89 ns 0.86 0.00 True

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'
Details

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.Indexer5(length: 1024)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 1012.2516095370784 (T) = (0 -792.2556437768623) / Math.Sqrt((4.192152244834137 / (299)) + (0.04488686982677043 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.14217194148578266 = (923.5599557667438 - 792.2556437768623) / 923.5599557667438 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### Span.IndexerBench.Fixed2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 32.15634749462845 (T) = (0 -966.3020986865672) / Math.Sqrt((37.61139641141697 / (299)) + (1453.691443334185 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.23544661878626397 = (1263.8778696558154 - 966.3020986865672) / 1263.8778696558154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.CoveredIndex1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 143.43228390895982 (T) = (0 -1583.8682548318968) / Math.Sqrt((452.18874915698774 / (299)) + (2.400694324558875 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10430330229695033 = (1768.308690758394 - 1583.8682548318968) / 1768.308690758394 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### Span.IndexerBench.Fixed1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 23.03718283952055 (T) = (0 -941.8437401171865) / Math.Sqrt((31.340125419406824 / (299)) + (3309.8456655801283 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.25450290831856126 = (1263.3768134398697 - 941.8437401171865) / 1263.3768134398697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer6(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 133.8094386794249 (T) = (0 -792.40434010995) / Math.Sqrt((16.627129738494016 / (299)) + (15.076155431738766 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.1408442369903772 = (922.3057962553349 - 792.40434010995) / 922.3057962553349 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2f21e533978677667d698f86064bf96932bed4e4
Compare c0db07b3f3f093b5f4cd27e1f5e8aa54adad049d
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 3.31 μs 1.82 μs 0.55 0.02 True
SingleParallel - Duration of single invocation 3.42 μs 2.24 μs 0.66 0.02 True
SingleSerial - Duration of single invocation 4.83 μs 3.29 μs 0.68 0.09 True
MultipleSerial - Duration of single invocation 28.98 μs 18.52 μs 0.64 0.00 True
ProducerConsumer - Duration of single invocation 4.28 μs 3.48 μs 0.81 0.55 False
MultipleSerial - Duration of single invocation 29.15 μs 18.92 μs 0.65 0.01 True
MultipleSerial - Duration of single invocation 27.54 μs 17.08 μs 0.62 0.00 True
ProducerConsumer - Duration of single invocation 3.99 μs 2.78 μs 0.70 0.03 True
SingleSerial - Duration of single invocation 3.41 μs 1.85 μs 0.54 0.01 True
ProducerConsumer - Duration of single invocation 8.75 μs 6.82 μs 0.78 0.65 False
SingleSerial - Duration of single invocation 4.55 μs 3.20 μs 0.70 0.09 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 78.74625111055448 (T) = (0 -1814.8191174710191) / Math.Sqrt((99.49227864788256 / (299)) + (6179.631540557241 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.4528523311677933 = (3316.8726120033384 - 1814.8191174710191) / 3316.8726120033384 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 67.01293396083874 (T) = (0 -2211.9637140318832) / Math.Sqrt((232.81787729748933 / (299)) + (5523.51221554674 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.353480380612911 = (3421.340432218995 - 2211.9637140318832) / 3421.340432218995 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 67.22922486215384 (T) = (0 -3366.453944755666) / Math.Sqrt((10604.922571561303 / (299)) + (6288.814231082381 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.29248616709386405 = (4758.145760808446 - 3366.453944755666) / 4758.145760808446 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 410.6173824394327 (T) = (0 -18484.324240164617) / Math.Sqrt((7994.979790127664 / (299)) + (10466.402394102905 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.3602214146624097 = (28891.75202763475 - 18484.324240164617) / 28891.75202763475 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.ProducerConsumer(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 13.056022408314096 (T) = (0 -3615.5557357939456) / Math.Sqrt((82045.80807416352 / (299)) + (57102.55844223259 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.1787568460328365 = (4402.539879119053 - 3615.5557357939456) / 4402.539879119053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 127.7128364237592 (T) = (0 -18560.294277260935) / Math.Sqrt((63279.18805718723 / (299)) + (119560.49859883924 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.36935396494469974 = (29430.604880649753 - 18560.294277260935) / 29430.604880649753 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 745.3214370562816 (T) = (0 -17066.58257351325) / Math.Sqrt((2746.587537335524 / (299)) + (3195.9153174097855 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.38012890098107643 = (27532.47021925156 - 17066.58257351325) / 27532.47021925156 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.ProducerConsumer(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 75.35002186656249 (T) = (0 -2867.6997718374605) / Math.Sqrt((4504.502259630132 / (299)) + (3781.2578652199886 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2882181128346196 = (4028.9024257948827 - 2867.6997718374605) / 4028.9024257948827 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 105.11280222394153 (T) = (0 -1867.6450309765135) / Math.Sqrt((33.51090402642433 / (299)) + (3697.1046772759973 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.45360982333298744 = (3418.152651222929 - 1867.6450309765135) / 3418.152651222929 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.ProducerConsumer(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 9.250252963195182 (T) = (0 -6818.248459882355) / Math.Sqrt((261997.39447023533 / (299)) + (404264.61854643724 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.17562004189053032 = (8270.759608856184 - 6818.248459882355) / 8270.759608856184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 65.22726683366132 (T) = (0 -3277.2149817170525) / Math.Sqrt((11204.705483925267 / (299)) + (6917.4288083683105 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2955602825312979 = (4652.229141044503 - 3277.2149817170525) / 4652.229141044503 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2f21e533978677667d698f86064bf96932bed4e4
Compare c0db07b3f3f093b5f4cd27e1f5e8aa54adad049d
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 43.38 μs 38.63 μs 0.89 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: StringProperties)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/3/2021 12:49:43 AM, 12/7/2021 2:09:29 PM, 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 10.891358276801416 (T) = (0 -39201.4000774971) / Math.Sqrt((743167.9584109371 / (299)) + (757752.4556069308 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.05684419385742714 = (41564.076499541996 - 39201.4000774971) / 41564.076499541996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 2f21e533978677667d698f86064bf96932bed4e4
Compare c0db07b3f3f093b5f4cd27e1f5e8aa54adad049d
Diff Diff

Improvements in System.Memory.Slice<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlyMemoryStart - Duration of single invocation 23.29 ns 18.51 ns 0.79 0.50 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Slice&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Slice<String>.ReadOnlyMemoryStart


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/4/2022 9:46:30 AM falls between 3/22/2022 12:26:23 PM and 4/4/2022 9:46:30 AM.
IsImprovementStdDev: Marked as improvement because 29.04017056706458 (T) = (0 -18.330036449252802) / Math.Sqrt((1.1336357181924002 / (299)) + (0.27721100173795876 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.18714077678804766 = (22.5500750016996 - 18.330036449252802) / 22.5500750016996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions