Skip to content

[Perf] Changes at 3/17/2022 11:24:40 PM #4218

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 4.69 μs 4.18 μs 0.89 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.180895110408344 < 4.315811713357666.
IsChangePoint: Marked as a change because one of 2/10/2022 7:02:09 PM, 2/12/2022 12:07:16 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 20.63971664666077 (T) = (0 -4227.071542218082) / Math.Sqrt((72833.82537538528 / (299)) + (5164.171745799637 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.09978770311993203 = (4695.638525343583 - 4227.071542218082) / 4695.638525343583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 58.49 μs 54.03 μs 0.92 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json40KB)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.03174194059976 < 55.685091026191024.
IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 50.291398571843054 (T) = (0 -53440.6206771865) / Math.Sqrt((547922.1396962462 / (299)) + (178858.2139337474 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.09074239600247935 = (58773.90570311054 - 53440.6206771865) / 58773.90570311054 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCredential_Uri - Duration of single invocation 608.84 ns 552.95 ns 0.91 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 10)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 552.948936272765 < 578.9638711291733.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 51.07943613254676 (T) = (0 -557.7586601840494) / Math.Sqrt((42.76080495593513 / (299)) + (11.38107238836583 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.07311904688761899 = (601.7586814262901 - 557.7586601840494) / 601.7586814262901 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DivideBenchmark - Duration of single invocation 42.91 ns 38.10 ns 0.89 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.DivideBenchmark


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.10417303318036 < 40.01222531426329.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 11.021525868155441 (T) = (0 -38.71668146447877) / Math.Sqrt((0.2198849060578672 / (227)) + (1.9233838639554397 / (18))) is greater than 1.9697743954265454 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (227) + (18) - 2, .975) and 0.0854851541326367 = (42.335760473913666 - 38.71668146447877) / 42.335760473913666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DivideBenchmark - Duration of single invocation 87.92 ns 76.59 ns 0.87 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.DivideBenchmark


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.58797973688681 < 83.50385380175607.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 476.20184970368933 (T) = (0 -76.40034199423667) / Math.Sqrt((0.030828026395456235 / (239)) + (0.00899018647502874 / (19))) is greater than 1.9692738896555546 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (239) + (19) - 2, .975) and 0.13265984180573986 = (88.0857888020505 - 76.40034199423667) / 88.0857888020505 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 4.82 μs 4.51 μs 0.94 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.5080620325473655 < 4.632911729179898.
IsChangePoint: Marked as a change because one of 2/5/2022 11:25:50 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 24.937950566688222 (T) = (0 -4532.239085813868) / Math.Sqrt((10838.77310157084 / (299)) + (1547.493857707327 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.058448151857028924 = (4813.58418524995 - 4532.239085813868) / 4813.58418524995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_Start_And_Length - Duration of single invocation 5.71 ns 4.44 ns 0.78 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Single)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.43726819473486 < 5.307940608849598.
IsChangePoint: Marked as a change because one of 2/4/2022 1:40:45 AM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 24.215139028515697 (T) = (0 -4.246351586884894) / Math.Sqrt((0.11859256430348898 / (299)) + (0.03211063136850006 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.20666007289742913 = (5.352499529922038 - 4.246351586884894) / 5.352499529922038 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 1.69 μs 1.47 μs 0.87 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.Stack(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4675077664159697 < 1.6057225504629287.
IsChangePoint: Marked as a change because one of 1/25/2022 2:47:22 PM, 2/8/2022 3:28:17 PM, 3/9/2022 10:22:25 AM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 28.13995812341157 (T) = (0 -1476.025382622646) / Math.Sqrt((2986.485786270234 / (299)) + (131.46037992020126 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.07345380588324092 = (1593.0402520617813 - 1476.025382622646) / 1593.0402520617813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 13.02 μs 11.24 μs 0.86 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.IDictionary(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.238923596570395 < 12.535246841274713.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 17.93353788026687 (T) = (0 -11518.175173306983) / Math.Sqrt((271894.4131655773 / (299)) + (102438.71912730271 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.11477890314662222 = (13011.636543965895 - 11518.175173306983) / 13011.636543965895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse3 - Duration of single invocation 45.78 ns 42.32 ns 0.92 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.TryParse3


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.31662137780571 < 43.7159239237716.
IsChangePoint: Marked as a change because one of 1/26/2022 10:02:18 PM, 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 43.704975473920015 (T) = (0 -42.699262725261306) / Math.Sqrt((2.2123257644177117 / (299)) + (0.06678235194314329 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.0966008668812725 = (47.26511367999026 - 42.699262725261306) / 47.26511367999026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SliceMemory - Duration of single invocation 8.49 ns 6.86 ns 0.81 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.SliceMemory


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.861159189728481 < 8.06154886638971.
IsChangePoint: Marked as a change because one of 2/4/2022 3:19:13 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 29.19868222552355 (T) = (0 -6.922449114198339) / Math.Sqrt((0.17523061958302644 / (299)) + (0.012151319129049409 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.12866560911805727 = (7.944652692052715 - 6.922449114198339) / 7.944652692052715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 18.60 μs 17.56 μs 0.94 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableDictionary(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.563634419676447 < 17.630245972897164.
IsChangePoint: Marked as a change because one of 1/31/2022 1:37:35 PM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 27.167847398632684 (T) = (0 -17048.994213766924) / Math.Sqrt((329971.6225986415 / (299)) + (57913.12364451031 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.09311481450253895 = (18799.506802412812 - 17048.994213766924) / 18799.506802412812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline c032e0d89f5fc90b596b67a37684ee3c05c6b3ea
Compare 5371203d5820a21922357e954e8c43eb4b76fd1d
Diff Diff

Improvements in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 543.40 ns 481.40 ns 0.89 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;String, String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<String, String>.Enumerate(Size: 100)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 481.398622936912 < 526.9843514695369.
IsChangePoint: Marked as a change because one of 3/1/2022 1:15:47 PM, 3/7/2022 10:58:04 AM, 3/17/2022 6:54:53 PM, 3/22/2022 2:14:22 AM falls between 3/13/2022 11:50:20 AM and 3/22/2022 2:14:22 AM.
IsImprovementStdDev: Marked as improvement because 27.33867623581682 (T) = (0 -493.4683981903391) / Math.Sqrt((511.14472978408384 / (299)) + (99.76459396958376 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.12752223857270692 = (565.5942420619069 - 493.4683981903391) / 565.5942420619069 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions