Skip to content

[Perf] Windows/x64: 9 Improvements on 11/14/2022 6:23:13 PM #10064

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4ff7eca4e02f8e09d134e4fac0dd202d484df899
Compare 5d1b7e77e054f74de05d6cd34de11c55ffbd125f
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBoolean - Duration of single invocation 127.63 ns 99.74 ns 0.78 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetBoolean


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.73504272108488 < 122.2453611726073.
IsChangePoint: Marked as a change because one of 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 25.776419416601573 (T) = (0 -101.92128460654257) / Math.Sqrt((14.318803130367185 / (299)) + (49.41738001032315 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2365728997133398 = (133.50493395934205 - 101.92128460654257) / 133.50493395934205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 4ff7eca4e02f8e09d134e4fac0dd202d484df899
Compare 5d1b7e77e054f74de05d6cd34de11c55ffbd125f
Diff Diff

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_DecodingRequired - Duration of single invocation 110.72 ns 87.89 ns 0.79 0.05 False
Decode_NoDecodingRequired - Duration of single invocation 65.53 ns 61.78 ns 0.94 0.09 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'
Details

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.88875413237298 < 104.6755196865118.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 31.770290414927796 (T) = (0 -87.68871223252478) / Math.Sqrt((49.81462130312615 / (299)) + (1.8287556561306895 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.14536364342313143 = (102.60353606269474 - 87.68871223252478) / 102.60353606269474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.78104647485643 < 62.186162783561876.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/13/2022 11:44:30 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.25341163566923 (T) = (0 -60.90430094422205) / Math.Sqrt((39.88441343232153 / (299)) + (13.648902336541328 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.1577512573910721 = (72.31153679798496 - 60.90430094422205) / 72.31153679798496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4ff7eca4e02f8e09d134e4fac0dd202d484df899
Compare 5d1b7e77e054f74de05d6cd34de11c55ffbd125f
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 112.50 ns 104.01 ns 0.92 0.05 False
Utf8JsonReaderCommentParsing - Duration of single invocation 125.59 ns 113.91 ns 0.91 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 143.60 ns 136.06 ns 0.95 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 127.99 ns 113.23 ns 0.88 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 115.94 ns 104.56 ns 0.90 0.11 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortMultiLine)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.0090975520643 < 106.97694227458074.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/2/2022 4:42:49 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 19.830854421002673 (T) = (0 -105.1777564121733) / Math.Sqrt((17.7383596066609 / (299)) + (2.951797950966978 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.0672326417517424 = (112.7588304651845 - 105.1777564121733) / 112.7588304651845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.90810489055681 < 119.3456618104302.
IsChangePoint: Marked as a change because one of 9/6/2022 1:06:44 PM, 10/25/2022 2:34:53 PM, 10/31/2022 4:38:04 PM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 37.3609307500711 (T) = (0 -114.56750298835152) / Math.Sqrt((12.945967032337588 / (299)) + (1.2342007126633001 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.08425233801083865 = (125.10815778605561 - 114.56750298835152) / 125.10815778605561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 136.06457500370206 < 136.459763061062.
IsChangePoint: Marked as a change because one of 10/17/2022 3:02:22 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 29.626232899160705 (T) = (0 -135.50595257544003) / Math.Sqrt((7.764817221149648 / (299)) + (0.8145417615700853 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.0560945064704041 = (143.55881336036666 - 135.50595257544003) / 143.55881336036666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.22847750269429 < 121.5173496957571.
IsChangePoint: Marked as a change because one of 10/31/2022 10:23:25 PM, 11/2/2022 9:35:40 AM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.43450958166973 (T) = (0 -116.4080314865383) / Math.Sqrt((10.294253151704769 / (299)) + (6.915864709044514 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.06441631853273934 = (124.42289641475735 - 116.4080314865383) / 124.42289641475735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.56258115320546 < 110.76626491296417.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 57.48701931638531 (T) = (0 -104.51672996022025) / Math.Sqrt((8.817789651080043 / (299)) + (0.20437837334692852 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.10158913672428713 = (116.33511373531239 - 104.51672996022025) / 116.33511373531239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4ff7eca4e02f8e09d134e4fac0dd202d484df899
Compare 5d1b7e77e054f74de05d6cd34de11c55ffbd125f
Diff Diff

Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 31.38 ns 29.45 ns 0.94 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 10)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.44943182171611 < 29.82317022857172.
IsChangePoint: Marked as a change because one of 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 18.882609050692356 (T) = (0 -29.840178560435994) / Math.Sqrt((0.1190768982624598 / (299)) + (0.2633524267482087 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.05402052294483224 = (31.544213467852824 - 29.840178560435994) / 31.544213467852824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions