-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 4ff7eca4e02f8e09d134e4fac0dd202d484df899 |
| Compare | 5d1b7e77e054f74de05d6cd34de11c55ffbd125f |
| Diff | Diff |
Improvements in System.Text.Json.Tests.Perf_Get
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetBoolean - Duration of single invocation | 127.63 ns | 99.74 ns | 0.78 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'Details
Payloads
Histogram
System.Text.Json.Tests.Perf_Get.GetBoolean
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.73504272108488 < 122.2453611726073.
IsChangePoint: Marked as a change because one of 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 25.776419416601573 (T) = (0 -101.92128460654257) / Math.Sqrt((14.318803130367185 / (299)) + (49.41738001032315 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.2365728997133398 = (133.50493395934205 - 101.92128460654257) / 133.50493395934205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 4ff7eca4e02f8e09d134e4fac0dd202d484df899 |
| Compare | 5d1b7e77e054f74de05d6cd34de11c55ffbd125f |
| Diff | Diff |
Improvements in System.Net.Tests.Perf_WebUtility
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Decode_DecodingRequired - Duration of single invocation | 110.72 ns | 87.89 ns | 0.79 | 0.05 | False | |||||
| Decode_NoDecodingRequired - Duration of single invocation | 65.53 ns | 61.78 ns | 0.94 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'Details
Payloads
Histogram
System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.88875413237298 < 104.6755196865118.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 31.770290414927796 (T) = (0 -87.68871223252478) / Math.Sqrt((49.81462130312615 / (299)) + (1.8287556561306895 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.14536364342313143 = (102.60353606269474 - 87.68871223252478) / 102.60353606269474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.78104647485643 < 62.186162783561876.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/13/2022 11:44:30 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.25341163566923 (T) = (0 -60.90430094422205) / Math.Sqrt((39.88441343232153 / (299)) + (13.648902336541328 / (38))) is greater than 1.967070609662329 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (38) - 2, .975) and 0.1577512573910721 = (72.31153679798496 - 60.90430094422205) / 72.31153679798496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 4ff7eca4e02f8e09d134e4fac0dd202d484df899 |
| Compare | 5d1b7e77e054f74de05d6cd34de11c55ffbd125f |
| Diff | Diff |
Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Utf8JsonReaderCommentParsing - Duration of single invocation | 112.50 ns | 104.01 ns | 0.92 | 0.05 | False | |||||
| Utf8JsonReaderCommentParsing - Duration of single invocation | 125.59 ns | 113.91 ns | 0.91 | 0.04 | False | |||||
| Utf8JsonReaderCommentParsing - Duration of single invocation | 143.60 ns | 136.06 ns | 0.95 | 0.04 | False | |||||
| Utf8JsonReaderCommentParsing - Duration of single invocation | 127.99 ns | 113.23 ns | 0.88 | 0.06 | False | |||||
| Utf8JsonReaderCommentParsing - Duration of single invocation | 115.94 ns | 104.56 ns | 0.90 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'Details
Payloads
Histogram
System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortMultiLine)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.0090975520643 < 106.97694227458074.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/2/2022 4:42:49 PM, 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 19.830854421002673 (T) = (0 -105.1777564121733) / Math.Sqrt((17.7383596066609 / (299)) + (2.951797950966978 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.0672326417517424 = (112.7588304651845 - 105.1777564121733) / 112.7588304651845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortMultiLine)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.90810489055681 < 119.3456618104302.
IsChangePoint: Marked as a change because one of 9/6/2022 1:06:44 PM, 10/25/2022 2:34:53 PM, 10/31/2022 4:38:04 PM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 37.3609307500711 (T) = (0 -114.56750298835152) / Math.Sqrt((12.945967032337588 / (299)) + (1.2342007126633001 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.08425233801083865 = (125.10815778605561 - 114.56750298835152) / 125.10815778605561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 136.06457500370206 < 136.459763061062.
IsChangePoint: Marked as a change because one of 10/17/2022 3:02:22 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 29.626232899160705 (T) = (0 -135.50595257544003) / Math.Sqrt((7.764817221149648 / (299)) + (0.8145417615700853 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.0560945064704041 = (143.55881336036666 - 135.50595257544003) / 143.55881336036666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortSingleLine)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.22847750269429 < 121.5173496957571.
IsChangePoint: Marked as a change because one of 10/31/2022 10:23:25 PM, 11/2/2022 9:35:40 AM, 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 16.43450958166973 (T) = (0 -116.4080314865383) / Math.Sqrt((10.294253151704769 / (299)) + (6.915864709044514 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.06441631853273934 = (124.42289641475735 - 116.4080314865383) / 124.42289641475735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.56258115320546 < 110.76626491296417.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 57.48701931638531 (T) = (0 -104.51672996022025) / Math.Sqrt((8.817789651080043 / (299)) + (0.20437837334692852 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.10158913672428713 = (116.33511373531239 - 104.51672996022025) / 116.33511373531239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 4ff7eca4e02f8e09d134e4fac0dd202d484df899 |
| Compare | 5d1b7e77e054f74de05d6cd34de11c55ffbd125f |
| Diff | Diff |
Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Enumerate - Duration of single invocation | 31.38 ns | 29.45 ns | 0.94 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>*'Details
Payloads
Histogram
System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 10)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.44943182171611 < 29.82317022857172.
IsChangePoint: Marked as a change because one of 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM.
IsImprovementStdDev: Marked as improvement because 18.882609050692356 (T) = (0 -29.840178560435994) / Math.Sqrt((0.1190768982624598 / (299)) + (0.2633524267482087 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.05402052294483224 = (31.544213467852824 - 29.840178560435994) / 31.544213467852824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository








_1.png)