[Perf] Linux/arm64: 49 Improvements on 6/29/2023 8:27:03 AM #19587
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Guid
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| GuidToString - Duration of single invocation | 38.38 ns | 21.16 ns | 0.55 | 0.33 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'Details
Payloads
System.Tests.Perf_Guid.GuidToString
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.164160526024023 < 36.499539982447914.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 20.371981144127183 (T) = (0 -21.16773824845084) / Math.Sqrt((27.383449548939417 / (299)) + (0.8970607255522256 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2783272901129322 = (29.33149329113929 - 21.16773824845084) / 29.33149329113929 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| Enumerate - Duration of single invocation | 294.89 ns | 162.35 ns | 0.55 | 0.02 | True | |||
| Enumerate - Duration of single invocation | 43.16 ns | 17.98 ns | 0.42 | 0.08 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>*'Details
Payloads
System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 100)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 162.3528974848099 < 278.40317526176784.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 64.79431440176215 (T) = (0 -162.66068161221958) / Math.Sqrt((1067.2417325608753 / (299)) + (4.854128653082194 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44164036723489536 = (291.31884195620034 - 162.66068161221958) / 291.31884195620034 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 10)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.98057055402939 < 41.03266236551283.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 8.135325198940713 (T) = (0 -21.810067506858754) / Math.Sqrt((11.970350089206132 / (299)) + (79.2324209603121 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4802192621998911 = (41.96013034104817 - 21.810067506858754) / 41.96013034104817 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| IterateForEachArray - Duration of single invocation | 17.33 ns | 4.73 ns | 0.27 | 0.12 | True | |||
| IterateForEachTenSegments - Duration of single invocation | 69.18 ns | 47.39 ns | 0.69 | 0.02 | True | |||
| IterateForEachSingleSegment - Duration of single invocation | 24.19 ns | 10.39 ns | 0.43 | 0.04 | True | |||
| IterateForEachMemory - Duration of single invocation | 29.22 ns | 19.33 ns | 0.66 | 0.39 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'Details
Payloads
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachArray
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.726460174061214 < 16.617630919893617.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 422.26601069419496 (T) = (0 -4.656651352996352) / Math.Sqrt((0.22908110386055186 / (299)) + (0.0027382636180881856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7391819528257508 = (17.854022769694705 - 4.656651352996352) / 17.854022769694705 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachTenSegments
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 47.39114875395558 < 64.11722901336069.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 36.64644226629069 (T) = (0 -46.592283499362495) / Math.Sqrt((1.5565882804277735 / (299)) + (4.79036987311246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32469570424869726 = (68.99450187493142 - 46.592283499362495) / 68.99450187493142 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachSingleSegment
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.387676395875422 < 22.81533878340542.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 242.26623925917195 (T) = (0 -10.59233048852666) / Math.Sqrt((0.41497481226572697 / (299)) + (0.01828393576157915 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5473145138775712 = (23.398873640189926 - 10.59233048852666) / 23.398873640189926 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateForEachMemory
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.332638454163806 < 30.520985374539993.
IsChangePoint: Marked as a change because one of 4/9/2023 11:24:15 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 48.346302211034185 (T) = (0 -15.262344012389443) / Math.Sqrt((7.072165910474404 / (299)) + (1.5444840572586356 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5445444096011574 = (33.51005967239133 - 15.262344012389443) / 33.51005967239133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Binary.Tests.BinaryReadAndWriteTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| ReadStructAndReverseBE - Duration of single invocation | 11.12 ns | 1.82 ns | 0.16 | 0.23 | True | |||
| ReadStructFieldByFieldUsingBitConverterLE - Duration of single invocation | 8.76 ns | 1.93 ns | 0.22 | 0.35 | True | |||
| ReadStructFieldByFieldUsingBitConverterBE - Duration of single invocation | 18.09 ns | 2.13 ns | 0.12 | 0.14 | True | |||
| ReadStructFieldByFieldBE - Duration of single invocation | 9.01 ns | 5.89 ns | 0.65 | 0.41 | False | |||
| ReadStructFieldByFieldLE - Duration of single invocation | 9.15 ns | 5.32 ns | 0.58 | 0.31 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'Details
Payloads
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8238390012589043 < 11.011082214402222.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 113.53978440526505 (T) = (0 -2.429658571280879) / Math.Sqrt((0.17864075078957542 / (299)) + (0.07809191505732772 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7915679916695139 = (11.656839996611533 - 2.429658571280879) / 11.656839996611533 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9267575116476343 < 8.537915763616553.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 92.28637060645275 (T) = (0 -1.9041677289769827) / Math.Sqrt((0.21444198876151996 / (299)) + (0.06559029581381513 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7862837592095383 = (8.909794229648302 - 1.9041677289769827) / 8.909794229648302 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1332528244143134 < 17.355999833040364.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 203.57637293376578 (T) = (0 -2.0383873453740136) / Math.Sqrt((1.0158377971978847 / (299)) + (0.04066272283584183 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8897168490990426 = (18.483216418114832 - 2.0383873453740136) / 18.483216418114832 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.886265079270616 < 8.387061174979525.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 39.26949896593799 (T) = (0 -5.936906531550207) / Math.Sqrt((0.3103882325095821 / (299)) + (0.05926678461907569 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33103757279805013 = (8.87479818019427 - 5.936906531550207) / 8.87479818019427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.323512341990691 < 8.355354323001189.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 30.065627011269456 (T) = (0 -5.715554141206766) / Math.Sqrt((0.19323592754286922 / (299)) + (0.1443437890171988 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36313687413548057 = (8.97454085357525 - 5.715554141206766) / 8.97454085357525 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| EnumerateProperties - Duration of single invocation | 1.30 μs | 706.82 ns | 0.54 | 0.00 | True | |||
| EnumerateProperties - Duration of single invocation | 2.30 μs | 1.30 μs | 0.57 | 0.01 | True | |||
| EnumerateProperties - Duration of single invocation | 1.30 μs | 706.53 ns | 0.54 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'Details
Payloads
System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 706.8178703297953 < 1.2345553313757465.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 638.9221938280333 (T) = (0 -707.7282333462473) / Math.Sqrt((242.9481094183197 / (299)) + (0.321182534187444 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4523701985498161 = (1292.3479172829987 - 707.7282333462473) / 1292.3479172829987 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3018600067422916 < 2.183620923128144.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 196.38829341563343 (T) = (0 -1301.4255267230278) / Math.Sqrt((6725.969558477173 / (299)) + (3.062820674735137 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41841867533016597 = (2237.736102447671 - 1301.4255267230278) / 2237.736102447671 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 706.5322187043392 < 1.235752596686803.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 664.9773687861947 (T) = (0 -707.5979909515504) / Math.Sqrt((222.9150124373356 / (299)) + (0.3685640137651414 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4525709510697263 = (1292.583929066719 - 707.5979909515504) / 1292.583929066719 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| IterateForEachTenSegments - Duration of single invocation | 69.70 ns | 45.37 ns | 0.65 | 0.03 | True | |||
| IterateGetPositionMemory - Duration of single invocation | 31.76 ns | 27.37 ns | 0.86 | 0.37 | False | |||
| IterateForEachMemory - Duration of single invocation | 30.30 ns | 14.49 ns | 0.48 | 0.47 | False | |||
| IterateForEachArray - Duration of single invocation | 16.74 ns | 5.24 ns | 0.31 | 0.18 | True | |||
| IterateForEachSingleSegment - Duration of single invocation | 24.08 ns | 10.54 ns | 0.44 | 0.07 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'Details
Payloads
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachTenSegments
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 45.37023310536621 < 65.69228500037413.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 208.2610712548734 (T) = (0 -45.23438574658516) / Math.Sqrt((1.6874736898605143 / (299)) + (0.10442247392954851 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34998313010646953 = (69.58955658180737 - 45.23438574658516) / 69.58955658180737 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionMemory
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.36634231105157 < 30.9423074121565.
IsChangePoint: Marked as a change because one of 4/28/2023 10:02:43 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 31.371216242831682 (T) = (0 -27.379791326786307) / Math.Sqrt((9.814479308297178 / (299)) + (0.16595962997405925 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19655875189821187 = (34.078149947458954 - 27.379791326786307) / 34.078149947458954 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachMemory
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.486798119937893 < 28.250639234407423.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 80.58752189103146 (T) = (0 -14.50773606208521) / Math.Sqrt((9.067828762921291 / (299)) + (0.2596793318718425 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5547327342911166 = (32.58208536616387 - 14.50773606208521) / 32.58208536616387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachArray
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.242900143811187 < 16.22287694875414.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 149.01794226159242 (T) = (0 -5.079773965874079) / Math.Sqrt((1.0295783476527356 / (299)) + (0.04604912599931054 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7103019668335125 = (17.53472024076452 - 5.079773965874079) / 17.53472024076452 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.54251827455374 < 22.689202804370403.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 235.79854133271184 (T) = (0 -10.452394308402774) / Math.Sqrt((0.4908363954267743 / (299)) + (0.016714398926590168 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5496637299419125 = (23.210198696752872 - 10.452394308402774) / 23.210198696752872 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc |
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| Enumerate - Duration of single invocation | 241.55 ns | 118.64 ns | 0.49 | 0.00 | True | |||
| Enumerate - Duration of single invocation | 33.89 ns | 14.55 ns | 0.43 | 0.08 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>*'Details
Payloads
System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 100)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 118.64196054480946 < 230.90995927842906.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 510.05271694517245 (T) = (0 -119.41618338327262) / Math.Sqrt((8.985698362899361 / (299)) + (0.33632521896532913 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5025038818284261 = (240.03440232289205 - 119.41618338327262) / 240.03440232289205 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 10)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.548790462287723 < 33.56658911351127.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 208.18700848051893 (T) = (0 -14.772665400491018) / Math.Sqrt((0.6840387646109801 / (299)) + (0.09096530645538403 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5759050223180624 = (34.83338916493892 - 14.772665400491018) / 34.83338916493892 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository