[Perf] Windows/x64: 7 Improvements on 6/19/2023 6:37:24 PM #19277
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 919d316fa81bb0f77361e43b4eb8d1faf8d1b126 |
| Compare | 8b791c7254136de82a0b278bbc3bff93ed0db0d9 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Net.Http.Tests.SocketsHttpHandlerPerfTest
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Get_EnumerateHeaders_Validated - Duration of single invocation | 49.92 μs | 41.72 μs | 0.84 | 0.43 | False | |||||
| Get_EnumerateHeaders_Validated - Duration of single invocation | 43.97 μs | 33.13 μs | 0.75 | 0.44 | False | |||||
| Get_EnumerateHeaders_Unvalidated - Duration of single invocation | 42.39 μs | 33.84 μs | 0.80 | 0.30 | False | |||||
| Get - Duration of single invocation | 43.07 μs | 34.43 μs | 0.80 | 0.38 | False | |||||
| Get_EnumerateHeaders_Unvalidated - Duration of single invocation | 43.82 μs | 34.24 μs | 0.78 | 0.38 | False | |||||
| Get - Duration of single invocation | 40.33 μs | 33.20 μs | 0.82 | 0.26 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Http.Tests.SocketsHttpHandlerPerfTest*'Details
Payloads
Histogram
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Validated(ssl: True, chunkedResponse: False, responseLength: 1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 41.71864286862839 < 42.39866846415134.
IsChangePoint: Marked as a change because one of 6/19/2023 1:02:34 PM, 6/26/2023 10:42:28 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 10:42:28 PM.
IsImprovementStdDev: Marked as improvement because 11.844544096588251 (T) = (0 -35322.3740011501) / Math.Sqrt((4020056.2937255385 / (299)) + (12310133.702459833 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18681726131610465 = (43437.19107751597 - 35322.3740011501) / 43437.19107751597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Validated(ssl: True, chunkedResponse: True, responseLength: 1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.13455485169907 < 41.09606638396785.
IsChangePoint: Marked as a change because one of 6/19/2023 1:02:34 PM, 6/26/2023 10:42:28 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 10:42:28 PM.
IsImprovementStdDev: Marked as improvement because 12.976785347608159 (T) = (0 -35327.49975431021) / Math.Sqrt((4296652.351633681 / (299)) + (9788868.291390657 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1840178325943126 = (43294.450743488116 - 35327.49975431021) / 43294.450743488116 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl: True, chunkedResponse: True, responseLength: 1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.83686266917756 < 41.46404353616189.
IsChangePoint: Marked as a change because one of 6/19/2023 1:02:34 PM, 6/26/2023 10:42:28 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 10:42:28 PM.
IsImprovementStdDev: Marked as improvement because 15.290060837001144 (T) = (0 -34961.520788020964) / Math.Sqrt((3068779.826544535 / (299)) + (8372934.155386611 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.19842282106689094 = (43615.91336040079 - 34961.520788020964) / 43615.91336040079 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get(ssl: True, chunkedResponse: False, responseLength: 1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.43279658432449 < 41.635030791525345.
IsChangePoint: Marked as a change because one of 6/19/2023 1:02:34 PM, 6/26/2023 10:42:28 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 10:42:28 PM.
IsImprovementStdDev: Marked as improvement because 16.114220196861673 (T) = (0 -34566.4809258534) / Math.Sqrt((3233535.3992223702 / (299)) + (8203016.17703095 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.20728626054281665 = (43605.25017457507 - 34566.4809258534) / 43605.25017457507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl: True, chunkedResponse: False, responseLength: 1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.23954165611547 < 40.83686402365553.
IsChangePoint: Marked as a change because one of 6/19/2023 1:02:34 PM, 6/26/2023 10:42:28 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 10:42:28 PM.
IsImprovementStdDev: Marked as improvement because 20.32321140941084 (T) = (0 -34542.03630261782) / Math.Sqrt((3887461.6980474866 / (299)) + (4813166.903011834 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.20465410059563682 = (43430.20606315623 - 34542.03630261782) / 43430.20606315623 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get(ssl: True, chunkedResponse: True, responseLength: 1)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.200944372574384 < 40.44815643905197.
IsChangePoint: Marked as a change because one of 6/19/2023 1:02:34 PM, 6/26/2023 10:42:28 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 10:42:28 PM.
IsImprovementStdDev: Marked as improvement because 13.760326727676075 (T) = (0 -35153.64864674416) / Math.Sqrt((2244386.9478463433 / (299)) + (10128212.79995901 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1949300514736639 = (43665.33456017354 - 35153.64864674416) / 43665.33456017354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 919d316fa81bb0f77361e43b4eb8d1faf8d1b126 |
| Compare | 8b791c7254136de82a0b278bbc3bff93ed0db0d9 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Net.Security.Tests.SslStreamTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ReadWriteAsync - Duration of single invocation | 10.01 μs | 8.45 μs | 0.84 | 0.13 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Security.Tests.SslStreamTests*'Details
Payloads
Histogram
System.Net.Security.Tests.SslStreamTests.ReadWriteAsync
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.447512153846153 < 9.038304686666667.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/19/2023 1:02:34 PM, 6/26/2023 10:42:28 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 10:42:28 PM.
IsImprovementStdDev: Marked as improvement because 19.224840965480226 (T) = (0 -8718.048696343423) / Math.Sqrt((78554.06536510495 / (299)) + (77261.17667130604 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1097328238235794 = (9792.620608328261 - 8718.048696343423) / 9792.620608328261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository