-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 736883624a7ef175729cf082e12ce18f236d8524 |
| Compare | e44810457ae2f3adf58601e08aa5cadcd6185121 |
| Diff | Diff |
Improvements in System.Net.NetworkInformation.Tests.PhysicalAddressTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| PALong - Duration of single invocation | 808.70 ns | 401.64 ns | 0.50 | 0.00 | True | |||||
| PAMedium - Duration of single invocation | 33.98 ns | 23.94 ns | 0.70 | 0.02 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'Details
Payloads
Histogram
System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 401.6430689877499 < 767.8209685584123.
IsChangePoint: Marked as a change because one of 4/16/2022 3:23:01 AM, 4/20/2022 10:05:23 PM falls between 4/12/2022 9:30:02 AM and 4/20/2022 10:05:23 PM.
IsImprovementStdDev: Marked as improvement because 1110.6138163159048 (T) = (0 -401.0452119535823) / Math.Sqrt((15.173706960971115 / (299)) + (1.325329280896813 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.5030177948736565 = (806.9609088953758 - 401.0452119535823) / 806.9609088953758 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.944814841373205 < 32.26041964013999.
IsChangePoint: Marked as a change because one of 4/16/2022 3:23:01 AM, 4/20/2022 10:05:23 PM falls between 4/12/2022 9:30:02 AM and 4/20/2022 10:05:23 PM.
IsImprovementStdDev: Marked as improvement because 87.98658019150041 (T) = (0 -24.100656630478777) / Math.Sqrt((0.20773158205750292 / (299)) + (0.19355829973428412 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.29224269507384937 = (34.05214819081734 - 24.100656630478777) / 34.05214819081734 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 736883624a7ef175729cf082e12ce18f236d8524 |
| Compare | e44810457ae2f3adf58601e08aa5cadcd6185121 |
| Diff | Diff |
Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CertProp - Duration of single invocation | 80.81 ns | 48.39 ns | 0.60 | 0.01 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'Details
Payloads
Histogram
System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.38873077408526 < 76.2695698585456.
IsChangePoint: Marked as a change because one of 4/16/2022 3:23:01 AM, 4/20/2022 10:05:23 PM falls between 4/12/2022 9:30:02 AM and 4/20/2022 10:05:23 PM.
IsImprovementStdDev: Marked as improvement because 144.25833515265265 (T) = (0 -48.64040677264793) / Math.Sqrt((0.9474538084095793 / (299)) + (0.7645071077467721 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.40100006638324753 = (81.2026914242843 - 48.64040677264793) / 81.2026914242843 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 736883624a7ef175729cf082e12ce18f236d8524 |
| Compare | e44810457ae2f3adf58601e08aa5cadcd6185121 |
| Diff | Diff |
Improvements in System.Perf_Convert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToHexString - Duration of single invocation | 211.35 ns | 111.57 ns | 0.53 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'Details
Payloads
Histogram
System.Perf_Convert.ToHexString
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.56850354202848 < 200.58035724415996.
IsChangePoint: Marked as a change because one of 4/16/2022 3:23:01 AM, 4/20/2022 10:05:23 PM falls between 4/12/2022 9:30:02 AM and 4/20/2022 10:05:23 PM.
IsImprovementStdDev: Marked as improvement because 452.390722166874 (T) = (0 -111.19334697851421) / Math.Sqrt((1.5310359963958664 / (299)) + (0.6833463175019683 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.470838405226232 = (210.13117368438768 - 111.19334697851421) / 210.13117368438768 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository



