-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 736883624a7ef175729cf082e12ce18f236d8524 |
| Compare | e44810457ae2f3adf58601e08aa5cadcd6185121 |
| Diff | Diff |
Improvements in System.Text.Json.Tests.Perf_Basic
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteBasicUtf8 - Duration of single invocation | 3.39 ms | 2.74 ms | 0.81 | 0.26 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'Details
Payloads
Histogram
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7370071868131873 < 3.0595743602687433.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 3:23:01 AM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsImprovementStdDev: Marked as improvement because 14.096951754957521 (T) = (0 -2775535.7722858028) / Math.Sqrt((17361864692.351986 / (279)) + (10038893225.887302 / (9))) is greater than 1.9682932552436228 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (279) + (9) - 2, .975) and 0.1484262998074934 = (3259301.8920832872 - 2775535.7722858028) / 3259301.8920832872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 736883624a7ef175729cf082e12ce18f236d8524 |
| Compare | e44810457ae2f3adf58601e08aa5cadcd6185121 |
| Diff | Diff |
Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CertProp - Duration of single invocation | 122.33 ns | 81.86 ns | 0.67 | 0.44 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'Details
Payloads
Histogram
System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 81.85984041558085 < 122.70081308406493.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/16/2022 3:23:01 AM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsImprovementStdDev: Marked as improvement because 30.971036375267467 (T) = (0 -83.01845523505584) / Math.Sqrt((55.58192039327701 / (299)) + (24.538995542341482 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.3890009698399109 = (135.87330116269416 - 83.01845523505584) / 135.87330116269416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 736883624a7ef175729cf082e12ce18f236d8524 |
| Compare | e44810457ae2f3adf58601e08aa5cadcd6185121 |
| Diff | Diff |
Improvements in System.Perf_Convert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToHexString - Duration of single invocation | 256.11 ns | 120.32 ns | 0.47 | 0.14 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'Details
Payloads
Histogram
System.Perf_Convert.ToHexString
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 120.31802416798789 < 244.05928104496496.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/16/2022 3:23:01 AM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsImprovementStdDev: Marked as improvement because 91.73764448926471 (T) = (0 -123.61101308541235) / Math.Sqrt((385.58835007407765 / (299)) + (19.589981111397915 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.5801358127198216 = (294.40713647464736 - 123.61101308541235) / 294.40713647464736 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


