-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-x64branch-refs/heads/mainkind-microos-linuxperf-improvementruntime-coreclruntriaged
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | e1ddbca5a008b4b15e8e6b309cbab12ba8b9b842 |
| Compare | 13f648fa15c2989455415f4941dad2abfbaa6bba |
| Diff | Diff |
Improvements in System.Tests.Perf_Boolean
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryParse - Duration of single invocation | 3.71 ns | 1.65 ns | 0.45 | 0.03 | False | |||||
| TryParse - Duration of single invocation | 2.36 ns | 1.26 ns | 0.53 | 0.01 | False | |||||
| TryParse - Duration of single invocation | 2.62 ns | 1.42 ns | 0.54 | 0.06 | False | |||||
| TryParse - Duration of single invocation | 3.70 ns | 2.44 ns | 0.66 | 0.03 | False | |||||
| TryParse - Duration of single invocation | 7.73 ns | 5.85 ns | 0.76 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Boolean*'Related Issues
Regressions
Improvements
Details
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Boolean.TryParse(value: "false")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6504297179929168 < 3.519500426003292.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 10/2/2022 4:08:26 PM, 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 32.47166457195949 (T) = (0 -1.7248107257379335) / Math.Sqrt((0.03946836547922099 / (299)) + (0.041708324004581784 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4823172245744967 = (3.3317908333346526 - 1.7248107257379335) / 3.3317908333346526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Boolean.TryParse(value: "TRUE")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2596304760656933 < 2.239149826040456.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 52.187464879009525 (T) = (0 -1.2757795195474277) / Math.Sqrt((0.059939634884224546 / (299)) + (4.915728209267149E-05 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.368331827645947 = (2.019698910573489 - 1.2757795195474277) / 2.019698910573489 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Boolean.TryParse(value: "0")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4182718744318865 < 2.491752545290171.
IsChangePoint: Marked as a change because one of 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 64.10735471777033 (T) = (0 -1.4529093735338252) / Math.Sqrt((0.009849568984330237 / (299)) + (0.005826554803320315 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4545255365506479 = (2.6635699210302803 - 1.4529093735338252) / 2.6635699210302803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Boolean.TryParse(value: "False")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4443662739564163 < 3.5179672430589193.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 10/2/2022 4:08:26 PM, 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 36.2242148758075 (T) = (0 -1.7032618183747514) / Math.Sqrt((0.03844962219741705 / (299)) + (0.03423164711209614 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4893540317097453 = (3.3355042909231503 - 1.7032618183747514) / 3.3355042909231503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Boolean.TryParse(value: "Bogus")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.850626237883877 < 7.342644799658513.
IsChangePoint: Marked as a change because one of 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 131.7220101491379 (T) = (0 -5.956321242342658) / Math.Sqrt((0.01368865094079572 / (299)) + (0.002124447387593275 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.2206001685497826 = (7.642189543792718 - 5.956321242342658) / 7.642189543792718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-x64branch-refs/heads/mainkind-microos-linuxperf-improvementruntime-coreclruntriaged




