Skip to content

[Perf] Linux/x64: 5 Improvements on 10/27/2022 2:21:25 PM #9336

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e1ddbca5a008b4b15e8e6b309cbab12ba8b9b842
Compare 13f648fa15c2989455415f4941dad2abfbaa6bba
Diff Diff

Improvements in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 3.71 ns 1.65 ns 0.45 0.03 False
TryParse - Duration of single invocation 2.36 ns 1.26 ns 0.53 0.01 False
TryParse - Duration of single invocation 2.62 ns 1.42 ns 0.54 0.06 False
TryParse - Duration of single invocation 3.70 ns 2.44 ns 0.66 0.03 False
TryParse - Duration of single invocation 7.73 ns 5.85 ns 0.76 0.01 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Boolean*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Boolean.TryParse(value: "false")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6504297179929168 < 3.519500426003292.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 10/2/2022 4:08:26 PM, 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 32.47166457195949 (T) = (0 -1.7248107257379335) / Math.Sqrt((0.03946836547922099 / (299)) + (0.041708324004581784 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4823172245744967 = (3.3317908333346526 - 1.7248107257379335) / 3.3317908333346526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Boolean.TryParse(value: "TRUE")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2596304760656933 < 2.239149826040456.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 52.187464879009525 (T) = (0 -1.2757795195474277) / Math.Sqrt((0.059939634884224546 / (299)) + (4.915728209267149E-05 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.368331827645947 = (2.019698910573489 - 1.2757795195474277) / 2.019698910573489 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Boolean.TryParse(value: "0")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4182718744318865 < 2.491752545290171.
IsChangePoint: Marked as a change because one of 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 64.10735471777033 (T) = (0 -1.4529093735338252) / Math.Sqrt((0.009849568984330237 / (299)) + (0.005826554803320315 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4545255365506479 = (2.6635699210302803 - 1.4529093735338252) / 2.6635699210302803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Boolean.TryParse(value: "False")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4443662739564163 < 3.5179672430589193.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 10/2/2022 4:08:26 PM, 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 36.2242148758075 (T) = (0 -1.7032618183747514) / Math.Sqrt((0.03844962219741705 / (299)) + (0.03423164711209614 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4893540317097453 = (3.3355042909231503 - 1.7032618183747514) / 3.3355042909231503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Boolean.TryParse(value: "Bogus")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.850626237883877 < 7.342644799658513.
IsChangePoint: Marked as a change because one of 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsImprovementStdDev: Marked as improvement because 131.7220101491379 (T) = (0 -5.956321242342658) / Math.Sqrt((0.01368865094079572 / (299)) + (0.002124447387593275 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.2206001685497826 = (7.642189543792718 - 5.956321242342658) / 7.642189543792718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions