Skip to content

[Perf] Changes at 4/16/2022 2:55:52 PM #4750

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 736883624a7ef175729cf082e12ce18f236d8524
Compare e44810457ae2f3adf58601e08aa5cadcd6185121
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf8 - Duration of single invocation 3.39 ms 2.74 ms 0.81 0.26 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7370071868131873 < 3.0595743602687433.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 3:23:01 AM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsImprovementStdDev: Marked as improvement because 14.096951754957521 (T) = (0 -2775535.7722858028) / Math.Sqrt((17361864692.351986 / (279)) + (10038893225.887302 / (9))) is greater than 1.9682932552436228 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (279) + (9) - 2, .975) and 0.1484262998074934 = (3259301.8920832872 - 2775535.7722858028) / 3259301.8920832872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 736883624a7ef175729cf082e12ce18f236d8524
Compare e44810457ae2f3adf58601e08aa5cadcd6185121
Diff Diff

Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CertProp - Duration of single invocation 122.33 ns 81.86 ns 0.67 0.44 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'
Details

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 81.85984041558085 < 122.70081308406493.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/16/2022 3:23:01 AM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsImprovementStdDev: Marked as improvement because 30.971036375267467 (T) = (0 -83.01845523505584) / Math.Sqrt((55.58192039327701 / (299)) + (24.538995542341482 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.3890009698399109 = (135.87330116269416 - 83.01845523505584) / 135.87330116269416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 736883624a7ef175729cf082e12ce18f236d8524
Compare e44810457ae2f3adf58601e08aa5cadcd6185121
Diff Diff

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToHexString - Duration of single invocation 256.11 ns 120.32 ns 0.47 0.14 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'
Details

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToHexString


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 120.31802416798789 < 244.05928104496496.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/16/2022 3:23:01 AM, 4/19/2022 4:53:57 AM falls between 4/10/2022 3:30:35 PM and 4/19/2022 4:53:57 AM.
IsImprovementStdDev: Marked as improvement because 91.73764448926471 (T) = (0 -123.61101308541235) / Math.Sqrt((385.58835007407765 / (299)) + (19.589981111397915 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.5801358127198216 = (294.40713647464736 - 123.61101308541235) / 294.40713647464736 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions