Skip to content

[Perf] Changes at 3/18/2022 1:59:24 AM #4260

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsDefinedMethodOverrideMissInherit - Duration of single invocation 1.62 μs 1.35 μs 0.83 0.47 False
GetCustomAttributesMethodOverrideHit - Duration of single invocation 5.79 μs 5.42 μs 0.94 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Attributes*'
Details

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3479277335572228 < 1.5275106267426564.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 24.443760397292365 (T) = (0 -1347.3631374591114) / Math.Sqrt((22322.450982389604 / (299)) + (817.2095583411191 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21871224714979393 = (1724.5414798117758 - 1347.3631374591114) / 1724.5414798117758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.424505461874098 < 5.52650802884869.
IsChangePoint: Marked as a change because one of 1/21/2022 1:44:30 AM, 2/13/2022 12:11:49 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 9.860798058745871 (T) = (0 -5456.715271358875) / Math.Sqrt((30279.05003853173 / (299)) + (16242.060048586287 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09468518699748901 = (6027.4229394982185 - 5456.715271358875) / 6027.4229394982185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetUInt32 - Duration of single invocation 1.24 μs 1.16 μs 0.93 0.01 False
GetSByte - Duration of single invocation 1.56 μs 1.44 μs 0.93 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetUInt32


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1561631167538886 < 1.1827897467688357.
IsChangePoint: Marked as a change because one of 11/29/2021 10:23:30 PM, 2/19/2022 1:08:37 AM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 12.677809229775786 (T) = (0 -1159.779005635974) / Math.Sqrt((346.87832684793 / (299)) + (333.76540373818403 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08263922954832893 = (1264.2561607086666 - 1159.779005635974) / 1264.2561607086666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetSByte

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.441577125373692 < 1.4793809206331394.
IsChangePoint: Marked as a change because one of 2/4/2022 3:19:13 PM, 3/18/2022 1:59:24 AM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 13.259588490896663 (T) = (0 -1447.630314812063) / Math.Sqrt((1148.5260358986075 / (299)) + (150.5559912689964 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.055705778156234925 = (1533.0288815974304 - 1447.630314812063) / 1533.0288815974304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Memory.Slice<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlyMemoryStartSpan - Duration of single invocation 31.26 ns 22.74 ns 0.73 0.30 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Slice&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Slice<String>.ReadOnlyMemoryStartSpan


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.742028412799524 < 29.739230570546876.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 3.646540944003522 (T) = (0 -24.39800699244101) / Math.Sqrt((0.8958633357604083 / (299)) + (11.047480099206753 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.19905258003206747 = (30.461434026989977 - 24.39800699244101) / 30.461434026989977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Security.Cryptography.Tests.Perf_CryptoConfig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromName - Duration of single invocation 2.64 μs 2.21 μs 0.84 0.33 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Tests.Perf_CryptoConfig*'
Details

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "RSA")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2147719226647395 < 2.617523996938975.
IsChangePoint: Marked as a change because one of 1/30/2022 12:18:49 AM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 6.5276129336056306 (T) = (0 -2258.2566048906265) / Math.Sqrt((31805.183286626703 / (299)) + (12387.312308963628 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12810796847946443 = (2590.064507129789 - 2258.2566048906265) / 2590.064507129789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions