Skip to content

[Perf] Changes at 5/5/2022 2:36:12 AM #5282

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 6111bc7fcfc5445e8d48b2321edb5ba2b1461ae5
Compare d66d72dcc26cb1443f3ed3df92a2d91c17b2f7c4
Diff Diff

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 16.22 ns 3.47 ns 0.21 0.82 False
TryFormat - Duration of single invocation 66.64 ns 41.92 ns 0.63 0.41 True
TryFormat - Duration of single invocation 25.06 ns 9.95 ns 0.40 0.78 False
ToString - Duration of single invocation 35.24 ns 28.92 ns 0.82 0.58 False
ToString - Duration of single invocation 85.85 ns 68.36 ns 0.80 0.48 False
ToString - Duration of single invocation 13.09 ns 3.54 ns 0.27 0.83 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryFormat(value: 0)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4665684696318304 < 14.028606844601768.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 21.266340115464686 (T) = (0 -2.57192927689081) / Math.Sqrt((8.711951755108869 / (299)) + (4.757408586640993 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.7867349395731131 = (12.059777967111204 - 2.57192927689081) / 12.059777967111204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.91975302634436 < 63.571802047728966.
IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 62.467283467108025 (T) = (0 -42.97236417381113) / Math.Sqrt((16.44035103608032 / (290)) + (3.5456136688105215 / (28))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (28) - 2, .975) and 0.38362844228545506 = (69.71827891142337 - 42.97236417381113) / 69.71827891142337 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.95276842171851 < 22.049936238156256.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 23.197629177318678 (T) = (0 -8.94652077951926) / Math.Sqrt((10.040396931827939 / (275)) + (7.346370346563901 / (28))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (275) + (28) - 2, .975) and 0.5863581592891736 = (21.628664943916295 - 8.94652077951926) / 21.628664943916295 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.918512355551762 < 33.35829453426279.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 30.595087762649904 (T) = (0 -29.018131182520893) / Math.Sqrt((29.524225448516383 / (299)) + (0.33443931987097253 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.2596855393661085 = (39.19703413286595 - 29.018131182520893) / 39.19703413286595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.35994018325134 < 77.226127744924.
IsChangePoint: Marked as a change because one of 1/27/2022 11:49:03 PM, 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 43.1325674281997 (T) = (0 -67.1638575213719) / Math.Sqrt((58.32764880014823 / (299)) + (2.0790268220890233 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.24997005739675038 = (89.54823495213469 - 67.1638575213719) / 89.54823495213469 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.5439418410955996 < 9.328477804713275.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 14.492154853291531 (T) = (0 -4.545899375837662) / Math.Sqrt((5.625824517841812 / (299)) + (2.275923221023555 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5021453051505455 = (9.130976212270708 - 4.545899375837662) / 9.130976212270708 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 6111bc7fcfc5445e8d48b2321edb5ba2b1461ae5
Compare d66d72dcc26cb1443f3ed3df92a2d91c17b2f7c4
Diff Diff

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 73.64 ns 43.09 ns 0.59 0.45 False
TryFormat - Duration of single invocation 21.84 ns 13.86 ns 0.63 0.73 False
ToString - Duration of single invocation 88.55 ns 72.61 ns 0.82 0.47 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.08853281035844 < 60.752758808970654.
IsChangePoint: Marked as a change because one of 1/28/2022 4:34:00 PM, 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 56.949381621800754 (T) = (0 -41.239425601221704) / Math.Sqrt((16.388150087727134 / (299)) + (5.315770234673443 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.4058429929163168 = (69.40829630813964 - 41.239425601221704) / 69.40829630813964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.863692250415282 < 21.82158734640099.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 17.879463712784954 (T) = (0 -11.3278117417261) / Math.Sqrt((9.773724256259978 / (299)) + (12.011001114325884 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5174729558628449 = (23.47601420347798 - 11.3278117417261) / 23.47601420347798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.61198840971196 < 85.22773409594217.
IsChangePoint: Marked as a change because one of 1/28/2022 9:02:40 AM, 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 41.65058829694537 (T) = (0 -70.05438928433128) / Math.Sqrt((26.478729788444145 / (299)) + (7.722696953128098 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.26410288284755606 = (95.19590123604091 - 70.05438928433128) / 95.19590123604091 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 6111bc7fcfc5445e8d48b2321edb5ba2b1461ae5
Compare d66d72dcc26cb1443f3ed3df92a2d91c17b2f7c4
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString2 - Duration of single invocation 75.09 ns 65.37 ns 0.87 0.51 False
ToString4 - Duration of single invocation 100.12 ns 87.65 ns 0.88 0.50 False
ToString3 - Duration of single invocation 88.69 ns 80.57 ns 0.91 0.45 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.ToString2


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.36856615508697 < 70.90750268598526.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 12.569728449318905 (T) = (0 -65.6509830588634) / Math.Sqrt((24.355234602100786 / (275)) + (6.824289301555473 / (28))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (275) + (28) - 2, .975) and 0.0993975827089026 = (72.89674311149817 - 65.6509830588634) / 72.89674311149817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString4

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.65206635608342 < 93.65231997840058.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 17.05433188576439 (T) = (0 -82.66660460152009) / Math.Sqrt((38.737021869579124 / (299)) + (18.756542150702074 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.15573132716330482 = (97.91504441798718 - 82.66660460152009) / 97.91504441798718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString3

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.56997386336843 < 83.62590549092006.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 13.91601619719142 (T) = (0 -75.09268908011379) / Math.Sqrt((31.73953649621583 / (291)) + (18.320504383826623 / (28))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (291) + (28) - 2, .975) and 0.1393515564782871 = (87.25129249387797 - 75.09268908011379) / 87.25129249387797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 6111bc7fcfc5445e8d48b2321edb5ba2b1461ae5
Compare d66d72dcc26cb1443f3ed3df92a2d91c17b2f7c4
Diff Diff

Improvements in GuardedDevirtualization.TwoClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 5.72 ns 4.08 ns 0.71 0.41 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'GuardedDevirtualization.TwoClassInterface*'
Details

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.90)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.084067636278195 < 5.486973504155973.
IsChangePoint: Marked as a change because one of 4/22/2022 1:40:09 PM, 5/1/2022 10:22:11 AM, 5/5/2022 12:13:02 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 4.855352520355236 (T) = (0 -4.493065192247391) / Math.Sqrt((1.117472670289252 / (299)) + (0.6272157914660806 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1487248689599573 = (5.278041174253502 - 4.493065192247391) / 5.278041174253502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions