Skip to content

[Perf] Changes at 4/26/2022 4:04:55 PM #5072

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 8006e6a89bc02e410331e6323e3f6321b224b327
Compare e4163ea55ebb3673c29e1c2a850a6a790029d278
Diff Diff

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 514.46 μs 461.12 μs 0.90 0.03 False
ICollection - Duration of single invocation 494.01 μs 455.23 μs 0.92 0.04 False
Span - Duration of single invocation 473.20 μs 424.89 μs 0.90 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.Stack(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 461.12389705882356 < 488.2838155241936.
IsChangePoint: Marked as a change because one of 2/25/2022 7:19:51 AM, 3/3/2022 7:35:51 PM, 4/26/2022 12:44:27 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 67.9937603115472 (T) = (0 -462611.0872621107) / Math.Sqrt((174228088.07872295 / (299)) + (935941.6289839866 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.10299045612258091 = (515725.9367190511 - 462611.0872621107) / 515725.9367190511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 455.22952380952376 < 469.0407380022321.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/3/2022 7:35:51 PM, 3/31/2022 1:45:25 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 30.73076162823169 (T) = (0 -456325.51539034926) / Math.Sqrt((591438002.8495884 / (299)) + (624645.9023218165 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.08682370455498944 = (499712.3969013803 - 456325.51539034926) / 499712.3969013803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 424.8935360360361 < 449.5544779040404.
IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 27.197387271793236 (T) = (0 -424371.5657012943) / Math.Sqrt((499090848.58157533 / (299)) + (212733.06170352397 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.07658447455680696 = (459567.2847254948 - 424371.5657012943) / 459567.2847254948 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions