Skip to content

[Perf] Changes at 5/30/2022 8:02:30 PM #5733

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 486b4d1a36aef5bbe6a77bb2c3412772e712961e
Compare afacf6a99e1b98331af958caec638bd93dc46ffe
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 37.08 ns 45.06 ns 1.22 0.57 False
EndsWith - Duration of single invocation 18.04 ns 26.80 ns 1.49 0.48 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.SequenceEqual(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.06174756628269 > 41.958255508755855.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/3/2022 4:51:57 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.4980369363118236 (T) = (0 -46.08692775655312) / Math.Sqrt((11.598757652991601 / (35)) + (9.469691475620555 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -0.1988203431081291 = (38.44356497743904 - 46.08692775655312) / 38.44356497743904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.EndsWith(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.802941449421375 > 18.837351561927637.
IsChangePoint: Marked as a change because one of 5/11/2022 7:18:06 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -8.405727835826475 (T) = (0 -28.432927701587342) / Math.Sqrt((17.050773127924558 / (35)) + (4.018919597645898 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -0.39990737674085497 = (20.31057780964228 - 28.432927701587342) / 20.31057780964228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 486b4d1a36aef5bbe6a77bb2c3412772e712961e
Compare afacf6a99e1b98331af958caec638bd93dc46ffe
Diff Diff

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 109.29 ns 119.80 ns 1.10 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTimeOffset*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.ToString(format: "o")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 119.80125038160551 > 118.40146029074235.
IsChangePoint: Marked as a change because one of 4/14/2022 10:59:17 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -4.8241602028968265 (T) = (0 -123.00904044674265) / Math.Sqrt((14.58988274976727 / (35)) + (13.606232354573587 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -0.05760156993481805 = (116.30943442559746 - 123.00904044674265) / 116.30943442559746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 486b4d1a36aef5bbe6a77bb2c3412772e712961e
Compare afacf6a99e1b98331af958caec638bd93dc46ffe
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsPrefix_FirstHalf - Duration of single invocation 105.88 ns 122.95 ns 1.16 0.42 False
LastIndexOf_Word_NotFound - Duration of single invocation 1.37 μs 1.47 μs 1.07 0.05 False
IndexOf_Word_NotFound - Duration of single invocation 1.35 μs 1.46 μs 1.09 0.04 False
IsSuffix_DifferentLastChar - Duration of single invocation 199.00 ns 229.35 ns 1.15 0.25 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
Details

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False))


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.94581374994831 > 111.36901042141324.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -16.208659837381624 (T) = (0 -123.29210492483091) / Math.Sqrt((22.78707133346674 / (35)) + (3.0110607952242603 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -0.15010867935959676 = (107.20039517785605 - 123.29210492483091) / 107.20039517785605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4650698761268908 > 1.4429595939916373.
IsChangePoint: Marked as a change because one of 5/11/2022 9:40:50 AM, 5/31/2022 10:47:41 AM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -17.216019143276824 (T) = (0 -1453.8714104609878) / Math.Sqrt((265.16797471754535 / (38)) + (85.88916264323159 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.057800075344225446 = (1374.429293728188 - 1453.8714104609878) / 1374.429293728188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.463692586585138 > 1.4146869995546683.
IsChangePoint: Marked as a change because one of 5/11/2022 7:18:06 PM, 5/18/2022 4:10:31 AM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -6.194208403561538 (T) = (0 -1436.7330071103509) / Math.Sqrt((355.98044295006343 / (35)) + (1206.3006873794677 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -0.05459919688485606 = (1362.3498020425832 - 1436.7330071103509) / 1362.3498020425832 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.3486096113699 > 209.52285727169405.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -15.011214601621642 (T) = (0 -228.09060658056262) / Math.Sqrt((19.257547242145574 / (35)) + (20.852167233213496 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -0.12541331676517625 = (202.67274536627417 - 228.09060658056262) / 202.67274536627417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 486b4d1a36aef5bbe6a77bb2c3412772e712961e
Compare afacf6a99e1b98331af958caec638bd93dc46ffe
Diff Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryWriteBytes - Duration of single invocation 1.54 ns 7.32 ns 4.75 0.72 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 143.24.20.36)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.315948877958643 > 1.6170291583684282.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -7.526027501568083 (T) = (0 -7.530726040732879) / Math.Sqrt((2.106672665466187 / (35)) + (3.192432394932188 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -1.806856751271793 = (2.6829748391401487 - 7.530726040732879) / 2.6829748391401487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 486b4d1a36aef5bbe6a77bb2c3412772e712961e
Compare afacf6a99e1b98331af958caec638bd93dc46ffe
Diff Diff

Regressions in System.Memory.Constructors<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayAsMemory - Duration of single invocation 2.88 ns 4.18 ns 1.45 0.70 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Constructors&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Constructors<Byte>.ArrayAsMemory


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.1845164717833505 > 3.2290531190329466.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -4.147465424628626 (T) = (0 -3.900546656234951) / Math.Sqrt((0.09560300997426746 / (35)) + (0.2623833906569381 / (9))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (9) - 2, .025) and -0.23436691968515952 = (3.1599572169593086 - 3.900546656234951) / 3.1599572169593086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions