Skip to content

[Perf] Linux/x64: 87 Regressions on 8/17/2022 6:09:21 PM #7976

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 2.00 μs 4.42 μs 2.22 0.05 False
IndexOfAnyFiveValues - Duration of single invocation 5.98 μs 16.58 μs 2.77 0.03 False
LastIndexOfValue - Duration of single invocation 1.76 μs 4.38 μs 2.50 0.08 False
IndexOfAnyThreeValues - Duration of single invocation 4.72 μs 9.95 μs 2.11 0.06 False
IndexOfAnyFourValues - Duration of single invocation 5.15 μs 16.80 μs 3.26 0.02 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfValue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.424370395797761 > 2.099354007601711.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -228.9822018722831 (T) = (0 -4440.129849755984) / Math.Sqrt((746.1326660454284 / (16)) + (1921.3579389411896 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2120651354916854 = (2007.2328696456113 - 4440.129849755984) / 2007.2328696456113 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFiveValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -230.9342858971992 (T) = (0 -16559.50904449143) / Math.Sqrt((83.31507440812801 / (2)) + (51518.820613645454 / (25))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (2) + (25) - 2, .025) and -1.7734624046482563 = (5970.6989417769255 - 16559.50904449143) / 5970.6989417769255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.382803354060675 > 1.8459576629846874.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -608.8437463151625 (T) = (0 -4393.6243695298745) / Math.Sqrt((83.06115909458961 / (16)) + (378.2744720148661 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -1.4953964456371467 = (1760.6919242076806 - 4393.6243695298745) / 1760.6919242076806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.953077892514834 > 4.958038374220887.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -133.06334302856706 (T) = (0 -10035.178960318004) / Math.Sqrt((10566.657385247858 / (16)) + (27466.34550316625 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.1350840643169582 = (4700.132949345201 - 10035.178960318004) / 4700.132949345201 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.79938062185671 > 5.4029949952717935.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -120.59949876258965 (T) = (0 -16744.636886163873) / Math.Sqrt((144.12597451887595 / (16)) + (249584.24096087858 / (27))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (27) - 2, .025) and -2.255209824331321 = (5143.9500953225115 - 16744.636886163873) / 5143.9500953225115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 3.91 μs 8.65 μs 2.21 0.06 False
IndexOfString - Duration of single invocation 578.96 ns 1.03 μs 1.79 0.04 False
IndexOfString - Duration of single invocation 1.04 μs 2.35 μs 2.26 0.06 False
IndexOfString - Duration of single invocation 511.54 ns 677.87 ns 1.33 0.11 False
IndexOfString - Duration of single invocation 3.90 μs 8.63 μs 2.21 0.05 False
IndexOfString - Duration of single invocation 1.60 μs 3.37 μs 2.10 0.06 False
IndexOfString - Duration of single invocation 388.82 ns 415.19 ns 1.07 0.12 False
IndexOfString - Duration of single invocation 587.29 ns 1.07 μs 1.82 0.07 False
IndexOfString - Duration of single invocation 366.00 ns 495.09 ns 1.35 0.13 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.64741195028587 > 4.1013109440625.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -180.96434391360944 (T) = (0 -8706.652358439575) / Math.Sqrt((1324.2543448862339 / (16)) + (17940.025527011872 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2244650627025373 = (3914.0432027562288 - 8706.652358439575) / 3914.0432027562288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0348012839410983 > 603.3218214752392.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -214.46461206020024 (T) = (0 -1023.6016826715849) / Math.Sqrt((36.39556865114193 / (16)) + (62.77360309679295 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.7903560279728833 = (571.7307991699003 - 1023.6016826715849) / 571.7307991699003 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.351876672460109 > 1.1005424139574933.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -138.67827147386146 (T) = (0 -2357.7106617874088) / Math.Sqrt((525.5180591270678 / (16)) + (1612.733691558525 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2381400077496516 = (1053.424117179327 - 2357.7106617874088) / 1053.424117179327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 677.8736925409745 > 536.477630083975.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -38.38513981916017 (T) = (0 -664.3106228947553) / Math.Sqrt((198.4434352488016 / (16)) + (72.66655300880606 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -0.2882830899330542 = (515.6557810048382 - 664.3106228947553) / 515.6557810048382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.630530967689898 > 4.096269678979396.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -260.7095987365435 (T) = (0 -8688.069538319753) / Math.Sqrt((1323.1846778303511 / (16)) + (7342.706292114539 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2220554931170406 = (3909.9246464508224 - 8688.069538319753) / 3909.9246464508224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3717654301364943 > 1.6824693754498397.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -132.24447565482643 (T) = (0 -3398.9921041228167) / Math.Sqrt((2351.302386780355 / (16)) + (982.3132722364186 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -1.1049970280982135 = (1614.7253695620082 - 3398.9921041228167) / 1614.7253695620082 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 415.18956022735387 > 410.1973734939151.
IsChangePoint: Marked as a change because one of 6/17/2022 8:24:28 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -4.450843903159241 (T) = (0 -427.223316492107) / Math.Sqrt((104.43290618479654 / (16)) + (1179.447940174716 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -0.07835981751772005 = (396.17881671030153 - 427.223316492107) / 396.17881671030153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.069781383349324 > 614.5315261211443.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -147.23790642148083 (T) = (0 -1067.3376950804065) / Math.Sqrt((127.05744436804608 / (16)) + (81.3158167840278 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.8254564622692572 = (584.696330557005 - 1067.3376950804065) / 584.696330557005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 495.0899798707861 > 384.86005499687917.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.43734624155144 (T) = (0 -498.9181598445892) / Math.Sqrt((97.6860874663925 / (16)) + (523.0212412338225 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.3702401017648088 = (364.1100265581226 - 498.9181598445892) / 364.1100265581226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 1.35 ms 1.62 ms 1.20 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'
Details

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.619453370673077 > 1.4160329543694912.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -52.032986115816996 (T) = (0 -1586243.5162880719) / Math.Sqrt((152836733.53902644 / (16)) + (317923584.2416149 / (27))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (27) - 2, .025) and -0.17853275682885175 = (1345947.753337185 - 1586243.5162880719) / 1345947.753337185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare f142128e89b63577a9bbba7e2b760ec82102a7a9
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfChar - Duration of single invocation 643.14 ns 1.28 μs 1.99 0.06 False
IndexOfShort - Duration of single invocation 644.02 ns 1.27 μs 1.97 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfChar


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2804434792942432 > 674.4623342442469.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -275.8648770624791 (T) = (0 -1270.0653190761518) / Math.Sqrt((27.01499653376699 / (16)) + (104.50117188523133 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.9986530990935593 = (635.4606107744056 - 1270.0653190761518) / 635.4606107744056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfShort

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2715719728990553 > 673.0728550867216.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -210.83791530025124 (T) = (0 -1271.2441752009593) / Math.Sqrt((30.694512461879636 / (16)) + (206.4508123471615 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.9943664505987144 = (637.4175492268877 - 1271.2441752009593) / 637.4175492268877 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions