-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-wasmarch-x64branch-refs/heads/mainkind-microos-linuxperf-regression
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 543bcc5ee7d6a2b9471b016770227421c43a756e |
| Compare | f142128e89b63577a9bbba7e2b760ec82102a7a9 |
| Diff | Diff |
Regressions in System.Memory.Span<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfValue - Duration of single invocation | 2.00 μs | 4.42 μs | 2.22 | 0.05 | False | |||||
| IndexOfAnyFiveValues - Duration of single invocation | 5.98 μs | 16.58 μs | 2.77 | 0.03 | False | |||||
| LastIndexOfValue - Duration of single invocation | 1.76 μs | 4.38 μs | 2.50 | 0.08 | False | |||||
| IndexOfAnyThreeValues - Duration of single invocation | 4.72 μs | 9.95 μs | 2.11 | 0.06 | False | |||||
| IndexOfAnyFourValues - Duration of single invocation | 5.15 μs | 16.80 μs | 3.26 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Char>*'Details
Payloads
Histogram
System.Memory.Span<Char>.IndexOfValue(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.424370395797761 > 2.099354007601711.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -228.9822018722831 (T) = (0 -4440.129849755984) / Math.Sqrt((746.1326660454284 / (16)) + (1921.3579389411896 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2120651354916854 = (2007.2328696456113 - 4440.129849755984) / 2007.2328696456113 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -230.9342858971992 (T) = (0 -16559.50904449143) / Math.Sqrt((83.31507440812801 / (2)) + (51518.820613645454 / (25))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (2) + (25) - 2, .025) and -1.7734624046482563 = (5970.6989417769255 - 16559.50904449143) / 5970.6989417769255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Char>.LastIndexOfValue(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.382803354060675 > 1.8459576629846874.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -608.8437463151625 (T) = (0 -4393.6243695298745) / Math.Sqrt((83.06115909458961 / (16)) + (378.2744720148661 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -1.4953964456371467 = (1760.6919242076806 - 4393.6243695298745) / 1760.6919242076806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.953077892514834 > 4.958038374220887.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -133.06334302856706 (T) = (0 -10035.178960318004) / Math.Sqrt((10566.657385247858 / (16)) + (27466.34550316625 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.1350840643169582 = (4700.132949345201 - 10035.178960318004) / 4700.132949345201 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.79938062185671 > 5.4029949952717935.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -120.59949876258965 (T) = (0 -16744.636886163873) / Math.Sqrt((144.12597451887595 / (16)) + (249584.24096087858 / (27))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (27) - 2, .025) and -2.255209824331321 = (5143.9500953225115 - 16744.636886163873) / 5143.9500953225115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 543bcc5ee7d6a2b9471b016770227421c43a756e |
| Compare | f142128e89b63577a9bbba7e2b760ec82102a7a9 |
| Diff | Diff |
Regressions in System.Memory.ReadOnlySpan
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfString - Duration of single invocation | 3.91 μs | 8.65 μs | 2.21 | 0.06 | False | |||||
| IndexOfString - Duration of single invocation | 578.96 ns | 1.03 μs | 1.79 | 0.04 | False | |||||
| IndexOfString - Duration of single invocation | 1.04 μs | 2.35 μs | 2.26 | 0.06 | False | |||||
| IndexOfString - Duration of single invocation | 511.54 ns | 677.87 ns | 1.33 | 0.11 | False | |||||
| IndexOfString - Duration of single invocation | 3.90 μs | 8.63 μs | 2.21 | 0.05 | False | |||||
| IndexOfString - Duration of single invocation | 1.60 μs | 3.37 μs | 2.10 | 0.06 | False | |||||
| IndexOfString - Duration of single invocation | 388.82 ns | 415.19 ns | 1.07 | 0.12 | False | |||||
| IndexOfString - Duration of single invocation | 587.29 ns | 1.07 μs | 1.82 | 0.07 | False | |||||
| IndexOfString - Duration of single invocation | 366.00 ns | 495.09 ns | 1.35 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'Details
Payloads
Histogram
System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.64741195028587 > 4.1013109440625.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -180.96434391360944 (T) = (0 -8706.652358439575) / Math.Sqrt((1324.2543448862339 / (16)) + (17940.025527011872 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2244650627025373 = (3914.0432027562288 - 8706.652358439575) / 3914.0432027562288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0348012839410983 > 603.3218214752392.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -214.46461206020024 (T) = (0 -1023.6016826715849) / Math.Sqrt((36.39556865114193 / (16)) + (62.77360309679295 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.7903560279728833 = (571.7307991699003 - 1023.6016826715849) / 571.7307991699003 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.351876672460109 > 1.1005424139574933.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -138.67827147386146 (T) = (0 -2357.7106617874088) / Math.Sqrt((525.5180591270678 / (16)) + (1612.733691558525 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2381400077496516 = (1053.424117179327 - 2357.7106617874088) / 1053.424117179327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 677.8736925409745 > 536.477630083975.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -38.38513981916017 (T) = (0 -664.3106228947553) / Math.Sqrt((198.4434352488016 / (16)) + (72.66655300880606 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -0.2882830899330542 = (515.6557810048382 - 664.3106228947553) / 515.6557810048382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.630530967689898 > 4.096269678979396.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -260.7095987365435 (T) = (0 -8688.069538319753) / Math.Sqrt((1323.1846778303511 / (16)) + (7342.706292114539 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -1.2220554931170406 = (3909.9246464508224 - 8688.069538319753) / 3909.9246464508224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3717654301364943 > 1.6824693754498397.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -132.24447565482643 (T) = (0 -3398.9921041228167) / Math.Sqrt((2351.302386780355 / (16)) + (982.3132722364186 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -1.1049970280982135 = (1614.7253695620082 - 3398.9921041228167) / 1614.7253695620082 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 415.18956022735387 > 410.1973734939151.
IsChangePoint: Marked as a change because one of 6/17/2022 8:24:28 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -4.450843903159241 (T) = (0 -427.223316492107) / Math.Sqrt((104.43290618479654 / (16)) + (1179.447940174716 / (28))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (28) - 2, .025) and -0.07835981751772005 = (396.17881671030153 - 427.223316492107) / 396.17881671030153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.069781383349324 > 614.5315261211443.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -147.23790642148083 (T) = (0 -1067.3376950804065) / Math.Sqrt((127.05744436804608 / (16)) + (81.3158167840278 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.8254564622692572 = (584.696330557005 - 1067.3376950804065) / 584.696330557005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 495.0899798707861 > 384.86005499687917.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -27.43734624155144 (T) = (0 -498.9181598445892) / Math.Sqrt((97.6860874663925 / (16)) + (523.0212412338225 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.3702401017648088 = (364.1100265581226 - 498.9181598445892) / 364.1100265581226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 543bcc5ee7d6a2b9471b016770227421c43a756e |
| Compare | f142128e89b63577a9bbba7e2b760ec82102a7a9 |
| Diff | Diff |
Regressions in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| XmlSerializer_ - Duration of single invocation | 1.35 ms | 1.62 ms | 1.20 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>*'Details
Payloads
Histogram
MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.619453370673077 > 1.4160329543694912.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -52.032986115816996 (T) = (0 -1586243.5162880719) / Math.Sqrt((152836733.53902644 / (16)) + (317923584.2416149 / (27))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (27) - 2, .025) and -0.17853275682885175 = (1345947.753337185 - 1586243.5162880719) / 1345947.753337185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 543bcc5ee7d6a2b9471b016770227421c43a756e |
| Compare | f142128e89b63577a9bbba7e2b760ec82102a7a9 |
| Diff | Diff |
Regressions in System.Tests.Perf_Array
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfChar - Duration of single invocation | 643.14 ns | 1.28 μs | 1.99 | 0.06 | False | |||||
| IndexOfShort - Duration of single invocation | 644.02 ns | 1.27 μs | 1.97 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'Details
Payloads
Histogram
System.Tests.Perf_Array.IndexOfChar
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2804434792942432 > 674.4623342442469.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -275.8648770624791 (T) = (0 -1270.0653190761518) / Math.Sqrt((27.01499653376699 / (16)) + (104.50117188523133 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.9986530990935593 = (635.4606107744056 - 1270.0653190761518) / 635.4606107744056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Array.IndexOfShort
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2715719728990553 > 673.0728550867216.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/23/2022 11:50:53 AM falls between 8/5/2022 10:47:59 PM and 8/23/2022 11:50:53 AM.
IsRegressionStdDev: Marked as regression because -210.83791530025124 (T) = (0 -1271.2441752009593) / Math.Sqrt((30.694512461879636 / (16)) + (206.4508123471615 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.9943664505987144 = (637.4175492268877 - 1271.2441752009593) / 637.4175492268877 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-wasmarch-x64branch-refs/heads/mainkind-microos-linuxperf-regression
_1.png)
_2.png)
_3.png)
_4.png)
_5.png)







_1.png)

