Skip to content

[Perf] Changes at 4/19/2022 9:35:37 PM #4828

New issue
New issue
Closed
Closed
[Perf] Changes at 4/19/2022 9:35:37 PM#4828
Labels
arch-x64branch-refs/heads/mainkind-microos-windowsperf-regressionruntime-coreclr
@performanceautofiler

Description

@performanceautofiler
performanceautofiler
bot
opened on Apr 26, 2022

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline f1e10a53132cc648cea914e75f953ff5a5087399
Compare de0b85f2b1690355f05bf13d3a88b8828b9853b1
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 107.76 ns 123.16 ns 1.14 0.02 False
IsMatch - Duration of single invocation 206.67 ns 228.16 ns 1.10 0.02 False
IsMatch - Duration of single invocation 172.10 ns 189.64 ns 1.10 0.01 False
IsMatch - Duration of single invocation 123.29 ns 138.07 ns 1.12 0.02 False
IsMatch - Duration of single invocation 844.56 ns 1.01 μs 1.19 0.08 False
IsMatch - Duration of single invocation 173.50 ns 190.16 ns 1.10 0.02 False
IsMatch - Duration of single invocation 226.37 ns 245.24 ns 1.08 0.06 False
IsMatch - Duration of single invocation 155.01 ns 170.95 ns 1.10 0.01 False
IsMatch - Duration of single invocation 174.34 ns 187.25 ns 1.07 0.01 False
IsMatch - Duration of single invocation 134.47 ns 146.73 ns 1.09 0.01 False
IsMatch - Duration of single invocation 154.58 ns 171.06 ns 1.11 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.16351722751426 > 113.16584949501438.
IsChangePoint: Marked as a change because one of 2/21/2022 9:58:09 PM, 2/25/2022 11:02:46 PM, 3/20/2022 10:41:02 AM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -41.486728130531844 (T) = (0 -120.58285797291992) / Math.Sqrt((1.0706065110637655 / (23)) + (1.709905955109555 / (34))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (34) - 2, .025) and -0.11990304709419411 = (107.67258673489242 - 120.58285797291992) / 107.67258673489242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 228.1632358622983 > 218.45807520293263.
IsChangePoint: Marked as a change because one of 2/21/2022 9:58:09 PM, 3/17/2022 6:54:53 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -28.196260536954156 (T) = (0 -226.7204635907672) / Math.Sqrt((3.483197439325087 / (20)) + (7.279326005571756 / (35))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (35) - 2, .025) and -0.08328257621169882 = (209.2902337482632 - 226.7204635907672) / 209.2902337482632 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 189.641401152105 > 183.63129989869475.
IsChangePoint: Marked as a change because one of 2/21/2022 9:58:09 PM, 3/21/2022 1:28:14 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -34.91638553755683 (T) = (0 -193.3717868715988) / Math.Sqrt((1.7169040349728846 / (23)) + (9.022327753172414 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.11620638453735374 = (173.24017274076758 - 193.3717868715988) / 173.24017274076758 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 138.06866580527839 > 129.77215364664303.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -34.1112201811742 (T) = (0 -139.1601443814752) / Math.Sqrt((1.266681546523059 / (23)) + (4.728764220861995 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.11969169461121369 = (124.28434099423703 - 139.1601443814752) / 124.28434099423703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0088931531240644 > 888.4325834575927.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -12.617272469885975 (T) = (0 -919.5168594362143) / Math.Sqrt((21.129287319310183 / (23)) + (1245.5504085423647 / (34))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (34) - 2, .025) and -0.09180591344682233 = (842.1980941038386 - 919.5168594362143) / 842.1980941038386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.16056990489298 > 181.62804320518106.
IsChangePoint: Marked as a change because one of 2/21/2022 9:58:09 PM, 3/21/2022 1:28:14 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -47.8672827724406 (T) = (0 -193.145970573192) / Math.Sqrt((0.4929435060591149 / (23)) + (5.506601490870105 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.11704822481978501 = (172.90745939312737 - 193.145970573192) / 172.90745939312737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 245.24029610411657 > 234.741172728848.
IsChangePoint: Marked as a change because one of 2/22/2022 2:43:53 PM, 3/16/2022 5:02:20 PM, 4/19/2022 10:22:09 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -21.023971449582415 (T) = (0 -247.81243155578542) / Math.Sqrt((9.588044359272425 / (20)) + (26.089433840753177 / (35))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (35) - 2, .025) and -0.1036207433529303 = (224.54492002651378 - 247.81243155578542) / 224.54492002651378 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 170.94560345945143 > 164.16905538063438.
IsChangePoint: Marked as a change because one of 3/17/2022 2:30:39 AM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -33.85184298141109 (T) = (0 -173.29614251369898) / Math.Sqrt((2.208286669966173 / (23)) + (5.354143208290989 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.10799962019608404 = (156.40451436529423 - 173.29614251369898) / 156.40451436529423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 187.2485338906169 > 178.1037441318872.
IsChangePoint: Marked as a change because one of 2/21/2022 9:58:09 PM, 3/16/2022 5:02:20 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -27.73617472566482 (T) = (0 -190.37630609344976) / Math.Sqrt((7.237851517868094 / (23)) + (5.347105485276277 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.11063115269016147 = (171.41271936440992 - 190.37630609344976) / 171.41271936440992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 146.72655981551628 > 139.2063483120434.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 3/17/2022 6:54:53 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -38.0937167745193 (T) = (0 -148.9378566792904) / Math.Sqrt((2.512078029703644 / (23)) + (1.7772521388960625 / (33))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (33) - 2, .025) and -0.11518366232849116 = (133.5545540259349 - 148.9378566792904) / 133.5545540259349 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 171.05582164096415 > 165.37147124355644.
IsChangePoint: Marked as a change because one of 2/21/2022 9:58:09 PM, 3/21/2022 1:28:14 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -37.464461714897396 (T) = (0 -172.7687119365087) / Math.Sqrt((1.8835114112020939 / (23)) + (4.104244862695936 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.10714052791048564 = (156.04948746892714 - 172.7687119365087) / 156.04948746892714 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline f1e10a53132cc648cea914e75f953ff5a5087399
Compare de0b85f2b1690355f05bf13d3a88b8828b9853b1
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 4.33 ms 5.05 ms 1.17 0.02 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.053543673469388 > 4.543151014754642.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -58.172904780781266 (T) = (0 -5138770.362191907) / Math.Sqrt((478944485.2954231 / (23)) + (6073802810.215456 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.1873966411006722 = (4327762.252576746 - 5138770.362191907) / 4327762.252576746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline f1e10a53132cc648cea914e75f953ff5a5087399
Compare de0b85f2b1690355f05bf13d3a88b8828b9853b1
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 6.56 ms 11.63 ms 1.77 0.03 True
Count - Duration of single invocation 11.15 ms 12.76 ms 1.14 0.04 False
Count - Duration of single invocation 4.80 ms 9.47 ms 1.97 0.04 True
Count - Duration of single invocation 6.52 ms 11.57 ms 1.77 0.04 True

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.626785 > 6.904031174342106.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -272.8071418487432 (T) = (0 -11661868.188620903) / Math.Sqrt((721266657.3168335 / (20)) + (11012507690.189392 / (35))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (35) - 2, .025) and -0.7796320279850919 = (6552966.009397194 - 11661868.188620903) / 6552966.009397194 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.7556275 > 11.723420625.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -18.644745914313862 (T) = (0 -12838999.476190478) / Math.Sqrt((43199876587.162865 / (20)) + (170832572025.93277 / (35))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (35) - 2, .025) and -0.13876292646467123 = (11274514.807089474 - 12838999.476190478) / 11274514.807089474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.469877591973244 > 5.070148989062501.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -152.87186428181613 (T) = (0 -9484390.155569041) / Math.Sqrt((601154613.3762816 / (21)) + (30696870398.228657 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.9682352237140112 = (4818727.986013903 - 9484390.155569041) / 4818727.986013903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.56996 > 6.862255565789474.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 4/19/2022 4:39:14 PM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -42.23133445427519 (T) = (0 -11678674.785583466) / Math.Sqrt((316493464463.6055 / (23)) + (10574291029.031921 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.750763661006315 = (6670617.540045767 - 11678674.785583466) / 6670617.540045767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Labels

    arch-x64branch-refs/heads/mainkind-microos-windowsperf-regressionruntime-coreclr

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions