Skip to content

[Perf] Changes at 6/7/2022 3:01:09 PM #6037

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 128.79 ns 98.92 ns 0.77 0.06 False
IsMatch - Duration of single invocation 115.38 ns 93.65 ns 0.81 0.06 False
IsMatch - Duration of single invocation 265.11 ns 205.40 ns 0.77 0.06 False
IsMatch - Duration of single invocation 140.59 ns 104.49 ns 0.74 0.04 False
IsMatch - Duration of single invocation 190.17 ns 159.06 ns 0.84 0.04 False
IsMatch - Duration of single invocation 109.16 ns 90.56 ns 0.83 0.05 False
IsMatch - Duration of single invocation 138.27 ns 97.79 ns 0.71 0.04 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.92074438049907 < 122.21021845090507.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 55.267370443558285 (T) = (0 -98.48400122188127) / Math.Sqrt((25.73155328255756 / (299)) + (3.92186051416707 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.19111725460311765 = (121.75312402486668 - 98.48400122188127) / 121.75312402486668 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 93.64592532498004 < 109.92103618163414.
IsChangePoint: Marked as a change because one of 5/19/2022 10:26:24 PM, 5/23/2022 3:43:39 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 26.063978814645314 (T) = (0 -94.4173289357728) / Math.Sqrt((44.44705838892128 / (299)) + (2.885743341664365 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.11365300984734236 = (106.52411525593502 - 94.4173289357728) / 106.52411525593502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 205.40047812002712 < 254.53369593338508.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 14.55889552646745 (T) = (0 -206.1070978688446) / Math.Sqrt((532.3730314000824 / (299)) + (6.563455008796191 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.08942969979468873 = (226.34946233406964 - 206.1070978688446) / 226.34946233406964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.49281753746006 < 133.65514924587734.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 70.69626645115824 (T) = (0 -104.66731478750127) / Math.Sqrt((26.630277572149325 / (299)) + (2.8839671150257398 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.21066591256013148 = (132.60204576617227 - 104.66731478750127) / 132.60204576617227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 159.05985892215455 < 180.963529105905.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 22.694672078780602 (T) = (0 -161.0941313212835) / Math.Sqrt((56.67910402629442 / (206)) + (29.59825023924954 / (43))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (43) - 2, .975) and 0.12148264540079563 = (183.37045987529476 - 161.0941313212835) / 183.37045987529476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 90.55579886085745 < 105.57994379662496.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 17.849712973744673 (T) = (0 -91.47648663463912) / Math.Sqrt((41.52597453368952 / (299)) + (3.360853668008145 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.08333097342859488 = (99.79227396477701 - 91.47648663463912) / 99.79227396477701 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.78837366875236 < 123.33333772966445.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 63.85076769190806 (T) = (0 -97.45662855985977) / Math.Sqrt((28.199008628664025 / (299)) + (2.0874628090494305 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.19848098098020542 = (121.58991395992432 - 97.45662855985977) / 121.58991395992432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 1.64 ms 1.13 ms 0.69 0.02 False
Count - Duration of single invocation 8.53 ms 6.38 ms 0.75 0.03 False
Count - Duration of single invocation 3.97 ms 2.77 ms 0.70 0.04 False
Count - Duration of single invocation 12.12 ms 4.18 ms 0.35 0.09 False
Count - Duration of single invocation 3.59 ms 2.48 ms 0.69 0.05 False
Count - Duration of single invocation 6.33 ms 4.55 ms 0.72 0.04 False
Count - Duration of single invocation 20.08 ms 14.22 ms 0.71 0.02 False
Count - Duration of single invocation 77.90 μs 65.55 μs 0.84 0.02 False
Count - Duration of single invocation 13.83 ms 6.05 ms 0.44 0.09 False
Count - Duration of single invocation 167.71 μs 131.43 μs 0.78 0.02 False
Count - Duration of single invocation 187.29 μs 152.86 μs 0.82 0.02 False
Count - Duration of single invocation 14.16 ms 6.06 ms 0.43 0.08 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1272076636363637 < 1.5669384880557429.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 25.578539161608226 (T) = (0 -1143664.3202319562) / Math.Sqrt((25675901097.843983 / (206)) + (255921865.155267 / (43))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (43) - 2, .975) and 0.20355851418089133 = (1435967.7899698338 - 1143664.3202319562) / 1435967.7899698338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.3847110290598295 < 8.119492688063662.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 44.8388350956208 (T) = (0 -6350506.359621917) / Math.Sqrt((25127578148.045795 / (299)) + (85262861064.36815 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.24299783122648413 = (8389020.033999264 - 6350506.359621917) / 8389020.033999264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.766230671296296 < 3.764930398650794.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 159.73446850016433 (T) = (0 -2780387.608071488) / Math.Sqrt((11858842843.083397 / (299)) + (692836578.9367582 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.3002353458702641 = (3973318.160131029 - 2780387.608071488) / 3973318.160131029 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.18208790934066 < 11.568782893166667.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 24.74980548159321 (T) = (0 -4235133.721514514) / Math.Sqrt((6184710866087.804 / (299)) + (6532223366.823128 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.4575722127378336 = (7807737.400199941 - 4235133.721514514) / 7807737.400199941 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.479822222142857 < 3.4208463132014124.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 78.93000614434807 (T) = (0 -2504656.029648117) / Math.Sqrt((11096955452.318413 / (299)) + (7174386081.645925 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.3103713264551259 = (3631890.792436923 - 2504656.029648117) / 3631890.792436923 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.548624353846154 < 6.036068608012821.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 164.9685520333669 (T) = (0 -4530699.386052391) / Math.Sqrt((17070287203.6908 / (299)) + (2966643421.7742653 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.29020128795022115 = (6383076.36959286 - 4530699.386052391) / 6383076.36959286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.21675971372549 < 19.181736432916665.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 13.9605515068372 (T) = (0 -14190013.005983021) / Math.Sqrt((4761897455869.668 / (299)) + (11124144505.12191 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.11125805833883017 = (15966404.127907041 - 14190013.005983021) / 15966404.127907041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.55188356661482 < 73.74658522704445.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 39.51685164109948 (T) = (0 -63795.07992944102) / Math.Sqrt((17633443.467195764 / (299)) + (1801799.299234921 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.1643961810628506 = (76346.08469188843 - 63795.07992944102) / 76346.08469188843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.050822451219513 < 13.430135623266807.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 21.42146966593976 (T) = (0 -6094242.438316677) / Math.Sqrt((7278501932134.859 / (299)) + (5207642708.850972 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.35474905276300134 = (9444763.257477684 - 6094242.438316677) / 9444763.257477684 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.43271802417237 < 156.40974713126548.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 80.5606513169636 (T) = (0 -128576.29785639886) / Math.Sqrt((19932471.968351144 / (206)) + (5404065.552167598 / (43))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (43) - 2, .975) and 0.2280999485862024 = (166571.17410589743 - 128576.29785639886) / 166571.17410589743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 152.86434574375866 < 176.94066107585743.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 25.795235432619922 (T) = (0 -152066.26812506764) / Math.Sqrt((119168114.22926652 / (206)) + (4438708.956551232 / (43))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (43) - 2, .975) and 0.12285141792468834 = (173364.3207463012 - 152066.26812506764) / 173364.3207463012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.05859241788618 < 13.63852336240741.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 20.471306232266695 (T) = (0 -6111167.575063219) / Math.Sqrt((7559130126280.408 / (299)) + (48425946928.62312 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.3524821957518649 = (9437837.129682014 - 6111167.575063219) / 9437837.129682014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 8.28 ms 2.93 ms 0.35 0.04 False
Count - Duration of single invocation 15.48 ms 14.52 ms 0.94 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9296303126984133 < 7.89916093011111.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 14.619041212609403 (T) = (0 -2944680.030035502) / Math.Sqrt((4014700912005.5093 / (299)) + (1977455903.0425296 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.3655846311814417 = (4641564.777220388 - 2944680.030035502) / 4641564.777220388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.516920023529412 < 14.622699098974358.
IsChangePoint: Marked as a change because one of 5/12/2022 3:44:01 PM, 5/19/2022 10:26:24 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 59.37049869174771 (T) = (0 -14533802.754440699) / Math.Sqrt((71563241932.8031 / (299)) + (4730770102.068234 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.0709370427281733 = (15643506.869673178 - 14533802.754440699) / 15643506.869673178 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 0971e5e22794fc7d7d0744376be2f452e26159ad
Compare 17d6d50eb9d74905f83e709093d1cc32489226ee
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 9.08 ms 8.48 ms 0.93 0.01 False
Count - Duration of single invocation 90.58 ms 60.86 ms 0.67 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.481948564986737 < 8.611463546780044.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 41.80435693517815 (T) = (0 -8603230.328366393) / Math.Sqrt((12562465204.342283 / (206)) + (5164767785.157387 / (43))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (206) + (43) - 2, .975) and 0.06137663207816931 = (9165796.01828417 - 8603230.328366393) / 9165796.01828417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 60.85996335714286 < 85.80685108458333.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM.
IsImprovementStdDev: Marked as improvement because 153.297958060084 (T) = (0 -61006416.00385678) / Math.Sqrt((9350036414130.4 / (299)) + (297378458561.1847 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.32932750494908797 = (90963050.45165996 - 61006416.00385678) / 90963050.45165996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions