Skip to content

[Perf] Changes at 6/7/2022 6:41:39 PM #6176

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 33.95 ms 14.95 ms 0.44 0.58 False
Count - Duration of single invocation 9.82 ms 7.44 ms 0.76 0.67 False
Count - Duration of single invocation 28.96 ms 11.24 ms 0.39 0.57 False
Count - Duration of single invocation 8.06 ms 5.96 ms 0.74 0.69 False
Count - Duration of single invocation 21.28 ms 13.50 ms 0.63 0.59 False
Count - Duration of single invocation 625.18 μs 445.64 μs 0.71 0.64 False
Count - Duration of single invocation 4.31 ms 2.35 ms 0.55 0.70 False
Count - Duration of single invocation 559.30 μs 376.40 μs 0.67 0.68 False
Count - Duration of single invocation 41.50 ms 31.74 ms 0.76 0.48 False
Count - Duration of single invocation 35.70 ms 16.03 ms 0.45 0.52 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.949135 < 34.35006295.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 16.816052841389407 (T) = (0 -16130710.154443791) / Math.Sqrt((45942003127454.06 / (118)) + (1027378458299.2097 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.40333699984771204 = (27034875.885259695 - 16130710.154443791) / 27034875.885259695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.444156400000001 < 9.325991587499999.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 21.73463337525289 (T) = (0 -5977929.67458055) / Math.Sqrt((888216086378.3363 / (118)) + (494293320906.88074 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.3529362176370841 = (9238547.786974939 - 5977929.67458055) / 9238547.786974939 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.243155545454547 < 28.58250462625.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 19.02509425756189 (T) = (0 -11156311.3393178) / Math.Sqrt((38907123662421.97 / (118)) + (432674021360.77325 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.4996242897141782 = (22295869.10392824 - 11156311.3393178) / 22295869.10392824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.962015121951219 < 7.805529362500001.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 28.968423920841698 (T) = (0 -5561351.793448391) / Math.Sqrt((1732932408373.6067 / (118)) + (153377836753.3853 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.4200451414775848 = (9589283.910160476 - 5561351.793448391) / 9589283.910160476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.5049425 < 19.9999293875.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 35.12905048818115 (T) = (0 -13865258.170536637) / Math.Sqrt((1573176000722.5203 / (118)) + (708906605481.088 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.32099009800681577 = (20419817.339682646 - 13865258.170536637) / 20419817.339682646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 445.6442897526502 < 577.8202654908056.
IsChangePoint: Marked as a change because one of 5/31/2022 10:14:36 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 9.075829223550338 (T) = (0 -476320.4862610404) / Math.Sqrt((2433135523.9181185 / (117)) + (1394942705.7532969 / (32))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (32) - 2, .975) and 0.13261731633956422 = (549146.8705034824 - 476320.4862610404) / 549146.8705034824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.350046 < 3.693440806428989.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 15.414955922904518 (T) = (0 -2581316.3450049423) / Math.Sqrt((245026887956.07703 / (117)) + (37092383597.12713 / (32))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (32) - 2, .975) and 0.2540758468960663 = (3460561.4180256655 - 2581316.3450049423) / 3460561.4180256655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 376.40239877300615 < 527.7793880092405.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 14.744889518402884 (T) = (0 -432411.7157227637) / Math.Sqrt((1397638909.6096232 / (117)) + (1054749986.8528003 / (32))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (32) - 2, .975) and 0.18600364929746446 = (531220.7055345669 - 432411.7157227637) / 531220.7055345669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.7416095 < 41.1756483625.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 12.820712334059474 (T) = (0 -32726676.38992051) / Math.Sqrt((27106121530884.332 / (67)) + (2727284095970.024 / (33))) is greater than 1.9844674545082206 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (33) - 2, .975) and 0.2147285997022374 = (41675624.98457357 - 32726676.38992051) / 41675624.98457357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.026320428571427 < 34.123455175.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 16.788015063803538 (T) = (0 -16139676.578082643) / Math.Sqrt((44284792700521.04 / (118)) + (1093409336344.4398 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.39931151426834693 = (26868629.84967013 - 16139676.578082643) / 26868629.84967013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.25094
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 377.57 ns 293.41 ns 0.78 0.64 False
IsMatch - Duration of single invocation 300.49 ns 251.74 ns 0.84 0.60 False
IsMatch - Duration of single invocation 528.52 ns 418.17 ns 0.79 0.58 False
IsMatch - Duration of single invocation 473.91 ns 343.62 ns 0.73 0.56 False
IsMatch - Duration of single invocation 347.76 ns 274.02 ns 0.79 0.64 False
IsMatch - Duration of single invocation 722.17 ns 546.18 ns 0.76 0.61 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 293.4142351746574 < 347.06144035387325.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 12.153004658502136 (T) = (0 -298.5639434119885) / Math.Sqrt((1223.7708930680315 / (118)) + (307.83940841691066 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.15301935449823595 = (352.50385589993584 - 298.5639434119885) / 352.50385589993584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 251.73897196750676 < 296.1311854767676.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 10.600318060702367 (T) = (0 -258.5564750591894) / Math.Sqrt((708.4440792641307 / (117)) + (259.82280204116677 / (32))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (32) - 2, .975) and 0.13371463548478993 = (298.4657084722683 - 258.5564750591894) / 298.4657084722683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 418.171639341193 < 527.6370925060457.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 6.899474306727786 (T) = (0 -438.6085449599551) / Math.Sqrt((4188.014474899558 / (118)) + (555.3070961830231 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.1021573093596223 = (488.5138003931644 - 438.6085449599551) / 488.5138003931644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 343.62019178248465 < 456.2724873222921.
IsChangePoint: Marked as a change because one of 6/8/2022 4:28:13 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 11.770373772023396 (T) = (0 -366.1018490471923) / Math.Sqrt((1169.8858644958775 / (120)) + (1055.0107076664121 / (29))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (120) + (29) - 2, .975) and 0.17922486595191967 = (446.0440306488943 - 366.1018490471923) / 446.0440306488943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 274.02024847893165 < 315.9315486283984.
IsChangePoint: Marked as a change because one of 6/8/2022 9:45:10 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 7.947951577389894 (T) = (0 -274.0226588281993) / Math.Sqrt((829.1462186906484 / (121)) + (556.678252119152 / (29))) is greater than 1.9761224936134032 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (121) + (29) - 2, .975) and 0.12894476280177883 = (314.58700565259505 - 274.0226588281993) / 314.58700565259505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 546.1751856483016 < 622.2190493292782.
IsChangePoint: Marked as a change because one of 5/27/2022 3:44:45 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 12.38265322380186 (T) = (0 -507.0815200057685) / Math.Sqrt((3808.5346295897116 / (118)) + (1345.1011509566604 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.17265926564409975 = (612.9052988071966 - 507.0815200057685) / 612.9052988071966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 236.61 ms 137.86 ms 0.58 0.59 False
Count - Duration of single invocation 236.98 ms 148.41 ms 0.63 0.53 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 137.858195 < 222.61348955.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 44.09265934172429 (T) = (0 -145890037.98281875) / Math.Sqrt((169762776289177.7 / (118)) + (72808491503129.88 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.36589140043023605 = (230071060.51203787 - 145890037.98281875) / 230071060.51203787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 148.40532 < 224.84043584999998.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 51.20673462157009 (T) = (0 -144989458.22766513) / Math.Sqrt((178798483322549.5 / (118)) + (44739476782392.9 / (33))) is greater than 1.9760131776890653 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (33) - 2, .975) and 0.3743821344213607 = (231754024.62901717 - 144989458.22766513) / 231754024.62901717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Threading.Tests.Perf_ThreadStatic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetThreadStatic - Duration of single invocation 7.80 ns 4.71 ns 0.60 0.83 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_ThreadStatic*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_ThreadStatic.GetThreadStatic


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.707167465920035 < 6.956796849339756.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 7.007251033248686 (T) = (0 -4.724594745223051) / Math.Sqrt((2.1138381942515383 / (117)) + (0.5084768653973245 / (33))) is greater than 1.9761224936134032 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (33) - 2, .975) and 0.21344074817709596 = (6.006660953098555 - 4.724594745223051) / 6.006660953098555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline bab1a1f239325746873d2ae795ea343e7901dbce
Compare 31cd7eb907133929846e5d229d18cc8bc77bde63
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 19.10 ms 6.39 ms 0.33 0.67 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.391856217391304 < 18.17505757777778.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/15/2022 7:24:26 PM falls between 6/7/2022 5:42:36 AM and 6/15/2022 7:24:26 PM.
IsImprovementStdDev: Marked as improvement because 16.26798923594773 (T) = (0 -7292880.099205563) / Math.Sqrt((17418223781077.723 / (67)) + (236102717025.63278 / (33))) is greater than 1.9844674545082206 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (33) - 2, .975) and 0.5355119402726913 = (15700898.971411798 - 7292880.099205563) / 15700898.971411798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions