Skip to content

[Perf] Changes at 3/31/2022 1:23:38 AM #4365

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 834.55 ns 768.14 ns 0.92 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 768.1417822085888 < 795.6179367555519.
IsChangePoint: Marked as a change because one of 2/7/2022 4:59:24 AM, 2/25/2022 11:02:46 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 34.982134189923464 (T) = (0 -766.5502892981608) / Math.Sqrt((2009.191871316471 / (299)) + (7.556719634583893 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1080638244156681 = (859.4228043233841 - 766.5502892981608) / 859.4228043233841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in SeekUnroll

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.30 secs 869.74 ms 0.67 0.22 False
Test - Duration of single invocation 1.74 secs 1.14 secs 0.66 0.18 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'SeekUnroll*'
Details

Payloads

Baseline
Compare

Histogram

SeekUnroll.Test(boxedIndex: 1)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 869.7364220833334 < 1.2390576240535716.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 42.561718860726124 (T) = (0 -885476376.7591575) / Math.Sqrt((11027978543954628 / (299)) + (610638799716690.5 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.27684149968317584 = (1224456846.4191737 - 885476376.7591575) / 1224456846.4191737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### SeekUnroll.Test(boxedIndex: 11)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1420968009285715 < 1.652185020576923.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 41.19062317982063 (T) = (0 -1167144120.5803792) / Math.Sqrt((11453671757602684 / (299)) + (2277152473179574 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.2925622504629506 = (1649818830.4824896 - 1167144120.5803792) / 1649818830.4824896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 11.93 μs 10.06 μs 0.84 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'
Details

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.ScaleB


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.059887710918115 < 11.340290403902397.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 1161.0443150650563 (T) = (0 -10068.378824008221) / Math.Sqrt((327.1211478025036 / (299)) + (33.420035136829505 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15543383307409406 = (11921.361781109003 - 10068.378824008221) / 11921.361781109003 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Average - Duration of single invocation 390.26 ns 352.86 ns 0.90 0.02 False
Select - Duration of single invocation 541.64 ns 506.12 ns 0.93 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 352.8629388550097 < 370.03977011862366.
IsChangePoint: Marked as a change because one of 2/25/2022 7:19:51 AM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 41.50561970934662 (T) = (0 -353.0525733867992) / Math.Sqrt((80.32766383205193 / (290)) + (3.132766968716355 / (23))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (23) - 2, .975) and 0.07026009470317812 = (379.7326234739664 - 353.0525733867992) / 379.7326234739664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Select(input: Array)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 506.1216150562042 < 513.0298949538152.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/21/2022 1:28:14 PM, 3/31/2022 1:45:25 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 36.66032917165368 (T) = (0 -511.05334884772293) / Math.Sqrt((212.89425613698708 / (299)) + (34.768093598523734 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.10095593615250338 = (568.4408244248327 - 511.05334884772293) / 568.4408244248327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Xml.Linq.Perf_XDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateWithRootlEement - Duration of single invocation 48.68 ns 44.15 ns 0.91 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XDocument*'
Details

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XDocument.CreateWithRootlEement


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.14553855360638 < 46.00081470005472.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 29.278409023099574 (T) = (0 -45.470117488401954) / Math.Sqrt((2.9308401442501433 / (299)) + (0.5346002698829703 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10478643728430827 = (50.79248056794988 - 45.470117488401954) / 50.79248056794988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions