Skip to content

[Perf] Windows/arm64: 33 Improvements on 12/5/2022 9:44:53 PM #10542

New issue
New issue
Open
Open
[Perf] Windows/arm64: 33 Improvements on 12/5/2022 9:44:53 PM#10542
Assignees
EgorBo
Labels
arch-arm64branch-refs/heads/mainkind-microos-windowsperf-improvementruntime-coreclruntriaged
@performanceautofiler

Description

@performanceautofiler
performanceautofiler
bot
opened on Dec 8, 2022

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d47a24c1c241cfe54247b452bb65aaa26fbc906d
Compare 2e2f9a4893f4fe72ccddb5ad86b4447a3cb7eb8f
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 521.80 ns 374.17 ns 0.72 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 374.17081070170286 < 487.1458310080382.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 64.95106219296936 (T) = (0 -374.52237872134555) / Math.Sqrt((105.97080077758665 / (30)) + (6.56163715896749 / (8))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (8) - 2, .975) and 0.26568341198175854 = (510.0284874839868 - 374.52237872134555) / 510.0284874839868 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline d47a24c1c241cfe54247b452bb65aaa26fbc906d
Compare 2e2f9a4893f4fe72ccddb5ad86b4447a3cb7eb8f
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 53.87 ns 48.44 ns 0.90 0.09 False
IsMatch - Duration of single invocation 57.07 ns 51.35 ns 0.90 0.14 False
IsMatch - Duration of single invocation 52.02 ns 45.99 ns 0.88 0.14 False
IsMatch - Duration of single invocation 93.19 ns 88.17 ns 0.95 0.07 False
IsMatch - Duration of single invocation 57.09 ns 52.24 ns 0.91 0.13 False
IsMatch - Duration of single invocation 56.71 ns 47.00 ns 0.83 0.10 False
IsMatch - Duration of single invocation 51.30 ns 45.15 ns 0.88 0.12 False
IsMatch - Duration of single invocation 89.12 ns 80.53 ns 0.90 0.02 False
IsMatch - Duration of single invocation 60.03 ns 50.12 ns 0.83 0.09 False
IsMatch - Duration of single invocation 51.77 ns 44.96 ns 0.87 0.13 False
IsMatch - Duration of single invocation 50.31 ns 44.53 ns 0.89 0.16 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.44093151184738 < 51.32102078055121.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 34.5193949402024 (T) = (0 -48.822367745096) / Math.Sqrt((1.2843438574189638 / (214)) + (0.19438181826389384 / (8))) is greater than 1.970805592384696 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (214) + (8) - 2, .975) and 0.10958550207050204 = (54.831056613098525 - 48.822367745096) / 54.831056613098525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.34510297380288 < 54.34785109400787.
IsChangePoint: Marked as a change because one of 9/26/2022 8:26:31 AM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 48.247761900905616 (T) = (0 -51.355516108592184) / Math.Sqrt((1.800400950561434 / (299)) + (0.10018762972208113 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.11342698320199945 = (57.92587314925374 - 51.355516108592184) / 57.92587314925374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.987169438292014 < 48.80416620790428.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 21.879908056221165 (T) = (0 -45.87119660820969) / Math.Sqrt((2.5274981143683735 / (299)) + (0.6093367540504335 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.12184649807844238 = (52.235966158348475 - 45.87119660820969) / 52.235966158348475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.16518187585915 < 88.90155136313511.
IsChangePoint: Marked as a change because one of 9/3/2022 11:44:47 AM, 9/13/2022 2:36:07 PM, 10/17/2022 8:10:07 PM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 24.39487006639447 (T) = (0 -87.39155770311021) / Math.Sqrt((2.6665553355043032 / (299)) + (0.3582011911441299 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.060753203147685175 = (93.04429676628588 - 87.39155770311021) / 93.04429676628588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.23618344827755 < 54.22554732157271.
IsChangePoint: Marked as a change because one of 9/16/2022 1:47:08 AM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 17.882062587586695 (T) = (0 -51.69149636367514) / Math.Sqrt((1.6917669264798025 / (299)) + (0.9173726509671576 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.10714365311941731 = (57.89452753992547 - 51.69149636367514) / 57.89452753992547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.99659215124652 < 49.58964782606426.
IsChangePoint: Marked as a change because one of 9/13/2022 2:36:07 PM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 42.617664012653606 (T) = (0 -46.82649862271023) / Math.Sqrt((1.465072715196273 / (299)) + (0.16617745258387975 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.12726550720301202 = (53.65491911822812 - 46.82649862271023) / 53.65491911822812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.14936964984747 < 48.677222169751246.
IsChangePoint: Marked as a change because one of 9/14/2022 7:20:13 PM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 18.610244433506757 (T) = (0 -45.666728851869344) / Math.Sqrt((1.6982502959895978 / (299)) + (0.8566574800305123 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.12037371508880113 = (51.91605757492746 - 45.666728851869344) / 51.91605757492746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.5312681234903 < 84.57691959856041.
IsChangePoint: Marked as a change because one of 9/13/2022 2:36:07 PM, 10/27/2022 12:34:03 AM, 11/21/2022 12:46:21 PM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 13.542072694098579 (T) = (0 -81.54248787259885) / Math.Sqrt((2.999585128665004 / (299)) + (1.044466498561869 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.05861976144924706 = (86.62013980464774 - 81.54248787259885) / 86.62013980464774 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.12438282587573 < 53.09769329324161.
IsChangePoint: Marked as a change because one of 9/13/2022 2:36:07 PM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 37.24054174051443 (T) = (0 -50.23242512087904) / Math.Sqrt((1.5435200640318971 / (299)) + (0.1802882648517617 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.1098323483390383 = (56.43029717733562 - 50.23242512087904) / 56.43029717733562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.96258279598192 < 49.07205706448382.
IsChangePoint: Marked as a change because one of 9/25/2022 1:34:22 AM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 13.941171968087115 (T) = (0 -45.99913211718633) / Math.Sqrt((2.0714494742792526 / (299)) + (1.5815887378849405 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.12056819776021512 = (52.305513628269104 - 45.99913211718633) / 52.305513628269104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.52690783072624 < 48.11725741578078.
IsChangePoint: Marked as a change because one of 9/26/2022 8:26:31 AM, 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 39.47516772894293 (T) = (0 -45.223738448710456) / Math.Sqrt((1.8396493015638045 / (299)) + (0.16438507616188427 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.12482828313400332 = (51.67413157575219 - 45.223738448710456) / 51.67413157575219 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d47a24c1c241cfe54247b452bb65aaa26fbc906d
Compare 2e2f9a4893f4fe72ccddb5ad86b4447a3cb7eb8f
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 11.36 μs 9.31 μs 0.82 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.30719266638406 < 10.812089834148424.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 123.41033245648134 (T) = (0 -9316.857381295004) / Math.Sqrt((4924.298361604434 / (30)) + (996.390925105465 / (8))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (8) - 2, .975) and 0.18371388011686787 = (11413.715306869239 - 9316.857381295004) / 11413.715306869239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d47a24c1c241cfe54247b452bb65aaa26fbc906d
Compare 2e2f9a4893f4fe72ccddb5ad86b4447a3cb7eb8f
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Cache

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch_Multithreading - Duration of single invocation 57.50 ms 0.00 ns 0.00 0.03 False
IsMatch_Multithreading - Duration of single invocation 56.73 ms 0.00 ns 0.00 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 400000, unique: 7, cacheSize: 15)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 54.52570540625.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 1181.4134682867616 (T) = (0 -0) / Math.Sqrt((714408679145.1837 / (299)) + (0 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 1 = (57748363.88889185 - 0) / 57748363.88889185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 400000, unique: 1, cacheSize: 15)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 53.685199437499996.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 1029.21572113035 (T) = (0 -0) / Math.Sqrt((625416016693.5277 / (214)) + (0 / (8))) is greater than 1.970805592384696 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (214) + (8) - 2, .975) and 1 = (55639603.2235952 - 0) / 55639603.2235952 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d47a24c1c241cfe54247b452bb65aaa26fbc906d
Compare 2e2f9a4893f4fe72ccddb5ad86b4447a3cb7eb8f
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 941.45 ns 539.95 ns 0.57 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 539.9502628571125 < 898.8775359580919.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 143.6058832744654 (T) = (0 -535.4289750909716) / Math.Sqrt((213.75747773511105 / (30)) + (6.5428364555505505 / (8))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (8) - 2, .975) and 0.4304921276739345 = (940.1607969071542 - 535.4289750909716) / 940.1607969071542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d47a24c1c241cfe54247b452bb65aaa26fbc906d
Compare 2e2f9a4893f4fe72ccddb5ad86b4447a3cb7eb8f
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 7.09 μs 5.12 μs 0.72 0.20 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.12311745021221 < 6.810129502418989.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsImprovementStdDev: Marked as improvement because 46.71856069214439 (T) = (0 -5129.167806315404) / Math.Sqrt((57327.05519874596 / (30)) + (980.0839272823303 / (8))) is greater than 2.0280940009779607 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (8) - 2, .975) and 0.2911465891833013 = (7235.865311568272 - 5129.167806315404) / 7235.865311568272 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Labels

arch-arm64branch-refs/heads/mainkind-microos-windowsperf-improvementruntime-coreclruntriaged

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions