Skip to content

[Perf] Linux/x64: 35 Improvements on 2/9/2023 12:40:40 PM #12928

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_Comparison - Duration of single invocation 22.08 μs 18.66 μs 0.84 0.00 False
LinqOrderByExtension - Duration of single invocation 30.04 μs 27.85 μs 0.93 0.01 False
LinqQuery - Duration of single invocation 30.25 μs 28.52 μs 0.94 0.01 False
Array_ComparerStruct - Duration of single invocation 26.25 μs 23.23 μs 0.88 0.00 False
Array_ComparerClass - Duration of single invocation 22.41 μs 18.85 μs 0.84 0.00 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.Array_Comparison(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.655065342857146 < 20.967218087461536.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 69.99078576466177 (T) = (0 -18768.33706986463) / Math.Sqrt((39027.68161447508 / (299)) + (50117.57421660486 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15197959428030494 = (22131.94039114705 - 18768.33706986463) / 22131.94039114705 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqOrderByExtension(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.84911717142857 < 28.61036853135714.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 29.528199544175816 (T) = (0 -26622.730915785953) / Math.Sqrt((98332.70299911403 / (299)) + (276805.6569484318 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10978815423105835 = (29906.062295531392 - 26622.730915785953) / 29906.062295531392 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqQuery(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.51633588571429 < 28.551109341333333.
IsChangePoint: Marked as a change because one of 12/14/2022 9:46:59 AM, 2/9/2023 4:10:34 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 21.499707826249114 (T) = (0 -26692.354737209134) / Math.Sqrt((99334.45218583963 / (299)) + (458750.70866149216 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10710335985945138 = (29894.114880987298 - 26692.354737209134) / 29894.114880987298 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.Array_ComparerStruct(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.229056 < 24.95388532864102.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 99.93137308151854 (T) = (0 -23451.190323080253) / Math.Sqrt((56179.63162196762 / (299)) + (12654.805974799996 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10531728160177288 = (26211.739470128032 - 23451.190323080253) / 26211.739470128032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.Array_ComparerClass(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.849780340000002 < 21.31952050033333.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 36.656607892664454 (T) = (0 -19204.649088620794) / Math.Sqrt((60089.946219705154 / (299)) + (157534.88462198214 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13813105117844057 = (22282.562928945834 - 19204.649088620794) / 22282.562928945834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringHex - Duration of single invocation 17.25 ns 15.80 ns 0.92 0.01 False
ToStringHex - Duration of single invocation 22.55 ns 20.02 ns 0.89 0.02 False
ToStringHex - Duration of single invocation 20.96 ns 18.54 ns 0.88 0.03 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToStringHex(value: 12345)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.79533917123477 < 16.26100018047935.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 12/23/2022 9:40:21 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 18.87755606527024 (T) = (0 -15.820643049756933) / Math.Sqrt((0.20914443356288187 / (299)) + (0.06587108048487181 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06649271372633969 = (16.947530332526046 - 15.820643049756933) / 16.947530332526046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.020708872276398 < 21.393379434145977.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/19/2023 8:10:32 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 39.26653500426708 (T) = (0 -19.960346784674453) / Math.Sqrt((0.4175038024902332 / (299)) + (0.03098536239220023 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09435408984831584 = (22.0399016447072 - 19.960346784674453) / 22.0399016447072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.53841874996217 < 19.898824355459393.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 1/19/2023 8:10:32 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 33.79620344354659 (T) = (0 -18.20398231791182) / Math.Sqrt((0.5991548697624273 / (299)) + (0.07137304692148153 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11713349757468898 = (20.619178854225297 - 18.20398231791182) / 20.619178854225297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same - Duration of single invocation 1.13 μs 535.87 ns 0.48 0.00 False
Compare_Same_Upper - Duration of single invocation 1.58 μs 1.15 μs 0.73 0.03 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'
Details

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 535.8743546417638 < 1.0720016370784626.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 3487.3599285972577 (T) = (0 -536.4763573925535) / Math.Sqrt((1.0886625566404384 / (299)) + (0.580976541189526 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5249631853700703 = (1129.3363816665185 - 536.4763573925535) / 1129.3363816665185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1450196802069574 < 1.4975452737365693.
IsChangePoint: Marked as a change because one of 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 66.9211134857743 (T) = (0 -1160.9637479934192) / Math.Sqrt((11133.809904163594 / (299)) + (363.38547072142063 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.29707219479102626 = (1651.611644026908 - 1160.9637479934192) / 1651.611644026908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToByteArray - Duration of single invocation 64.21 ns 48.15 ns 0.75 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.1549099088941 < 60.946301909687676.
IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/10/2022 11:25:25 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 29.78404923699472 (T) = (0 -48.54087514668297) / Math.Sqrt((10.087626824463158 / (299)) + (4.465098810156508 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.22654611978931888 = (62.758590251639724 - 48.54087514668297) / 62.758590251639724 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFullPathForLegacyLength - Duration of single invocation 300.19 ns 248.83 ns 0.83 0.00 False
GetFullPathForReallyLongPath - Duration of single invocation 1.48 μs 1.23 μs 0.83 0.01 False
GetFullPathForTypicalLongPath - Duration of single invocation 770.67 ns 653.08 ns 0.85 0.00 False
GetFullPathNoRedundantSegments - Duration of single invocation 146.22 ns 125.13 ns 0.86 0.01 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFullPathForLegacyLength


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 248.83103081927348 < 285.05177116127015.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 109.53129757140499 (T) = (0 -253.8915599553503) / Math.Sqrt((1.6704124024779252 / (299)) + (3.800891071427518 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.15411450284434566 = (300.1488508894849 - 253.8915599553503) / 300.1488508894849 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathForReallyLongPath

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2324496193505676 < 1.4115456997230962.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 131.96200564352864 (T) = (0 -1235.2296779536077) / Math.Sqrt((125.568833676854 / (299)) + (69.6682621385252 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.16555636971312299 = (1480.3033220218156 - 1235.2296779536077) / 1480.3033220218156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathForTypicalLongPath

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 653.083375864628 < 738.4537710720456.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 129.41980486701416 (T) = (0 -645.675443755768) / Math.Sqrt((53.373639101637636 / (299)) + (16.87932608789476 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.16069325755413097 = (769.2961477637728 - 645.675443755768) / 769.2961477637728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathNoRedundantSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.12822962095733 < 138.913588434111.
IsChangePoint: Marked as a change because one of 1/19/2023 8:10:32 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 76.29094099234786 (T) = (0 -124.22482538983486) / Math.Sqrt((44.70316149146055 / (299)) + (0.30189563470810293 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1985052497137453 = (154.991439863415 - 124.22482538983486) / 154.991439863415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_ComparerStruct - Duration of single invocation 29.81 μs 27.51 μs 0.92 0.02 False
LinqQuery - Duration of single invocation 39.46 μs 36.11 μs 0.92 0.06 False
Array_Comparison - Duration of single invocation 25.95 μs 23.38 μs 0.90 0.03 False
LinqOrderByExtension - Duration of single invocation 39.66 μs 35.93 μs 0.91 0.09 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<BigStruct>.Array_ComparerStruct(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.50707044 < 28.598639285333334.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 25.95910142654973 (T) = (0 -27602.938063956048) / Math.Sqrt((154288.86483542123 / (299)) + (195671.01281154572 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08200858368365112 = (30068.841138753964 - 27602.938063956048) / 30068.841138753964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;BigStruct&gt;.LinqQuery(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.1051265 < 37.69769950928571.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 11.603820516565936 (T) = (0 -36165.62763385571) / Math.Sqrt((1115807.6498390615 / (299)) + (1557644.8135222695 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07899251540992734 = (39267.463336579196 - 36165.62763385571) / 39267.463336579196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;BigStruct&gt;.Array_Comparison(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.382021742857145 < 24.63448381321428.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 42.75097915346008 (T) = (0 -23703.506257923233) / Math.Sqrt((201489.89844685057 / (299)) + (62325.32905708994 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09495108319261958 = (26190.303991014 - 23703.506257923233) / 26190.303991014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;BigStruct&gt;.LinqOrderByExtension(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.92643344285714 < 37.64318471346154.
IsChangePoint: Marked as a change because one of 1/8/2023 4:08:48 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 28.225837299239902 (T) = (0 -35960.87148888358) / Math.Sqrt((2534322.6614197455 / (299)) + (160346.12525611857 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08888339223729712 = (39469.011082113284 - 35960.87148888358) / 39469.011082113284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 120.39 μs 98.70 μs 0.82 0.13 False
GetString - Duration of single invocation 228.45 μs 212.10 μs 0.93 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.70416877246183 < 114.09442162793074.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/17/2023 8:36:05 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 56.943352161552575 (T) = (0 -99777.41095685864) / Math.Sqrt((17434373.297598973 / (299)) + (1095462.248076929 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15665413382736282 = (118311.37728780152 - 99777.41095685864) / 118311.37728780152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 212.09657851914412 < 217.0696836963987.
IsChangePoint: Marked as a change because one of 1/19/2023 12:32:21 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 9.61617591326962 (T) = (0 -198393.07146143966) / Math.Sqrt((56470019.345327795 / (299)) + (97349485.24763575 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09249287216742653 = (218613.23771116574 - 198393.07146143966) / 218613.23771116574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions