-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Collections.Sort<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Array_Comparison - Duration of single invocation | 22.08 μs | 18.66 μs | 0.84 | 0.00 | False | |||||
| LinqOrderByExtension - Duration of single invocation | 30.04 μs | 27.85 μs | 0.93 | 0.01 | False | |||||
| LinqQuery - Duration of single invocation | 30.25 μs | 28.52 μs | 0.94 | 0.01 | False | |||||
| Array_ComparerStruct - Duration of single invocation | 26.25 μs | 23.23 μs | 0.88 | 0.00 | False | |||||
| Array_ComparerClass - Duration of single invocation | 22.41 μs | 18.85 μs | 0.84 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<Int32>*'Details
Payloads
Histogram
System.Collections.Sort<Int32>.Array_Comparison(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.655065342857146 < 20.967218087461536.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 69.99078576466177 (T) = (0 -18768.33706986463) / Math.Sqrt((39027.68161447508 / (299)) + (50117.57421660486 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15197959428030494 = (22131.94039114705 - 18768.33706986463) / 22131.94039114705 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.84911717142857 < 28.61036853135714.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 29.528199544175816 (T) = (0 -26622.730915785953) / Math.Sqrt((98332.70299911403 / (299)) + (276805.6569484318 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10978815423105835 = (29906.062295531392 - 26622.730915785953) / 29906.062295531392 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Sort<Int32>.LinqQuery(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.51633588571429 < 28.551109341333333.
IsChangePoint: Marked as a change because one of 12/14/2022 9:46:59 AM, 2/9/2023 4:10:34 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 21.499707826249114 (T) = (0 -26692.354737209134) / Math.Sqrt((99334.45218583963 / (299)) + (458750.70866149216 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.10710335985945138 = (29894.114880987298 - 26692.354737209134) / 29894.114880987298 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Sort<Int32>.Array_ComparerStruct(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.229056 < 24.95388532864102.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 99.93137308151854 (T) = (0 -23451.190323080253) / Math.Sqrt((56179.63162196762 / (299)) + (12654.805974799996 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10531728160177288 = (26211.739470128032 - 23451.190323080253) / 26211.739470128032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Sort<Int32>.Array_ComparerClass(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.849780340000002 < 21.31952050033333.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 36.656607892664454 (T) = (0 -19204.649088620794) / Math.Sqrt((60089.946219705154 / (299)) + (157534.88462198214 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13813105117844057 = (22282.562928945834 - 19204.649088620794) / 22282.562928945834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Tests.Perf_Int32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToStringHex - Duration of single invocation | 17.25 ns | 15.80 ns | 0.92 | 0.01 | False | |||||
| ToStringHex - Duration of single invocation | 22.55 ns | 20.02 ns | 0.89 | 0.02 | False | |||||
| ToStringHex - Duration of single invocation | 20.96 ns | 18.54 ns | 0.88 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'Details
Payloads
Histogram
System.Tests.Perf_Int32.ToStringHex(value: 12345)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.79533917123477 < 16.26100018047935.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 12/23/2022 9:40:21 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 18.87755606527024 (T) = (0 -15.820643049756933) / Math.Sqrt((0.20914443356288187 / (299)) + (0.06587108048487181 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06649271372633969 = (16.947530332526046 - 15.820643049756933) / 16.947530332526046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.020708872276398 < 21.393379434145977.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/19/2023 8:10:32 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 39.26653500426708 (T) = (0 -19.960346784674453) / Math.Sqrt((0.4175038024902332 / (299)) + (0.03098536239220023 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09435408984831584 = (22.0399016447072 - 19.960346784674453) / 22.0399016447072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToStringHex(value: -2147483648)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.53841874996217 < 19.898824355459393.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 1/19/2023 8:10:32 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 33.79620344354659 (T) = (0 -18.20398231791182) / Math.Sqrt((0.5991548697624273 / (299)) + (0.07137304692148153 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11713349757468898 = (20.619178854225297 - 18.20398231791182) / 20.619178854225297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Globalization.Tests.StringEquality
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compare_Same - Duration of single invocation | 1.13 μs | 535.87 ns | 0.48 | 0.00 | False | |||||
| Compare_Same_Upper - Duration of single invocation | 1.58 μs | 1.15 μs | 0.73 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'Details
Payloads
Histogram
System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 535.8743546417638 < 1.0720016370784626.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 3487.3599285972577 (T) = (0 -536.4763573925535) / Math.Sqrt((1.0886625566404384 / (299)) + (0.580976541189526 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5249631853700703 = (1129.3363816665185 - 536.4763573925535) / 1129.3363816665185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1450196802069574 < 1.4975452737365693.
IsChangePoint: Marked as a change because one of 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 66.9211134857743 (T) = (0 -1160.9637479934192) / Math.Sqrt((11133.809904163594 / (299)) + (363.38547072142063 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.29707219479102626 = (1651.611644026908 - 1160.9637479934192) / 1651.611644026908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Numerics.Tests.Perf_BigInteger
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToByteArray - Duration of single invocation | 64.21 ns | 48.15 ns | 0.75 | 0.12 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'Details
Payloads
Histogram
System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.1549099088941 < 60.946301909687676.
IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/10/2022 11:25:25 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 29.78404923699472 (T) = (0 -48.54087514668297) / Math.Sqrt((10.087626824463158 / (299)) + (4.465098810156508 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.22654611978931888 = (62.758590251639724 - 48.54087514668297) / 62.758590251639724 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.IO.Tests.Perf_Path
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetFullPathForLegacyLength - Duration of single invocation | 300.19 ns | 248.83 ns | 0.83 | 0.00 | False | |||||
| GetFullPathForReallyLongPath - Duration of single invocation | 1.48 μs | 1.23 μs | 0.83 | 0.01 | False | |||||
| GetFullPathForTypicalLongPath - Duration of single invocation | 770.67 ns | 653.08 ns | 0.85 | 0.00 | False | |||||
| GetFullPathNoRedundantSegments - Duration of single invocation | 146.22 ns | 125.13 ns | 0.86 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'Details
Payloads
Histogram
System.IO.Tests.Perf_Path.GetFullPathForLegacyLength
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 248.83103081927348 < 285.05177116127015.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 109.53129757140499 (T) = (0 -253.8915599553503) / Math.Sqrt((1.6704124024779252 / (299)) + (3.800891071427518 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.15411450284434566 = (300.1488508894849 - 253.8915599553503) / 300.1488508894849 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.Perf_Path.GetFullPathForReallyLongPath
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2324496193505676 < 1.4115456997230962.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 131.96200564352864 (T) = (0 -1235.2296779536077) / Math.Sqrt((125.568833676854 / (299)) + (69.6682621385252 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.16555636971312299 = (1480.3033220218156 - 1235.2296779536077) / 1480.3033220218156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.Perf_Path.GetFullPathForTypicalLongPath
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 653.083375864628 < 738.4537710720456.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 129.41980486701416 (T) = (0 -645.675443755768) / Math.Sqrt((53.373639101637636 / (299)) + (16.87932608789476 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.16069325755413097 = (769.2961477637728 - 645.675443755768) / 769.2961477637728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.Perf_Path.GetFullPathNoRedundantSegments
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.12822962095733 < 138.913588434111.
IsChangePoint: Marked as a change because one of 1/19/2023 8:10:32 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 76.29094099234786 (T) = (0 -124.22482538983486) / Math.Sqrt((44.70316149146055 / (299)) + (0.30189563470810293 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1985052497137453 = (154.991439863415 - 124.22482538983486) / 154.991439863415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Collections.Sort<BigStruct>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Array_ComparerStruct - Duration of single invocation | 29.81 μs | 27.51 μs | 0.92 | 0.02 | False | |||||
| LinqQuery - Duration of single invocation | 39.46 μs | 36.11 μs | 0.92 | 0.06 | False | |||||
| Array_Comparison - Duration of single invocation | 25.95 μs | 23.38 μs | 0.90 | 0.03 | False | |||||
| LinqOrderByExtension - Duration of single invocation | 39.66 μs | 35.93 μs | 0.91 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<BigStruct>*'Details
Payloads
Histogram
System.Collections.Sort<BigStruct>.Array_ComparerStruct(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.50707044 < 28.598639285333334.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 25.95910142654973 (T) = (0 -27602.938063956048) / Math.Sqrt((154288.86483542123 / (299)) + (195671.01281154572 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08200858368365112 = (30068.841138753964 - 27602.938063956048) / 30068.841138753964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.1051265 < 37.69769950928571.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 11.603820516565936 (T) = (0 -36165.62763385571) / Math.Sqrt((1115807.6498390615 / (299)) + (1557644.8135222695 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07899251540992734 = (39267.463336579196 - 36165.62763385571) / 39267.463336579196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.382021742857145 < 24.63448381321428.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 42.75097915346008 (T) = (0 -23703.506257923233) / Math.Sqrt((201489.89844685057 / (299)) + (62325.32905708994 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09495108319261958 = (26190.303991014 - 23703.506257923233) / 26190.303991014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.92643344285714 < 37.64318471346154.
IsChangePoint: Marked as a change because one of 1/8/2023 4:08:48 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 28.225837299239902 (T) = (0 -35960.87148888358) / Math.Sqrt((2534322.6614197455 / (299)) + (160346.12525611857 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08888339223729712 = (39469.011082113284 - 35960.87148888358) / 39469.011082113284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Text.Perf_Utf8Encoding
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetBytes - Duration of single invocation | 120.39 μs | 98.70 μs | 0.82 | 0.13 | False | |||||
| GetString - Duration of single invocation | 228.45 μs | 212.10 μs | 0.93 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'Details
Payloads
Histogram
System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.70416877246183 < 114.09442162793074.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/17/2023 8:36:05 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 56.943352161552575 (T) = (0 -99777.41095685864) / Math.Sqrt((17434373.297598973 / (299)) + (1095462.248076929 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15665413382736282 = (118311.37728780152 - 99777.41095685864) / 118311.37728780152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 212.09657851914412 < 217.0696836963987.
IsChangePoint: Marked as a change because one of 1/19/2023 12:32:21 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 9.61617591326962 (T) = (0 -198393.07146143966) / Math.Sqrt((56470019.345327795 / (299)) + (97349485.24763575 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09249287216742653 = (218613.23771116574 - 198393.07146143966) / 218613.23771116574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
_1.png)
_2.png)
_3.png)
_4.png)
_5.png)










_1.png)
_2.png)
_3.png)
_4.png)

