-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 70065c1b180ebe66dcf68ce03a47e54c23135d79 |
| Compare | abb7df59eb4f8c813c07ddd6f46f94e59604577d |
| Diff | Diff |
Improvements in System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentStack - Duration of single invocation | 271.02 ms | 206.88 ms | 0.76 | 0.01 | True | |||||
| ConcurrentBag - Duration of single invocation | 375.89 ms | 348.20 ms | 0.93 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>*'Details
Payloads
Histogram
System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>.ConcurrentStack(Size: 2000000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 16.67948171180381 (T) = (0 -215443465.83333334) / Math.Sqrt((10546042819398.842 / (299)) + (95493177328510.33 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.21127829658274336 = (273155239.5476019 - 215443465.83333334) / 273155239.5476019 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>.ConcurrentBag(Size: 2000000)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 79.06356046101595 (T) = (0 -348663795.66620874) / Math.Sqrt((2438879688158.7295 / (299)) + (970959441663.8026 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07545326945332011 = (377118629.2120092 - 348663795.66620874) / 377118629.2120092 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 70065c1b180ebe66dcf68ce03a47e54c23135d79 |
| Compare | abb7df59eb4f8c813c07ddd6f46f94e59604577d |
| Diff | Diff |
Improvements in System.Collections.Concurrent.Count<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 12.59 ns | 5.72 ns | 0.45 | 0.07 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count<Int32>*'Details
Payloads
Histogram
System.Collections.Concurrent.Count<Int32>.Queue(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 176.103931468525 (T) = (0 -5.654123445769828) / Math.Sqrt((0.08495375148919211 / (299)) + (0.010836976491446562 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5576864764089658 = (12.7830670874935 - 5.654123445769828) / 12.7830670874935 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 70065c1b180ebe66dcf68ce03a47e54c23135d79 |
| Compare | abb7df59eb4f8c813c07ddd6f46f94e59604577d |
| Diff | Diff |
Improvements in System.Collections.CtorFromCollection<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentBag - Duration of single invocation | 27.69 μs | 22.26 μs | 0.80 | 0.01 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection<String>*'Details
Payloads
Histogram
System.Collections.CtorFromCollection<String>.ConcurrentBag(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 12.787278352807247 (T) = (0 -19400.039240822825) / Math.Sqrt((20194.143552133308 / (299)) + (3355232.232347668 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.2991803402211076 = (27681.92782568844 - 19400.039240822825) / 27681.92782568844 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 70065c1b180ebe66dcf68ce03a47e54c23135d79 |
| Compare | abb7df59eb4f8c813c07ddd6f46f94e59604577d |
| Diff | Diff |
Improvements in System.Linq.Tests.Perf_Enumerable
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CastToSameType - Duration of single invocation | 924.66 ns | 721.65 ns | 0.78 | 0.19 | False | |||||
| Reverse - Duration of single invocation | 1.76 μs | 1.52 μs | 0.87 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'Details
Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 11/13/2021 7:30:09 PM, 11/14/2021 9:27:23 PM, 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 17.367621436107566 (T) = (0 -735.6843592021042) / Math.Sqrt((3382.2786564589946 / (299)) + (1106.5902015446243 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.22407213142223642 = (948.134986504063 - 735.6843592021042) / 948.134986504063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/7/2021 2:09:29 PM, 12/8/2021 2:58:05 AM, 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 32.919980304971006 (T) = (0 -1524.822385350095) / Math.Sqrt((3594.218187031065 / (299)) + (353.76514803907315 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.13934672045501095 = (1771.7034508440372 - 1524.822385350095) / 1771.7034508440372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 70065c1b180ebe66dcf68ce03a47e54c23135d79 |
| Compare | abb7df59eb4f8c813c07ddd6f46f94e59604577d |
| Diff | Diff |
Improvements in System.Collections.Concurrent.Count<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 13.46 ns | 7.23 ns | 0.54 | 0.04 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count<String>*'Details
Payloads
Histogram
System.Collections.Concurrent.Count<String>.Queue(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 1/11/2022 6:41:22 PM falls between 1/3/2022 4:56:39 AM and 1/11/2022 6:41:22 PM.
IsImprovementStdDev: Marked as improvement because 162.30410568234697 (T) = (0 -7.422319921367619) / Math.Sqrt((0.08810113342015431 / (299)) + (0.008679943105738509 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.448190567333139 = (13.450875396413585 - 7.422319921367619) / 13.450875396413585 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
_1.png)
_2.png)
_1.png)
_1.png)


_1.png)