-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | a9d25b6324708127c0aada0244e7a76af390b56d |
| Compare | 3f4ac7f4c38d4e8043175639dd3a47f921500442 |
| Diff | Diff |
Regressions in System.Text.Tests.Perf_Encoding
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetChars - Duration of single invocation | 98.26 ns | 110.39 ns | 1.12 | 0.07 | False | |||||
| GetString - Duration of single invocation | 90.52 ns | 105.21 ns | 1.16 | 0.09 | False | |||||
| GetChars - Duration of single invocation | 29.83 ns | 36.72 ns | 1.23 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'Details
Payloads
Histogram
System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.38604202467967 > 103.14220061275307.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -73.55434422020438 (T) = (0 -110.4705984435926) / Math.Sqrt((1.3181753708610513 / (55)) + (0.007581557031792949 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.12154973336553813 = (98.49817191083739 - 110.4705984435926) / 98.49817191083739 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.20531467795311 > 95.02835537614881.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/7/2022 10:45:01 PM, 3/21/2022 1:28:14 PM, 3/25/2022 7:34:40 AM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -62.973701577965585 (T) = (0 -105.2643868896783) / Math.Sqrt((2.982663266993408 / (55)) + (0.002725657231597837 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.16343650781164562 = (90.47712202849326 - 105.2643868896783) / 90.47712202849326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.72061018190148 > 31.21869926682091.
IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -10.109131726201786 (T) = (0 -36.976784843415736) / Math.Sqrt((0.1427228925743686 / (55)) + (1.3688149389557562 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.2272866624618307 = (30.128889993185055 - 36.976784843415736) / 30.128889993185055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | a9d25b6324708127c0aada0244e7a76af390b56d |
| Compare | 3f4ac7f4c38d4e8043175639dd3a47f921500442 |
| Diff | Diff |
Regressions in PerfLabTests.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IFooObjIsIFoo - Duration of single invocation | 501.17 μs | 746.78 μs | 1.49 | 0.38 | False | |||||
| CheckArrayIsNonvariantGenericInterfaceNo - Duration of single invocation | 7.59 ns | 12.14 ns | 1.60 | 0.41 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'Details
Payloads
Histogram
PerfLabTests.CastingPerf.IFooObjIsIFoo
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 746.7817708333333 > 539.0385636268029.
IsChangePoint: Marked as a change because one of 3/8/2022 5:51:58 AM, 3/11/2022 11:18:20 AM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -3.4853443562581 (T) = (0 -947343.0085784314) / Math.Sqrt((14419596474.537815 / (55)) + (30180846071.982964 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.5969134144392011 = (593233.797156821 - 947343.0085784314) / 593233.797156821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.CastingPerf.CheckArrayIsNonvariantGenericInterfaceNo
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.143733030819977 > 7.730138894624011.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/3/2022 3:34:31 AM, 4/4/2022 4:14:58 AM, 4/25/2022 2:48:42 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -12.085092571856082 (T) = (0 -11.707063810166687) / Math.Sqrt((0.9530977214325811 / (52)) + (0.9227826681232261 / (6))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (6) - 2, .025) and -0.7490999033509673 = (6.693193332032101 - 11.707063810166687) / 6.693193332032101 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | a9d25b6324708127c0aada0244e7a76af390b56d |
| Compare | 3f4ac7f4c38d4e8043175639dd3a47f921500442 |
| Diff | Diff |
Regressions in System.Linq.Tests.Perf_Enumerable
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Select - Duration of single invocation | 1.01 μs | 1.11 μs | 1.10 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'Details
Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.Select(input: IList)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.112483387355955 > 1.059702745925858.
IsChangePoint: Marked as a change because one of 2/19/2022 1:52:17 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -12.495458940095526 (T) = (0 -1112.6850838105906) / Math.Sqrt((2252.3991069586846 / (55)) + (0.25533249089619414 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.07751685974488794 = (1032.6382123376047 - 1112.6850838105906) / 1032.6382123376047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | a9d25b6324708127c0aada0244e7a76af390b56d |
| Compare | 3f4ac7f4c38d4e8043175639dd3a47f921500442 |
| Diff | Diff |
Regressions in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ICollection - Duration of single invocation | 16.92 μs | 22.12 μs | 1.31 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse<Int32>*'Details
Payloads
Histogram
System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.117128418198963 > 17.76386576704546.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 4/26/2022 6:18:08 AM falls between 4/17/2022 3:25:40 PM and 4/26/2022 6:18:08 AM.
IsRegressionStdDev: Marked as regression because -424.3460759080571 (T) = (0 -22108.99384134842) / Math.Sqrt((7217.817995957334 / (55)) + (56.36996203148867 / (3))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (55) + (3) - 2, .025) and -0.30733968310025006 = (16911.437881942613 - 22108.99384134842) / 16911.437881942613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository






_1.png)