-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Closed
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMICLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtime
Milestone
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 4019e83878a81465f6e42e8502b53bc5d1752f81 |
| Compare | 019bc7f143ae5840fe7d0e8e720e4854b1a91c26 |
| Diff | Diff |
Regressions in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsSuffix_DifferentLastChar - Duration of single invocation | 16.64 ns | 19.98 ns | 1.20 | 0.29 | False | |||||
| IsPrefix_FirstHalf - Duration of single invocation | 12.03 ns | 13.96 ns | 1.16 | 0.18 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'Details
Payloads
Histogram
System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False))
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.97911225341915 > 17.44699084664059.
IsChangePoint: Marked as a change because one of 3/2/2022 3:17:47 PM, 3/28/2022 7:37:41 AM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -13.002365424661688 (T) = (0 -19.115487599893818) / Math.Sqrt((0.32787265954056094 / (19)) + (0.5051995568270351 / (36))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (36) - 2, .025) and -0.13677715168127164 = (16.8155100334506 - 19.115487599893818) / 16.8155100334506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.957168843212287 > 12.539578215438409.
IsChangePoint: Marked as a change because one of 2/4/2022 6:00:25 PM, 3/2/2022 3:17:47 PM, 3/30/2022 4:48:39 AM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -10.00018437580746 (T) = (0 -14.18687111136129) / Math.Sqrt((0.34624076266092135 / (28)) + (0.5333189236045973 / (27))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (27) - 2, .025) and -0.1445935477736601 = (12.39468031158839 - 14.18687111136129) / 12.39468031158839 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 4019e83878a81465f6e42e8502b53bc5d1752f81 |
| Compare | 019bc7f143ae5840fe7d0e8e720e4854b1a91c26 |
| Diff | Diff |
Regressions in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SequenceEqual - Duration of single invocation | 12.39 ns | 14.71 ns | 1.19 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Byte>*'Details
Payloads
Histogram
System.Memory.Span<Byte>.SequenceEqual(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.712574604126564 > 13.072225077291382.
IsChangePoint: Marked as a change because one of 1/28/2022 6:30:08 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/28/2022 7:37:41 AM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -8.217728367436298 (T) = (0 -13.685223290119147) / Math.Sqrt((0.21000824790973555 / (18)) + (0.39593923483020577 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.09938774412884786 = (12.448040614608892 - 13.685223290119147) / 12.448040614608892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 4019e83878a81465f6e42e8502b53bc5d1752f81 |
| Compare | 019bc7f143ae5840fe7d0e8e720e4854b1a91c26 |
| Diff | Diff |
Regressions in System.Globalization.Tests.StringEquality
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compare_Same - Duration of single invocation | 54.44 ns | 67.42 ns | 1.24 | 0.17 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'Details
Payloads
Histogram
System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, Ordinal))
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.42402506213939 > 57.61935682181894.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 3/17/2022 6:54:53 PM, 3/28/2022 7:37:41 AM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -24.152920412669175 (T) = (0 -71.68631198538739) / Math.Sqrt((2.5467050998996177 / (19)) + (13.176893030965825 / (36))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (36) - 2, .025) and -0.31277837319955654 = (54.606560748461 - 71.68631198538739) / 54.606560748461 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMICLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtime


_1.png)
