-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 4ff5a3a85a9b0de7ab9e9267959e668142815f4f |
| Compare | 6e4eacf77636a1858851b0ea18907c198e61a6e6 |
| Diff | Diff |
Improvements in PerfLabTests.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ObjObjIsFoo - Duration of single invocation | 1.25 ms | 846.21 μs | 0.68 | 0.32 | False | |||||
| FooObjIsFoo - Duration of single invocation | 413.54 μs | 322.61 μs | 0.78 | 0.12 | False | |||||
| FooObjIsFoo2 - Duration of single invocation | 413.53 μs | 322.21 μs | 0.78 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'Details
Payloads
Histogram
Edge Detector Info
PerfLabTests.CastingPerf.ObjObjIsFoo
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 846.2142712550608 < 1.1829973506098768.
IsChangePoint: Marked as a change because one of 8/31/2022 1:36:07 AM, 9/2/2022 3:08:54 AM, 9/14/2022 9:08:24 AM, 9/15/2022 1:10:12 AM, 9/22/2022 3:16:12 AM falls between 9/13/2022 2:36:07 PM and 9/22/2022 3:16:12 AM.
IsImprovementStdDev: Marked as improvement because 7.899268754659466 (T) = (0 -826184.4066056567) / Math.Sqrt((55801254980.91177 / (299)) + (90251062.38136558 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11634588692947283 = (934963.5727205814 - 826184.4066056567) / 934963.5727205814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.CastingPerf.FooObjIsFoo
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 322.6121193910256 < 392.8613870192308.
IsChangePoint: Marked as a change because one of 9/20/2022 10:35:57 PM, 9/22/2022 3:16:12 AM falls between 9/13/2022 2:36:07 PM and 9/22/2022 3:16:12 AM.
IsImprovementStdDev: Marked as improvement because 97.87788864264385 (T) = (0 -322586.5286494348) / Math.Sqrt((287349367.03919613 / (299)) + (799.151755503869 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.2292732811614827 = (418548.5214987378 - 322586.5286494348) / 418548.5214987378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.CastingPerf.FooObjIsFoo2
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 322.21438775510205 < 392.8587087912088.
IsChangePoint: Marked as a change because one of 9/20/2022 10:35:57 PM, 9/22/2022 3:16:12 AM falls between 9/13/2022 2:36:07 PM and 9/22/2022 3:16:12 AM.
IsImprovementStdDev: Marked as improvement because 106.10813662660829 (T) = (0 -322205.7243937238) / Math.Sqrt((241974267.03083745 / (299)) + (1760.205721116171 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.22859426883563264 = (417686.4539331116 - 322205.7243937238) / 417686.4539331116 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


