-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 8cada8ea5cbb72a6cc3f14af65e48b5076273ef9 |
| Compare | 8e624b33465546e9d05da2afea5faea233d9f847 |
| Diff | Diff |
Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| EncodeUtf8 - Duration of single invocation | 58.98 ns | 50.83 ns | 0.86 | 0.00 | True | |||||
| EncodeUtf8 - Duration of single invocation | 73.62 ns | 66.52 ns | 0.90 | 0.00 | True | |||||
| EncodeUtf8 - Duration of single invocation | 85.21 ns | 79.20 ns | 0.93 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'Details
Payloads
Histogram
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.833972865792845 < 55.81928872569761.
IsChangePoint: Marked as a change because one of 4/29/2022 5:18:09 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 67.17281737947953 (T) = (0 -51.0122138204924) / Math.Sqrt((0.3117810562666219 / (299)) + (0.20115845415698883 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.1270597010591373 = (58.437230910734044 - 51.0122138204924) / 58.437230910734044 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.51780598843564 < 69.92170472922085.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 4/29/2022 5:18:09 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 102.0901063073727 (T) = (0 -65.96116241928499) / Math.Sqrt((0.7011638526837292 / (299)) + (0.06273835092002156 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.10569047970963816 = (73.75652491977152 - 65.96116241928499) / 73.75652491977152 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.20360497654697 < 80.94849351651644.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 1:54:35 PM, 4/7/2022 3:14:09 PM, 4/29/2022 5:18:09 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 37.59292467311908 (T) = (0 -78.90182675752015) / Math.Sqrt((6.51701911656081 / (299)) + (0.4101397825110382 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.09140447262914747 = (86.83932991155433 - 78.90182675752015) / 86.83932991155433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 8cada8ea5cbb72a6cc3f14af65e48b5076273ef9 |
| Compare | 8e624b33465546e9d05da2afea5faea233d9f847 |
| Diff | Diff |
Improvements in System.Collections.ContainsTrue<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| HashSet - Duration of single invocation | 4.88 μs | 3.63 μs | 0.74 | 0.23 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*'Details
Payloads
Histogram
System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.625816312793292 < 4.649977229936858.
IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 4/29/2022 5:18:09 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 8.722351545035762 (T) = (0 -3613.3535435574618) / Math.Sqrt((746130.9920785124 / (299)) + (455.53264303335186 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.10809481790324442 = (4051.275422644061 - 3613.3535435574618) / 4051.275422644061 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 8cada8ea5cbb72a6cc3f14af65e48b5076273ef9 |
| Compare | 8e624b33465546e9d05da2afea5faea233d9f847 |
| Diff | Diff |
Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| DivideBenchmark - Duration of single invocation | 115.77 ns | 109.72 ns | 0.95 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<SByte>*'Details
Payloads
Histogram
System.Numerics.Tests.Perf_VectorOf<SByte>.DivideBenchmark
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 109.71815185690187 < 110.27037573908156.
IsChangePoint: Marked as a change because one of 2/24/2022 10:17:16 AM, 3/17/2022 6:54:53 PM, 4/29/2022 5:18:09 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM.
IsImprovementStdDev: Marked as improvement because 35.180076522563034 (T) = (0 -109.70083990782406) / Math.Sqrt((21.572281629690913 / (299)) + (0.004298038848506047 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.07942814941499995 = (119.16597258335888 - 109.70083990782406) / 119.16597258335888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository