-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 87aa786b3997a23501912abd4654e4fd9958230c |
| Compare | 55bf5d8a28f4a8c519dbc6edfc1d3d64519f530f |
| Diff | Diff |
Improvements in System.IO.Compression.Brotli
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Decompress - Duration of single invocation | 1.00 ms | 508.33 μs | 0.51 | 0.00 | False | |||||
| Decompress_WithoutState - Duration of single invocation | 953.78 μs | 469.22 μs | 0.49 | 0.01 | False | |||||
| Decompress - Duration of single invocation | 217.78 μs | 193.14 μs | 0.89 | 0.00 | False | |||||
| Decompress_WithState - Duration of single invocation | 958.57 μs | 474.95 μs | 0.50 | 0.01 | False | |||||
| Decompress_WithoutState - Duration of single invocation | 195.16 μs | 168.81 μs | 0.87 | 0.00 | False | |||||
| Decompress_WithState - Duration of single invocation | 192.89 μs | 168.74 μs | 0.87 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Brotli*'Details
Payloads
Histogram
System.IO.Compression.Brotli.Decompress(level: Optimal, file: "TestDocument.pdf")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 508.3324596774193 < 0.9545202320033481.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 485.56901975944515 (T) = (0 -508891.0183215828) / Math.Sqrt((272590893.84665054 / (299)) + (400569.91755718115 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.4826780784431704 = (983702.7914651774 - 508891.0183215828) / 983702.7914651774 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "TestDocument.pdf")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 469.2193167892157 < 912.5678233924278.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 263.4697258129493 (T) = (0 -474615.68573643616) / Math.Sqrt((385225052.7104263 / (299)) + (16302716.945762765 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.4942753590265815 = (938486.3763468122 - 474615.68573643616) / 938486.3763468122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "sum")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.14255198915006 < 207.36133362676054.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 41.3862194073105 (T) = (0 -190724.02870597685) / Math.Sqrt((10986296.697508018 / (299)) + (3054626.160739282 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.11745359973553528 = (216106.5170611135 - 190724.02870597685) / 216106.5170611135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress_WithState(level: Optimal, file: "TestDocument.pdf")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 474.9479637096775 < 922.2844808485243.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 259.3953402498576 (T) = (0 -469201.1182795337) / Math.Sqrt((490445557.35002965 / (299)) + (15143145.630686754 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.5019335949099965 = (942045.3045708761 - 469201.1182795337) / 942045.3045708761 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "sum")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.81175702811248 < 184.96796753416575.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 101.02885535733051 (T) = (0 -169201.48035737994) / Math.Sqrt((8909881.586432531 / (299)) + (254418.42292921885 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.12578444156075344 = (193546.63586571088 - 169201.48035737994) / 193546.63586571088 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress_WithState(level: Optimal, file: "sum")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.74373655913976 < 184.93206781193487.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 81.85660222662749 (T) = (0 -168929.9144326456) / Math.Sqrt((8173328.794981199 / (299)) + (513549.0348509702 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1233989562097316 = (192710.14520154166 - 168929.9144326456) / 192710.14520154166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository





