-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 87aa786b3997a23501912abd4654e4fd9958230c |
| Compare | 55bf5d8a28f4a8c519dbc6edfc1d3d64519f530f |
| Diff | Diff |
Improvements in System.IO.Compression.Brotli
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Decompress_WithoutState - Duration of single invocation | 1.54 ms | 703.88 μs | 0.46 | 0.58 | False | |||||
| Decompress_WithState - Duration of single invocation | 1.49 ms | 736.95 μs | 0.49 | 0.64 | False | |||||
| Decompress - Duration of single invocation | 325.70 μs | 287.18 μs | 0.88 | 0.42 | False | |||||
| Decompress_WithoutState - Duration of single invocation | 299.98 μs | 254.18 μs | 0.85 | 0.62 | False | |||||
| Decompress - Duration of single invocation | 1.55 ms | 791.81 μs | 0.51 | 0.45 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Brotli*'Details
Payloads
Histogram
System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "TestDocument.pdf")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 703.87986875 < 1.4473511975694444.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 34.98453228980313 (T) = (0 -673426.5994923287) / Math.Sqrt((10487049303.945114 / (273)) + (4698053899.421693 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.551533934974679 = (1501622.200677115 - 673426.5994923287) / 1501622.200677115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress_WithState(level: Optimal, file: "TestDocument.pdf")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 736.952185 < 1.4067932084374999.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 39.55718479327341 (T) = (0 -673450.3846676587) / Math.Sqrt((13312549212.714314 / (274)) + (3277385870.3824153 / (9))) is greater than 1.968442091833948 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (274) + (9) - 2, .975) and 0.5440708023419356 = (1477094.2245570547 - 673450.3846676587) / 1477094.2245570547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "sum")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 287.1777524038461 < 313.53668856218434.
IsChangePoint: Marked as a change because one of 6/20/2022 12:28:47 PM, 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 10.976743212702594 (T) = (0 -272789.41522804624) / Math.Sqrt((237503586.78508574 / (273)) + (154747078.2595305 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.14604568991886638 = (319442.6352882143 - 272789.41522804624) / 319442.6352882143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "sum")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 254.1807564935065 < 265.0342402421875.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 9.912754842964036 (T) = (0 -247977.04503339052) / Math.Sqrt((471118785.775026 / (273)) + (131504468.62753949 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.13909967931812686 = (288043.85255308205 - 247977.04503339052) / 288043.85255308205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "TestDocument.pdf")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 791.81243125 < 1.5307552211458335.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 63.327411595730766 (T) = (0 -770675.125579754) / Math.Sqrt((6492377875.459285 / (273)) + (1302768432.4690928 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.516149151816183 = (1592794.821942673 - 770675.125579754) / 1592794.821942673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository




