Skip to content

[Perf] Changes at 7/15/2022 10:58:28 PM #6980

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline 87aa786b3997a23501912abd4654e4fd9958230c
Compare 55bf5d8a28f4a8c519dbc6edfc1d3d64519f530f
Diff Diff

Improvements in System.IO.Compression.Brotli

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decompress_WithoutState - Duration of single invocation 1.54 ms 703.88 μs 0.46 0.58 False
Decompress_WithState - Duration of single invocation 1.49 ms 736.95 μs 0.49 0.64 False
Decompress - Duration of single invocation 325.70 μs 287.18 μs 0.88 0.42 False
Decompress_WithoutState - Duration of single invocation 299.98 μs 254.18 μs 0.85 0.62 False
Decompress - Duration of single invocation 1.55 ms 791.81 μs 0.51 0.45 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Brotli*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "TestDocument.pdf")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 703.87986875 < 1.4473511975694444.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 34.98453228980313 (T) = (0 -673426.5994923287) / Math.Sqrt((10487049303.945114 / (273)) + (4698053899.421693 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.551533934974679 = (1501622.200677115 - 673426.5994923287) / 1501622.200677115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress_WithState(level: Optimal, file: "TestDocument.pdf")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 736.952185 < 1.4067932084374999.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 39.55718479327341 (T) = (0 -673450.3846676587) / Math.Sqrt((13312549212.714314 / (274)) + (3277385870.3824153 / (9))) is greater than 1.968442091833948 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (274) + (9) - 2, .975) and 0.5440708023419356 = (1477094.2245570547 - 673450.3846676587) / 1477094.2245570547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "sum")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 287.1777524038461 < 313.53668856218434.
IsChangePoint: Marked as a change because one of 6/20/2022 12:28:47 PM, 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 10.976743212702594 (T) = (0 -272789.41522804624) / Math.Sqrt((237503586.78508574 / (273)) + (154747078.2595305 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.14604568991886638 = (319442.6352882143 - 272789.41522804624) / 319442.6352882143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "sum")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 254.1807564935065 < 265.0342402421875.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 9.912754842964036 (T) = (0 -247977.04503339052) / Math.Sqrt((471118785.775026 / (273)) + (131504468.62753949 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.13909967931812686 = (288043.85255308205 - 247977.04503339052) / 288043.85255308205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "TestDocument.pdf")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 791.81243125 < 1.5307552211458335.
IsChangePoint: Marked as a change because one of 7/15/2022 6:44:31 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 63.327411595730766 (T) = (0 -770675.125579754) / Math.Sqrt((6492377875.459285 / (273)) + (1302768432.4690928 / (9))) is greater than 1.9684724997369398 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (9) - 2, .975) and 0.516149151816183 = (1592794.821942673 - 770675.125579754) / 1592794.821942673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions