Skip to content

[Perf] Linux/arm64: 86 Improvements on 12/9/2022 1:45:57 AM #10694

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 83b7b5378d10906176a9ec5638f98d7db1e27c3b
Compare 9b5b9775d4b53cee164d5df693ca236cb26492b4
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetTempPath - Duration of single invocation 1.86 μs 1.66 μs 0.89 0.04 False
GetFileNameWithoutExtension - Duration of single invocation 647.54 ns 540.79 ns 0.84 0.16 False
ChangeExtension - Duration of single invocation 571.69 ns 452.73 ns 0.79 0.05 False
GetRandomFileName - Duration of single invocation 1.23 μs 1.13 μs 0.92 0.08 False
GetFileName - Duration of single invocation 559.74 ns 425.26 ns 0.76 0.16 False
GetDirectoryName - Duration of single invocation 1.14 μs 1.00 μs 0.88 0.16 False
GetExtension - Duration of single invocation 398.98 ns 272.64 ns 0.68 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetTempPath


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 18.547930538758205 (T) = (0 -1634.245119075875) / Math.Sqrt((1329.5785504894004 / (21)) + (394.63899720243495 / (3))) is greater than 2.0738730678945014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (3) - 2, .975) and 0.1367631609060157 = (1893.1596116670685 - 1634.245119075875) / 1893.1596116670685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 18.4921070920953 (T) = (0 -549.2587024976693) / Math.Sqrt((594.4499089197083 / (32)) + (67.76304750247705 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.1776368814468287 = (667.9028887676899 - 549.2587024976693) / 667.9028887676899 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.ChangeExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 16.547093772221498 (T) = (0 -464.3774588901456) / Math.Sqrt((124.60821648846847 / (32)) + (176.8570232074095 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.22026189764045004 = (595.5556839981299 - 464.3774588901456) / 595.5556839981299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetRandomFileName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 12.599789712128068 (T) = (0 -1131.1430830483926) / Math.Sqrt((812.0825079495704 / (31)) + (139.02214182426923 / (3))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .975) and 0.08664890405593831 = (1238.4537425656843 - 1131.1430830483926) / 1238.4537425656843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFileName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 11.113961798475602 (T) = (0 -439.5243705866639) / Math.Sqrt((329.70395029145936 / (32)) + (373.8061578368274 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.22702185245249012 = (568.611637963089 - 439.5243705866639) / 568.611637963089 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetDirectoryName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 9.410898963881742 (T) = (0 -1020.4846066219774) / Math.Sqrt((855.009731524351 / (32)) + (952.648166748076 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.1461086346622041 = (1195.0988709415956 - 1020.4846066219774) / 1195.0988709415956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 25.841900618008985 (T) = (0 -280.53143766820114) / Math.Sqrt((41.63520953620577 / (32)) + (55.55373138250246 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.2908276508215903 = (395.57582581047103 - 280.53143766820114) / 395.57582581047103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 83b7b5378d10906176a9ec5638f98d7db1e27c3b
Compare 9b5b9775d4b53cee164d5df693ca236cb26492b4
Diff Diff

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 1.95 ms 1.61 ms 0.83 0.05 False
ReadLine - Duration of single invocation 3.24 ms 2.20 ms 0.68 0.08 False
ReadLine - Duration of single invocation 426.71 μs 330.29 μs 0.77 0.08 False
ReadLine - Duration of single invocation 1.37 ms 964.09 μs 0.70 0.05 False
ReadLineAsync - Duration of single invocation 591.77 μs 493.73 μs 0.83 0.03 False
ReadLineAsync - Duration of single invocation 4.62 ms 3.78 ms 0.82 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 8])


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 29.7176471067134 (T) = (0 -1622841.1501785715) / Math.Sqrt((2123474056.7377908 / (32)) + (198706102.31178403 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.17414269082134487 = (1965038.187762176 - 1622841.1501785715) / 1965038.187762176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 26.706856842348316 (T) = (0 -2241348.872867063) / Math.Sqrt((9711298406.32178 / (31)) + (3317646662.4042487 / (3))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .975) and 0.30981033749109627 = (3247439.0658352533 - 2241348.872867063) / 3247439.0658352533 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 18.76309023491619 (T) = (0 -340771.8701278181) / Math.Sqrt((250061092.67564538 / (32)) + (102089901.8986174 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.2626301384097371 = (462145.10231389967 - 340771.8701278181) / 462145.10231389967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 29.61733901218243 (T) = (0 -986322.7996629901) / Math.Sqrt((1372674909.2051854 / (32)) + (405275461.2240482 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.2860255775972484 = (1381453.9690983598 - 986322.7996629901) / 1381453.9690983598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 15.45509967076963 (T) = (0 -494099.32772028324) / Math.Sqrt((177630603.80413917 / (32)) + (127689573.17027865 / (3))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (3) - 2, .975) and 0.17828511941820716 = (601302.6408508626 - 494099.32772028324) / 601302.6408508626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 25.663582270167893 (T) = (0 -3838575.906329365) / Math.Sqrt((6471545960.531114 / (30)) + (2551817655.5180416 / (3))) is greater than 2.0395134463927453 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (3) - 2, .975) and 0.17919698444245513 = (4676610.384675481 - 3838575.906329365) / 4676610.384675481 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions