Skip to content

[Perf] Windows/x64: 7 Regressions on 10/26/2022 10:56:11 AM #9430

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanSingleSegment - Duration of single invocation 8.73 ns 9.97 ns 1.14 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanSingleSegment


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 9.973660381699359 > 9.164718786176035.
IsChangePoint: Marked as a change because one of 9/20/2022 10:35:57 PM, 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -16.07581830331667 (T) = (0 -10.169804069534665) / Math.Sqrt((0.10813553542001815 / (25)) + (0.015792721577036383 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.12456651094116591 = (9.043310440592268 - 10.169804069534665) / 9.043310440592268 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Unescape - Duration of single invocation 67.97 ns 80.77 ns 1.19 0.25 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Uri*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Uri.Unescape


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 80.76766083907398 > 75.83828757703182.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 10/10/2022 9:53:50 PM, 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -18.973335157463776 (T) = (0 -81.54424517074615) / Math.Sqrt((4.969338225427989 / (25)) + (0.9056978909890566 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.12631389778706556 = (72.39921777664368 - 81.54424517074615) / 72.39921777664368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 19.22 μs 21.28 μs 1.11 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 21.27680385524055 > 20.179183324425765.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 10/19/2022 7:27:15 PM, 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -49.690585370148426 (T) = (0 -21343.618782351867) / Math.Sqrt((27013.434996932338 / (25)) + (20743.650894956918 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.11124009916900655 = (19207.027174696792 - 21343.618782351867) / 19207.027174696792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAny - Duration of single invocation 6.85 ns 10.63 ns 1.55 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_String.IndexOfAny


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.630860304944065 > 7.135247195809612.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -112.8367098261711 (T) = (0 -10.512279802540613) / Math.Sqrt((0.011527383183701279 / (25)) + (0.01750742688703658 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.5565094570698715 = (6.753752606379903 - 10.512279802540613) / 6.753752606379903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 64.40 ns 68.14 ns 1.06 0.13 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Base64*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 68.13883521871462 > 67.86109970154617.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 8/29/2022 8:26:03 AM, 9/1/2022 8:04:48 PM, 10/22/2022 11:54:51 PM, 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -4.181519877915932 (T) = (0 -73.38975640255418) / Math.Sqrt((33.379937325999826 / (25)) + (7.163957344240375 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.077682146002548 = (68.09963092993532 - 73.38975640255418) / 68.09963092993532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 52.32 ns 55.34 ns 1.06 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 55.33758790827824 > 54.42609381581579.
IsChangePoint: Marked as a change because one of 8/29/2022 8:26:03 AM, 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -20.53286426822604 (T) = (0 -62.73256804228831) / Math.Sqrt((0.11351816220476034 / (25)) + (7.256086044065866 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.20642621995312818 = (51.998677585708954 - 62.73256804228831) / 51.998677585708954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Divide - Duration of single invocation 8.98 ns 10.32 ns 1.15 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.317619199287382 > 9.411518428431545.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -83.09435162863014 (T) = (0 -10.27436219317706) / Math.Sqrt((0.0016801480377506407 / (25)) + (0.00491422258311315 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.1463606246797247 = (8.962591676635402 - 10.27436219317706) / 8.962591676635402 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions