Skip to content

[Perf] Windows/arm64: 23 Regressions on 12/21/2022 4:32:37 PM #11194

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 279fb0436f475fbc35ffeff68330f970ee77831a
Compare 8d66372138fca4c9c73eef07c1632f13a0887d8e
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Join_Enumerable - Duration of single invocation 296.07 ns 334.66 ns 1.13 0.06 False
ToUpperInvariant - Duration of single invocation 19.83 ns 25.41 ns 1.28 0.31 False
ToLowerInvariant - Duration of single invocation 6.32 ns 9.04 ns 1.43 0.34 False
ToLower - Duration of single invocation 17.23 ns 19.51 ns 1.13 0.22 False
ToUpper - Duration of single invocation 29.71 ns 33.00 ns 1.11 0.05 False
ToLower - Duration of single invocation 30.42 ns 33.42 ns 1.10 0.03 False
ToUpperInvariant - Duration of single invocation 21.16 ns 24.29 ns 1.15 0.32 False
ToLowerInvariant - Duration of single invocation 19.80 ns 25.28 ns 1.28 0.24 False
ToLower - Duration of single invocation 30.85 ns 33.68 ns 1.09 0.04 False
ToLowerInvariant - Duration of single invocation 19.48 ns 23.06 ns 1.18 0.31 False
ToUpper - Duration of single invocation 29.30 ns 32.33 ns 1.10 0.04 False
ToUpper - Duration of single invocation 14.76 ns 18.94 ns 1.28 0.24 False
ToUpperInvariant - Duration of single invocation 5.65 ns 9.16 ns 1.62 0.40 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Join_Enumerable


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 334.66333073105886 > 310.1232281009754.
IsChangePoint: Marked as a change because one of 9/30/2022 1:24:23 AM, 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -35.56753947678202 (T) = (0 -338.1024718087037) / Math.Sqrt((12.24625632510734 / (13)) + (10.160627020295303 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.1450514697595924 = (295.27272855227153 - 338.1024718087037) / 295.27272855227153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.414427047072177 > 20.597503598316973.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -22.061815479173696 (T) = (0 -23.994170338302084) / Math.Sqrt((0.12916561174130706 / (13)) + (0.602321354753627 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.22550875253312355 = (19.578946530333784 - 23.994170338302084) / 19.578946530333784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.036497944769593 > 6.627082228036932.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -22.50111154160463 (T) = (0 -9.171730434726742) / Math.Sqrt((0.20891689611225828 / (13)) + (0.024832697954338973 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.476660305783434 = (6.2111308869108734 - 9.171730434726742) / 6.2111308869108734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.51009981849415 > 17.985569481689346.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -19.179701037557003 (T) = (0 -19.597615576673654) / Math.Sqrt((0.14031857224319072 / (13)) + (0.18774109945438738 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.16148029786276744 = (16.872964279062764 - 19.597615576673654) / 16.872964279062764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.000620663713114 > 31.055951448408138.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -19.43446867925207 (T) = (0 -33.52746689619404) / Math.Sqrt((0.12438905590055883 / (13)) + (0.7008816314786561 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.13951471056601336 = (29.422583653650655 - 33.52746689619404) / 29.422583653650655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.417354494004385 > 31.84558462930623.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -13.395060246232482 (T) = (0 -34.13213888290169) / Math.Sqrt((0.2832731455885848 / (13)) + (1.3289671016411684 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.13195973790609833 = (30.153138614310965 - 34.13213888290169) / 30.153138614310965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.290574363224817 > 20.791499677158964.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -19.518210726867927 (T) = (0 -23.98887070096966) / Math.Sqrt((0.3014290855452062 / (13)) + (0.432212644573708 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.2080350723349983 = (19.857760134895607 - 23.98887070096966) / 19.857760134895607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.27565528156059 > 20.77821741123003.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -19.773528258430883 (T) = (0 -23.9518979242687) / Math.Sqrt((0.4383164479841955 / (13)) + (0.3000113879439696 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.2228161388680815 = (19.587489208672157 - 23.9518979242687) / 19.587489208672157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.67716767418449 > 32.01994829408373.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -17.14699364826293 (T) = (0 -33.84112233688593) / Math.Sqrt((0.26412430448800156 / (13)) + (0.44141920804607765 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.11646963280450641 = (30.310830982369858 - 33.84112233688593) / 30.310830982369858 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.063296915172234 > 20.505618850660184.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -33.090668274750065 (T) = (0 -23.911675199947346) / Math.Sqrt((0.08327327970194669 / (13)) + (0.21458909181712643 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.22122514391528453 = (19.580071143381307 - 23.911675199947346) / 19.580071143381307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.332151632345 > 30.804338657715615.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -26.337359233834636 (T) = (0 -33.62495721567597) / Math.Sqrt((0.10092346314378493 / (13)) + (0.3675918892346647 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.14500962102741463 = (29.36652810437031 - 33.62495721567597) / 29.36652810437031 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.942335073449645 > 16.817424168362383.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -14.595238090343788 (T) = (0 -18.545804408767488) / Math.Sqrt((0.2629325554235216 / (13)) + (0.26753598986410165 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.16858133008526954 = (15.8703582979665 - 18.545804408767488) / 15.8703582979665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.155163945008864 > 6.44893053461952.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -25.68438599814117 (T) = (0 -9.30690673955127) / Math.Sqrt((0.1734794542635362 / (13)) + (0.02865013532801322 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.5048307453179597 = (6.184686728728943 - 9.30690673955127) / 6.184686728728943 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 279fb0436f475fbc35ffeff68330f970ee77831a
Compare 8d66372138fca4c9c73eef07c1632f13a0887d8e
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 17.92 μs 34.17 μs 1.91 0.01 False
GetByteCount - Duration of single invocation 23.79 μs 40.12 μs 1.69 0.00 False
GetByteCount - Duration of single invocation 51.81 μs 86.77 μs 1.67 0.00 False
GetBytes - Duration of single invocation 123.69 μs 141.71 μs 1.15 0.00 False
GetBytes - Duration of single invocation 197.98 μs 215.88 μs 1.09 0.00 False
GetBytes - Duration of single invocation 146.39 μs 181.08 μs 1.24 0.00 False
GetByteCount - Duration of single invocation 25.96 μs 43.57 μs 1.68 0.00 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.16599150031192 > 18.81367854071101.
IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -8568.995089522856 (T) = (0 -34164.00743610237) / Math.Sqrt((30.758538662959406 / (13)) + (24.571060388198948 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.9067190563681012 = (17917.69339169328 - 34164.00743610237) / 17917.69339169328 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.1208949704142 > 24.9784349552892.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -6727.035352665285 (T) = (0 -40134.01645034987) / Math.Sqrt((27.045655028749024 / (13)) + (76.45384947871726 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.6870280536832469 = (23789.774190610675 - 40134.01645034987) / 23789.774190610675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.76546221389108 > 54.466236225961545.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -915.7129729032915 (T) = (0 -86769.46212714349) / Math.Sqrt((18520.589505035812 / (13)) + (505.19599847967487 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.6718229719017443 = (51901.10650797007 - 86769.46212714349) / 51901.10650797007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 141.70833333333337 > 129.93362967519684.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -154.90269403697508 (T) = (0 -141518.9849586993) / Math.Sqrt((29511.042669360813 / (13)) + (215669.12084546566 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.14293172819960412 = (123821.03100911037 - 141518.9849586993) / 123821.03100911037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 215.87748842592592 > 208.3405915033674.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -52.80859080276642 (T) = (0 -217844.13584237377) / Math.Sqrt((842773.0640384926 / (13)) + (1498820.1294783743 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.09966069955928718 = (198101.22879691847 - 217844.13584237377) / 198101.22879691847 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 181.0798782771536 > 154.0526226635514.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -301.52435481089407 (T) = (0 -180895.914878383) / Math.Sqrt((49613.73812359402 / (13)) + (178722.17948667676 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.23187820694107752 = (146845.61660326173 - 180895.914878383) / 146845.61660326173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Greek)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.57316968430826 > 27.2476458519678.
IsChangePoint: Marked as a change because one of 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -1860.5546188179449 (T) = (0 -43550.95653844046) / Math.Sqrt((700.9654185582541 / (13)) + (710.035346761315 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.6778156796942642 = (25956.93738323892 - 43550.95653844046) / 25956.93738323892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 279fb0436f475fbc35ffeff68330f970ee77831a
Compare 8d66372138fca4c9c73eef07c1632f13a0887d8e
Diff Diff

Regressions in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetConverter - Duration of single invocation 173.99 ns 194.21 ns 1.12 0.03 False
GetConverter - Duration of single invocation 116.55 ns 129.12 ns 1.11 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'

Payloads

Baseline
Compare

Histogram

System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Guid))


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 194.21294329564145 > 181.62521817889.
IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -39.33973176724806 (T) = (0 -194.72908774562407) / Math.Sqrt((2.2746909683452006 / (13)) + (2.0519011197646444 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.11911380118318617 = (174.00293655546577 - 194.72908774562407) / 174.00293655546577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(string))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.11781944754324 > 122.06454926686476.
IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 9:53:04 AM, 10/6/2022 2:13:23 PM, 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -31.88883289065751 (T) = (0 -127.99747529264737) / Math.Sqrt((0.4536949040804344 / (13)) + (1.9810635166438846 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.10033133067123312 = (116.32630256430606 - 127.99747529264737) / 116.32630256430606 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 279fb0436f475fbc35ffeff68330f970ee77831a
Compare 8d66372138fca4c9c73eef07c1632f13a0887d8e
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 10.60 ns 11.90 ns 1.12 0.11 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.90207453945245 > 11.175408462193328.
IsChangePoint: Marked as a change because one of 9/13/2022 2:36:07 PM, 11/21/2022 10:59:14 PM, 12/21/2022 10:31:08 AM, 12/28/2022 6:20:56 PM falls between 12/19/2022 10:34:07 PM and 12/28/2022 6:20:56 PM.
IsRegressionStdDev: Marked as regression because -13.338569703714693 (T) = (0 -12.233820410584944) / Math.Sqrt((0.04504923734971284 / (13)) + (0.1980317719117278 / (20))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (20) - 2, .025) and -0.14423778044593555 = (10.691676694871187 - 12.233820410584944) / 10.691676694871187 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions