Skip to content

[Perf] Changes at 3/18/2022 1:59:24 AM #4251

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 119.61 μs 112.03 μs 0.94 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 4.56 μs 3.95 μs 0.87 0.07 False
ReadSpanEmptyLoop - Duration of single invocation 897.81 ns 823.19 ns 0.92 0.08 False
ReadReturnBytes - Duration of single invocation 13.58 μs 11.57 μs 0.85 0.09 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 12.80 μs 11.72 μs 0.92 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 1.81 μs 1.67 μs 0.92 0.00 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 17.61 μs 15.56 μs 0.88 0.00 True
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 5.17 μs 4.63 μs 0.89 0.08 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 1.80 μs 1.71 μs 0.95 0.02 False
ReadSpanEmptyLoop - Duration of single invocation 3.00 μs 2.78 μs 0.93 0.03 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 7.73 μs 6.95 μs 0.90 0.13 False
ReadSpanEmptyLoop - Duration of single invocation 11.39 μs 9.25 μs 0.81 0.11 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 12.23 μs 11.42 μs 0.93 0.00 False
ReadReturnBytes - Duration of single invocation 133.72 ns 122.16 ns 0.91 0.16 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 222.28 ns 207.16 ns 0.93 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 112.0282124738884 < 114.39731849195317.
IsChangePoint: Marked as a change because one of 2/7/2022 11:21:03 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 58.38380026287903 (T) = (0 -111654.90319866486) / Math.Sqrt((1584844.9153430301 / (299)) + (103822.89041206148 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07784783490640589 = (121080.77975106463 - 111654.90319866486) / 121080.77975106463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.9525678765257157 < 4.325494912782394.
IsChangePoint: Marked as a change because one of 2/17/2022 1:06:33 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 84.58535086685272 (T) = (0 -3956.5994676881137) / Math.Sqrt((8792.065578459602 / (299)) + (197.02006481465585 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15062436210191266 = (4658.244587140899 - 3956.5994676881137) / 4658.244587140899 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 823.1870583778258 < 861.2195141018266.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 29.486583591294437 (T) = (0 -817.7049450190023) / Math.Sqrt((332.17599862182414 / (299)) + (51.386012773234235 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10848801900393974 = (917.2113919382266 - 817.7049450190023) / 917.2113919382266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.565342537682236 < 12.882832075518374.
IsChangePoint: Marked as a change because one of 2/3/2022 8:57:20 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 91.89144595784903 (T) = (0 -11580.48892900516) / Math.Sqrt((65638.94925669061 / (299)) + (1748.2712903269019 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15917645546041428 = (13772.793357429531 - 11580.48892900516) / 13772.793357429531 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.719595460676874 < 12.122135718291512.
IsChangePoint: Marked as a change because one of 1/26/2022 8:19:47 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 42.80684154741216 (T) = (0 -11734.30032690979) / Math.Sqrt((10194.326324612362 / (299)) + (2681.8305350406995 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08014701087705363 = (12756.712720038135 - 11734.30032690979) / 12756.712720038135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6688428303905831 < 1.7074532301820229.
IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 20.56429446632099 (T) = (0 -1678.560148151829) / Math.Sqrt((809.6136848862623 / (299)) + (206.68348040083222 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07519237865047715 = (1815.03710544945 - 1678.560148151829) / 1815.03710544945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.563716193794589 < 16.894335071630422.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 93.73472756556362 (T) = (0 -15519.58265822232) / Math.Sqrt((29351.40991028442 / (299)) + (2748.1046371883235 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13324014853484734 = (17905.285566686485 - 15519.58265822232) / 17905.285566686485 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.628131612474379 < 4.910118667777208.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/22/2022 8:48:40 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 37.42466031647536 (T) = (0 -4672.853993292345) / Math.Sqrt((29295.10571189089 / (299)) + (786.6206038609563 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11345016239390619 = (5270.830578357804 - 4672.853993292345) / 5270.830578357804 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7103424105999048 < 1.7234759928554606.
IsChangePoint: Marked as a change because one of 2/11/2022 11:08:57 PM, 2/12/2022 1:03:18 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 20.017590503203255 (T) = (0 -1693.639804130693) / Math.Sqrt((1136.5424709352264 / (299)) + (264.9497707869608 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08178527882291776 = (1844.4921052448103 - 1693.639804130693) / 1844.4921052448103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7801914092721547 < 2.8496567275536524.
IsChangePoint: Marked as a change because one of 2/4/2022 3:19:13 PM, 2/5/2022 9:44:06 AM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 49.87680191311344 (T) = (0 -2789.42407095217) / Math.Sqrt((2286.529702118468 / (299)) + (65.37766950432555 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07527019284399239 = (3016.474703601262 - 2789.42407095217) / 3016.474703601262 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.950626352664977 < 7.342531326941735.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 45.30464592356835 (T) = (0 -6943.916806349082) / Math.Sqrt((39377.811977049554 / (299)) + (1662.9881576772805 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12325610829431649 = (7920.1199712265625 - 6943.916806349082) / 7920.1199712265625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.254980111962285 < 10.737538508234731.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 78.62453789637816 (T) = (0 -9241.245140006544) / Math.Sqrt((43826.620711252486 / (299)) + (3270.892610082547 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1940399629583023 = (11466.133201748851 - 9241.245140006544) / 11466.133201748851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.418194174378453 < 11.619008453549194.
IsChangePoint: Marked as a change because one of 1/7/2022 3:54:44 AM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 27.194248262678716 (T) = (0 -11437.365507851728) / Math.Sqrt((14068.662632874115 / (299)) + (6057.516956594175 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07778890100572779 = (12402.112184861879 - 11437.365507851728) / 12402.112184861879 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 122.1573625583415 < 123.5693371215141.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 7.277454238898236 (T) = (0 -119.384351322881) / Math.Sqrt((17.166515905064234 / (299)) + (14.087749006709211 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09367665373086342 = (131.7237957229332 - 119.384351322881) / 131.7237957229332 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 207.16100460682716 < 210.99201762405932.
IsChangePoint: Marked as a change because one of 12/2/2021 4:22:41 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 56.223226384443024 (T) = (0 -206.93052077794232) / Math.Sqrt((12.022219836251129 / (299)) + (0.18964977952517767 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07058798391368518 = (222.64670264250663 - 206.93052077794232) / 222.64670264250663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanTenSegments - Duration of single invocation 8.08 ns 7.07 ns 0.87 0.20 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanTenSegments


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.071643186123248 < 7.684266342894704.
IsChangePoint: Marked as a change because one of 2/20/2022 8:03:17 PM, 2/21/2022 7:08:33 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 13.510364534840807 (T) = (0 -7.186945812945304) / Math.Sqrt((0.10795780262723319 / (299)) + (0.02105629015377879 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11277776102110047 = (8.100502328725135 - 7.186945812945304) / 8.100502328725135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Segment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 122.19 μs 112.97 μs 0.92 0.00 True
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 11.62 μs 10.55 μs 0.91 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Segment*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json40KB)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 112.96679411866938 < 115.9534162569395.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 113.35951289877586 (T) = (0 -113118.62804673414) / Math.Sqrt((450374.14785118634 / (299)) + (30665.93713445671 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08053595197798932 = (123026.70048935534 - 113118.62804673414) / 123026.70048935534 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.55101805338624 < 11.04294321743934.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 73.03225216360774 (T) = (0 -10579.77113337209) / Math.Sqrt((14270.221008575792 / (299)) + (985.9248020888199 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09749707474401986 = (11722.700101355696 - 10579.77113337209) / 11722.700101355696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 1.20 ms 1.11 ms 0.93 0.07 False
Parse - Duration of single invocation 331.72 ns 301.95 ns 0.91 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.110222448377581 < 1.1217352710860289.
IsChangePoint: Marked as a change because one of 2/3/2022 8:57:20 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 37.87891068772597 (T) = (0 -1106316.0302944593) / Math.Sqrt((1104415212.920732 / (299)) + (19536926.778786045 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08625440307692392 = (1210748.4118334907 - 1106316.0302944593) / 1210748.4118334907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 301.95479508512165 < 315.18811986278666.
IsChangePoint: Marked as a change because one of 12/15/2021 12:53:06 PM, 12/17/2021 12:35:23 PM, 3/11/2022 5:51:22 PM, 3/15/2022 12:53:39 AM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 11.88001572011978 (T) = (0 -300.3025707886901) / Math.Sqrt((43.27284021675095 / (299)) + (11.018755077037037 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.05715960651384992 = (318.5083847312927 - 300.3025707886901) / 318.5083847312927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 14.87 μs 14.02 μs 0.94 0.08 False
ParseThenWrite - Duration of single invocation 5.39 μs 4.89 μs 0.91 0.08 False
ParseThenWrite - Duration of single invocation 19.91 μs 18.14 μs 0.91 0.12 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.022364434482114 < 14.138914772501517.
IsChangePoint: Marked as a change because one of 12/3/2021 5:08:27 AM, 12/3/2021 7:52:18 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 19.89273426223492 (T) = (0 -14112.660614457516) / Math.Sqrt((91655.30031095645 / (299)) + (10786.423422534468 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06539134017904942 = (15100.074738405672 - 14112.660614457516) / 15100.074738405672 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.891058609840743 < 5.138688789587792.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 60.4625586973938 (T) = (0 -4870.192110768488) / Math.Sqrt((16311.240602273643 / (299)) + (155.8853534565931 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10309818539469964 = (5430.017011295394 - 4870.192110768488) / 5430.017011295394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.14050380804558 < 19.007016646187036.
IsChangePoint: Marked as a change because one of 2/4/2022 3:19:13 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 37.67189780659914 (T) = (0 -18192.79415968631) / Math.Sqrt((356168.5428672985 / (299)) + (9608.060426766884 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10356505998409432 = (20294.606276015424 - 18192.79415968631) / 20294.606276015424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline ac84ea6e241ad2d2cde346144b2c4b3a5d64fa1d
Compare 0b12d37843e7165fb4c8b794186f19ef43af6c73
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 18.49 μs 16.96 μs 0.92 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfNumbers)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.959427754677755 < 17.588678318997182.
IsChangePoint: Marked as a change because one of 2/5/2022 11:25:50 PM, 3/17/2022 6:54:53 PM, 3/22/2022 12:26:23 PM falls between 3/13/2022 11:50:20 AM and 3/22/2022 12:26:23 PM.
IsImprovementStdDev: Marked as improvement because 11.648584915774936 (T) = (0 -17201.681003187805) / Math.Sqrt((24030.5718133202 / (299)) + (80087.32453165742 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07912046548110668 = (18679.6213385008 - 17201.681003187805) / 18679.6213385008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions