Skip to content

[Perf] Changes at 3/31/2022 1:23:38 AM #4399

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 523.71 μs 437.39 μs 0.84 0.13 False
Count - Duration of single invocation 23.55 ms 20.83 ms 0.88 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: None)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 437.3923842592592 < 496.3691274321503.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 3/4/2022 7:13:54 PM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 25.62424172507734 (T) = (0 -440031.22309217934) / Math.Sqrt((761980341.5855137 / (299)) + (35559703.255763926 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10466686378425742 = (491472.06251299503 - 440031.22309217934) / 491472.06251299503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.82669944444444 < 22.3471722962963.
IsChangePoint: Marked as a change because one of 2/10/2022 10:02:40 PM, 2/19/2022 11:51:48 PM, 3/4/2022 3:52:45 PM, 3/31/2022 1:45:25 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 33.14533635364267 (T) = (0 -20779207.738603987) / Math.Sqrt((1767437200428.412 / (299)) + (8001719803.894442 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.1123185970918928 = (23408407.194889776 - 20779207.738603987) / 23408407.194889776 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 2.57 μs 2.26 μs 0.88 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2581460946234655 < 2.346521443082046.
IsChangePoint: Marked as a change because one of 1/25/2022 6:54:35 PM, 1/29/2022 3:20:10 AM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 22.221540635245404 (T) = (0 -2284.4013953603776) / Math.Sqrt((4382.602351382528 / (299)) + (1140.336129074743 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07123651850642668 = (2459.615866557071 - 2284.4013953603776) / 2459.615866557071 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CoveredIndex2 - Duration of single invocation 869.42 ns 755.26 ns 0.87 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'
Details

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.CoveredIndex2(length: 1024)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 755.2604850873352 < 825.3199452715636.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 2/24/2022 10:17:16 AM, 3/3/2022 10:59:13 PM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 38.13298823782115 (T) = (0 -755.4647147173491) / Math.Sqrt((1755.7189566535042 / (299)) + (1.0888929572131516 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10935856960009929 = (848.2254349858205 - 755.4647147173491) / 848.2254349858205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 221.31 ns 206.65 ns 0.93 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 206.64791265423142 < 210.4283324022768.
IsChangePoint: Marked as a change because one of 2/25/2022 11:02:46 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 33.24176559850509 (T) = (0 -205.83785111847146) / Math.Sqrt((127.19467067054035 / (299)) + (2.703446702518897 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1059126052292898 = (230.2211756058353 - 205.83785111847146) / 230.2211756058353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 2453f16807b85b279efc26d17d6f20de87801c09
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GuidToString - Duration of single invocation 32.32 ns 30.16 ns 0.93 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Guid*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.GuidToString


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.15943631267518 < 30.742586401388166.
IsChangePoint: Marked as a change because one of 2/1/2022 3:55:45 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 11.299721217692284 (T) = (0 -30.203335563737394) / Math.Sqrt((0.4413718906261661 / (299)) + (0.5812426349589209 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05657703983909956 = (32.014628474365544 - 30.203335563737394) / 32.014628474365544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 11.64 μs 10.69 μs 0.92 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToStream


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.685015873015875 < 10.999141549688089.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 9.024737383421206 (T) = (0 -10387.950480581656) / Math.Sqrt((204017.16771602997 / (299)) + (85260.91028554642 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05351067660483074 = (10975.243168426716 - 10387.950480581656) / 10975.243168426716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TypeReflectionObjectGetType - Duration of single invocation 167.06 μs 139.29 μs 0.83 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'
Details

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.TypeReflectionObjectGetType


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 139.2871202064897 < 158.71134378957697.
IsChangePoint: Marked as a change because one of 2/2/2022 4:36:37 PM, 2/15/2022 7:45:46 PM, 2/25/2022 1:44:54 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 25.694861699608055 (T) = (0 -139244.24929963946) / Math.Sqrt((166806412.783788 / (299)) + (1175.6913799162671 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12113779117538351 = (158436.95166488457 - 139244.24929963946) / 158436.95166488457 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 1.51 μs 1.37 μs 0.91 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.ICollection(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.368974950655985 < 1.4356157973786325.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/24/2022 10:17:16 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 24.5204387114148 (T) = (0 -1370.8379456083385) / Math.Sqrt((6719.030708360176 / (299)) + (2.342595258835878 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0783214098782792 = (1487.3275351088494 - 1370.8379456083385) / 1487.3275351088494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 127.34 ns 118.54 ns 0.93 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 118.53915613456581 < 120.91215556400887.
IsChangePoint: Marked as a change because one of 2/1/2022 2:18:14 AM, 2/24/2022 3:52:46 AM, 3/15/2022 12:53:39 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 25.330440201969175 (T) = (0 -119.64353331504473) / Math.Sqrt((16.870961879603588 / (299)) + (2.857609166198442 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08146610415007816 = (130.2548919050377 - 119.64353331504473) / 130.2548919050377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in Microsoft.Extensions.Logging.ScopesOverheadBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FilteredByLevel_InsideScope - Duration of single invocation 23.11 ns 20.10 ns 0.87 0.17 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.ScopesOverheadBenchmark*'
Details

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel_InsideScope(HasISupportLoggingScopeLogger: True, CaptureScopes: False)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.09651187249087 < 21.33115566068042.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/25/2022 11:02:46 PM, 3/3/2022 10:59:13 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 29.39292251671711 (T) = (0 -20.039454920180336) / Math.Sqrt((1.911513764190251 / (299)) + (0.07253345926741997 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12459027954905731 = (22.891515198000743 - 20.039454920180336) / 22.891515198000743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CastToBaseClass - Duration of single invocation 2.43 μs 2.14 μs 0.88 0.20 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.CastToBaseClass(input: IEnumerable)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1410495077685505 < 2.279781917176136.
IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/14/2022 2:55:27 AM, 3/22/2022 2:14:22 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsImprovementStdDev: Marked as improvement because 15.665289789581038 (T) = (0 -2092.6323923244336) / Math.Sqrt((29923.26907172203 / (299)) + (1486.746638426955 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08699795765152181 = (2292.034732958143 - 2092.6323923244336) / 2292.034732958143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions