[Perf] Linux/x64: 14 Improvements on 8/16/2023 3:30:19 PM #20687
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2470610ea493b650177a5303684f87ee9fcec966 |
| Compare | 374b1116d14a2912f8b36ffbd9523001cceb8316 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in PerfLabTests.CastingPerf2.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 284.10 μs | 263.39 μs | 0.93 | 0.04 | True | ||||
| 262.74 μs | 197.30 μs | 0.75 | 0.04 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'Details
Payloads
PerfLabTests.CastingPerf2.CastingPerf.FooObjCastIfIsa
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 9.494813040091731 (T) = (0 -263292.68133247213) / Math.Sqrt((37983993.93114604 / (12)) + (157494.67324158226 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.06039134534189695 = (280215.25773224904 - 263292.68133247213) / 280215.25773224904 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 925.0650528613221 (T) = (0 -197284.71624611624) / Math.Sqrt((48148.56048974083 / (12)) + (13930.72248670721 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.24914079980954146 = (262745.28726034675 - 197284.71624611624) / 262745.28726034675 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2470610ea493b650177a5303684f87ee9fcec966 |
| Compare | 374b1116d14a2912f8b36ffbd9523001cceb8316 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Linq.Tests.Perf_Enumerable
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 142.13 ns | 106.48 ns | 0.75 | 0.04 | True | ||||
| 120.91 ns | 99.65 ns | 0.82 | 0.04 | True | ||||
| 159.35 ns | 138.06 ns | 0.87 | 0.04 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'Details
Payloads
System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 75.7009147103962 (T) = (0 -106.89806289493576) / Math.Sqrt((1.739466336191545 / (12)) + (1.1858806376126108 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.2533810790993385 = (143.17620395419723 - 106.89806289493576) / 143.17620395419723 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 44.388262705935716 (T) = (0 -101.71821652646386) / Math.Sqrt((1.0556236641214536 / (12)) + (1.582356725562636 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.16362892930307377 = (121.61852566433787 - 101.71821652646386) / 121.61852566433787 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 54.559133713635724 (T) = (0 -138.4091012823689) / Math.Sqrt((1.3455071778623504 / (12)) + (0.6116076204468842 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.13464691705547968 = (159.94523392856811 - 138.4091012823689) / 159.94523392856811 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2470610ea493b650177a5303684f87ee9fcec966 |
| Compare | 374b1116d14a2912f8b36ffbd9523001cceb8316 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 157.95 ns | 123.41 ns | 0.78 | 0.25 | True | ||||
| 28.54 ns | 24.52 ns | 0.86 | 0.14 | True | ||||
| 22.47 ns | 19.10 ns | 0.85 | 0.21 | True | ||||
| 20.60 ns | 17.98 ns | 0.87 | 0.14 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'Details
Payloads
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 6.014983560580014 (T) = (0 -128.19914603351714) / Math.Sqrt((101.54829333457236 / (12)) + (168.0138870611611 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.17508431353573928 = (155.40878678523146 - 128.19914603351714) / 155.40878678523146 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False))
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 6.276982990039618 (T) = (0 -23.856979244812518) / Math.Sqrt((5.366606212959045 / (12)) + (0.1305410794945401 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.1509428588462258 = (28.098202215687788 - 23.856979244812518) / 28.098202215687788 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 7.901123872415249 (T) = (0 -19.285460762198692) / Math.Sqrt((1.2134373732907595 / (12)) + (0.17561570436383034 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.12136129229541678 = (21.94925012190889 - 19.285460762198692) / 21.94925012190889 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 7.112611568208002 (T) = (0 -18.315414631699998) / Math.Sqrt((0.9421052810494398 / (12)) + (0.1342598700414755 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.10335207337868169 = (20.42653988027918 - 18.315414631699998) / 20.42653988027918 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2470610ea493b650177a5303684f87ee9fcec966 |
| Compare | 374b1116d14a2912f8b36ffbd9523001cceb8316 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.AddGivenSize<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 1.44 μs | 1.36 μs | 0.94 | 0.04 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<String>*'Details
Payloads
System.Collections.AddGivenSize<String>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 46.739509744546524 (T) = (0 -1346.8668945236454) / Math.Sqrt((13.387186643725055 / (12)) + (42.00009385481513 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.06577032768604833 = (1441.6871294482128 - 1346.8668945236454) / 1441.6871294482128 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2470610ea493b650177a5303684f87ee9fcec966 |
| Compare | 374b1116d14a2912f8b36ffbd9523001cceb8316 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.IndexerSetReverse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 1.63 μs | 1.47 μs | 0.90 | 0.11 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse<String>*'Details
Payloads
System.Collections.IndexerSetReverse<String>.IList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 4.392962314145936 (T) = (0 -1468.7321758928558) / Math.Sqrt((13967.605190909264 / (12)) + (112.65845370867737 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.09288462276642113 = (1619.1238873846835 - 1468.7321758928558) / 1619.1238873846835 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2470610ea493b650177a5303684f87ee9fcec966 |
| Compare | 374b1116d14a2912f8b36ffbd9523001cceb8316 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in PerfLabTests.DelegatePerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 182.08 ms | 169.79 ms | 0.93 | 0.32 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'Details
Payloads
PerfLabTests.DelegatePerf.MulticastDelegateCombineInvoke
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 7.913382005031206 (T) = (0 -161895246.05491892) / Math.Sqrt((22674177261221.266 / (12)) + (29983713259305.64 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.08936922519382426 = (177783631.44972527 - 161895246.05491892) / 177783631.44972527 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 22.04 |
| Queue | TigerUbuntu |
| Baseline | 2470610ea493b650177a5303684f87ee9fcec966 |
| Compare | 374b1116d14a2912f8b36ffbd9523001cceb8316 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Tests.Perf_Get
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 2.99 μs | 2.61 μs | 0.87 | 0.36 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'Details
Payloads
System.Text.Json.Tests.Perf_Get.GetDateTime
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 5.8997892803552 (T) = (0 -2725.9438347077967) / Math.Sqrt((3170.354807182759 / (12)) + (31137.387225590537 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.09744189657756022 = (3020.2419371907476 - 2725.9438347077967) / 3020.2419371907476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository