-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-x64branch-refs/heads/mainkind-microos-linuxperf-improvementruntime-coreclr
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 5a0564b01442f8ea9247e27c4fab85ee0d457265 |
| Compare | 43ab6b874e08f55fe88081f75ddde8e4a2fe726e |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsMatch - Duration of single invocation | 834.55 ns | 768.14 ns | 0.92 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'Details
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: NonBacktracking)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 768.1417822085888 < 795.6179367555519.
IsChangePoint: Marked as a change because one of 2/7/2022 4:59:24 AM, 2/25/2022 11:02:46 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 34.982134189923464 (T) = (0 -766.5502892981608) / Math.Sqrt((2009.191871316471 / (299)) + (7.556719634583893 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1080638244156681 = (859.4228043233841 - 766.5502892981608) / 859.4228043233841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 5a0564b01442f8ea9247e27c4fab85ee0d457265 |
| Compare | 43ab6b874e08f55fe88081f75ddde8e4a2fe726e |
| Diff | Diff |
Improvements in SeekUnroll
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Test - Duration of single invocation | 1.30 secs | 869.74 ms | 0.67 | 0.22 | False | |||||
| Test - Duration of single invocation | 1.74 secs | 1.14 secs | 0.66 | 0.18 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'SeekUnroll*'Details
Payloads
Histogram
SeekUnroll.Test(boxedIndex: 1)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 869.7364220833334 < 1.2390576240535716.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 42.561718860726124 (T) = (0 -885476376.7591575) / Math.Sqrt((11027978543954628 / (299)) + (610638799716690.5 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.27684149968317584 = (1224456846.4191737 - 885476376.7591575) / 1224456846.4191737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### SeekUnroll.Test(boxedIndex: 11)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1420968009285715 < 1.652185020576923.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 41.19062317982063 (T) = (0 -1167144120.5803792) / Math.Sqrt((11453671757602684 / (299)) + (2277152473179574 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.2925622504629506 = (1649818830.4824896 - 1167144120.5803792) / 1649818830.4824896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 5a0564b01442f8ea9247e27c4fab85ee0d457265 |
| Compare | 43ab6b874e08f55fe88081f75ddde8e4a2fe726e |
| Diff | Diff |
Improvements in System.MathBenchmarks.Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ScaleB - Duration of single invocation | 11.93 μs | 10.06 μs | 0.84 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'Details
Payloads
Histogram
System.MathBenchmarks.Single.ScaleB
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.059887710918115 < 11.340290403902397.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 1161.0443150650563 (T) = (0 -10068.378824008221) / Math.Sqrt((327.1211478025036 / (299)) + (33.420035136829505 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15543383307409406 = (11921.361781109003 - 10068.378824008221) / 11921.361781109003 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 5a0564b01442f8ea9247e27c4fab85ee0d457265 |
| Compare | 43ab6b874e08f55fe88081f75ddde8e4a2fe726e |
| Diff | Diff |
Improvements in System.Linq.Tests.Perf_Enumerable
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Average - Duration of single invocation | 390.26 ns | 352.86 ns | 0.90 | 0.02 | False | |||||
| Select - Duration of single invocation | 541.64 ns | 506.12 ns | 0.93 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'Details
Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 352.8629388550097 < 370.03977011862366.
IsChangePoint: Marked as a change because one of 2/25/2022 7:19:51 AM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 41.50561970934662 (T) = (0 -353.0525733867992) / Math.Sqrt((80.32766383205193 / (290)) + (3.132766968716355 / (23))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (23) - 2, .975) and 0.07026009470317812 = (379.7326234739664 - 353.0525733867992) / 379.7326234739664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Select(input: Array)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 506.1216150562042 < 513.0298949538152.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/21/2022 1:28:14 PM, 3/31/2022 1:45:25 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 36.66032917165368 (T) = (0 -511.05334884772293) / Math.Sqrt((212.89425613698708 / (299)) + (34.768093598523734 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.10095593615250338 = (568.4408244248327 - 511.05334884772293) / 568.4408244248327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 5a0564b01442f8ea9247e27c4fab85ee0d457265 |
| Compare | 43ab6b874e08f55fe88081f75ddde8e4a2fe726e |
| Diff | Diff |
Improvements in System.Xml.Linq.Perf_XDocument
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CreateWithRootlEement - Duration of single invocation | 48.68 ns | 44.15 ns | 0.91 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XDocument*'Details
Payloads
Histogram
System.Xml.Linq.Perf_XDocument.CreateWithRootlEement
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.14553855360638 < 46.00081470005472.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsImprovementStdDev: Marked as improvement because 29.278409023099574 (T) = (0 -45.470117488401954) / Math.Sqrt((2.9308401442501433 / (299)) + (0.5346002698829703 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10478643728430827 = (50.79248056794988 - 45.470117488401954) / 50.79248056794988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-x64branch-refs/heads/mainkind-microos-linuxperf-improvementruntime-coreclr






