-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-arm64branch-refs/heads/mainkind-microos-linuxperf-improvementruntime-coreclr
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Transient - Duration of single invocation | 19.80 μs | 17.49 μs | 0.88 | 0.07 | False | |||||
| Singleton - Duration of single invocation | 19.62 μs | 17.77 μs | 0.91 | 0.08 | False | |||||
| Scoped - Duration of single invocation | 22.16 μs | 19.23 μs | 0.87 | 0.05 | False | |||||
| Scoped - Duration of single invocation | 20.90 μs | 19.10 μs | 0.91 | 0.07 | False | |||||
| Transient - Duration of single invocation | 19.21 μs | 17.42 μs | 0.91 | 0.04 | False | |||||
| Singleton - Duration of single invocation | 20.60 μs | 17.83 μs | 0.87 | 0.07 | False | |||||
| Scoped - Duration of single invocation | 20.97 μs | 19.05 μs | 0.91 | 0.10 | False | |||||
| Singleton - Duration of single invocation | 19.38 μs | 17.65 μs | 0.91 | 0.08 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'Details
Payloads
Histogram
Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.491877129850675 < 18.715122886106357.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 18.554997205851834 (T) = (0 -17685.501387752956) / Math.Sqrt((734811.4572642763 / (299)) + (82169.41642523912 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.09779621258862439 = (19602.557243188498 - 17685.501387752956) / 19602.557243188498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.77077748650641 < 18.495480918878503.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 7:50:08 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 14.40038647713384 (T) = (0 -17954.38163217237) / Math.Sqrt((715373.4506473293 / (299)) + (141715.5234594861 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09748979694779836 = (19893.826763899622 - 17954.38163217237) / 19893.826763899622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.22800646739409 < 20.409235335744842.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 26.221379897697325 (T) = (0 -19063.41137096281) / Math.Sqrt((830759.1099644577 / (299)) + (43593.40545094593 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1041098552087028 = (21278.73766867225 - 19063.41137096281) / 21278.73766867225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.09922034797833 < 19.85200385237886.
IsChangePoint: Marked as a change because one of 3/18/2022 1:59:24 AM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 17.00499668511749 (T) = (0 -19192.635568195208) / Math.Sqrt((939063.3649704024 / (299)) + (128573.30240406218 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1007730151753979 = (21343.482671328875 - 19192.635568195208) / 21343.482671328875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.424959823198748 < 18.258711692160663.
IsChangePoint: Marked as a change because one of 3/18/2022 1:59:24 AM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 28.8544742053622 (T) = (0 -17507.501095329237) / Math.Sqrt((759687.9363724379 / (299)) + (27017.707445490352 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10661059998247835 = (19596.719073436365 - 17507.501095329237) / 19596.719073436365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.830469975533862 < 18.652257173162123.
IsChangePoint: Marked as a change because one of 3/18/2022 7:38:50 AM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 26.364770814567457 (T) = (0 -17834.389718529397) / Math.Sqrt((786666.3107330895 / (299)) + (34622.214958016244 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10345703273215306 = (19892.398211408065 - 17834.389718529397) / 19892.398211408065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.04537123408993 < 19.75063235578536.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 21.60329004127775 (T) = (0 -19112.158218484336) / Math.Sqrt((878079.1354744541 / (299)) + (77075.2527616029 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10443901272138864 = (21340.99016144223 - 19112.158218484336) / 21340.99016144223 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.654722372507226 < 18.482527028628855.
IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 17.641845193887608 (T) = (0 -17938.819433900364) / Math.Sqrt((767244.109391679 / (299)) + (97492.34173125267 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.09839813828525037 = (19896.60868687944 - 17938.819433900364) / 19896.60868687944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in System.Security.Cryptography.Tests.Perf_CryptoConfig
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CreateFromName - Duration of single invocation | 3.37 μs | 2.67 μs | 0.79 | 0.44 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Tests.Perf_CryptoConfig*'Details
Payloads
Histogram
System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "SHA512")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.665732005349223 < 3.056728571911875.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/19/2022 11:38:14 PM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 12.94629529177374 (T) = (0 -2521.007310263533) / Math.Sqrt((141389.87753420658 / (288)) + (32359.635532626482 / (10))) is greater than 1.9680107275502179 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (10) - 2, .975) and 0.2386785439091181 = (3311.357232998554 - 2521.007310263533) / 3311.357232998554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in Exceptions.Handling
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ThrowAndCatchDeep - Duration of single invocation | 201.79 μs | 167.72 μs | 0.83 | 0.38 | False | |||||
| ThrowAndCatch - Duration of single invocation | 178.92 μs | 139.33 μs | 0.78 | 0.44 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Exceptions.Handling*'Details
Payloads
Histogram
Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionSoftware)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 167.71671798281508 < 194.26622396827202.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/9/2022 9:57:04 AM, 5/15/2022 9:56:58 AM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 7.939071781694362 (T) = (0 -168889.0910310067) / Math.Sqrt((454650671.87515366 / (299)) + (55091537.75133066 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.11082200243939593 = (189938.45045012562 - 168889.0910310067) / 189938.45045012562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 139.3298702591656 < 167.35181348999487.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 8.916560016453184 (T) = (0 -143053.6975545382) / Math.Sqrt((464085725.64172137 / (299)) + (33131540.82877634 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.12086676027682568 = (162721.29307678508 - 143053.6975545382) / 162721.29307678508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-arm64branch-refs/heads/mainkind-microos-linuxperf-improvementruntime-coreclr










