-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-arm64branch-refs/heads/mainkind-microos-windowsperf-improvementruntime-coreclr
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'Details
Payloads
Histogram
Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/9/2022 4:16:21 PM, 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 11.063584442368505 (T) = (0 -3812.104499994258) / Math.Sqrt((47958.03026668814 / (299)) + (36968.7103297134 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15272056003665102 = (4499.229321744381 - 3812.104499994258) / 4499.229321744381 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 12.225197532115534 (T) = (0 -3782.687901364997) / Math.Sqrt((41790.15030257124 / (299)) + (34862.457846128156 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.16290824589868205 = (4518.845016489264 - 3782.687901364997) / 4518.845016489264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/4/2022 11:38:38 PM, 2/6/2022 2:48:36 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 16.348770677499797 (T) = (0 -1587.1678858585874) / Math.Sqrt((4623.432495235621 / (299)) + (791.0218607182782 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11952005889865731 = (1802.6167454459992 - 1587.1678858585874) / 1802.6167454459992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 13.270400493380892 (T) = (0 -3742.9317377910193) / Math.Sqrt((31298.036805555974 / (299)) + (29132.008699012145 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.16301907383465972 = (4471.943888780599 - 3742.9317377910193) / 4471.943888780599 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 9.003769843408284 (T) = (0 -1602.3821433324879) / Math.Sqrt((4507.069956460819 / (299)) + (2386.2281039719105 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11085479169264348 = (1802.160241500826 - 1602.3821433324879) / 1802.160241500826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 11.046481249067197 (T) = (0 -3837.9297196441257) / Math.Sqrt((41863.62869606312 / (299)) + (32758.6993424279 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.14399706214121244 = (4483.547368709216 - 3837.9297196441257) / 4483.547368709216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 10.476558856753423 (T) = (0 -3836.7985918177537) / Math.Sqrt((43878.56993745887 / (299)) + (42061.64658461563 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15265173042948724 = (4528.0066409558785 - 3836.7985918177537) / 4528.0066409558785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/13/2022 1:54:51 AM, 3/14/2022 12:52:16 AM, 5/24/2022 10:45:34 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 9.579761448739117 (T) = (0 -1614.0861691663365) / Math.Sqrt((4693.39451099283 / (299)) + (1531.8926141385248 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.10594905583495753 = (1805.3626358772403 - 1614.0861691663365) / 1805.3626358772403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 8.892012384362452 (T) = (0 -3897.9042956433027) / Math.Sqrt((41512.52751867679 / (299)) + (47244.353089364726 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.13725159102794893 = (4518.0080949527155 - 3897.9042956433027) / 4518.0080949527155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 9.884041360645115 (T) = (0 -4229.575776221897) / Math.Sqrt((43325.66902366773 / (299)) + (59022.63279508133 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15377912768698832 = (4998.1936331362485 - 4229.575776221897) / 4998.1936331362485 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 14.15912959460845 (T) = (0 -4182.653335070775) / Math.Sqrt((54037.04024938592 / (299)) + (33140.4979286211 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.16675122145157498 = (5019.693328992616 - 4182.653335070775) / 5019.693328992616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 11.803763947882807 (T) = (0 -3774.8779091799574) / Math.Sqrt((39771.50541999204 / (299)) + (34116.57522160727 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1569492938597617 = (4477.6404096291935 - 3774.8779091799574) / 4477.6404096291935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 14.291087491161006 (T) = (0 -4163.321988772691) / Math.Sqrt((40106.345461177836 / (299)) + (33112.914458187115 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.16769794220262457 = (5002.176733517406 - 4163.321988772691) / 5002.176733517406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 10.844139537652955 (T) = (0 -4254.139300842291) / Math.Sqrt((43225.018992941055 / (299)) + (47234.225824432026 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15099671662015335 = (5010.745404784231 - 4254.139300842291) / 5010.745404784231 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 12.772341230812684 (T) = (0 -3796.408653049059) / Math.Sqrt((44089.65207379211 / (299)) + (30692.11277611683 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.16023533574531593 = (4520.800665527507 - 3796.408653049059) / 4520.800665527507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in System.Security.Cryptography.Tests.Perf_CryptoConfig
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CreateFromName - Duration of single invocation | 633.88 ns | 529.05 ns | 0.83 | 0.22 | False | |||||
| CreateFromName - Duration of single invocation | 1.12 μs | 933.00 ns | 0.83 | 0.13 | False | |||||
| CreateFromName - Duration of single invocation | 595.93 ns | 424.30 ns | 0.71 | 0.19 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Tests.Perf_CryptoConfig*'Details
Payloads
Histogram
System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "RSA")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 32.832339384605184 (T) = (0 -535.193800494278) / Math.Sqrt((2275.7490711895416 / (299)) + (182.18717843454263 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.2376787273267962 = (702.0580687949763 - 535.193800494278) / 702.0580687949763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "SHA512")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 29.745461979424782 (T) = (0 -938.6516739949329) / Math.Sqrt((1579.4835665848825 / (299)) + (272.51215197045684 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15308220416941526 = (1108.3149729713537 - 938.6516739949329) / 1108.3149729713537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "X509Chain")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/29/2022 1:50:33 AM, 4/8/2022 8:30:12 AM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 35.87145869775903 (T) = (0 -430.25227262468854) / Math.Sqrt((1874.4941767513344 / (299)) + (128.41504946651654 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.2671158576996061 = (587.0672426806816 - 430.25227262468854) / 587.0672426806816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| XmlSerializer_ - Duration of single invocation | 1.48 μs | 1.21 μs | 0.82 | 0.30 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>*'Details
Payloads
Histogram
MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>.XmlSerializer_
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 18.108284870962777 (T) = (0 -1217.1783420651223) / Math.Sqrt((3296.0105833458247 / (299)) + (1656.4376680982427 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.16509634028243414 = (1457.866818402585 - 1217.1783420651223) / 1457.866818402585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| XmlSerializer_ - Duration of single invocation | 1.47 μs | 1.27 μs | 0.86 | 0.15 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>*'Details
Payloads
Histogram
MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.XmlSerializer_
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/11/2022 11:18:20 AM, 4/8/2022 1:52:24 AM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 24.876378365706554 (T) = (0 -1305.2437584714685) / Math.Sqrt((3981.7695140325777 / (299)) + (787.3470647165292 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15458919579988023 = (1543.916581130539 - 1305.2437584714685) / 1543.916581130539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 4aeec6397c7dd6198129219ed806f93d6ed675c0 |
| Compare | 110b4cfaa627deed4b06b864b06128267d0a64c2 |
| Diff | Diff |
Improvements in MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| XmlSerializer_ - Duration of single invocation | 2.15 μs | 1.90 μs | 0.88 | 0.18 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>*'Details
Payloads
Histogram
MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.XmlSerializer_
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/2/2022 3:48:06 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 18.000728854306423 (T) = (0 -1966.282938860606) / Math.Sqrt((8607.470310154671 / (299)) + (2156.1411552305535 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.12520014730244755 = (2247.6946387191674 - 1966.282938860606) / 2247.6946387191674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
PGOApplied if there were any profile guided optimization updates in the observed interval.Applied if there were any profile guided optimization updates in the observed interval.arch-arm64branch-refs/heads/mainkind-microos-windowsperf-improvementruntime-coreclr


















_1.png)
_1.png)
_1.png)