Skip to content

[Perf] Changes at 5/23/2022 7:50:08 PM #5550

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 4aeec6397c7dd6198129219ed806f93d6ed675c0
Compare 110b4cfaa627deed4b06b864b06128267d0a64c2
Diff Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 294.02 ns 271.75 ns 0.92 0.27 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 1.83 μs 1.65 μs 0.90 0.42 False
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 1.82 μs 1.66 μs 0.91 0.44 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 271.7535724566645 < 287.41069050748797.
IsChangePoint: Marked as a change because one of 3/18/2022 3:36:38 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 23.80653825049907 (T) = (0 -266.72890668805377) / Math.Sqrt((229.86367115440143 / (299)) + (36.3775488577703 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.15779472228216063 = (316.70296273946514 - 266.72890668805377) / 316.70296273946514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6466836258051478 < 1.7423819840792572.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 14.316911607890582 (T) = (0 -1692.3610800767212) / Math.Sqrt((109559.7957773481 / (299)) + (8973.169599105144 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.19453685615223726 = (2101.1030647438133 - 1692.3610800767212) / 2101.1030647438133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.657099897452371 < 1.7248047122007892.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 21.15028129050498 (T) = (0 -1677.7249475064168) / Math.Sqrt((113194.47594703273 / (278)) + (398.7973200356347 / (20))) is greater than 1.9680107275502179 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (278) + (20) - 2, .975) and 0.206686554560048 = (2114.832361849095 - 1677.7249475064168) / 2114.832361849095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 4aeec6397c7dd6198129219ed806f93d6ed675c0
Compare 110b4cfaa627deed4b06b864b06128267d0a64c2
Diff Diff

Improvements in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ThrowAndCatch - Duration of single invocation 178.92 μs 139.33 μs 0.78 0.44 False
ThrowAndCatchDeep - Duration of single invocation 201.79 μs 167.72 μs 0.83 0.38 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Exceptions.Handling*'
Details

Payloads

Baseline
Compare

Histogram

Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 139.3298702591656 < 167.35181348999487.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 8.916560016453184 (T) = (0 -143053.6975545382) / Math.Sqrt((464085725.64172137 / (299)) + (33131540.82877634 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.12086676027682568 = (162721.29307678508 - 143053.6975545382) / 162721.29307678508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionSoftware)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 167.71671798281508 < 194.26622396827202.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/9/2022 9:57:04 AM, 5/15/2022 9:56:58 AM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 7.939071781694362 (T) = (0 -168889.0910310067) / Math.Sqrt((454650671.87515366 / (299)) + (55091537.75133066 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.11082200243939593 = (189938.45045012562 - 168889.0910310067) / 189938.45045012562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 4aeec6397c7dd6198129219ed806f93d6ed675c0
Compare 110b4cfaa627deed4b06b864b06128267d0a64c2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Scoped - Duration of single invocation 22.16 μs 19.23 μs 0.87 0.05 False
Singleton - Duration of single invocation 20.60 μs 17.83 μs 0.87 0.07 False
Transient - Duration of single invocation 19.21 μs 17.42 μs 0.91 0.04 False
Singleton - Duration of single invocation 19.62 μs 17.77 μs 0.91 0.08 False
Scoped - Duration of single invocation 20.97 μs 19.05 μs 0.91 0.10 False
Transient - Duration of single invocation 19.80 μs 17.49 μs 0.88 0.07 False
Singleton - Duration of single invocation 19.38 μs 17.65 μs 0.91 0.08 False
Scoped - Duration of single invocation 20.90 μs 19.10 μs 0.91 0.07 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
Details

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.22800646739409 < 20.409235335744842.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 26.221379897697325 (T) = (0 -19063.41137096281) / Math.Sqrt((830759.1099644577 / (299)) + (43593.40545094593 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1041098552087028 = (21278.73766867225 - 19063.41137096281) / 21278.73766867225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.830469975533862 < 18.652257173162123.
IsChangePoint: Marked as a change because one of 3/18/2022 7:38:50 AM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 26.364770814567457 (T) = (0 -17834.389718529397) / Math.Sqrt((786666.3107330895 / (299)) + (34622.214958016244 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10345703273215306 = (19892.398211408065 - 17834.389718529397) / 19892.398211408065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.424959823198748 < 18.258711692160663.
IsChangePoint: Marked as a change because one of 3/18/2022 1:59:24 AM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 28.8544742053622 (T) = (0 -17507.501095329237) / Math.Sqrt((759687.9363724379 / (299)) + (27017.707445490352 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10661059998247835 = (19596.719073436365 - 17507.501095329237) / 19596.719073436365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.77077748650641 < 18.495480918878503.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 7:50:08 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 14.40038647713384 (T) = (0 -17954.38163217237) / Math.Sqrt((715373.4506473293 / (299)) + (141715.5234594861 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09748979694779836 = (19893.826763899622 - 17954.38163217237) / 19893.826763899622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.04537123408993 < 19.75063235578536.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 21.60329004127775 (T) = (0 -19112.158218484336) / Math.Sqrt((878079.1354744541 / (299)) + (77075.2527616029 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.10443901272138864 = (21340.99016144223 - 19112.158218484336) / 21340.99016144223 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.491877129850675 < 18.715122886106357.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 18.554997205851834 (T) = (0 -17685.501387752956) / Math.Sqrt((734811.4572642763 / (299)) + (82169.41642523912 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.09779621258862439 = (19602.557243188498 - 17685.501387752956) / 19602.557243188498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.654722372507226 < 18.482527028628855.
IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 17.641845193887608 (T) = (0 -17938.819433900364) / Math.Sqrt((767244.109391679 / (299)) + (97492.34173125267 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.09839813828525037 = (19896.60868687944 - 17938.819433900364) / 19896.60868687944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.09922034797833 < 19.85200385237886.
IsChangePoint: Marked as a change because one of 3/18/2022 1:59:24 AM, 5/9/2022 9:57:04 AM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 17.00499668511749 (T) = (0 -19192.635568195208) / Math.Sqrt((939063.3649704024 / (299)) + (128573.30240406218 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1007730151753979 = (21343.482671328875 - 19192.635568195208) / 21343.482671328875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 4aeec6397c7dd6198129219ed806f93d6ed675c0
Compare 110b4cfaa627deed4b06b864b06128267d0a64c2
Diff Diff

Improvements in System.Security.Cryptography.Tests.Perf_CryptoConfig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromName - Duration of single invocation 3.37 μs 2.67 μs 0.79 0.44 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Tests.Perf_CryptoConfig*'
Details

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "SHA512")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.665732005349223 < 3.056728571911875.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/19/2022 11:38:14 PM, 5/15/2022 3:27:46 PM, 5/23/2022 1:47:22 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsImprovementStdDev: Marked as improvement because 12.94629529177374 (T) = (0 -2521.007310263533) / Math.Sqrt((141389.87753420658 / (288)) + (32359.635532626482 / (10))) is greater than 1.9680107275502179 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (10) - 2, .975) and 0.2386785439091181 = (3311.357232998554 - 2521.007310263533) / 3311.357232998554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions