Skip to content

[Perf] Changes at 5/9/2022 3:38:36 PM #5284

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 3e5517beb897faf4592d23f036446561da1e5c23
Compare 0533792282ba5546e10cb5c3d10748aff46ddc1a
Diff Diff

Improvements in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ThrowAndCatch - Duration of single invocation 178.09 μs 137.24 μs 0.77 0.46 False
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation 200.19 μs 135.89 μs 0.68 0.36 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Exceptions.Handling*'
Details

Payloads

Baseline
Compare

Histogram

Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 137.2443082315163 < 178.38886739410003.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 5.493051663467603 (T) = (0 -144709.21474880382) / Math.Sqrt((459639543.50362307 / (299)) + (60558153.23931924 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.09151647731392606 = (159286.55956351123 - 144709.21474880382) / 159286.55956351123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionSoftware)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 135.88835321906353 < 189.12500181201315.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 15.261202368787597 (T) = (0 -138343.09820499978) / Math.Sqrt((500851545.2415736 / (299)) + (2616435.919967634 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1323773167760695 = (159450.7622725372 - 138343.09820499978) / 159450.7622725372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 18.04
Baseline 3e5517beb897faf4592d23f036446561da1e5c23
Compare 0533792282ba5546e10cb5c3d10748aff46ddc1a
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Transient - Duration of single invocation 19.74 μs 17.19 μs 0.87 0.06 False
Singleton - Duration of single invocation 19.93 μs 17.60 μs 0.88 0.06 False
Singleton - Duration of single invocation 19.68 μs 17.97 μs 0.91 0.08 False
Scoped - Duration of single invocation 21.01 μs 18.89 μs 0.90 0.04 False
Singleton - Duration of single invocation 19.93 μs 17.53 μs 0.88 0.06 False
Scoped - Duration of single invocation 21.07 μs 18.84 μs 0.89 0.07 False
Singleton - Duration of single invocation 19.72 μs 17.76 μs 0.90 0.06 False
Scoped - Duration of single invocation 21.08 μs 19.00 μs 0.90 0.09 False
Transient - Duration of single invocation 19.37 μs 17.14 μs 0.88 0.06 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
Details

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.187070024595965 < 18.548053560155118.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 36.05107076720448 (T) = (0 -17498.342493605916) / Math.Sqrt((371525.11458023137 / (299)) + (41062.882259744605 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1268857373042857 = (20041.29727486103 - 17498.342493605916) / 20041.29727486103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.603181839007544 < 18.70145217958402.
IsChangePoint: Marked as a change because one of 3/18/2022 7:38:50 AM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 23.283670684304365 (T) = (0 -17811.10270683597) / Math.Sqrt((336196.0135842124 / (299)) + (112204.16902273285 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1221258983897214 = (20288.903242691842 - 17811.10270683597) / 20288.903242691842 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.973728042024835 < 18.653811883958515.
IsChangePoint: Marked as a change because one of 3/18/2022 1:59:24 AM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 19.28748721574007 (T) = (0 -18029.67547330024) / Math.Sqrt((353946.8399368647 / (299)) + (139756.450349394 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11195801147382291 = (20302.728594199547 - 18029.67547330024) / 20302.728594199547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.891674214289324 < 20.0380591865709.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 57.691265456114 (T) = (0 -19039.669275490145) / Math.Sqrt((382122.16323418764 / (299)) + (9475.76898387818 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12292416487626694 = (21708.122049450983 - 19039.669275490145) / 21708.122049450983 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.526823101817524 < 18.92236482904492.
IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 58.29102626506418 (T) = (0 -17714.872052853923) / Math.Sqrt((330575.5860110873 / (299)) + (9533.075737284435 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12749989964528982 = (20303.57594876154 - 17714.872052853923) / 20303.57594876154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.83900167476038 < 20.244983613584.
IsChangePoint: Marked as a change because one of 3/18/2022 1:59:24 AM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 18.141402899256132 (T) = (0 -19237.845471632947) / Math.Sqrt((405703.7387961346 / (299)) + (197718.04102508974 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11591467702617689 = (21760.168359001884 - 19237.845471632947) / 21760.168359001884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.75608424597276 < 18.65047475657848.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 27.11249704668036 (T) = (0 -17875.125913270425) / Math.Sqrt((334765.6166177906 / (299)) + (74342.04807290909 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11865208561403992 = (20281.57736746234 - 17875.125913270425) / 20281.57736746234 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.00322544563955 < 20.037165961730448.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 23.8627474547782 (T) = (0 -19215.69487926475) / Math.Sqrt((434460.8046669556 / (299)) + (108998.9054549722 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.11689710837085308 = (21759.29335234727 - 19215.69487926475) / 21759.29335234727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.140703491132378 < 18.491480644835573.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 23.340792544231437 (T) = (0 -17581.398567453554) / Math.Sqrt((352303.1776981717 / (299)) + (105726.7170832878 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12118924092757948 = (20005.898182232788 - 17581.398567453554) / 20005.898182232788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 3e5517beb897faf4592d23f036446561da1e5c23
Compare 0533792282ba5546e10cb5c3d10748aff46ddc1a
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateInstance - Duration of single invocation 2.68 μs 1.99 μs 0.74 0.48 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
Details

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9921491510865452 < 2.6459349244007577.
IsChangePoint: Marked as a change because one of 4/7/2022 2:11:06 AM, 5/9/2022 9:57:04 AM, 5/12/2022 5:52:45 AM falls between 5/3/2022 7:02:53 AM and 5/12/2022 5:52:45 AM.
IsImprovementStdDev: Marked as improvement because 41.60724521394111 (T) = (0 -2043.1054846403565) / Math.Sqrt((116689.74443120719 / (299)) + (2982.151543826062 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.34371233405079393 = (3113.1249155587275 - 2043.1054846403565) / 3113.1249155587275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions