Skip to content

[IAST] Move analyzers init to an explicit call (#5920 -> v2)#5937

Merged
andrewlock merged 1 commit into
release/2.xfrom
dani/iast/hardcodedsecrets_init_fix-v2
Aug 23, 2024
Merged

[IAST] Move analyzers init to an explicit call (#5920 -> v2)#5937
andrewlock merged 1 commit into
release/2.xfrom
dani/iast/hardcodedsecrets_init_fix-v2

Conversation

@andrewlock

@andrewlock andrewlock commented Aug 23, 2024

Copy link
Copy Markdown
Member

Summary of changes

Make IAST analyzers initialization explicit, and call it only in Instrumentation.Initialize to avoid multiple instances scenarios

Reason for change

A crash under these circumstances has been detected

Implementation details

Move IAST analyzers init inside Instrumentation.Initialize only to make sure they are initialized only once

Test coverage

Other details

Backport of #5920

@dromanol @andrewlock
[IAST] Move analyzers init to an explicit call (#5920)
ace0982 
## Summary of changes
Make IAST analyzers initialization explicit, and call it only in
`Instrumentation.Initialize` to avoid multiple instances scenarios

## Reason for change
A crash under these circumstances has been detected

## Implementation details
Move IAST analyzers init inside `Instrumentation.Initialize` only to
make sure they are initialized only once

## Test coverage

## Other details
<!-- Fixes #{issue} -->

<!-- ⚠️ Note: where possible, please obtain 2 approvals prior to
merging. Unless CODEOWNERS specifies otherwise, for external teams it is
typically best to have one review from a team member, and one review
from apm-dotnet. Trivial changes do not require 2 reviews. -->
@andrewlock
andrewlock requested review from a team as code owners August 23, 2024 15:28
dromanol
dromanol approved these changes Aug 23, 2024
View reviewed changes

@dromanol dromanol left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving my own changes feels like cheating XD

@datadog-ddstaging

datadog-ddstaging Bot commented Aug 23, 2024 •
edited
Loading

Copy link
Copy Markdown

Datadog Report

Branch report: dani/iast/hardcodedsecrets_init_fix-v2
Commit report: ace0982
Test service: dd-trace-dotnet

✅ 0 Failed, 346055 Passed, 1775 Skipped, 15h 5m 2.16s Total Time

@andrewlock

andrewlock commented Aug 23, 2024

Copy link
Copy Markdown
Member Author

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5937) - mean (74ms)  : 66, 83
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5937) - mean (1,032ms)  : 996, 1068
     .   : milestone, 1032,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5937) - mean (110ms)  : 107, 113
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5937) - mean (720ms)  : 679, 762
     .   : milestone, 720,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5937) - mean (94ms)  : 91, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (5937) - mean (675ms)  : 635, 716
     .   : milestone, 675,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5937) - mean (190ms)  : 186, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (5937) - mean (1,112ms)  : 1090, 1134
     .   : milestone, 1112,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5937) - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (5937) - mean (877ms)  : 854, 899
     .   : milestone, 877,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5937) - mean (267ms)  : 261, 273
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (5937) - mean (865ms)  : 850, 880
     .   : milestone, 865,
Loading

@andrewlock

andrewlock commented Aug 23, 2024

Copy link
Copy Markdown
Member Author

Benchmarks Report for appsec 🐌

Benchmarks for #5937 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.158
  • 2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5937

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net472 1.158 4,526.68 3,907.45

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 73.6μs 87ns 337ns 0.0733 0 0 6 KB
master AllCycleSimpleBody netcoreapp3.1 62.6μs 119ns 462ns 0.094 0 0 6.95 KB
master AllCycleSimpleBody net472 48.5μs 27.5ns 103ns 1.32 0 0 8.34 KB
master AllCycleMoreComplexBody net6.0 78.7μs 121ns 470ns 0.119 0 0 9.51 KB
master AllCycleMoreComplexBody netcoreapp3.1 70.9μs 165ns 638ns 0.141 0 0 10.37 KB
master AllCycleMoreComplexBody net472 56.2μs 40.4ns 156ns 1.86 0.0282 0 11.85 KB
master ObjectExtractorSimpleBody net6.0 149ns 0.125ns 0.486ns 0.00396 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 198ns 0.163ns 0.611ns 0.0037 0 0 272 B
master ObjectExtractorSimpleBody net472 191ns 0.149ns 0.577ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.16μs 1.56ns 5.82ns 0.0525 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.89μs 2.32ns 8.68ns 0.0507 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.53μs 2.58ns 9.31ns 0.602 0.00679 0 3.8 KB
#5937 AllCycleSimpleBody net6.0 72.1μs 148ns 572ns 0.0734 0 0 6.01 KB
#5937 AllCycleSimpleBody netcoreapp3.1 62.2μs 78.8ns 305ns 0.0934 0 0 6.95 KB
#5937 AllCycleSimpleBody net472 49.4μs 70.3ns 272ns 1.32 0 0 8.34 KB
#5937 AllCycleMoreComplexBody net6.0 80μs 113ns 437ns 0.12 0 0 9.51 KB
#5937 AllCycleMoreComplexBody netcoreapp3.1 69.3μs 49.4ns 185ns 0.138 0 0 10.37 KB
#5937 AllCycleMoreComplexBody net472 56μs 48.6ns 182ns 1.87 0.0279 0 11.85 KB
#5937 ObjectExtractorSimpleBody net6.0 140ns 0.107ns 0.4ns 0.00396 0 0 280 B
#5937 ObjectExtractorSimpleBody netcoreapp3.1 205ns 0.188ns 0.679ns 0.00379 0 0 272 B
#5937 ObjectExtractorSimpleBody net472 200ns 0.105ns 0.406ns 0.0446 0 0 281 B
#5937 ObjectExtractorMoreComplexBody net6.0 3.27μs 1.37ns 5.12ns 0.0527 0 0 3.78 KB
#5937 ObjectExtractorMoreComplexBody netcoreapp3.1 3.94μs 2.26ns 8.76ns 0.0491 0 0 3.69 KB
#5937 ObjectExtractorMoreComplexBody net472 3.91μs 3.58ns 13.4ns 0.602 0.00583 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 39.1μs 17.4ns 65.1ns 0.464 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.2μs 17.1ns 66.1ns 0.43 0 0 32.4 KB
master EncodeArgs net472 65.6μs 24.2ns 90.7ns 5.16 0.0653 0 32.5 KB
master EncodeLegacyArgs net6.0 78μs 356ns 2.1μs 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 105μs 382ns 1.48μs 0 0 0 2.14 KB
master EncodeLegacyArgs net472 150μs 79.9ns 309ns 0.299 0 0 2.15 KB
#5937 EncodeArgs net6.0 38.1μs 24.4ns 94.4ns 0.459 0 0 32.4 KB
#5937 EncodeArgs netcoreapp3.1 54.4μs 27.9ns 108ns 0.437 0 0 32.4 KB
#5937 EncodeArgs net472 67.1μs 95.3ns 369ns 5.15 0.0668 0 32.5 KB
#5937 EncodeLegacyArgs net6.0 75.9μs 410ns 2.32μs 0 0 0 2.14 KB
#5937 EncodeLegacyArgs netcoreapp3.1 107μs 358ns 1.39μs 0 0 0 2.14 KB
#5937 EncodeLegacyArgs net472 155μs 181ns 703ns 0.311 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 183μs 98.6ns 369ns 0 0 0 2.42 KB
master RunWafRealisticBenchmark netcoreapp3.1 197μs 117ns 436ns 0 0 0 2.37 KB
master RunWafRealisticBenchmark net472 209μs 160ns 620ns 0.311 0 0 2.43 KB
master RunWafRealisticBenchmarkWithAttack net6.0 122μs 50.7ns 176ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 131μs 137ns 529ns 0 0 0 1.45 KB
master RunWafRealisticBenchmarkWithAttack net472 138μs 53.3ns 206ns 0.206 0 0 1.48 KB
#5937 RunWafRealisticBenchmark net6.0 183μs 147ns 568ns 0 0 0 2.42 KB
#5937 RunWafRealisticBenchmark netcoreapp3.1 196μs 288ns 1.12μs 0 0 0 2.37 KB
#5937 RunWafRealisticBenchmark net472 210μs 167ns 646ns 0.312 0 0 2.43 KB
#5937 RunWafRealisticBenchmarkWithAttack net6.0 122μs 27.4ns 106ns 0 0 0 1.46 KB
#5937 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 129μs 76.7ns 297ns 0 0 0 1.45 KB
#5937 RunWafRealisticBenchmarkWithAttack net472 140μs 38.7ns 145ns 0.207 0 0 1.48 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5937

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 255.61 KB 264.35 KB 8.74 KB 3.42%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.26 KB 59.09 KB 1.83 KB 3.20%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 57.8μs 727ns 7.2μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 62.1μs 861ns 8.57μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 38.5μs 195ns 892ns 0 0 0 57.26 KB
master StringConcatAspectBenchmark net6.0 299μs 5.67μs 56.1μs 0 0 0 256.54 KB
master StringConcatAspectBenchmark netcoreapp3.1 346μs 1.75μs 8.03μs 0 0 0 255.61 KB
master StringConcatAspectBenchmark net472 290μs 6.76μs 65.6μs 0 0 0 278.53 KB
#5937 StringConcatBenchmark net6.0 60.9μs 652ns 6.22μs 0 0 0 43.44 KB
#5937 StringConcatBenchmark netcoreapp3.1 54.4μs 289ns 1.53μs 0 0 0 42.64 KB
#5937 StringConcatBenchmark net472 36.8μs 64.5ns 224ns 0 0 0 59.09 KB
#5937 StringConcatAspectBenchmark net6.0 315μs 1.68μs 12μs 0 0 0 255.64 KB
#5937 StringConcatAspectBenchmark netcoreapp3.1 331μs 2.63μs 25.5μs 0 0 0 264.35 KB
#5937 StringConcatAspectBenchmark net472 284μs 6.39μs 61.6μs 0 0 0 278.53 KB

@andrewlock

andrewlock commented Aug 23, 2024

Copy link
Copy Markdown
Member Author

Benchmarks Report for tracer 🐌

Benchmarks for #5937 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.147
  • 1 benchmarks are slower, with geometric mean 1.202
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.98μs 45ns 305ns 0.0188 0.0075 0 5.43 KB
master StartStopWithChild netcoreapp3.1 9.93μs 56ns 376ns 0.0297 0.0149 0 5.62 KB
master StartStopWithChild net472 15.8μs 55.7ns 216ns 1.02 0.306 0.0943 6.06 KB
#5937 StartStopWithChild net6.0 7.75μs 43.5ns 304ns 0.016 0.00798 0 5.42 KB
#5937 StartStopWithChild netcoreapp3.1 10.1μs 54.5ns 294ns 0.0151 0.00502 0 5.61 KB
#5937 StartStopWithChild net472 16μs 32.5ns 126ns 1.02 0.29 0.0887 6.06 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 459μs 209ns 809ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 635μs 426ns 1.59μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 845μs 532ns 1.99μs 0.422 0 0 3.3 KB
#5937 WriteAndFlushEnrichedTraces net6.0 470μs 434ns 1.68μs 0 0 0 2.7 KB
#5937 WriteAndFlushEnrichedTraces netcoreapp3.1 652μs 409ns 1.58μs 0 0 0 2.7 KB
#5937 WriteAndFlushEnrichedTraces net472 828μs 336ns 1.26μs 0.411 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 192μs 1.07μs 7.01μs 0.198 0 0 18.45 KB
master SendRequest netcoreapp3.1 214μs 1.22μs 9.35μs 0.211 0 0 20.61 KB
master SendRequest net472 0.000893ns 0.000468ns 0.00175ns 0 0 0 0 b
#5937 SendRequest net6.0 189μs 1.08μs 8.26μs 0.191 0 0 18.45 KB
#5937 SendRequest netcoreapp3.1 211μs 1.2μs 8.91μs 0.203 0 0 20.61 KB
#5937 SendRequest net472 0.00272ns 0.000814ns 0.00315ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5937

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.42 KB 41.84 KB 422 B 1.02%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 573μs 989ns 3.57μs 0.536 0 0 41.76 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 731μs 3.79μs 19.3μs 0.368 0 0 41.42 KB
master WriteAndFlushEnrichedTraces net472 880μs 3.72μs 14.4μs 8.8 2.64 0.44 53.34 KB
#5937 WriteAndFlushEnrichedTraces net6.0 568μs 2.64μs 10.9μs 0.561 0 0 41.58 KB
#5937 WriteAndFlushEnrichedTraces netcoreapp3.1 671μs 3.29μs 13.6μs 0.332 0 0 41.84 KB
#5937 WriteAndFlushEnrichedTraces net472 865μs 4.14μs 17.5μs 8.25 2.6 0.434 53.29 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.39μs 0.903ns 3.5ns 0.0139 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.74μs 1.3ns 5.03ns 0.014 0 0 1.02 KB
master ExecuteNonQuery net472 2.07μs 2.93ns 11.4ns 0.156 0 0 987 B
#5937 ExecuteNonQuery net6.0 1.28μs 1.02ns 3.94ns 0.0147 0 0 1.02 KB
#5937 ExecuteNonQuery netcoreapp3.1 1.62μs 4.37ns 16.9ns 0.0138 0 0 1.02 KB
#5937 ExecuteNonQuery net472 2.01μs 2.73ns 10.6ns 0.156 0 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.22μs 1.52ns 5.7ns 0.0138 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.61μs 3.11ns 12.1ns 0.0131 0 0 976 B
master CallElasticsearch net472 2.4μs 1.63ns 6.32ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.2μs 0.715ns 2.68ns 0.0132 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.61μs 0.613ns 2.3ns 0.0137 0 0 1.02 KB
master CallElasticsearchAsync net472 2.57μs 1.35ns 5.24ns 0.167 0 0 1.05 KB
#5937 CallElasticsearch net6.0 1.23μs 0.723ns 2.7ns 0.0136 0 0 976 B
#5937 CallElasticsearch netcoreapp3.1 1.56μs 0.689ns 2.49ns 0.0132 0 0 976 B
#5937 CallElasticsearch net472 2.46μs 1.7ns 6.58ns 0.157 0 0 995 B
#5937 CallElasticsearchAsync net6.0 1.32μs 0.83ns 3.22ns 0.0132 0 0 952 B
#5937 CallElasticsearchAsync netcoreapp3.1 1.56μs 0.735ns 2.75ns 0.0143 0 0 1.02 KB
#5937 CallElasticsearchAsync net472 2.56μs 1.75ns 6.54ns 0.167 0.00128 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.27μs 1.09ns 4.08ns 0.0133 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.65μs 0.858ns 3.21ns 0.0125 0 0 952 B
master ExecuteAsync net472 1.66μs 0.384ns 1.44ns 0.145 0 0 915 B
#5937 ExecuteAsync net6.0 1.26μs 0.699ns 2.61ns 0.0133 0 0 952 B
#5937 ExecuteAsync netcoreapp3.1 1.62μs 0.581ns 2.17ns 0.0122 0 0 952 B
#5937 ExecuteAsync net472 1.82μs 1.03ns 4ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.34μs 2.08ns 7.8ns 0.0304 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.08μs 1.51ns 5.64ns 0.0357 0 0 2.76 KB
master SendAsync net472 7.85μs 2.83ns 11ns 0.497 0 0 3.15 KB
#5937 SendAsync net6.0 4.02μs 1.08ns 3.9ns 0.0302 0 0 2.22 KB
#5937 SendAsync netcoreapp3.1 5.21μs 2.62ns 10.2ns 0.0363 0 0 2.76 KB
#5937 SendAsync net472 7.99μs 2.24ns 8.67ns 0.498 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.49μs 0.763ns 2.96ns 0.0232 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.24μs 1.61ns 6.02ns 0.0223 0 0 1.64 KB
master EnrichedLog net472 2.65μs 1.83ns 6.59ns 0.249 0 0 1.57 KB
#5937 EnrichedLog net6.0 1.5μs 0.717ns 2.78ns 0.023 0 0 1.64 KB
#5937 EnrichedLog netcoreapp3.1 2.28μs 8.6ns 33.3ns 0.0221 0 0 1.64 KB
#5937 EnrichedLog net472 2.64μs 1.82ns 6.82ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 116μs 200ns 774ns 0.0583 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 120μs 265ns 1.02μs 0 0 0 4.28 KB
master EnrichedLog net472 151μs 201ns 778ns 0.674 0.225 0 4.46 KB
#5937 EnrichedLog net6.0 115μs 417ns 1.61μs 0 0 0 4.28 KB
#5937 EnrichedLog netcoreapp3.1 120μs 223ns 865ns 0 0 0 4.28 KB
#5937 EnrichedLog net472 146μs 282ns 1.09μs 0.658 0.219 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.01μs 0.751ns 2.91ns 0.0301 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.3μs 1.37ns 5.32ns 0.0301 0 0 2.2 KB
master EnrichedLog net472 4.91μs 1.48ns 5.73ns 0.319 0 0 2.02 KB
#5937 EnrichedLog net6.0 3.06μs 0.839ns 3.14ns 0.0308 0 0 2.2 KB
#5937 EnrichedLog netcoreapp3.1 4.14μs 1.33ns 5.14ns 0.0288 0 0 2.2 KB
#5937 EnrichedLog net472 4.92μs 1.27ns 4.92ns 0.32 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.33μs 0.667ns 2.58ns 0.016 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.82μs 1.35ns 5.22ns 0.0154 0 0 1.14 KB
master SendReceive net472 2.21μs 2.01ns 7.77ns 0.183 0.00111 0 1.16 KB
#5937 SendReceive net6.0 1.47μs 4.35ns 16.9ns 0.016 0 0 1.14 KB
#5937 SendReceive netcoreapp3.1 1.78μs 1.62ns 6.27ns 0.0153 0 0 1.14 KB
#5937 SendReceive net472 2.23μs 2.71ns 10.5ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.88μs 0.836ns 3.24ns 0.0216 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 4μs 1.16ns 4.48ns 0.022 0 0 1.65 KB
master EnrichedLog net472 4.32μs 2.36ns 9.14ns 0.323 0 0 2.04 KB
#5937 EnrichedLog net6.0 2.87μs 1.04ns 4.03ns 0.0215 0 0 1.6 KB
#5937 EnrichedLog netcoreapp3.1 3.92μs 1.28ns 4.95ns 0.0214 0 0 1.65 KB
#5937 EnrichedLog net472 4.31μs 2.32ns 8.67ns 0.324 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5937

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.202 403.18 484.56

Faster 🎉 in #5937

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.147 535.57 466.94

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 403ns 0.148ns 0.574ns 0.00813 0 0 576 B
master StartFinishSpan netcoreapp3.1 623ns 0.358ns 1.34ns 0.00779 0 0 576 B
master StartFinishSpan net472 666ns 0.202ns 0.783ns 0.0916 0 0 578 B
master StartFinishScope net6.0 535ns 0.185ns 0.715ns 0.00981 0 0 696 B
master StartFinishScope netcoreapp3.1 737ns 0.29ns 1.12ns 0.00951 0 0 696 B
master StartFinishScope net472 795ns 1.02ns 3.94ns 0.104 0 0 658 B
#5937 StartFinishSpan net6.0 484ns 0.129ns 0.499ns 0.00803 0 0 576 B
#5937 StartFinishSpan netcoreapp3.1 597ns 0.168ns 0.651ns 0.00775 0 0 576 B
#5937 StartFinishSpan net472 646ns 0.292ns 1.13ns 0.0917 0 0 578 B
#5937 StartFinishScope net6.0 467ns 0.111ns 0.415ns 0.00965 0 0 696 B
#5937 StartFinishScope netcoreapp3.1 718ns 0.323ns 1.21ns 0.00933 0 0 696 B
#5937 StartFinishScope net472 766ns 0.392ns 1.47ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 681ns 0.43ns 1.67ns 0.00963 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 979ns 0.281ns 1.09ns 0.00936 0 0 696 B
master RunOnMethodBegin net472 1.09μs 0.426ns 1.65ns 0.104 0 0 658 B
#5937 RunOnMethodBegin net6.0 706ns 0.58ns 2.01ns 0.00955 0 0 696 B
#5937 RunOnMethodBegin netcoreapp3.1 889ns 0.841ns 3.26ns 0.00969 0 0 696 B
#5937 RunOnMethodBegin net472 1.09μs 0.164ns 0.612ns 0.104 0 0 658 B

@andrewlock

andrewlock commented Aug 23, 2024

Copy link
Copy Markdown
Member Author

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5937) (11.497M)   : 0, 11497140
    master (11.601M)   : 0, 11600674
    benchmarks/2.9.0 (11.883M)   : 0, 11882814

    section Automatic
    This PR (5937) (7.748M)   : 0, 7748289
    master (7.683M)   : 0, 7683307
    benchmarks/2.9.0 (8.446M)   : 0, 8445613

    section Trace stats
    master (8.064M)   : 0, 8064161

    section Manual
    This PR (5937) (9.810M)   : crit ,0, 9809785
    master (11.471M)   : 0, 11470876

    section Manual + Automatic
    This PR (5937) (7.334M)   : 0, 7334306
    master (7.151M)   : 0, 7150588
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5937) (9.549M)   : 0, 9548994
    benchmarks/2.9.0 (9.760M)   : 0, 9759697

    section Automatic
    This PR (5937) (6.578M)   : 0, 6577879

    section Manual
    This PR (5937) (8.157M)   : 0, 8157495

    section Manual + Automatic
    This PR (5937) (5.939M)   : 0, 5939303
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5937) (10.087M)   : 0, 10086814

    section Automatic
    This PR (5937) (6.881M)   : 0, 6880780

    section Manual
    This PR (5937) (8.678M)   : 0, 8678141

    section Manual + Automatic
    This PR (5937) (6.611M)   : 0, 6611489
Loading

@andrewlock
andrewlock merged commit ba1be9a into release/2.x Aug 23, 2024
@andrewlock
andrewlock deleted the dani/iast/hardcodedsecrets_init_fix-v2 branch August 23, 2024 22:13
@github-actions github-actions Bot added this to the vNext-v2 milestone Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bouwkast bouwkast bouwkast approved these changes
@dromanol dromanol dromanol approved these changes

Assignees

No one assigned

Labels

area:asm-iast identified-by:crashtracking

Projects

None yet

Milestone

2.58.0

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewlock @bouwkast @dromanol