Skip to content

Removing MetaStruct check from TestSessionTimeoutVulnerability#6395

Merged
link04 merged 2 commits into
masterfrom
maximo/update-iast-timeoutvul-test
Dec 4, 2024
Merged

Removing MetaStruct check from TestSessionTimeoutVulnerability#6395
link04 merged 2 commits into
masterfrom
maximo/update-iast-timeoutvul-test

Conversation

@link04

@link04 link04 commented Dec 3, 2024

Copy link
Copy Markdown
Contributor

Summary of changes

Disabling SpanMetaStructs from Agent used in related test and snapshot file.

Reason for change

After recent PR got merged test started flaking in master, but since the feature is already tested somewhere else as expected removing from this test to prevent flakes.

Implementation details

Setting:
await TryStartApp(newFixture, new MockTracerAgent.AgentConfiguration { SpanMetaStructs = false });

Removing the below from the received snapshot:

},
    MetaStruct: {
      iast:

Test coverage

The test.

@link04
link04 requested a review from a team as a code owner December 3, 2024 20:49
@github-actions

github-actions Bot commented Dec 3, 2024

Copy link
Copy Markdown
Contributor

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

1 occurrences of :

-    },
-    MetaStruct: {
-      iast:

@github-actions github-actions Bot added the area:tests unit tests, integration tests label Dec 3, 2024
@andrewlock

andrewlock commented Dec 3, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6395) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (69ms)  : 66, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6395) - mean (980ms)  : 957, 1003
     .   : milestone, 980,
    master - mean (977ms)  : 957, 998
     .   : milestone, 977,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6395) - mean (108ms)  : 106, 111
     .   : milestone, 108,
    master - mean (108ms)  : 106, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6395) - mean (685ms)  : 665, 706
     .   : milestone, 685,
    master - mean (679ms)  : 665, 692
     .   : milestone, 679,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6395) - mean (92ms)  : 90, 93
     .   : milestone, 92,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6395) - mean (636ms)  : 618, 654
     .   : milestone, 636,
    master - mean (633ms)  : 617, 648
     .   : milestone, 633,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6395) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (190ms)  : 186, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6395) - mean (1,093ms)  : 1064, 1121
     .   : milestone, 1093,
    master - mean (1,092ms)  : 1055, 1128
     .   : milestone, 1092,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6395) - mean (277ms)  : 272, 281
     .   : milestone, 277,
    master - mean (277ms)  : 273, 281
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (6395) - mean (872ms)  : 844, 900
     .   : milestone, 872,
    master - mean (869ms)  : 842, 895
     .   : milestone, 869,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6395) - mean (267ms)  : 258, 276
     .   : milestone, 267,
    master - mean (265ms)  : 261, 269
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (6395) - mean (853ms)  : 818, 888
     .   : milestone, 853,
    master - mean (852ms)  : 818, 886
     .   : milestone, 852,
Loading

@datadog-ddstaging

datadog-ddstaging Bot commented Dec 3, 2024 •
edited
Loading

Copy link
Copy Markdown

Datadog Report

Branch report: maximo/update-iast-timeoutvul-test
Commit report: 995e7dc
Test service: dd-trace-dotnet

✅ 0 Failed, 452169 Passed, 2739 Skipped, 19h 33m 38.58s Total Time

@andrewlock

andrewlock commented Dec 3, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Benchmarks Report for appsec 🐌

Benchmarks for #6395 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.192
  • 1 benchmarks are slower, with geometric mean 1.121
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6395

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.121 199.17 223.32

Faster 🎉 in #6395

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net6.0 1.192 181.67 152.36

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 218μs 168ns 628ns 2.5 0 0 176.82 KB
master AllCycleSimpleBody netcoreapp3.1 306μs 201ns 778ns 2.45 0 0 184.26 KB
master AllCycleSimpleBody net472 285μs 574ns 2.22μs 33.4 2.53 0 210.04 KB
master AllCycleMoreComplexBody net6.0 225μs 202ns 781ns 2.45 0 0 180.33 KB
master AllCycleMoreComplexBody netcoreapp3.1 336μs 125ns 469ns 2.51 0 0 187.68 KB
master AllCycleMoreComplexBody net472 291μs 511ns 1.98μs 33.9 2.49 0 213.56 KB
master ObjectExtractorSimpleBody net6.0 182ns 0.104ns 0.388ns 0.00394 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 199ns 0.203ns 0.759ns 0.00371 0 0 272 B
master ObjectExtractorSimpleBody net472 211ns 0.246ns 0.953ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.96μs 1.64ns 6.14ns 0.0534 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.86μs 2.11ns 7.89ns 0.0503 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.34μs 5.78ns 22.4ns 0.603 0.00651 0 3.8 KB
#6395 AllCycleSimpleBody net6.0 222μs 162ns 628ns 2.45 0 0 176.82 KB
#6395 AllCycleSimpleBody netcoreapp3.1 315μs 168ns 628ns 2.39 0 0 184.26 KB
#6395 AllCycleSimpleBody net472 289μs 447ns 1.73μs 33.3 2.58 0 210.04 KB
#6395 AllCycleMoreComplexBody net6.0 220μs 125ns 450ns 2.54 0 0 180.33 KB
#6395 AllCycleMoreComplexBody netcoreapp3.1 326μs 186ns 720ns 2.49 0 0 187.68 KB
#6395 AllCycleMoreComplexBody net472 302μs 871ns 3.37μs 33.9 2.63 0 213.56 KB
#6395 ObjectExtractorSimpleBody net6.0 152ns 0.0814ns 0.305ns 0.00391 0 0 280 B
#6395 ObjectExtractorSimpleBody netcoreapp3.1 223ns 0.114ns 0.44ns 0.00372 0 0 272 B
#6395 ObjectExtractorSimpleBody net472 215ns 0.54ns 2.09ns 0.0446 0 0 281 B
#6395 ObjectExtractorMoreComplexBody net6.0 2.99μs 2.06ns 7.71ns 0.0533 0 0 3.78 KB
#6395 ObjectExtractorMoreComplexBody netcoreapp3.1 3.68μs 2.39ns 8.94ns 0.0498 0 0 3.69 KB
#6395 ObjectExtractorMoreComplexBody net472 4.42μs 7.27ns 28.1ns 0.602 0.0065 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 38.6μs 21.2ns 79.5ns 0.457 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.3μs 30.9ns 107ns 0.431 0 0 32.4 KB
master EncodeArgs net472 68.2μs 56.8ns 213ns 5.14 0.068 0 32.5 KB
master EncodeLegacyArgs net6.0 79.5μs 142ns 548ns 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 105μs 327ns 1.27μs 0 0 0 2.14 KB
master EncodeLegacyArgs net472 160μs 83.9ns 325ns 0.321 0 0 2.15 KB
#6395 EncodeArgs net6.0 37.2μs 28.7ns 107ns 0.447 0 0 32.4 KB
#6395 EncodeArgs netcoreapp3.1 54.3μs 19.4ns 72.6ns 0.431 0 0 32.4 KB
#6395 EncodeArgs net472 69.2μs 77.9ns 302ns 5.14 0.0694 0 32.5 KB
#6395 EncodeLegacyArgs net6.0 71.9μs 48.4ns 181ns 0.0359 0 0 2.14 KB
#6395 EncodeLegacyArgs netcoreapp3.1 106μs 391ns 1.51μs 0 0 0 2.14 KB
#6395 EncodeLegacyArgs net472 156μs 48.5ns 181ns 0.315 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 186μs 310ns 1.2μs 0 0 0 2.44 KB
master RunWafRealisticBenchmark netcoreapp3.1 194μs 109ns 392ns 0 0 0 2.39 KB
master RunWafRealisticBenchmark net472 211μs 110ns 425ns 0.316 0 0 2.46 KB
master RunWafRealisticBenchmarkWithAttack net6.0 124μs 62.9ns 244ns 0 0 0 1.47 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 98.3ns 381ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack net472 141μs 180ns 696ns 0.209 0 0 1.49 KB
#6395 RunWafRealisticBenchmark net6.0 183μs 148ns 574ns 0 0 0 2.44 KB
#6395 RunWafRealisticBenchmark netcoreapp3.1 196μs 183ns 709ns 0 0 0 2.39 KB
#6395 RunWafRealisticBenchmark net472 209μs 119ns 459ns 0.311 0 0 2.46 KB
#6395 RunWafRealisticBenchmarkWithAttack net6.0 124μs 49.5ns 185ns 0 0 0 1.47 KB
#6395 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 129μs 117ns 439ns 0 0 0 1.46 KB
#6395 RunWafRealisticBenchmarkWithAttack net472 139μs 52.1ns 202ns 0.209 0 0 1.49 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6395

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 62.21 KB 60.91 KB -1.3 KB -2.08%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 57.4μs 625ns 6.15μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 58.6μs 673ns 6.45μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 37.9μs 49.4ns 171ns 0 0 0 62.21 KB
master StringConcatAspectBenchmark net6.0 317μs 1.44μs 8.4μs 0 0 0 255.74 KB
master StringConcatAspectBenchmark netcoreapp3.1 338μs 2.17μs 20.8μs 0 0 0 253.86 KB
master StringConcatAspectBenchmark net472 288μs 5.69μs 55.4μs 0 0 0 278.53 KB
#6395 StringConcatBenchmark net6.0 52.9μs 291ns 2.13μs 0 0 0 43.44 KB
#6395 StringConcatBenchmark netcoreapp3.1 59.5μs 712ns 6.87μs 0 0 0 42.64 KB
#6395 StringConcatBenchmark net472 37.4μs 77.9ns 291ns 0 0 0 60.91 KB
#6395 StringConcatAspectBenchmark net6.0 323μs 6.21μs 61.4μs 0 0 0 255.02 KB
#6395 StringConcatAspectBenchmark netcoreapp3.1 341μs 1.51μs 9.06μs 0 0 0 253.62 KB
#6395 StringConcatAspectBenchmark net472 307μs 7.05μs 69.4μs 0 0 0 278.53 KB

@andrewlock

andrewlock commented Dec 3, 2024

Copy link
Copy Markdown
Member

Benchmarks Report for tracer 🐌

Benchmarks for #6395 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.04μs 45.7ns 313ns 0.0127 0.00423 0 5.61 KB
master StartStopWithChild netcoreapp3.1 10.3μs 53.8ns 309ns 0.0241 0.00963 0 5.8 KB
master StartStopWithChild net472 16.3μs 42.8ns 166ns 1.07 0.345 0.102 6.21 KB
#6395 StartStopWithChild net6.0 8.22μs 45.2ns 268ns 0.0242 0.0121 0 5.61 KB
#6395 StartStopWithChild netcoreapp3.1 10.2μs 57.4ns 406ns 0.0194 0.00971 0 5.8 KB
#6395 StartStopWithChild net472 16.2μs 56.4ns 218ns 1.04 0.297 0.0962 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 503μs 446ns 1.67μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 648μs 456ns 1.77μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 850μs 346ns 1.34μs 0.422 0 0 3.3 KB
#6395 WriteAndFlushEnrichedTraces net6.0 487μs 449ns 1.74μs 0 0 0 2.7 KB
#6395 WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 406ns 1.57μs 0 0 0 2.7 KB
#6395 WriteAndFlushEnrichedTraces net472 882μs 2.13μs 8.25μs 0.437 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 159μs 1.27μs 12.3μs 0.147 0 0 14.47 KB
master SendRequest netcoreapp3.1 176μs 1.29μs 12.7μs 0.174 0 0 17.27 KB
master SendRequest net472 0.00173ns 0.000792ns 0.00296ns 0 0 0 0 b
#6395 SendRequest net6.0 151μs 1.01μs 9.84μs 0.15 0 0 14.47 KB
#6395 SendRequest netcoreapp3.1 169μs 972ns 7.89μs 0.16 0 0 17.27 KB
#6395 SendRequest net472 0.00101ns 0.000324ns 0.00125ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 575μs 3.12μs 17.1μs 0.571 0 0 41.63 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 710μs 4.15μs 37.1μs 0.365 0 0 41.79 KB
master WriteAndFlushEnrichedTraces net472 849μs 3.89μs 15.1μs 8.45 2.53 0.422 53.33 KB
#6395 WriteAndFlushEnrichedTraces net6.0 575μs 3.04μs 15.5μs 0.556 0 0 41.47 KB
#6395 WriteAndFlushEnrichedTraces netcoreapp3.1 674μs 3.39μs 14.4μs 0.327 0 0 41.73 KB
#6395 WriteAndFlushEnrichedTraces net472 857μs 4.16μs 16.6μs 8.87 2.53 0.422 53.27 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.42μs 1.23ns 4.78ns 0.0141 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.75μs 1.5ns 5.6ns 0.0139 0 0 1.02 KB
master ExecuteNonQuery net472 2.13μs 2.01ns 7.77ns 0.156 0.00106 0 987 B
#6395 ExecuteNonQuery net6.0 1.32μs 1.13ns 4.36ns 0.0145 0 0 1.02 KB
#6395 ExecuteNonQuery netcoreapp3.1 1.72μs 1.92ns 7.19ns 0.0136 0 0 1.02 KB
#6395 ExecuteNonQuery net472 2.08μs 1.88ns 7.28ns 0.157 0.00104 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.2μs 0.671ns 2.51ns 0.0133 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.53μs 0.756ns 2.93ns 0.0126 0 0 976 B
master CallElasticsearch net472 2.64μs 2.53ns 9.78ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.39μs 0.905ns 3.51ns 0.0132 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.64μs 0.907ns 3.39ns 0.014 0 0 1.02 KB
master CallElasticsearchAsync net472 2.56μs 1.89ns 7.33ns 0.166 0 0 1.05 KB
#6395 CallElasticsearch net6.0 1.12μs 1.25ns 4.68ns 0.0135 0 0 976 B
#6395 CallElasticsearch netcoreapp3.1 1.53μs 3.82ns 14.8ns 0.0129 0 0 976 B
#6395 CallElasticsearch net472 2.49μs 1.72ns 6.43ns 0.157 0 0 995 B
#6395 CallElasticsearchAsync net6.0 1.25μs 3.06ns 11.9ns 0.0133 0 0 952 B
#6395 CallElasticsearchAsync netcoreapp3.1 1.65μs 0.602ns 2.25ns 0.014 0 0 1.02 KB
#6395 CallElasticsearchAsync net472 2.52μs 1.94ns 7.01ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.33μs 1.38ns 5.17ns 0.0134 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.72μs 0.988ns 3.83ns 0.013 0 0 952 B
master ExecuteAsync net472 1.93μs 0.688ns 2.58ns 0.145 0 0 915 B
#6395 ExecuteAsync net6.0 1.36μs 0.469ns 1.82ns 0.0129 0 0 952 B
#6395 ExecuteAsync netcoreapp3.1 1.68μs 1.08ns 4.19ns 0.0126 0 0 952 B
#6395 ExecuteAsync net472 1.83μs 0.788ns 3.05ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.39μs 1.16ns 4.17ns 0.0329 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.48μs 2.27ns 8.8ns 0.0384 0 0 2.85 KB
master SendAsync net472 7.17μs 0.965ns 3.74ns 0.494 0 0 3.12 KB
#6395 SendAsync net6.0 4.36μs 3.14ns 11.3ns 0.0308 0 0 2.31 KB
#6395 SendAsync netcoreapp3.1 5.28μs 2.04ns 7.34ns 0.0372 0 0 2.85 KB
#6395 SendAsync net472 7.29μs 1.68ns 6.51ns 0.495 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.57μs 0.717ns 2.68ns 0.0236 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.29μs 4.95ns 19.2ns 0.0216 0 0 1.64 KB
master EnrichedLog net472 2.68μs 1.01ns 3.91ns 0.249 0 0 1.57 KB
#6395 EnrichedLog net6.0 1.56μs 0.612ns 2.21ns 0.0226 0 0 1.64 KB
#6395 EnrichedLog netcoreapp3.1 2.38μs 1.26ns 4.73ns 0.0226 0 0 1.64 KB
#6395 EnrichedLog net472 2.72μs 1.73ns 6.46ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 121μs 109ns 409ns 0.0604 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 124μs 117ns 455ns 0 0 0 4.28 KB
master EnrichedLog net472 154μs 164ns 634ns 0.688 0.229 0 4.46 KB
#6395 EnrichedLog net6.0 120μs 116ns 448ns 0.0601 0 0 4.28 KB
#6395 EnrichedLog netcoreapp3.1 125μs 167ns 647ns 0 0 0 4.28 KB
#6395 EnrichedLog net472 152μs 137ns 511ns 0.683 0.228 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.99μs 0.709ns 2.75ns 0.0299 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.15μs 1.29ns 4.83ns 0.0292 0 0 2.2 KB
master EnrichedLog net472 4.96μs 1.09ns 4.07ns 0.319 0 0 2.02 KB
#6395 EnrichedLog net6.0 3.07μs 1.16ns 4.51ns 0.0308 0 0 2.2 KB
#6395 EnrichedLog netcoreapp3.1 4.36μs 1.74ns 6.75ns 0.0285 0 0 2.2 KB
#6395 EnrichedLog net472 5.04μs 1.63ns 6.11ns 0.321 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.46μs 0.803ns 3.11ns 0.016 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.69μs 0.851ns 3.3ns 0.0152 0 0 1.14 KB
master SendReceive net472 2.06μs 1.25ns 4.83ns 0.184 0 0 1.16 KB
#6395 SendReceive net6.0 1.43μs 0.944ns 3.65ns 0.0164 0 0 1.14 KB
#6395 SendReceive netcoreapp3.1 1.8μs 0.734ns 2.84ns 0.0154 0 0 1.14 KB
#6395 SendReceive net472 2.14μs 1.87ns 7.02ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.67μs 1.05ns 4.06ns 0.0217 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.99μs 1.54ns 5.95ns 0.0219 0 0 1.65 KB
master EnrichedLog net472 4.46μs 3.88ns 15ns 0.323 0 0 2.04 KB
#6395 EnrichedLog net6.0 2.78μs 0.953ns 3.44ns 0.0223 0 0 1.6 KB
#6395 EnrichedLog netcoreapp3.1 3.99μs 1.47ns 5.5ns 0.0218 0 0 1.65 KB
#6395 EnrichedLog net472 4.33μs 1.22ns 4.42ns 0.324 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 392ns 0.195ns 0.756ns 0.00806 0 0 576 B
master StartFinishSpan netcoreapp3.1 606ns 3.34ns 18.9ns 0.00783 0 0 576 B
master StartFinishSpan net472 631ns 0.61ns 2.36ns 0.0918 0 0 578 B
master StartFinishScope net6.0 490ns 0.247ns 0.958ns 0.00978 0 0 696 B
master StartFinishScope netcoreapp3.1 785ns 0.477ns 1.85ns 0.00945 0 0 696 B
master StartFinishScope net472 853ns 0.628ns 2.43ns 0.104 0 0 658 B
#6395 StartFinishSpan net6.0 413ns 0.337ns 1.26ns 0.00814 0 0 576 B
#6395 StartFinishSpan netcoreapp3.1 618ns 0.516ns 1.86ns 0.00767 0 0 576 B
#6395 StartFinishSpan net472 656ns 0.678ns 2.63ns 0.0917 0 0 578 B
#6395 StartFinishScope net6.0 491ns 0.365ns 1.41ns 0.00985 0 0 696 B
#6395 StartFinishScope netcoreapp3.1 751ns 0.501ns 1.94ns 0.00951 0 0 696 B
#6395 StartFinishScope net472 815ns 0.557ns 2.16ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 696ns 0.399ns 1.55ns 0.0097 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 904ns 1.49ns 5.79ns 0.00946 0 0 696 B
master RunOnMethodBegin net472 1.1μs 0.804ns 3.11ns 0.104 0 0 658 B
#6395 RunOnMethodBegin net6.0 660ns 1.06ns 4.11ns 0.00977 0 0 696 B
#6395 RunOnMethodBegin netcoreapp3.1 894ns 1.81ns 6.76ns 0.00907 0 0 696 B
#6395 RunOnMethodBegin net472 1.03μs 0.784ns 3.04ns 0.104 0 0 658 B

andrewlock
andrewlock approved these changes Dec 4, 2024
View reviewed changes

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to understand why this is flaky 🤔 Can someone from ASM shed light on it?

@link04

link04 commented Dec 4, 2024

Copy link
Copy Markdown
Contributor Author

It would be good to understand why this is flaky 🤔 Can someone from ASM shed light on it?

So speaking with @e-n-0 in Slack and from my understanding on this it could be that the vulnerability is reported very early on this tests which could be happening before agent config is fetched so instead it falls back to the Json.

e-n-0
e-n-0 approved these changes Dec 4, 2024
View reviewed changes

@e-n-0 e-n-0 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR 😄

In the previous merged PR, I added meta struct support to iast. The vulnerability data is stored in meta struct if the config of the agent say it's compatible, so we need it.
As mentioned by Maximo, as the vulnerability is triggered at the start of the application, the agent config might not be loaded yet by the tracer, thus fallbacking to the json serialization (and the missing metastruct part from snapshot).

e-n-0
e-n-0 approved these changes Dec 4, 2024 •
edited
Loading
View reviewed changes
@link04
link04 merged commit 26acac5 into master Dec 4, 2024
@link04
link04 deleted the maximo/update-iast-timeoutvul-test branch December 4, 2024 18:27
@github-actions github-actions Bot added this to the vNext-v3 milestone Dec 4, 2024
@link04 link04 mentioned this pull request Dec 5, 2024
Merged
link04 added a commit that referenced this pull request Dec 5, 2024
@link04
Removing MetaStruct Check from TestDirectoryListingLeak (#6404)
af0bf3e 
## Summary of changes
Same as #6395 this test
is part of the same family and I just noticed it also sometimes flakes
without the expected snapshot.
veerbia pushed a commit that referenced this pull request Dec 16, 2024
@link04 @veerbia
Removing MetaStruct check from TestSessionTimeoutVulnerability (#6395)
3fee433 
## Summary of changes
Disabling SpanMetaStructs from Agent used in related test and snapshot
file.

## Reason for change
After recent PR got merged test started flaking in master, but since the
feature is already tested somewhere else as expected removing from this
test to prevent flakes.

## Implementation details
Setting:
```await TryStartApp(newFixture, new MockTracerAgent.AgentConfiguration { SpanMetaStructs = false });```

Removing the below from the received snapshot:
```    
},
    MetaStruct: {
      iast: 
```
## Test coverage
The test.
veerbia pushed a commit that referenced this pull request Dec 16, 2024
@link04 @veerbia
Removing MetaStruct Check from TestDirectoryListingLeak (#6404)
ee3ad4b 
## Summary of changes
Same as #6395 this test
is part of the same family and I just noticed it also sometimes flakes
without the expected snapshot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@e-n-0 e-n-0 e-n-0 approved these changes
@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:tests unit tests, integration tests

Projects

None yet

Milestone

3.7.0

Development

Successfully merging this pull request may close these issues.

3 participants

@link04 @andrewlock @e-n-0