[ASM] Appsec events in meta struct#5779
Merged
Merged
Conversation
Member
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5779) - mean (76ms) : 67, 84
. : milestone, 76,
master - mean (75ms) : 64, 87
. : milestone, 75,
section CallTarget+Inlining+NGEN
This PR (5779) - mean (1,047ms) : 1024, 1070
. : milestone, 1047,
master - mean (1,048ms) : 1030, 1067
. : milestone, 1048,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5779) - mean (112ms) : 109, 115
. : milestone, 112,
master - mean (112ms) : 108, 116
. : milestone, 112,
section CallTarget+Inlining+NGEN
This PR (5779) - mean (737ms) : 713, 762
. : milestone, 737,
master - mean (735ms) : 713, 756
. : milestone, 735,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5779) - mean (93ms) : 91, 96
. : milestone, 93,
master - mean (95ms) : 92, 98
. : milestone, 95,
section CallTarget+Inlining+NGEN
This PR (5779) - mean (684ms) : 661, 708
. : milestone, 684,
master - mean (688ms) : 661, 716
. : milestone, 688,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5779) - mean (192ms) : 189, 196
. : milestone, 192,
master - mean (192ms) : 187, 196
. : milestone, 192,
section CallTarget+Inlining+NGEN
This PR (5779) - mean (1,156ms) : 1136, 1177
. : milestone, 1156,
master - mean (1,160ms) : 1140, 1180
. : milestone, 1160,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5779) - mean (277ms) : 274, 281
. : milestone, 277,
master - mean (277ms) : 273, 281
. : milestone, 277,
section CallTarget+Inlining+NGEN
This PR (5779) - mean (939ms) : 917, 962
. : milestone, 939,
master - mean (949ms) : 930, 967
. : milestone, 949,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5779) - mean (265ms) : 259, 271
. : milestone, 265,
master - mean (265ms) : 260, 269
. : milestone, 265,
section CallTarget+Inlining+NGEN
This PR (5779) - mean (914ms) : 891, 937
. : milestone, 914,
master - mean (922ms) : 902, 941
. : milestone, 922,
|
Datadog ReportBranch report: ✅ 0 Failed, 349036 Passed, 1752 Skipped, 15h 11m 26.68s Total Time |
Member
Benchmarks Report for tracer 🐌Benchmarks for #5779 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️Raw results
|
Member
Benchmarks Report for appsec 🐌Benchmarks for #5779 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations
|
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 | 59.14 KB | 60.5 KB | 1.35 KB | 2.29% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 | 254.48 KB | 256.98 KB | 2.5 KB | 0.98% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StringConcatBenchmark |
net6.0 | 58.9μs | 660ns | 6.57μs | 0 | 0 | 0 | 43.44 KB |
| master | StringConcatBenchmark |
netcoreapp3.1 | 67.4μs | 921ns | 9.02μs | 0 | 0 | 0 | 42.64 KB |
| master | StringConcatBenchmark |
net472 | 37.5μs | 70.1ns | 262ns | 0 | 0 | 0 | 59.14 KB |
| master | StringConcatAspectBenchmark |
net6.0 | 302μs | 1.58μs | 10.1μs | 0 | 0 | 0 | 254.48 KB |
| master | StringConcatAspectBenchmark |
netcoreapp3.1 | 340μs | 1.76μs | 11.4μs | 0 | 0 | 0 | 264.91 KB |
| master | StringConcatAspectBenchmark |
net472 | 303μs | 7.78μs | 75.1μs | 0 | 0 | 0 | 278.53 KB |
| #5779 | StringConcatBenchmark |
net6.0 | 53.7μs | 293ns | 1.74μs | 0 | 0 | 0 | 43.44 KB |
| #5779 | StringConcatBenchmark |
netcoreapp3.1 | 60μs | 688ns | 6.74μs | 0 | 0 | 0 | 42.64 KB |
| #5779 | StringConcatBenchmark |
net472 | 38.9μs | 203ns | 1.04μs | 0 | 0 | 0 | 60.5 KB |
| #5779 | StringConcatAspectBenchmark |
net6.0 | 313μs | 1.68μs | 11.6μs | 0 | 0 | 0 | 256.98 KB |
| #5779 | StringConcatAspectBenchmark |
netcoreapp3.1 | 348μs | 1.86μs | 11.6μs | 0 | 0 | 0 | 263.6 KB |
| #5779 | StringConcatAspectBenchmark |
net472 | 289μs | 6.6μs | 63μs | 0 | 0 | 0 | 278.53 KB |
e-n-0
force-pushed
the
flavien/asm/meta_struct_triggers
branch
from
July 9, 2024 16:00
fc5a80e to
61584be
Compare
e-n-0
force-pushed
the
flavien/asm/meta_struct_triggers
branch
from
July 11, 2024 16:39
61584be to
0bfd1fb
Compare
Contributor
Snapshots difference summaryThe following differences have been observed in committed snapshots. It is meant to help the reviewer. 8 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"rasp-001-001","name":"Path traversal attack","tags":{"category":"vulnerability_trigger","type":"lfi"}},"rule_matches":[{"operator":"lfi_detector","operator_value":"","parameters":[{"address":null,"highlight":["/etc/password"],"key_path":null,"value":null}]}],"span_id": XXX}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"rasp-001-001","name":"Path traversal attack","tags":{"category":"vulnerability_trigger","type":"lfi"}},"rule_matches":[{"operator":"lfi_detector","operator_value":"","parameters":[{"address":null,"highlight":["/etc/password"],"key_path":null,"value":null}]}],"span_id": XXX}]},
12 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"rasp-002-001","name":"Server-side request forgery","tags":{"category":"vulnerability_trigger","type":"ssrf"}},"rule_matches":[{"operator":"ssrf_detector","operator_value":"","parameters":[{"address":null,"highlight":["127.0.0.1"],"key_path":null,"value":null}]}],"span_id": XXX}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"rasp-002-001","name":"Server-side request forgery","tags":{"category":"vulnerability_trigger","type":"ssrf"}},"rule_matches":[{"operator":"ssrf_detector","operator_value":"","parameters":[{"address":null,"highlight":["127.0.0.1"],"key_path":null,"value":null}]}],"span_id": XXX}]},
8 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"rasp-942-100","name":"SQL injection exploit","tags":{"category":"vulnerability_trigger","type":"sql_injection"}},"rule_matches":[{"operator":"sqli_detector","operator_value":"","parameters":[{"address":null,"highlight":["' or '1'='1"],"key_path":null,"value":null}]}],"span_id": XXX}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"rasp-942-100","name":"SQL injection exploit","tags":{"category":"vulnerability_trigger","type":"sql_injection"}},"rule_matches":[{"operator":"sqli_detector","operator_value":"","parameters":[{"address":null,"highlight":["' or '1'='1"],"key_path":null,"value":null}]}],"span_id": XXX}]},
4 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-932-160","name":"Remote Command Execution: Unix Shell Code Found","tags":{"category":"attack_attempt","type":"command_injection"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dev/zero"],"key_path":["Property"],"value":"dev/zero"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-932-160","name":"Remote Command Execution: Unix Shell Code Found","tags":{"category":"attack_attempt","type":"command_injection"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dev/zero"],"key_path":["Property"],"value":"dev/zero"}]}]}]},
40 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"ublock","name":"Hello","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"match_regex","operator_value":"hello","parameters":[{"address":"server.request.headers.no_cookies","highlight":["hello"],"key_path":["user-agent","0"],"value":"mistake not... hello/v"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"ublock","name":"Hello","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"match_regex","operator_value":"hello","parameters":[{"address":"server.request.headers.no_cookies","highlight":["hello"],"key_path":["user-agent","0"],"value":"mistake not... hello/v"}]}]}]},
10 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"nfd-000-001","name":"Detect common directory discovery scans","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"match_regex","operator_value":"^404$","parameters":[{"address":"server.response.status","highlight":["404"],"key_path":[],"value":"404"}]},{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.uri.raw","highlight":["/login.php"],"key_path":[],"value":"/health/login.php"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"nfd-000-001","name":"Detect common directory discovery scans","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"match_regex","operator_value":"^404$","parameters":[{"address":"server.response.status","highlight":["404"],"key_path":[],"value":"404"}]},{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.uri.raw","highlight":["/login.php"],"key_path":[],"value":"/health/login.php"}]}]}]},
10 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"test-dummy-rule","name":"Dummy rule to test blocking","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dummy_rule"],"key_path":["Property2"],"value":"dummy_rule"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"test-dummy-rule","name":"Dummy rule to test blocking","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dummy_rule"],"key_path":["Property2"],"value":"dummy_rule"}]}]}]},
20 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"test-dummy-rule","name":"Dummy rule to test blocking","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dummy_rule"],"key_path":["Property"],"value":"dummy_rule"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"test-dummy-rule","name":"Dummy rule to test blocking","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dummy_rule"],"key_path":["Property"],"value":"dummy_rule"}]}]}]},
60 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["id"],"value":"appscan_fingerprint"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["id"],"value":"appscan_fingerprint"}]}]}]},
31 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[?\\$(?:(?:s(?:lic|iz)|wher)e|e(?:lemMatch|xists|q)|n(?:o[rt]|in?|e)|l(?:ike|te?)|t(?:ext|ype)|a(?:ll|nd)|jsonSchema|between|regex|x?or|div|mod)\\]?)\\b)","parameters":[{"address":"server.request.query","highlight":["[$slice"],"key_path":["[$slice]"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[?\\$(?:(?:s(?:lic|iz)|wher)e|e(?:lemMatch|xists|q)|n(?:o[rt]|in?|e)|l(?:ike|te?)|t(?:ext|ype)|a(?:ll|nd)|jsonSchema|between|regex|x?or|div|mod)\\]?)\\b)","parameters":[{"address":"server.request.query","highlight":["[$slice"],"key_path":["[$slice]"],"value":"[$slice]"}]}]}]},
10 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[?\\$(?:(?:s(?:lic|iz)|wher)e|e(?:lemMatch|xists|q)|n(?:o[rt]|in?|e)|l(?:ike|te?)|t(?:ext|ype)|a(?:ll|nd)|jsonSchema|between|regex|x?or|div|mod)\\]?)\\b)","parameters":[{"address":"server.request.query","highlight":["[$slice"],"key_path":["[$slice]"],"value":"[$slice]"}]}]},{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["id"],"value":"appscan_fingerprint"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[?\\$(?:(?:s(?:lic|iz)|wher)e|e(?:lemMatch|xists|q)|n(?:o[rt]|in?|e)|l(?:ike|te?)|t(?:ext|ype)|a(?:ll|nd)|jsonSchema|between|regex|x?or|div|mod)\\]?)\\b)","parameters":[{"address":"server.request.query","highlight":["[$slice"],"key_path":["[$slice]"],"value":"[$slice]"}]}]},{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["id"],"value":"appscan_fingerprint"}]}]}]},
10 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"tst-037-011","name":"No fun","tags":{"category":"attack_attempt","type":"lfi"}},"rule_matches":[{"operator":"match_regex","operator_value":"fun","parameters":[{"address":"server.request.uri.raw","highlight":["fun"],"key_path":[],"value":"/health?q=fun"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"tst-037-011","name":"No fun","tags":{"category":"attack_attempt","type":"lfi"}},"rule_matches":[{"operator":"match_regex","operator_value":"fun","parameters":[{"address":"server.request.uri.raw","highlight":["fun"],"key_path":[],"value":"/health?q=fun"}]}]}]},
20 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"tst-037-009","name":"Test block on response header","tags":{"category":"attack_attempt","type":"lfi"}},"rule_matches":[{"operator":"match_regex","operator_value":"en-us|krypton","parameters":[{"address":"server.response.headers.no_cookies","highlight":["krypton"],"key_path":["content-language","0"],"value":"krypton"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"tst-037-009","name":"Test block on response header","tags":{"category":"attack_attempt","type":"lfi"}},"rule_matches":[{"operator":"match_regex","operator_value":"en-us|krypton","parameters":[{"address":"server.response.headers.no_cookies","highlight":["krypton"],"key_path":["content-language","0"],"value":"krypton"}]}]}]},
10 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"tst-037-010","name":"No teapot","tags":{"category":"attack_attempt","type":"lfi"}},"rule_matches":[{"operator":"match_regex","operator_value":"418","parameters":[{"address":"server.response.status","highlight":["418"],"key_path":[],"value":"418"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"tst-037-010","name":"No teapot","tags":{"category":"attack_attempt","type":"lfi"}},"rule_matches":[{"operator":"match_regex","operator_value":"418","parameters":[{"address":"server.response.status","highlight":["418"],"key_path":[],"value":"418"}]}]}]},
5 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["s"],"value":"appscan_fingerprint"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["s"],"value":"appscan_fingerprint"}]}]}]},
4 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"test-dummy-rule-custom-block","name":"Dummy rule to test blocking with a custom action","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.query","highlight":["dummy_custom_action"],"key_path":["arg","0"],"value":"dummy_custom_action"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"test-dummy-rule-custom-block","name":"Dummy rule to test blocking with a custom action","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.query","highlight":["dummy_custom_action"],"key_path":["arg","0"],"value":"dummy_custom_action"}]}]}]},
12 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"test-dummy-rule","name":"Dummy rule to test blocking","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.query","highlight":["dummy_rule"],"key_path":["arg","0"],"value":"dummy_rule"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"test-dummy-rule","name":"Dummy rule to test blocking","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.query","highlight":["dummy_rule"],"key_path":["arg","0"],"value":"dummy_rule"}]}]}]},
1 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"test_custom_rule","name":"Test custom rule","tags":{"category":"attack_attempt","type":"custom_rule"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.query","highlight":["customrule"],"key_path":["arg","0"],"value":"customrule_trigger"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"test_custom_rule","name":"Test custom rule","tags":{"category":"attack_attempt","type":"custom_rule"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.query","highlight":["customrule"],"key_path":["arg","0"],"value":"customrule_trigger"}]}]}]},
9 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"blk-001-001","name":"Block IP Addresses","tags":{"category":"security_response","type":"block_ip"}},"rule_matches":[{"operator":"ip_match","operator_value":"","parameters":[{"address":"http.client_ip","highlight":["86.242.244.246"],"key_path":[],"value":"86.242.244.246"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"blk-001-001","name":"Block IP Addresses","tags":{"category":"security_response","type":"block_ip"}},"rule_matches":[{"operator":"ip_match","operator_value":"","parameters":[{"address":"http.client_ip","highlight":["86.242.244.246"],"key_path":[],"value":"86.242.244.246"}]}]}]},
7 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"blk-001-002","name":"Block User Addresses","tags":{"category":"security_response","type":"block_user"}},"rule_matches":[{"operator":"exact_match","operator_value":"","parameters":[{"address":"usr.id","highlight":["user3"],"key_path":[],"value":"user3"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"blk-001-002","name":"Block User Addresses","tags":{"category":"security_response","type":"block_user"}},"rule_matches":[{"operator":"exact_match","operator_value":"","parameters":[{"address":"usr.id","highlight":["user3"],"key_path":[],"value":"user3"}]}]}]},
1 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["[$slice]"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["[$slice]"],"value":"[$slice]"}]}]}]},
2 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290-new","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["[$slice]"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290-new","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["[$slice]"],"value":"[$slice]"}]}]}]},
55 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["arg","0"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["arg","0"],"value":"[$slice]"}]}]}]},
15 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["param","0"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":["param","0"],"value":"[$slice]"}]}]}]},
4 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-932-160","name":"Remote Command Execution: Unix Shell Code Found","tags":{"category":"attack_attempt","type":"command_injection"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dev/zero"],"key_path":["model","Dog2"],"value":"dev/zero"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-932-160","name":"Remote Command Execution: Unix Shell Code Found","tags":{"category":"attack_attempt","type":"command_injection"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.body","highlight":["dev/zero"],"key_path":["model","Dog2"],"value":"dev/zero"}]}]}]},
10 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.body","highlight":["[$slice]"],"key_path":["miscModel","DictionaryProperty","a"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.body","highlight":["[$slice]"],"key_path":["miscModel","DictionaryProperty","a"],"value":"[$slice]"}]}]}]},
35 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.body","highlight":["[$slice]"],"key_path":["miscModel","Property1"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.body","highlight":["[$slice]"],"key_path":["miscModel","Property1"],"value":"[$slice]"}]}]}]},
5 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"nfd-000-001","name":"Detect common directory discovery scans","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"match_regex","operator_value":"^404$","parameters":[{"address":"server.response.status","highlight":["404"],"key_path":[],"value":"404"}]},{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.uri.raw","highlight":["/wp-config"],"key_path":[],"value":"/health/wp-config"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"nfd-000-001","name":"Detect common directory discovery scans","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"match_regex","operator_value":"^404$","parameters":[{"address":"server.response.status","highlight":["404"],"key_path":[],"value":"404"}]},{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.uri.raw","highlight":["/wp-config"],"key_path":[],"value":"/health/wp-config"}]}]}]},
6 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"ua0-600-16x","name":"SQL power injector","tags":{"category":"attack_attempt","type":"attack_tool"}},"rule_matches":[{"operator":"match_regex","operator_value":"sql power injector","parameters":[{"address":"server.request.headers.no_cookies","highlight":["sql power injector"],"key_path":["user-agent","0"],"value":"Mistake Not... (sql power injector)"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"ua0-600-16x","name":"SQL power injector","tags":{"category":"attack_attempt","type":"attack_tool"}},"rule_matches":[{"operator":"match_regex","operator_value":"sql power injector","parameters":[{"address":"server.request.headers.no_cookies","highlight":["sql power injector"],"key_path":["user-agent","0"],"value":"Mistake Not... (sql power injector)"}]}]}]},
4 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["pathparam2"],"value":"appscan_fingerprint"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-913-120","name":"Known security scanner filename/argument","tags":{"category":"attack_attempt","type":"security_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.path_params","highlight":["appscan_fingerprint"],"key_path":["pathparam2"],"value":"appscan_fingerprint"}]}]}]},
5 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":[0,1],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.query","highlight":["[$slice]"],"key_path":[0,1],"value":"[$slice]"}]}]}]},
10 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.body","highlight":["[$slice]"],"key_path":["ctl00$MainContent$testBox"],"value":"[$slice]"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-942-290","name":"Finds basic MongoDB SQL injection attempts","tags":{"category":"attack_attempt","type":"nosql_injection"}},"rule_matches":[{"operator":"match_regex","operator_value":"(?i:(?:\\[\\$(?:ne|eq|lte?|gte?|n?in|mod|all|size|exists|type|slice|x?or|div|like|between|and)\\]))","parameters":[{"address":"server.request.body","highlight":["[$slice]"],"key_path":["ctl00$MainContent$testBox"],"value":"[$slice]"}]}]}]},
3 occurrences of : - _dd.appsec.json: {"triggers":[{"rule":{"id":"crs-913-110","name":"Acunetix","tags":{"category":"attack_attempt","type":"commercial_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.headers.no_cookies","highlight":["acunetix-product"],"key_path":["user-agent","0"],"value":"mistake not... acunetix-product"}]}]}]},
+ _dd.appsec.meta-struct.test: {"triggers":[{"rule":{"id":"crs-913-110","name":"Acunetix","tags":{"category":"attack_attempt","type":"commercial_scanner"}},"rule_matches":[{"operator":"phrase_match","operator_value":"","parameters":[{"address":"server.request.headers.no_cookies","highlight":["acunetix-product"],"key_path":["user-agent","0"],"value":"mistake not... acunetix-product"}]}]}]},
|
e-n-0
force-pushed
the
flavien/asm/meta_struct_triggers
branch
3 times, most recently
from
July 15, 2024 12:29
312e670 to
dd3a2cf
Compare
e-n-0
force-pushed
the
flavien/asm/meta_struct_triggers
branch
from
July 15, 2024 15:54
dd3a2cf to
252f1f8
Compare
e-n-0
marked this pull request as ready for review
July 15, 2024 16:02
Member
Throughput/Crank Report:zap:Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5779) (11.499M) : 0, 11498508
master (11.579M) : 0, 11579356
benchmarks/2.9.0 (11.793M) : 0, 11792634
section Automatic
This PR (5779) (7.746M) : 0, 7745797
master (7.775M) : 0, 7774636
benchmarks/2.9.0 (8.209M) : 0, 8209122
section Trace stats
master (8.206M) : 0, 8205782
section Manual
This PR (5779) (10.030M) : 0, 10029958
master (10.090M) : 0, 10090249
section Manual + Automatic
This PR (5779) (7.316M) : 0, 7315588
master (7.416M) : 0, 7415620
section Version Conflict
master (6.677M) : 0, 6676535
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5779) (9.629M) : 0, 9629118
section Automatic
This PR (5779) (6.451M) : 0, 6450673
section Manual
This PR (5779) (8.356M) : 0, 8356047
section Manual + Automatic
This PR (5779) (6.273M) : 0, 6273305
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5779) (10.115M) : 0, 10115417
master (10.110M) : 0, 10109641
benchmarks/2.9.0 (10.296M) : 0, 10295769
section Automatic
This PR (5779) (7.116M) : 0, 7115999
master (7.224M) : 0, 7223553
benchmarks/2.9.0 (7.267M) : 0, 7266503
section Trace stats
master (7.632M) : 0, 7631517
section Manual
This PR (5779) (8.708M) : 0, 8707786
master (8.931M) : 0, 8930758
section Manual + Automatic
This PR (5779) (6.742M) : 0, 6741748
master (6.943M) : 0, 6943118
section Version Conflict
master (6.220M) : 0, 6220018
|
GreenMatan
approved these changes
Jul 16, 2024
GreenMatan
left a comment
GreenMatan
left a comment
Contributor
There was a problem hiding this comment.
Reviewed Debugger's side of things, looking good
| // Fallback to the _dd.appsec.json tag | ||
| if (Security.Instance.IsMetaStructSupported()) | ||
| { | ||
| span.SetMetaStruct(AppsecKey, MetaStructHelper.ObjectToByteArray(new Dictionary<string, List<object>> { { "triggers", _wafSecurityEvents } })); |
Member
There was a problem hiding this comment.
Not something you want to do in this PR, but maybe in the future we should probably try to move this MessagePack/JSON serialization into the background thread we already use to serialize spans, instead of blocking the web request.
Member
Author
There was a problem hiding this comment.
Yep that's will be even better. It will be part of another PR 😄
dromanol
reviewed
Jul 16, 2024
lucaspimentel
approved these changes
Jul 16, 2024
robertpi
approved these changes
Jul 17, 2024
7 tasks
e-n-0
added a commit
that referenced
this pull request
Dec 3, 2024
## Summary of changes Move iast data from the `_dd.iast.json` tag to a messagepack-encoded data serialized into meta struct. [RFC Linked to that change.](https://docs.google.com/document/d/1iWQsOfT6Lg_IFyvQeqry9wVmXOE2Yav0X4MgOTk7mks/) - Added a new telemetry tag `_dd.iast.metastruct.size.exceeded` to reflect the exceeded size for meta struct when a truncation occured ## Reason for change Better performance. ## Implementation details - Check for meta struct compatibility with the connected agent - Fallback to json serialization if not supported ## Test coverage - By default, now the mocked trace agent is set to support meta struct (cf #5779), so all iast events will be set in meta struct - To keep all existing snapshots unchanged, meta struct iast events are set back as json on the same tag into snapshots - Add a new option in the iast test: "forceMetaStruct". When enabled, the tag `_dd.iast.json.metastruct.test` is set when iast data from metastruct is serialized back to json. This is to be sure that meta struct is used in the test. - Updated all unit tests of Vulnerability Batch to also use the meta struct serialization - New test for the telemetry tag `_dd.iast.metastruct.size.exceeded` ## Other details <!-- Fixes #{issue} --> <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
veerbia
pushed a commit
that referenced
this pull request
Dec 16, 2024
## Summary of changes Move iast data from the `_dd.iast.json` tag to a messagepack-encoded data serialized into meta struct. [RFC Linked to that change.](https://docs.google.com/document/d/1iWQsOfT6Lg_IFyvQeqry9wVmXOE2Yav0X4MgOTk7mks/) - Added a new telemetry tag `_dd.iast.metastruct.size.exceeded` to reflect the exceeded size for meta struct when a truncation occured ## Reason for change Better performance. ## Implementation details - Check for meta struct compatibility with the connected agent - Fallback to json serialization if not supported ## Test coverage - By default, now the mocked trace agent is set to support meta struct (cf #5779), so all iast events will be set in meta struct - To keep all existing snapshots unchanged, meta struct iast events are set back as json on the same tag into snapshots - Add a new option in the iast test: "forceMetaStruct". When enabled, the tag `_dd.iast.json.metastruct.test` is set when iast data from metastruct is serialized back to json. This is to be sure that meta struct is used in the test. - Updated all unit tests of Vulnerability Batch to also use the meta struct serialization - New test for the telemetry tag `_dd.iast.metastruct.size.exceeded` ## Other details <!-- Fixes #{issue} --> <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Move appsec event data from the
_dd.appsec.jsontag to a messagepack-encoded data serialized into meta struct.RFC Linked to that change.
Implementation details
Reason for change
Better performance.
Test coverage
_dd.appsec.json.metastruct.testis set when appsec events from metastruct are serialized back to json. This is to be sure that meta struct is used in the test.