Skip to content

Upgrade to AppSec rules v1.11.0#6754

Merged
smola merged 1 commit into
masterfrom
smola/appsec-rules-1.11.0
Mar 5, 2024
Merged

Upgrade to AppSec rules v1.11.0#6754
smola merged 1 commit into
masterfrom
smola/appsec-rules-1.11.0

Conversation

@smola

@smola smola commented Feb 28, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

What Does This Do

Upgrade to the latest Application Security rules.

Additional Notes

The new rules produce a small fixed increase in startup time for users with DD_APPSEC_ENABLED=true. This is 2-4ms in our benchmarks. We will try to improve this in future versions.

@smola smola added the comp: asm waf Application Security Management (WAF) label Feb 28, 2024
@pr-commenter

pr-commenter Bot commented Feb 28, 2024 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/appsec-rules-1.11.0
git_commit_date 1709558898 1709631287
git_commit_sha aeb7876 edc0cec
release_version 1.31.0~aeb7876f22 1.32.0-SNAPSHOT~edc0ceceed
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1709634472 1709634472
ci_job_id 450680068 450680068
ci_pipeline_id 29490470 29490470
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 49 metrics, 13 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:appsec:AppSec worse
[+4.028ms; +5.285ms] or [+2.663%; +3.493%]		 155.958ms 151.301ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1082529
Total [baseline] (9.158 s) : 0, 9157910
Agent [candidate] (1.093 s) : 0, 1092590
Total [candidate] (9.201 s) : 0, 9200707
section appsec
Agent [baseline] (1.198 s) : 0, 1198446
Total [baseline] (9.337 s) : 0, 9336610
Agent [candidate] (1.203 s) : 0, 1203114
Total [candidate] (9.297 s) : 0, 9297409
section iast
Agent [baseline] (1.204 s) : 0, 1204246
Total [baseline] (9.299 s) : 0, 9299252
Agent [candidate] (1.207 s) : 0, 1206566
Total [candidate] (9.29 s) : 0, 9289924
section profiling
Agent [baseline] (1.272 s) : 0, 1272260
Total [baseline] (9.333 s) : 0, 9333110
Agent [candidate] (1.284 s) : 0, 1283888
Total [candidate] (9.354 s) : 0, 9354089
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.198 s 115.918 ms (10.7%)
Agent iast 1.204 s 121.717 ms (11.2%)
Agent profiling 1.272 s 189.731 ms (17.5%)
Total tracing 9.158 s -
Total appsec 9.337 s 178.7 ms (2.0%)
Total iast 9.299 s 141.342 ms (1.5%)
Total profiling 9.333 s 175.199 ms (1.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.093 s -
Agent appsec 1.203 s 110.525 ms (10.1%)
Agent iast 1.207 s 113.976 ms (10.4%)
Agent profiling 1.284 s 191.298 ms (17.5%)
Total tracing 9.201 s -
Total appsec 9.297 s 96.702 ms (1.1%)
Total iast 9.29 s 89.217 ms (1.0%)
Total profiling 9.354 s 153.382 ms (1.7%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (696.688 ms) : 0, 696688
BytebuddyAgent [candidate] (703.408 ms) : 0, 703408
GlobalTracer [baseline] (291.378 ms) : 0, 291378
GlobalTracer [candidate] (294.886 ms) : 0, 294886
AppSec [baseline] (51.554 ms) : 0, 51554
AppSec [candidate] (51.04 ms) : 0, 51040
Remote Config [baseline] (746.106 µs) : 0, 746
Remote Config [candidate] (724.879 µs) : 0, 725
Telemetry [baseline] (7.77 ms) : 0, 7770
Telemetry [candidate] (7.839 ms) : 0, 7839
section appsec
BytebuddyAgent [baseline] (696.305 ms) : 0, 696305
BytebuddyAgent [candidate] (696.108 ms) : 0, 696108
GlobalTracer [baseline] (291.259 ms) : 0, 291259
GlobalTracer [candidate] (291.431 ms) : 0, 291431
AppSec [baseline] (151.301 ms) : 0, 151301
AppSec [candidate] (155.958 ms) : 0, 155958
Remote Config [baseline] (607.147 µs) : 0, 607
Remote Config [candidate] (606.682 µs) : 0, 607
Telemetry [baseline] (6.869 ms) : 0, 6869
Telemetry [candidate] (6.867 ms) : 0, 6867
IAST [baseline] (17.823 ms) : 0, 17823
IAST [candidate] (17.774 ms) : 0, 17774
section iast
BytebuddyAgent [baseline] (799.012 ms) : 0, 799012
BytebuddyAgent [candidate] (799.995 ms) : 0, 799995
GlobalTracer [baseline] (287.546 ms) : 0, 287546
GlobalTracer [candidate] (288.766 ms) : 0, 288766
AppSec [baseline] (54.284 ms) : 0, 54284
AppSec [candidate] (50.843 ms) : 0, 50843
Remote Config [baseline] (587.088 µs) : 0, 587
Remote Config [candidate] (587.507 µs) : 0, 588
Telemetry [baseline] (6.678 ms) : 0, 6678
Telemetry [candidate] (6.689 ms) : 0, 6689
IAST [baseline] (22.0 ms) : 0, 22000
IAST [candidate] (25.341 ms) : 0, 25341
section profiling
BytebuddyAgent [baseline] (688.227 ms) : 0, 688227
BytebuddyAgent [candidate] (694.378 ms) : 0, 694378
GlobalTracer [baseline] (373.962 ms) : 0, 373962
GlobalTracer [candidate] (377.484 ms) : 0, 377484
AppSec [baseline] (52.691 ms) : 0, 52691
AppSec [candidate] (53.259 ms) : 0, 53259
Remote Config [baseline] (777.59 µs) : 0, 778
Remote Config [candidate] (798.42 µs) : 0, 798
Telemetry [baseline] (7.444 ms) : 0, 7444
Telemetry [candidate] (7.425 ms) : 0, 7425
ProfilingAgent [baseline] (93.133 ms) : 0, 93133
ProfilingAgent [candidate] (93.993 ms) : 0, 93993
Profiling [baseline] (93.156 ms) : 0, 93156
Profiling [candidate] (94.017 ms) : 0, 94017
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.099 s) : 0, 1099344
Total [baseline] (8.637 s) : 0, 8636555
Agent [candidate] (1.083 s) : 0, 1083009
Total [candidate] (8.57 s) : 0, 8570468
section iast
Agent [baseline] (1.222 s) : 0, 1221527
Total [baseline] (9.076 s) : 0, 9076255
Agent [candidate] (1.209 s) : 0, 1209001
Total [candidate] (9.079 s) : 0, 9078707
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.202 s) : 0, 1201766
Total [baseline] (9.036 s) : 0, 9036197
Agent [candidate] (1.209 s) : 0, 1208951
Total [candidate] (9.028 s) : 0, 9028106
section iast_TELEMETRY_OFF
Agent [baseline] (1.208 s) : 0, 1208302
Total [baseline] (9.063 s) : 0, 9062667
Agent [candidate] (1.199 s) : 0, 1198681
Total [candidate] (9.03 s) : 0, 9029713
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.099 s -
Agent iast 1.222 s 122.183 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.202 s 102.423 ms (9.3%)
Agent iast_TELEMETRY_OFF 1.208 s 108.958 ms (9.9%)
Total tracing 8.637 s -
Total iast 9.076 s 439.701 ms (5.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.036 s 399.642 ms (4.6%)
Total iast_TELEMETRY_OFF 9.063 s 426.113 ms (4.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent iast 1.209 s 125.992 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.209 s 125.942 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.199 s 115.673 ms (10.7%)
Total tracing 8.57 s -
Total iast 9.079 s 508.238 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.028 s 457.638 ms (5.3%)
Total iast_TELEMETRY_OFF 9.03 s 459.244 ms (5.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (708.157 ms) : 0, 708157
BytebuddyAgent [candidate] (697.103 ms) : 0, 697103
GlobalTracer [baseline] (295.734 ms) : 0, 295734
GlobalTracer [candidate] (292.258 ms) : 0, 292258
AppSec [baseline] (51.993 ms) : 0, 51993
AppSec [candidate] (50.985 ms) : 0, 50985
Remote Config [baseline] (736.312 µs) : 0, 736
Remote Config [candidate] (737.98 µs) : 0, 738
Telemetry [baseline] (7.79 ms) : 0, 7790
Telemetry [candidate] (7.592 ms) : 0, 7592
section iast
BytebuddyAgent [baseline] (810.846 ms) : 0, 810846
BytebuddyAgent [candidate] (801.361 ms) : 0, 801361
GlobalTracer [baseline] (291.585 ms) : 0, 291585
GlobalTracer [candidate] (288.865 ms) : 0, 288865
AppSec [baseline] (52.477 ms) : 0, 52477
AppSec [candidate] (52.329 ms) : 0, 52329
IAST [baseline] (24.75 ms) : 0, 24750
IAST [candidate] (24.047 ms) : 0, 24047
Remote Config [baseline] (584.784 µs) : 0, 585
Remote Config [candidate] (591.18 µs) : 0, 591
Telemetry [baseline] (6.715 ms) : 0, 6715
Telemetry [candidate] (7.449 ms) : 0, 7449
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (796.986 ms) : 0, 796986
BytebuddyAgent [candidate] (802.206 ms) : 0, 802206
GlobalTracer [baseline] (287.524 ms) : 0, 287524
GlobalTracer [candidate] (288.781 ms) : 0, 288781
AppSec [baseline] (50.888 ms) : 0, 50888
AppSec [candidate] (51.258 ms) : 0, 51258
IAST [baseline] (23.978 ms) : 0, 23978
IAST [candidate] (24.413 ms) : 0, 24413
Remote Config [baseline] (592.976 µs) : 0, 593
Remote Config [candidate] (578.095 µs) : 0, 578
Telemetry [baseline] (7.516 ms) : 0, 7516
Telemetry [candidate] (7.399 ms) : 0, 7399
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.48 ms) : 0, 799480
BytebuddyAgent [candidate] (792.887 ms) : 0, 792887
GlobalTracer [baseline] (291.278 ms) : 0, 291278
GlobalTracer [candidate] (289.849 ms) : 0, 289849
AppSec [baseline] (54.043 ms) : 0, 54043
AppSec [candidate] (52.813 ms) : 0, 52813
IAST [baseline] (20.174 ms) : 0, 20174
IAST [candidate] (21.623 ms) : 0, 21623
Remote Config [baseline] (591.682 µs) : 0, 592
Remote Config [candidate] (587.678 µs) : 0, 588
Telemetry [baseline] (8.172 ms) : 0, 8172
Telemetry [candidate] (6.545 ms) : 0, 6545
Loading

Load

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.338 ms) : 1319, 1357
.   : milestone, 1338,
appsec (1.771 ms) : 1748, 1794
.   : milestone, 1771,
iast (1.512 ms) : 1489, 1536
.   : milestone, 1512,
profiling (1.519 ms) : 1495, 1542
.   : milestone, 1519,
tracing (1.497 ms) : 1474, 1520
.   : milestone, 1497,
section candidate
no_agent (1.355 ms) : 1336, 1374
.   : milestone, 1355,
appsec (1.769 ms) : 1745, 1793
.   : milestone, 1769,
iast (1.515 ms) : 1492, 1538
.   : milestone, 1515,
profiling (1.517 ms) : 1493, 1541
.   : milestone, 1517,
tracing (1.499 ms) : 1475, 1523
.   : milestone, 1499,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.338 ms [1.319 ms, 1.357 ms] -
appsec 1.771 ms [1.748 ms, 1.794 ms] 433.109 µs (32.4%)
iast 1.512 ms [1.489 ms, 1.536 ms] 173.95 µs (13.0%)
profiling 1.519 ms [1.495 ms, 1.542 ms] 180.358 µs (13.5%)
tracing 1.497 ms [1.474 ms, 1.52 ms] 158.643 µs (11.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.374 ms] -
appsec 1.769 ms [1.745 ms, 1.793 ms] 414.237 µs (30.6%)
iast 1.515 ms [1.492 ms, 1.538 ms] 159.982 µs (11.8%)
profiling 1.517 ms [1.493 ms, 1.541 ms] 161.985 µs (12.0%)
tracing 1.499 ms [1.475 ms, 1.523 ms] 144.069 µs (10.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22
    dateFormat X
    axisFormat %s
section baseline
no_agent (359.883 µs) : 340, 380
.   : milestone, 360,
iast (466.826 µs) : 447, 487
.   : milestone, 467,
iast_FULL (529.777 µs) : 509, 550
.   : milestone, 530,
iast_GLOBAL (484.918 µs) : 465, 505
.   : milestone, 485,
iast_HARDCODED_SECRET_DISABLED (470.301 µs) : 449, 491
.   : milestone, 470,
iast_INACTIVE (449.107 µs) : 427, 471
.   : milestone, 449,
iast_TELEMETRY_OFF (461.532 µs) : 441, 482
.   : milestone, 462,
tracing (438.866 µs) : 418, 459
.   : milestone, 439,
section candidate
no_agent (359.666 µs) : 339, 380
.   : milestone, 360,
iast (467.508 µs) : 447, 488
.   : milestone, 468,
iast_FULL (533.37 µs) : 513, 554
.   : milestone, 533,
iast_GLOBAL (492.695 µs) : 471, 514
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (475.248 µs) : 455, 496
.   : milestone, 475,
iast_INACTIVE (439.997 µs) : 420, 460
.   : milestone, 440,
iast_TELEMETRY_OFF (470.064 µs) : 449, 491
.   : milestone, 470,
tracing (444.516 µs) : 423, 466
.   : milestone, 445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.883 µs [340.184 µs, 379.582 µs] -
iast 466.826 µs [446.594 µs, 487.058 µs] 106.943 µs (29.7%)
iast_FULL 529.777 µs [509.347 µs, 550.206 µs] 169.894 µs (47.2%)
iast_GLOBAL 484.918 µs [464.657 µs, 505.178 µs] 125.035 µs (34.7%)
iast_HARDCODED_SECRET_DISABLED 470.301 µs [449.463 µs, 491.139 µs] 110.418 µs (30.7%)
iast_INACTIVE 449.107 µs [426.779 µs, 471.435 µs] 89.224 µs (24.8%)
iast_TELEMETRY_OFF 461.532 µs [440.841 µs, 482.222 µs] 101.649 µs (28.2%)
tracing 438.866 µs [418.481 µs, 459.252 µs] 78.983 µs (21.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.666 µs [339.164 µs, 380.168 µs] -
iast 467.508 µs [446.633 µs, 488.382 µs] 107.842 µs (30.0%)
iast_FULL 533.37 µs [512.873 µs, 553.866 µs] 173.704 µs (48.3%)
iast_GLOBAL 492.695 µs [471.373 µs, 514.018 µs] 133.03 µs (37.0%)
iast_HARDCODED_SECRET_DISABLED 475.248 µs [454.961 µs, 495.534 µs] 115.582 µs (32.1%)
iast_INACTIVE 439.997 µs [419.611 µs, 460.383 µs] 80.331 µs (22.3%)
iast_TELEMETRY_OFF 470.064 µs [449.273 µs, 490.856 µs] 110.399 µs (30.7%)
tracing 444.516 µs [423.392 µs, 465.639 µs] 84.85 µs (23.6%)

@smola
smola marked this pull request as ready for review February 28, 2024 17:35
@smola
smola requested a review from a team as a code owner February 28, 2024 17:35
@smola
smola enabled auto-merge (squash) February 28, 2024 17:46
@smola
smola disabled auto-merge February 28, 2024 17:47
@smola
smola force-pushed the smola/appsec-rules-1.11.0 branch from 23518b2 to 25fcf3b Compare March 1, 2024 07:50
@smola
smola force-pushed the smola/appsec-rules-1.11.0 branch from 6c3dbe0 to edc0cec Compare March 5, 2024 09:35
@smola
smola merged commit 7e58261 into master Mar 5, 2024
@smola
smola deleted the smola/appsec-rules-1.11.0 branch March 5, 2024 11:59
@github-actions github-actions Bot added this to the 1.32.0 milestone Mar 5, 2024
@smola smola mentioned this pull request Mar 5, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez manuel-alvarez-alvarez approved these changes
@jandro996 jandro996 jandro996 approved these changes

Assignees

No one assigned

Labels

comp: asm waf Application Security Management (WAF)

Projects

None yet

Milestone

1.32.0

Development

Successfully merging this pull request may close these issues.

3 participants

@smola @manuel-alvarez-alvarez @jandro996