Skip to content

Improve performance while computing IAST metrics#5784

Merged
manuel-alvarez-alvarez merged 1 commit into
masterfrom
malvarez/iast-improve-telemetry-perf
Aug 28, 2023
Merged

Improve performance while computing IAST metrics#5784
manuel-alvarez-alvarez merged 1 commit into
masterfrom
malvarez/iast-improve-telemetry-perf

Conversation

@manuel-alvarez-alvarez

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Aug 25, 2023

Copy link
Copy Markdown
Member

What Does This Do

Improve IAST metrics by:

  • Removing the need to have extra accesses to the current context in the tainted
  • Removing replaceAll method call with a simpler replace

Motivation

Additional Notes

@manuel-alvarez-alvarez manuel-alvarez-alvarez added tag: performance Performance related changes comp: asm iast Application Security Management (IAST) labels Aug 25, 2023
@manuel-alvarez-alvarez
manuel-alvarez-alvarez requested a review from a team as a code owner August 25, 2023 10:54
@pr-commenter

pr-commenter Bot commented Aug 25, 2023 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~72572d1d17 1.20.0-SNAPSHOT~5c1e684acb
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~5c1e684acb, baseline=1.20.0-SNAPSHOT~72572d1d17

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (991.218 ms) : 0, 991218
Total [baseline] (9.179 s) : 0, 9178519
Agent [candidate] (990.858 ms) : 0, 990858
Total [candidate] (9.24 s) : 0, 9240378
section appsec
Agent [baseline] (1.08 s) : 0, 1079757
Total [baseline] (9.331 s) : 0, 9330864
Agent [candidate] (1.074 s) : 0, 1074155
Total [candidate] (9.297 s) : 0, 9297186
section iast
Agent [baseline] (1.09 s) : 0, 1089794
Total [baseline] (9.407 s) : 0, 9407218
Agent [candidate] (1.096 s) : 0, 1095626
Total [candidate] (9.447 s) : 0, 9446609
section profiling
Agent [baseline] (1.156 s) : 0, 1155507
Total [baseline] (9.445 s) : 0, 9444976
Agent [candidate] (1.169 s) : 0, 1169282
Total [candidate] (9.45 s) : 0, 9450187
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 991.218 ms -
Agent appsec 1.08 s 88.54 ms (8.9%)
Agent iast 1.09 s 98.576 ms (9.9%)
Agent profiling 1.156 s 164.29 ms (16.6%)
Total tracing 9.179 s -
Total appsec 9.331 s 152.345 ms (1.7%)
Total iast 9.407 s 228.7 ms (2.5%)
Total profiling 9.445 s 266.458 ms (2.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 990.858 ms -
Agent appsec 1.074 s 83.297 ms (8.4%)
Agent iast 1.096 s 104.768 ms (10.6%)
Agent profiling 1.169 s 178.424 ms (18.0%)
Total tracing 9.24 s -
Total appsec 9.297 s 56.808 ms (0.6%)
Total iast 9.447 s 206.231 ms (2.2%)
Total profiling 9.45 s 209.809 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~5c1e684acb, baseline=1.20.0-SNAPSHOT~72572d1d17

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (616.414 ms) : 0, 616414
BytebuddyAgent [candidate] (614.053 ms) : 0, 614053
GlobalTracer [baseline] (283.958 ms) : 0, 283958
GlobalTracer [candidate] (285.921 ms) : 0, 285921
AppSec [baseline] (49.694 ms) : 0, 49694
AppSec [candidate] (49.803 ms) : 0, 49803
Remote Config [baseline] (660.551 µs) : 0, 661
Remote Config [candidate] (664.3 µs) : 0, 664
Telemetry [baseline] (5.945 ms) : 0, 5945
Telemetry [candidate] (6.019 ms) : 0, 6019
section appsec
BytebuddyAgent [baseline] (617.231 ms) : 0, 617231
BytebuddyAgent [candidate] (612.541 ms) : 0, 612541
GlobalTracer [baseline] (286.205 ms) : 0, 286205
GlobalTracer [candidate] (285.111 ms) : 0, 285111
AppSec [baseline] (135.287 ms) : 0, 135287
AppSec [candidate] (135.732 ms) : 0, 135732
Remote Config [baseline] (730.123 µs) : 0, 730
Remote Config [candidate] (729.404 µs) : 0, 729
Telemetry [baseline] (5.83 ms) : 0, 5830
Telemetry [candidate] (5.74 ms) : 0, 5740
section iast
BytebuddyAgent [baseline] (718.071 ms) : 0, 718071
BytebuddyAgent [candidate] (721.195 ms) : 0, 721195
GlobalTracer [baseline] (270.087 ms) : 0, 270087
GlobalTracer [candidate] (271.689 ms) : 0, 271689
AppSec [baseline] (47.122 ms) : 0, 47122
AppSec [candidate] (47.74 ms) : 0, 47740
Remote Config [baseline] (543.914 µs) : 0, 544
Remote Config [candidate] (553.633 µs) : 0, 554
Telemetry [baseline] (5.716 ms) : 0, 5716
Telemetry [candidate] (5.74 ms) : 0, 5740
IAST [baseline] (13.97 ms) : 0, 13970
IAST [candidate] (14.168 ms) : 0, 14168
section profiling
BytebuddyAgent [baseline] (624.097 ms) : 0, 624097
BytebuddyAgent [candidate] (630.083 ms) : 0, 630083
GlobalTracer [baseline] (346.698 ms) : 0, 346698
GlobalTracer [candidate] (350.58 ms) : 0, 350580
AppSec [baseline] (48.968 ms) : 0, 48968
AppSec [candidate] (49.335 ms) : 0, 49335
Remote Config [baseline] (638.921 µs) : 0, 639
Remote Config [candidate] (655.147 µs) : 0, 655
Telemetry [baseline] (5.952 ms) : 0, 5952
Telemetry [candidate] (6.013 ms) : 0, 6013
ProfilingAgent [baseline] (76.356 ms) : 0, 76356
ProfilingAgent [candidate] (79.409 ms) : 0, 79409
Profiling [baseline] (76.38 ms) : 0, 76380
Profiling [candidate] (79.434 ms) : 0, 79434
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~5c1e684acb, baseline=1.20.0-SNAPSHOT~72572d1d17

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (985.712 ms) : 0, 985712
Total [baseline] (8.632 s) : 0, 8631800
Agent [candidate] (990.413 ms) : 0, 990413
Total [candidate] (8.661 s) : 0, 8661486
section appsec
Agent [baseline] (1.072 s) : 0, 1072340
Total [baseline] (8.704 s) : 0, 8704241
Agent [candidate] (1.073 s) : 0, 1072550
Total [candidate] (8.759 s) : 0, 8759075
section iast
Agent [baseline] (1.089 s) : 0, 1089013
Total [baseline] (9.147 s) : 0, 9146865
Agent [candidate] (1.097 s) : 0, 1096746
Total [candidate] (9.209 s) : 0, 9208885
section profiling
Agent [baseline] (1.157 s) : 0, 1157175
Total [baseline] (8.852 s) : 0, 8851884
Agent [candidate] (1.169 s) : 0, 1169483
Total [candidate] (8.909 s) : 0, 8908712
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 985.712 ms -
Agent appsec 1.072 s 86.628 ms (8.8%)
Agent iast 1.089 s 103.3 ms (10.5%)
Agent profiling 1.157 s 171.463 ms (17.4%)
Total tracing 8.632 s -
Total appsec 8.704 s 72.442 ms (0.8%)
Total iast 9.147 s 515.065 ms (6.0%)
Total profiling 8.852 s 220.084 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 990.413 ms -
Agent appsec 1.073 s 82.136 ms (8.3%)
Agent iast 1.097 s 106.333 ms (10.7%)
Agent profiling 1.169 s 179.069 ms (18.1%)
Total tracing 8.661 s -
Total appsec 8.759 s 97.59 ms (1.1%)
Total iast 9.209 s 547.399 ms (6.3%)
Total profiling 8.909 s 247.226 ms (2.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~5c1e684acb, baseline=1.20.0-SNAPSHOT~72572d1d17

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (611.274 ms) : 0, 611274
BytebuddyAgent [candidate] (613.556 ms) : 0, 613556
GlobalTracer [baseline] (283.719 ms) : 0, 283719
GlobalTracer [candidate] (285.575 ms) : 0, 285575
AppSec [baseline] (49.9 ms) : 0, 49900
AppSec [candidate] (50.16 ms) : 0, 50160
Remote Config [baseline] (653.71 µs) : 0, 654
Remote Config [candidate] (674.274 µs) : 0, 674
Telemetry [baseline] (5.926 ms) : 0, 5926
Telemetry [candidate] (6.024 ms) : 0, 6024
section appsec
BytebuddyAgent [baseline] (612.257 ms) : 0, 612257
BytebuddyAgent [candidate] (612.301 ms) : 0, 612301
GlobalTracer [baseline] (284.07 ms) : 0, 284070
GlobalTracer [candidate] (284.115 ms) : 0, 284115
AppSec [baseline] (135.227 ms) : 0, 135227
AppSec [candidate] (135.365 ms) : 0, 135365
Remote Config [baseline] (742.043 µs) : 0, 742
Remote Config [candidate] (727.315 µs) : 0, 727
Telemetry [baseline] (5.754 ms) : 0, 5754
Telemetry [candidate] (5.723 ms) : 0, 5723
section iast
BytebuddyAgent [baseline] (717.215 ms) : 0, 717215
BytebuddyAgent [candidate] (722.328 ms) : 0, 722328
GlobalTracer [baseline] (270.004 ms) : 0, 270004
GlobalTracer [candidate] (272.138 ms) : 0, 272138
AppSec [baseline] (47.146 ms) : 0, 47146
AppSec [candidate] (47.45 ms) : 0, 47450
Remote Config [baseline] (574.624 µs) : 0, 575
Remote Config [candidate] (551.966 µs) : 0, 552
Telemetry [baseline] (5.736 ms) : 0, 5736
Telemetry [candidate] (5.742 ms) : 0, 5742
IAST [baseline] (14.038 ms) : 0, 14038
IAST [candidate] (14.109 ms) : 0, 14109
section profiling
BytebuddyAgent [baseline] (624.474 ms) : 0, 624474
BytebuddyAgent [candidate] (631.283 ms) : 0, 631283
GlobalTracer [baseline] (347.217 ms) : 0, 347217
GlobalTracer [candidate] (350.722 ms) : 0, 350722
AppSec [baseline] (48.89 ms) : 0, 48890
AppSec [candidate] (49.339 ms) : 0, 49339
Remote Config [baseline] (638.549 µs) : 0, 639
Remote Config [candidate] (646.568 µs) : 0, 647
Telemetry [baseline] (5.95 ms) : 0, 5950
Telemetry [candidate] (6.04 ms) : 0, 6040
ProfilingAgent [baseline] (77.23 ms) : 0, 77230
ProfilingAgent [candidate] (78.153 ms) : 0, 78153
Profiling [baseline] (77.254 ms) : 0, 77254
Profiling [candidate] (78.178 ms) : 0, 78178
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~72572d1d17 1.20.0-SNAPSHOT~5c1e684acb
config baseline candidate
end_time 2023-08-28T08:33:26 2023-08-28T08:51:32
start_time 2023-08-28T08:33:09 2023-08-28T08:51:15
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~5c1e684acb, baseline=1.20.0-SNAPSHOT~72572d1d17
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1334, 1372
.   : milestone, 1353,
appsec (1.689 ms) : 1665, 1713
.   : milestone, 1689,
iast (1.444 ms) : 1419, 1468
.   : milestone, 1444,
profiling (1.48 ms) : 1456, 1504
.   : milestone, 1480,
tracing (1.441 ms) : 1416, 1465
.   : milestone, 1441,
section candidate
no_agent (1.346 ms) : 1328, 1365
.   : milestone, 1346,
appsec (1.661 ms) : 1636, 1685
.   : milestone, 1661,
iast (1.448 ms) : 1423, 1472
.   : milestone, 1448,
profiling (1.478 ms) : 1452, 1504
.   : milestone, 1478,
tracing (1.453 ms) : 1429, 1477
.   : milestone, 1453,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.334 ms, 1.372 ms] -
appsec 1.689 ms [1.665 ms, 1.713 ms] 336.119 µs (24.8%)
iast 1.444 ms [1.419 ms, 1.468 ms] 90.582 µs (6.7%)
profiling 1.48 ms [1.456 ms, 1.504 ms] 126.804 µs (9.4%)
tracing 1.441 ms [1.416 ms, 1.465 ms] 87.705 µs (6.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.328 ms, 1.365 ms] -
appsec 1.661 ms [1.636 ms, 1.685 ms] 314.141 µs (23.3%)
iast 1.448 ms [1.423 ms, 1.472 ms] 101.18 µs (7.5%)
profiling 1.478 ms [1.452 ms, 1.504 ms] 131.242 µs (9.7%)
tracing 1.453 ms [1.429 ms, 1.477 ms] 106.635 µs (7.9%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~5c1e684acb, baseline=1.20.0-SNAPSHOT~72572d1d17
    dateFormat X
    axisFormat %s
section baseline
no_agent (361.595 µs) : 341, 382
.   : milestone, 362,
appsec (665.36 µs) : 644, 687
.   : milestone, 665,
iast (453.405 µs) : 433, 474
.   : milestone, 453,
iast_FULL (512.845 µs) : 492, 533
.   : milestone, 513,
iast_INACTIVE (432.895 µs) : 412, 454
.   : milestone, 433,
profiling (449.725 µs) : 428, 472
.   : milestone, 450,
tracing (435.509 µs) : 415, 456
.   : milestone, 436,
section candidate
no_agent (361.245 µs) : 342, 381
.   : milestone, 361,
appsec (667.78 µs) : 647, 689
.   : milestone, 668,
iast (454.918 µs) : 434, 476
.   : milestone, 455,
iast_FULL (509.891 µs) : 489, 530
.   : milestone, 510,
iast_INACTIVE (428.87 µs) : 408, 450
.   : milestone, 429,
profiling (429.641 µs) : 409, 450
.   : milestone, 430,
tracing (431.44 µs) : 410, 453
.   : milestone, 431,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.595 µs [340.735 µs, 382.454 µs] -
appsec 665.36 µs [644.186 µs, 686.534 µs] 303.765 µs (84.0%)
iast 453.405 µs [432.863 µs, 473.948 µs] 91.811 µs (25.4%)
iast_FULL 512.845 µs [492.456 µs, 533.234 µs] 151.25 µs (41.8%)
iast_INACTIVE 432.895 µs [411.798 µs, 453.993 µs] 71.301 µs (19.7%)
profiling 449.725 µs [427.745 µs, 471.704 µs] 88.13 µs (24.4%)
tracing 435.509 µs [414.953 µs, 456.065 µs] 73.914 µs (20.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.245 µs [341.524 µs, 380.965 µs] -
appsec 667.78 µs [646.649 µs, 688.912 µs] 306.536 µs (84.9%)
iast 454.918 µs [433.729 µs, 476.107 µs] 93.673 µs (25.9%)
iast_FULL 509.891 µs [489.413 µs, 530.37 µs] 148.647 µs (41.1%)
iast_INACTIVE 428.87 µs [408.166 µs, 449.574 µs] 67.626 µs (18.7%)
profiling 429.641 µs [408.824 µs, 450.458 µs] 68.396 µs (18.9%)
tracing 431.44 µs [410.34 µs, 452.54 µs] 70.196 µs (19.4%)

@manuel-alvarez-alvarez
manuel-alvarez-alvarez force-pushed the malvarez/iast-improve-telemetry-perf branch from 32801a4 to 359b776 Compare August 25, 2023 12:46
@manuel-alvarez-alvarez
manuel-alvarez-alvarez force-pushed the malvarez/iast-improve-telemetry-perf branch from 359b776 to 5c1e684 Compare August 28, 2023 08:11
@manuel-alvarez-alvarez
manuel-alvarez-alvarez merged commit d21bcc7 into master Aug 28, 2023
@manuel-alvarez-alvarez
manuel-alvarez-alvarez deleted the malvarez/iast-improve-telemetry-perf branch August 28, 2023 10:33
@github-actions github-actions Bot added this to the 1.20.0 milestone Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@smola smola smola approved these changes
@jandro996 jandro996 jandro996 approved these changes
@ValentinZakharov ValentinZakharov Awaiting requested review from ValentinZakharov ValentinZakharov is a code owner automatically assigned from DataDog/asm-java

Assignees

No one assigned

Labels

comp: asm iast Application Security Management (IAST) tag: performance Performance related changes

Projects

None yet

Milestone

1.20.0

Development

Successfully merging this pull request may close these issues.

3 participants

@manuel-alvarez-alvarez @smola @jandro996