Skip to content

Check for overhead constraints in weak randomness module#5783

Merged
manuel-alvarez-alvarez merged 1 commit into
masterfrom
malvarez/iast-weak-randomness-check-oce
Aug 25, 2023
Merged

Check for overhead constraints in weak randomness module#5783
manuel-alvarez-alvarez merged 1 commit into
masterfrom
malvarez/iast-weak-randomness-check-oce

Conversation

@manuel-alvarez-alvarez

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Aug 25, 2023

Copy link
Copy Markdown
Member

What Does This Do

Check for overhead limits before reporting weak randomness vulnerability

Motivation

Additional Notes

@manuel-alvarez-alvarez manuel-alvarez-alvarez added tag: performance Performance related changes comp: asm iast Application Security Management (IAST) labels Aug 25, 2023
@manuel-alvarez-alvarez
manuel-alvarez-alvarez requested a review from a team as a code owner August 25, 2023 10:42
smola
smola approved these changes Aug 25, 2023
View reviewed changes

@smola smola left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but we have to review quotas, possibly setting a separate one per vulnerability type. Approving to avoid the excessive overhead at the moment.

@manuel-alvarez-alvarez
manuel-alvarez-alvarez force-pushed the malvarez/iast-weak-randomness-check-oce branch from 6afd960 to af66281 Compare August 25, 2023 10:46
@pr-commenter

pr-commenter Bot commented Aug 25, 2023 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~acb417b5b2 1.20.0-SNAPSHOT~af66281dbb
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~af66281dbb, baseline=1.20.0-SNAPSHOT~acb417b5b2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (987.181 ms) : 0, 987181
Total [baseline] (9.155 s) : 0, 9154616
Agent [candidate] (994.224 ms) : 0, 994224
Total [candidate] (9.174 s) : 0, 9174043
section appsec
Agent [baseline] (1.072 s) : 0, 1072177
Total [baseline] (9.248 s) : 0, 9247670
Agent [candidate] (1.079 s) : 0, 1078624
Total [candidate] (9.322 s) : 0, 9321583
section iast
Agent [baseline] (1.091 s) : 0, 1090676
Total [baseline] (9.45 s) : 0, 9450476
Agent [candidate] (1.09 s) : 0, 1090047
Total [candidate] (9.403 s) : 0, 9403027
section profiling
Agent [baseline] (1.154 s) : 0, 1154152
Total [baseline] (9.429 s) : 0, 9428957
Agent [candidate] (1.158 s) : 0, 1157601
Total [candidate] (9.414 s) : 0, 9414295
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 987.181 ms -
Agent appsec 1.072 s 84.996 ms (8.6%)
Agent iast 1.091 s 103.494 ms (10.5%)
Agent profiling 1.154 s 166.97 ms (16.9%)
Total tracing 9.155 s -
Total appsec 9.248 s 93.054 ms (1.0%)
Total iast 9.45 s 295.861 ms (3.2%)
Total profiling 9.429 s 274.341 ms (3.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 994.224 ms -
Agent appsec 1.079 s 84.401 ms (8.5%)
Agent iast 1.09 s 95.824 ms (9.6%)
Agent profiling 1.158 s 163.377 ms (16.4%)
Total tracing 9.174 s -
Total appsec 9.322 s 147.539 ms (1.6%)
Total iast 9.403 s 228.983 ms (2.5%)
Total profiling 9.414 s 240.252 ms (2.6%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~af66281dbb, baseline=1.20.0-SNAPSHOT~acb417b5b2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (612.637 ms) : 0, 612637
BytebuddyAgent [candidate] (616.182 ms) : 0, 616182
GlobalTracer [baseline] (283.366 ms) : 0, 283366
GlobalTracer [candidate] (286.775 ms) : 0, 286775
AppSec [baseline] (50.223 ms) : 0, 50223
AppSec [candidate] (50.061 ms) : 0, 50061
Remote Config [baseline] (663.026 µs) : 0, 663
Remote Config [candidate] (666.411 µs) : 0, 666
Telemetry [baseline] (6.059 ms) : 0, 6059
Telemetry [candidate] (6.064 ms) : 0, 6064
section appsec
BytebuddyAgent [baseline] (612.947 ms) : 0, 612947
BytebuddyAgent [candidate] (616.202 ms) : 0, 616202
GlobalTracer [baseline] (283.76 ms) : 0, 283760
GlobalTracer [candidate] (286.175 ms) : 0, 286175
AppSec [baseline] (134.721 ms) : 0, 134721
AppSec [candidate] (135.293 ms) : 0, 135293
Remote Config [baseline] (757.604 µs) : 0, 758
Remote Config [candidate] (721.931 µs) : 0, 722
Telemetry [baseline] (5.762 ms) : 0, 5762
Telemetry [candidate] (5.773 ms) : 0, 5773
section iast
BytebuddyAgent [baseline] (718.45 ms) : 0, 718450
BytebuddyAgent [candidate] (717.278 ms) : 0, 717278
GlobalTracer [baseline] (270.518 ms) : 0, 270518
GlobalTracer [candidate] (271.102 ms) : 0, 271102
AppSec [baseline] (47.18 ms) : 0, 47180
AppSec [candidate] (47.265 ms) : 0, 47265
Remote Config [baseline] (557.732 µs) : 0, 558
Remote Config [candidate] (549.092 µs) : 0, 549
Telemetry [baseline] (5.741 ms) : 0, 5741
Telemetry [candidate] (5.7 ms) : 0, 5700
IAST [baseline] (13.868 ms) : 0, 13868
IAST [candidate] (13.845 ms) : 0, 13845
section profiling
BytebuddyAgent [baseline] (624.246 ms) : 0, 624246
BytebuddyAgent [candidate] (625.228 ms) : 0, 625228
GlobalTracer [baseline] (345.117 ms) : 0, 345117
GlobalTracer [candidate] (346.619 ms) : 0, 346619
AppSec [baseline] (48.743 ms) : 0, 48743
AppSec [candidate] (49.007 ms) : 0, 49007
Remote Config [baseline] (643.296 µs) : 0, 643
Remote Config [candidate] (651.068 µs) : 0, 651
Telemetry [baseline] (5.926 ms) : 0, 5926
Telemetry [candidate] (5.981 ms) : 0, 5981
ProfilingAgent [baseline] (76.66 ms) : 0, 76660
ProfilingAgent [candidate] (77.321 ms) : 0, 77321
Profiling [baseline] (76.685 ms) : 0, 76685
Profiling [candidate] (77.345 ms) : 0, 77345
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~af66281dbb, baseline=1.20.0-SNAPSHOT~acb417b5b2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (986.072 ms) : 0, 986072
Total [baseline] (8.618 s) : 0, 8618407
Agent [candidate] (986.248 ms) : 0, 986248
Total [candidate] (8.616 s) : 0, 8615995
section appsec
Agent [baseline] (1.071 s) : 0, 1071399
Total [baseline] (8.71 s) : 0, 8709666
Agent [candidate] (1.073 s) : 0, 1072978
Total [candidate] (8.718 s) : 0, 8718166
section iast
Agent [baseline] (1.088 s) : 0, 1087985
Total [baseline] (9.158 s) : 0, 9158474
Agent [candidate] (1.094 s) : 0, 1093995
Total [candidate] (9.173 s) : 0, 9173074
section profiling
Agent [baseline] (1.154 s) : 0, 1153702
Total [baseline] (8.827 s) : 0, 8827370
Agent [candidate] (1.164 s) : 0, 1163568
Total [candidate] (8.878 s) : 0, 8878200
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 986.072 ms -
Agent appsec 1.071 s 85.327 ms (8.7%)
Agent iast 1.088 s 101.913 ms (10.3%)
Agent profiling 1.154 s 167.63 ms (17.0%)
Total tracing 8.618 s -
Total appsec 8.71 s 91.258 ms (1.1%)
Total iast 9.158 s 540.067 ms (6.3%)
Total profiling 8.827 s 208.963 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 986.248 ms -
Agent appsec 1.073 s 86.73 ms (8.8%)
Agent iast 1.094 s 107.747 ms (10.9%)
Agent profiling 1.164 s 177.321 ms (18.0%)
Total tracing 8.616 s -
Total appsec 8.718 s 102.171 ms (1.2%)
Total iast 9.173 s 557.079 ms (6.5%)
Total profiling 8.878 s 262.205 ms (3.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~af66281dbb, baseline=1.20.0-SNAPSHOT~acb417b5b2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (611.617 ms) : 0, 611617
BytebuddyAgent [candidate] (611.049 ms) : 0, 611049
GlobalTracer [baseline] (283.598 ms) : 0, 283598
GlobalTracer [candidate] (283.965 ms) : 0, 283965
AppSec [baseline] (49.874 ms) : 0, 49874
AppSec [candidate] (50.268 ms) : 0, 50268
Remote Config [baseline] (664.78 µs) : 0, 665
Remote Config [candidate] (659.499 µs) : 0, 659
Telemetry [baseline] (5.998 ms) : 0, 5998
Telemetry [candidate] (6.038 ms) : 0, 6038
section appsec
BytebuddyAgent [baseline] (611.976 ms) : 0, 611976
BytebuddyAgent [candidate] (612.345 ms) : 0, 612345
GlobalTracer [baseline] (283.679 ms) : 0, 283679
GlobalTracer [candidate] (284.795 ms) : 0, 284795
AppSec [baseline] (135.017 ms) : 0, 135017
AppSec [candidate] (135.033 ms) : 0, 135033
Remote Config [baseline] (723.716 µs) : 0, 724
Remote Config [candidate] (712.352 µs) : 0, 712
Telemetry [baseline] (5.775 ms) : 0, 5775
Telemetry [candidate] (5.743 ms) : 0, 5743
section iast
BytebuddyAgent [baseline] (716.452 ms) : 0, 716452
BytebuddyAgent [candidate] (720.597 ms) : 0, 720597
GlobalTracer [baseline] (269.994 ms) : 0, 269994
GlobalTracer [candidate] (271.578 ms) : 0, 271578
AppSec [baseline] (47.168 ms) : 0, 47168
AppSec [candidate] (47.118 ms) : 0, 47118
Remote Config [baseline] (558.216 µs) : 0, 558
Remote Config [candidate] (555.706 µs) : 0, 556
Telemetry [baseline] (5.645 ms) : 0, 5645
Telemetry [candidate] (5.727 ms) : 0, 5727
IAST [baseline] (13.822 ms) : 0, 13822
IAST [candidate] (13.985 ms) : 0, 13985
section profiling
BytebuddyAgent [baseline] (623.877 ms) : 0, 623877
BytebuddyAgent [candidate] (629.63 ms) : 0, 629630
GlobalTracer [baseline] (345.019 ms) : 0, 345019
GlobalTracer [candidate] (348.097 ms) : 0, 348097
AppSec [baseline] (48.72 ms) : 0, 48720
AppSec [candidate] (49.107 ms) : 0, 49107
Remote Config [baseline] (644.57 µs) : 0, 645
Remote Config [candidate] (651.596 µs) : 0, 652
Telemetry [baseline] (6.006 ms) : 0, 6006
Telemetry [candidate] (6.025 ms) : 0, 6025
ProfilingAgent [baseline] (76.758 ms) : 0, 76758
ProfilingAgent [candidate] (77.003 ms) : 0, 77003
Profiling [baseline] (76.782 ms) : 0, 76782
Profiling [candidate] (77.027 ms) : 0, 77027
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~acb417b5b2 1.20.0-SNAPSHOT~af66281dbb
config baseline candidate
end_time 2023-08-25T11:09:41 2023-08-25T11:27:45
start_time 2023-08-25T11:09:24 2023-08-25T11:27:27
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~af66281dbb, baseline=1.20.0-SNAPSHOT~acb417b5b2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.342 ms) : 1323, 1361
.   : milestone, 1342,
appsec (1.669 ms) : 1645, 1693
.   : milestone, 1669,
iast (1.46 ms) : 1436, 1484
.   : milestone, 1460,
profiling (1.451 ms) : 1426, 1477
.   : milestone, 1451,
tracing (1.437 ms) : 1413, 1461
.   : milestone, 1437,
section candidate
no_agent (1.34 ms) : 1322, 1359
.   : milestone, 1340,
appsec (1.673 ms) : 1649, 1697
.   : milestone, 1673,
iast (1.468 ms) : 1444, 1492
.   : milestone, 1468,
profiling (1.468 ms) : 1443, 1494
.   : milestone, 1468,
tracing (1.443 ms) : 1419, 1467
.   : milestone, 1443,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.323 ms, 1.361 ms] -
appsec 1.669 ms [1.645 ms, 1.693 ms] 327.097 µs (24.4%)
iast 1.46 ms [1.436 ms, 1.484 ms] 118.02 µs (8.8%)
profiling 1.451 ms [1.426 ms, 1.477 ms] 109.397 µs (8.2%)
tracing 1.437 ms [1.413 ms, 1.461 ms] 94.491 µs (7.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.34 ms [1.322 ms, 1.359 ms] -
appsec 1.673 ms [1.649 ms, 1.697 ms] 332.686 µs (24.8%)
iast 1.468 ms [1.444 ms, 1.492 ms] 127.747 µs (9.5%)
profiling 1.468 ms [1.443 ms, 1.494 ms] 128.103 µs (9.6%)
tracing 1.443 ms [1.419 ms, 1.467 ms] 102.458 µs (7.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~af66281dbb, baseline=1.20.0-SNAPSHOT~acb417b5b2
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.704 µs) : 340, 381
.   : milestone, 361,
appsec (664.99 µs) : 644, 686
.   : milestone, 665,
iast (447.944 µs) : 427, 469
.   : milestone, 448,
iast_FULL (521.075 µs) : 500, 542
.   : milestone, 521,
iast_INACTIVE (430.951 µs) : 410, 452
.   : milestone, 431,
profiling (432.059 µs) : 411, 453
.   : milestone, 432,
tracing (442.797 µs) : 421, 464
.   : milestone, 443,
section candidate
no_agent (364.418 µs) : 344, 385
.   : milestone, 364,
appsec (667.551 µs) : 646, 689
.   : milestone, 668,
iast (451.622 µs) : 431, 472
.   : milestone, 452,
iast_FULL (517.578 µs) : 497, 538
.   : milestone, 518,
iast_INACTIVE (427.389 µs) : 406, 449
.   : milestone, 427,
profiling (448.533 µs) : 427, 470
.   : milestone, 449,
tracing (431.241 µs) : 410, 453
.   : milestone, 431,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.704 µs [340.328 µs, 381.08 µs] -
appsec 664.99 µs [643.9 µs, 686.08 µs] 304.286 µs (84.4%)
iast 447.944 µs [427.284 µs, 468.604 µs] 87.24 µs (24.2%)
iast_FULL 521.075 µs [500.431 µs, 541.718 µs] 160.37 µs (44.5%)
iast_INACTIVE 430.951 µs [410.0 µs, 451.902 µs] 70.247 µs (19.5%)
profiling 432.059 µs [411.333 µs, 452.784 µs] 71.355 µs (19.8%)
tracing 442.797 µs [421.325 µs, 464.269 µs] 82.093 µs (22.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.418 µs [344.217 µs, 384.619 µs] -
appsec 667.551 µs [646.445 µs, 688.657 µs] 303.133 µs (83.2%)
iast 451.622 µs [431.267 µs, 471.978 µs] 87.204 µs (23.9%)
iast_FULL 517.578 µs [496.936 µs, 538.22 µs] 153.16 µs (42.0%)
iast_INACTIVE 427.389 µs [406.243 µs, 448.536 µs] 62.971 µs (17.3%)
profiling 448.533 µs [426.752 µs, 470.314 µs] 84.115 µs (23.1%)
tracing 431.241 µs [409.971 µs, 452.51 µs] 66.823 µs (18.3%)

@manuel-alvarez-alvarez
manuel-alvarez-alvarez merged commit 4a1be49 into master Aug 25, 2023
@manuel-alvarez-alvarez
manuel-alvarez-alvarez deleted the malvarez/iast-weak-randomness-check-oce branch August 25, 2023 12:52
@github-actions github-actions Bot added this to the 1.20.0 milestone Aug 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@smola smola smola approved these changes
@jandro996 jandro996 Awaiting requested review from jandro996 jandro996 is a code owner automatically assigned from DataDog/asm-java

Assignees

No one assigned

Labels

comp: asm iast Application Security Management (IAST) tag: performance Performance related changes

Projects

None yet

Milestone

1.20.0

Development

Successfully merging this pull request may close these issues.

2 participants

@manuel-alvarez-alvarez @smola