Emit recording setting events for SSI details by jbachorik · Pull Request #7507 · DataDog/dd-trace-java

jbachorik · 2024-08-26T10:29:48Z

What Does This Do

It adds two more settings emitted in JFR recordings that are describing the SSI activation state.

Motivation

Enable tracking the progress and any potential issues with SSI activations.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Squash your commits prior merging or merge using GitHub's Squash and merge
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: PROF-10400

pr-commenter · 2024-08-26T11:06:16Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jb/profiling_ssi_settings
git_commit_date	1724748799	1724755236
git_commit_sha	`bb9b598`	`968a6af`
release_version	1.39.0-SNAPSHOT~bb9b598987	1.39.0-SNAPSHOT~968a6afce9

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1724757537	1724757537
ci_job_id	618206160	618206160
ci_pipeline_id	42909727	42909727
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~968a6afce9, baseline=1.39.0-SNAPSHOT~bb9b598987

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058001
Total [baseline] (10.358 s) : 0, 10358089
Agent [candidate] (1.055 s) : 0, 1054990
Total [candidate] (10.403 s) : 0, 10403462
section appsec
Agent [baseline] (1.175 s) : 0, 1175236
Total [baseline] (10.46 s) : 0, 10460219
Agent [candidate] (1.17 s) : 0, 1169846
Total [candidate] (10.44 s) : 0, 10439970
section iast
Agent [baseline] (1.172 s) : 0, 1172144
Total [baseline] (10.835 s) : 0, 10835302
Agent [candidate] (1.175 s) : 0, 1174724
Total [candidate] (10.876 s) : 0, 10875556
section profiling
Agent [baseline] (1.247 s) : 0, 1246889
Total [baseline] (10.618 s) : 0, 10618303
Agent [candidate] (1.247 s) : 0, 1246975
Total [candidate] (10.532 s) : 0, 10531753

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	appsec	1.175 s	117.234 ms (11.1%)
Agent	iast	1.172 s	114.143 ms (10.8%)
Agent	profiling	1.247 s	188.887 ms (17.9%)
Total	tracing	10.358 s	-
Total	appsec	10.46 s	102.13 ms (1.0%)
Total	iast	10.835 s	477.213 ms (4.6%)
Total	profiling	10.618 s	260.214 ms (2.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	appsec	1.17 s	114.856 ms (10.9%)
Agent	iast	1.175 s	119.734 ms (11.3%)
Agent	profiling	1.247 s	191.985 ms (18.2%)
Total	tracing	10.403 s	-
Total	appsec	10.44 s	36.507 ms (0.4%)
Total	iast	10.876 s	472.094 ms (4.5%)
Total	profiling	10.532 s	128.291 ms (1.2%)

gantt
    title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~968a6afce9, baseline=1.39.0-SNAPSHOT~bb9b598987

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (676.539 ms) : 0, 676539
BytebuddyAgent [candidate] (673.366 ms) : 0, 673366
GlobalTracer [baseline] (308.068 ms) : 0, 308068
GlobalTracer [candidate] (308.054 ms) : 0, 308054
AppSec [baseline] (51.59 ms) : 0, 51590
AppSec [candidate] (51.704 ms) : 0, 51704
Remote Config [baseline] (666.995 µs) : 0, 667
Remote Config [candidate] (676.417 µs) : 0, 676
Telemetry [baseline] (7.483 ms) : 0, 7483
Telemetry [candidate] (7.546 ms) : 0, 7546
section appsec
BytebuddyAgent [baseline] (681.367 ms) : 0, 681367
BytebuddyAgent [candidate] (678.911 ms) : 0, 678911
GlobalTracer [baseline] (302.486 ms) : 0, 302486
GlobalTracer [candidate] (300.3 ms) : 0, 300300
AppSec [baseline] (159.787 ms) : 0, 159787
AppSec [candidate] (159.067 ms) : 0, 159067
IAST [baseline] (18.422 ms) : 0, 18422
IAST [candidate] (18.494 ms) : 0, 18494
Remote Config [baseline] (635.267 µs) : 0, 635
Remote Config [candidate] (638.433 µs) : 0, 638
Telemetry [baseline] (8.129 ms) : 0, 8129
Telemetry [candidate] (8.099 ms) : 0, 8099
section iast
BytebuddyAgent [baseline] (778.322 ms) : 0, 778322
BytebuddyAgent [candidate] (779.505 ms) : 0, 779505
GlobalTracer [baseline] (296.131 ms) : 0, 296131
GlobalTracer [candidate] (296.408 ms) : 0, 296408
AppSec [baseline] (52.201 ms) : 0, 52201
AppSec [candidate] (49.087 ms) : 0, 49087
IAST [baseline] (23.319 ms) : 0, 23319
IAST [candidate] (24.36 ms) : 0, 24360
Remote Config [baseline] (591.007 µs) : 0, 591
Remote Config [candidate] (605.052 µs) : 0, 605
Telemetry [baseline] (8.018 ms) : 0, 8018
Telemetry [candidate] (11.148 ms) : 0, 11148
section profiling
BytebuddyAgent [baseline] (663.819 ms) : 0, 663819
BytebuddyAgent [candidate] (664.24 ms) : 0, 664240
GlobalTracer [baseline] (389.347 ms) : 0, 389347
GlobalTracer [candidate] (389.066 ms) : 0, 389066
AppSec [baseline] (52.275 ms) : 0, 52275
AppSec [candidate] (52.236 ms) : 0, 52236
Remote Config [baseline] (687.186 µs) : 0, 687
Remote Config [candidate] (699.963 µs) : 0, 700
Telemetry [baseline] (7.335 ms) : 0, 7335
Telemetry [candidate] (7.399 ms) : 0, 7399
ProfilingAgent [baseline] (96.065 ms) : 0, 96065
ProfilingAgent [candidate] (95.929 ms) : 0, 95929
Profiling [baseline] (96.089 ms) : 0, 96089
Profiling [candidate] (95.953 ms) : 0, 95953

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~968a6afce9, baseline=1.39.0-SNAPSHOT~bb9b598987

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1047489
Total [baseline] (8.483 s) : 0, 8482677
Agent [candidate] (1.049 s) : 0, 1048690
Total [candidate] (8.486 s) : 0, 8485713
section iast
Agent [baseline] (1.172 s) : 0, 1171683
Total [baseline] (8.952 s) : 0, 8952364
Agent [candidate] (1.17 s) : 0, 1170116
Total [candidate] (8.967 s) : 0, 8966614
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.188 s) : 0, 1188452
Total [baseline] (9.045 s) : 0, 9044922
Agent [candidate] (1.174 s) : 0, 1174315
Total [candidate] (8.938 s) : 0, 8937500
section iast_TELEMETRY_OFF
Agent [baseline] (1.17 s) : 0, 1169985
Total [baseline] (8.952 s) : 0, 8952125
Agent [candidate] (1.177 s) : 0, 1176778
Total [candidate] (8.944 s) : 0, 8944134

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	iast	1.172 s	124.194 ms (11.9%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.188 s	140.962 ms (13.5%)
Agent	iast_TELEMETRY_OFF	1.17 s	122.496 ms (11.7%)
Total	tracing	8.483 s	-
Total	iast	8.952 s	469.687 ms (5.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.045 s	562.245 ms (6.6%)
Total	iast_TELEMETRY_OFF	8.952 s	469.448 ms (5.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	iast	1.17 s	121.425 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.174 s	125.624 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.177 s	128.088 ms (12.2%)
Total	tracing	8.486 s	-
Total	iast	8.967 s	480.901 ms (5.7%)
Total	iast_HARDCODED_SECRET_DISABLED	8.938 s	451.787 ms (5.3%)
Total	iast_TELEMETRY_OFF	8.944 s	458.421 ms (5.4%)

gantt
    title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~968a6afce9, baseline=1.39.0-SNAPSHOT~bb9b598987

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.181 ms) : 0, 668181
BytebuddyAgent [candidate] (668.939 ms) : 0, 668939
GlobalTracer [baseline] (306.243 ms) : 0, 306243
GlobalTracer [candidate] (306.48 ms) : 0, 306480
AppSec [baseline] (51.328 ms) : 0, 51328
AppSec [candidate] (51.612 ms) : 0, 51612
Remote Config [baseline] (663.548 µs) : 0, 664
Remote Config [candidate] (664.072 µs) : 0, 664
Telemetry [baseline] (7.536 ms) : 0, 7536
Telemetry [candidate] (7.409 ms) : 0, 7409
section iast
BytebuddyAgent [baseline] (778.234 ms) : 0, 778234
BytebuddyAgent [candidate] (777.395 ms) : 0, 777395
GlobalTracer [baseline] (296.053 ms) : 0, 296053
GlobalTracer [candidate] (295.849 ms) : 0, 295849
AppSec [baseline] (49.689 ms) : 0, 49689
AppSec [candidate] (50.214 ms) : 0, 50214
IAST [baseline] (24.867 ms) : 0, 24867
IAST [candidate] (22.93 ms) : 0, 22930
Remote Config [baseline] (591.838 µs) : 0, 592
Remote Config [candidate] (597.112 µs) : 0, 597
Telemetry [baseline] (8.664 ms) : 0, 8664
Telemetry [candidate] (9.541 ms) : 0, 9541
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (790.335 ms) : 0, 790335
BytebuddyAgent [candidate] (778.9 ms) : 0, 778900
GlobalTracer [baseline] (300.41 ms) : 0, 300410
GlobalTracer [candidate] (296.754 ms) : 0, 296754
AppSec [baseline] (49.77 ms) : 0, 49770
AppSec [candidate] (50.767 ms) : 0, 50767
IAST [baseline] (25.447 ms) : 0, 25447
IAST [candidate] (24.215 ms) : 0, 24215
Remote Config [baseline] (598.387 µs) : 0, 598
Remote Config [candidate] (595.571 µs) : 0, 596
Telemetry [baseline] (8.126 ms) : 0, 8126
Telemetry [candidate] (9.465 ms) : 0, 9465
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (775.436 ms) : 0, 775436
BytebuddyAgent [candidate] (780.068 ms) : 0, 780068
GlobalTracer [baseline] (296.555 ms) : 0, 296555
GlobalTracer [candidate] (297.845 ms) : 0, 297845
AppSec [baseline] (49.636 ms) : 0, 49636
AppSec [candidate] (51.837 ms) : 0, 51837
IAST [baseline] (24.045 ms) : 0, 24045
IAST [candidate] (24.311 ms) : 0, 24311
Remote Config [baseline] (591.187 µs) : 0, 591
Remote Config [candidate] (585.275 µs) : 0, 585
Telemetry [baseline] (10.16 ms) : 0, 10160
Telemetry [candidate] (8.444 ms) : 0, 8444

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-08-27T10:47:37	2024-08-27T10:54:24
git_branch	master	jb/profiling_ssi_settings
git_commit_date	1724748799	1724755236
git_commit_sha	`bb9b598`	`968a6af`
release_version	1.39.0-SNAPSHOT~bb9b598987	1.39.0-SNAPSHOT~968a6afce9
start_time	2024-08-27T10:47:23	2024-08-27T10:54:11

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1724756406	1724756406
ci_job_id	618206161	618206161
ci_pipeline_id	42909727	42909727
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~968a6afce9, baseline=1.39.0-SNAPSHOT~bb9b598987
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.347 ms) : 1328, 1366
.   : milestone, 1347,
appsec (1.712 ms) : 1687, 1737
.   : milestone, 1712,
appsec_no_iast (1.73 ms) : 1706, 1754
.   : milestone, 1730,
iast (1.485 ms) : 1463, 1508
.   : milestone, 1485,
profiling (1.473 ms) : 1450, 1496
.   : milestone, 1473,
tracing (1.477 ms) : 1453, 1501
.   : milestone, 1477,
section candidate
no_agent (1.335 ms) : 1316, 1354
.   : milestone, 1335,
appsec (1.697 ms) : 1673, 1721
.   : milestone, 1697,
appsec_no_iast (1.701 ms) : 1676, 1725
.   : milestone, 1701,
iast (1.449 ms) : 1427, 1472
.   : milestone, 1449,
profiling (1.461 ms) : 1437, 1485
.   : milestone, 1461,
tracing (1.471 ms) : 1446, 1496
.   : milestone, 1471,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.347 ms [1.328 ms, 1.366 ms]	-
appsec	1.712 ms [1.687 ms, 1.737 ms]	364.826 µs (27.1%)
appsec_no_iast	1.73 ms [1.706 ms, 1.754 ms]	382.722 µs (28.4%)
iast	1.485 ms [1.463 ms, 1.508 ms]	137.976 µs (10.2%)
profiling	1.473 ms [1.45 ms, 1.496 ms]	126.051 µs (9.4%)
tracing	1.477 ms [1.453 ms, 1.501 ms]	129.868 µs (9.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.335 ms [1.316 ms, 1.354 ms]	-
appsec	1.697 ms [1.673 ms, 1.721 ms]	362.119 µs (27.1%)
appsec_no_iast	1.701 ms [1.676 ms, 1.725 ms]	365.745 µs (27.4%)
iast	1.449 ms [1.427 ms, 1.472 ms]	114.45 µs (8.6%)
profiling	1.461 ms [1.437 ms, 1.485 ms]	126.385 µs (9.5%)
tracing	1.471 ms [1.446 ms, 1.496 ms]	136.134 µs (10.2%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~968a6afce9, baseline=1.39.0-SNAPSHOT~bb9b598987
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.162 µs) : 353, 392
.   : milestone, 372,
iast (477.259 µs) : 456, 499
.   : milestone, 477,
iast_FULL (552.115 µs) : 531, 573
.   : milestone, 552,
iast_GLOBAL (502.292 µs) : 480, 524
.   : milestone, 502,
iast_HARDCODED_SECRET_DISABLED (486.886 µs) : 465, 509
.   : milestone, 487,
iast_INACTIVE (442.412 µs) : 421, 463
.   : milestone, 442,
iast_TELEMETRY_OFF (463.751 µs) : 442, 486
.   : milestone, 464,
tracing (437.249 µs) : 417, 458
.   : milestone, 437,
section candidate
no_agent (364.414 µs) : 345, 384
.   : milestone, 364,
iast (480.152 µs) : 458, 503
.   : milestone, 480,
iast_FULL (540.736 µs) : 520, 562
.   : milestone, 541,
iast_GLOBAL (502.762 µs) : 482, 524
.   : milestone, 503,
iast_HARDCODED_SECRET_DISABLED (480.788 µs) : 459, 503
.   : milestone, 481,
iast_INACTIVE (447.673 µs) : 427, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (464.521 µs) : 442, 487
.   : milestone, 465,
tracing (436.154 µs) : 416, 456
.   : milestone, 436,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	372.162 µs [352.556 µs, 391.768 µs]	-
iast	477.259 µs [455.795 µs, 498.722 µs]	105.097 µs (28.2%)
iast_FULL	552.115 µs [530.958 µs, 573.272 µs]	179.953 µs (48.4%)
iast_GLOBAL	502.292 µs [480.131 µs, 524.454 µs]	130.131 µs (35.0%)
iast_HARDCODED_SECRET_DISABLED	486.886 µs [464.629 µs, 509.142 µs]	114.724 µs (30.8%)
iast_INACTIVE	442.412 µs [421.479 µs, 463.344 µs]	70.25 µs (18.9%)
iast_TELEMETRY_OFF	463.751 µs [441.601 µs, 485.9 µs]	91.589 µs (24.6%)
tracing	437.249 µs [416.989 µs, 457.508 µs]	65.087 µs (17.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	364.414 µs [344.93 µs, 383.898 µs]	-
iast	480.152 µs [457.598 µs, 502.707 µs]	115.738 µs (31.8%)
iast_FULL	540.736 µs [519.823 µs, 561.648 µs]	176.322 µs (48.4%)
iast_GLOBAL	502.762 µs [481.946 µs, 523.578 µs]	138.348 µs (38.0%)
iast_HARDCODED_SECRET_DISABLED	480.788 µs [458.513 µs, 503.063 µs]	116.374 µs (31.9%)
iast_INACTIVE	447.673 µs [426.613 µs, 468.733 µs]	83.259 µs (22.8%)
iast_TELEMETRY_OFF	464.521 µs [442.139 µs, 486.903 µs]	100.107 µs (27.5%)
tracing	436.154 µs [415.963 µs, 456.346 µs]	71.74 µs (19.7%)

Dacapo

r1viollet · 2024-08-27T12:13:18Z


 public enum ProfilingEnablement {
-  ENABLED(true),
+  ENABLED(true, "manual"),


others default to their enum values I assume ?

r1viollet · 2024-08-27T12:15:14Z

+
+    @Override
+    public String toString() {
+      return "ProfilerActivationSetting{"


for naming @szegedi could be interested in reviewing.

r1viollet

LGTM

jbachorik added the comp: profiling Profiling label Aug 26, 2024

jbachorik force-pushed the jb/profiling_ssi_settings branch from a3a12a4 to c7df1b7 Compare August 26, 2024 10:54

jbachorik force-pushed the jb/profiling_ssi_settings branch from c7df1b7 to 0ebb499 Compare August 26, 2024 12:44

jbachorik marked this pull request as ready for review August 26, 2024 13:29

jbachorik requested review from a team as code owners August 26, 2024 13:29

jbachorik requested a review from nayeem-kamal August 26, 2024 13:29

Emit recording setting events for SSI details

093faf3

jbachorik force-pushed the jb/profiling_ssi_settings branch from 0ebb499 to 093faf3 Compare August 26, 2024 13:40

jbachorik and others added 4 commits August 27, 2024 10:39

Merge branch 'master' into jb/profiling_ssi_settings

34aa433

spotless

f2657ae

Improve code coverage

5719124

spotless

968a6af

jbachorik enabled auto-merge (squash) August 27, 2024 12:05

r1viollet reviewed Aug 27, 2024

View reviewed changes

r1viollet approved these changes Aug 27, 2024

View reviewed changes

jbachorik merged commit a795550 into master Aug 27, 2024

jbachorik deleted the jb/profiling_ssi_settings branch August 27, 2024 12:15

github-actions Bot added this to the 1.39.0 milestone Aug 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Emit recording setting events for SSI details#7507

Emit recording setting events for SSI details#7507
jbachorik merged 5 commits intomasterfrom
jb/profiling_ssi_settings

jbachorik commented Aug 26, 2024 •

edited by atlassian Bot

Loading

Uh oh!

pr-commenter Bot commented Aug 26, 2024 •

edited

Loading

Uh oh!

r1viollet Aug 27, 2024

Uh oh!

r1viollet Aug 27, 2024

Uh oh!

r1viollet left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jbachorik commented Aug 26, 2024 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter Bot commented Aug 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Uh oh!

r1viollet Aug 27, 2024

Choose a reason for hiding this comment

Uh oh!

r1viollet Aug 27, 2024

Choose a reason for hiding this comment

Uh oh!

r1viollet left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jbachorik commented Aug 26, 2024 •

edited by atlassian Bot

Loading

pr-commenter Bot commented Aug 26, 2024 •

edited

Loading