Populate tracer-flare zip with configuration details by mcculls · Pull Request #6359 · DataDog/dd-trace-java

mcculls · 2023-12-13T12:31:46Z

pr-commenter · 2023-12-13T13:16:18Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mcculls/tracer-flare-triage-report
git_commit_date	1702895354	1702902832
git_commit_sha	`6bb3505`	`3129c2c`
release_version	1.26.0-SNAPSHOT~6bb350544f	1.26.0-SNAPSHOT~3129c2c396

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702905439	1702905439
ci_job_id	391778374	391778374
ci_pipeline_id	25236002	25236002
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 5 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.26.0-SNAPSHOT~3129c2c396, baseline=1.26.0-SNAPSHOT~6bb350544f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1052574
Total [baseline] (8.732 s) : 0, 8731515
Agent [candidate] (1.051 s) : 0, 1050577
Total [candidate] (8.723 s) : 0, 8722826
section iast
Agent [baseline] (1.164 s) : 0, 1164040
Total [baseline] (9.265 s) : 0, 9264801
Agent [candidate] (1.167 s) : 0, 1167285
Total [candidate] (9.291 s) : 0, 9291031
section iast_TELEMETRY_OFF
Agent [baseline] (1.167 s) : 0, 1167447
Total [baseline] (9.294 s) : 0, 9293997
Agent [candidate] (1.158 s) : 0, 1157614
Total [candidate] (9.294 s) : 0, 9293574

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.053 s	-
Agent	iast	1.164 s	111.466 ms (10.6%)
Agent	iast_TELEMETRY_OFF	1.167 s	114.873 ms (10.9%)
Total	tracing	8.732 s	-
Total	iast	9.265 s	533.286 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.294 s	562.481 ms (6.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	iast	1.167 s	116.708 ms (11.1%)
Agent	iast_TELEMETRY_OFF	1.158 s	107.037 ms (10.2%)
Total	tracing	8.723 s	-
Total	iast	9.291 s	568.206 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.294 s	570.749 ms (6.5%)

gantt
    title insecure-bank - break down per module: candidate=1.26.0-SNAPSHOT~3129c2c396, baseline=1.26.0-SNAPSHOT~6bb350544f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (651.387 ms) : 0, 651387
BytebuddyAgent [candidate] (649.166 ms) : 0, 649166
GlobalTracer [baseline] (307.722 ms) : 0, 307722
GlobalTracer [candidate] (308.013 ms) : 0, 308013
AppSec [baseline] (51.106 ms) : 0, 51106
AppSec [candidate] (51.206 ms) : 0, 51206
Remote Config [baseline] (680.066 µs) : 0, 680
Remote Config [candidate] (678.148 µs) : 0, 678
Telemetry [baseline] (7.226 ms) : 0, 7226
Telemetry [candidate] (7.245 ms) : 0, 7245
section iast
BytebuddyAgent [baseline] (767.352 ms) : 0, 767352
BytebuddyAgent [candidate] (769.77 ms) : 0, 769770
GlobalTracer [baseline] (285.274 ms) : 0, 285274
GlobalTracer [candidate] (285.273 ms) : 0, 285273
AppSec [baseline] (48.94 ms) : 0, 48940
AppSec [candidate] (49.16 ms) : 0, 49160
Remote Config [baseline] (596.438 µs) : 0, 596
Remote Config [candidate] (608.882 µs) : 0, 609
Telemetry [baseline] (6.489 ms) : 0, 6489
Telemetry [candidate] (6.493 ms) : 0, 6493
IAST [baseline] (21.157 ms) : 0, 21157
IAST [candidate] (21.631 ms) : 0, 21631
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (766.816 ms) : 0, 766816
BytebuddyAgent [candidate] (761.16 ms) : 0, 761160
GlobalTracer [baseline] (287.906 ms) : 0, 287906
GlobalTracer [candidate] (285.849 ms) : 0, 285849
AppSec [baseline] (49.56 ms) : 0, 49560
AppSec [candidate] (49.062 ms) : 0, 49062
Remote Config [baseline] (610.709 µs) : 0, 611
Remote Config [candidate] (625.445 µs) : 0, 625
Telemetry [baseline] (8.057 ms) : 0, 8057
Telemetry [candidate] (8.28 ms) : 0, 8280
IAST [baseline] (20.05 ms) : 0, 20050
IAST [candidate] (18.386 ms) : 0, 18386

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.26.0-SNAPSHOT~3129c2c396, baseline=1.26.0-SNAPSHOT~6bb350544f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.05 s) : 0, 1049991
Total [baseline] (9.349 s) : 0, 9349073
Agent [candidate] (1.048 s) : 0, 1048439
Total [candidate] (9.326 s) : 0, 9326353
section appsec
Agent [baseline] (1.151 s) : 0, 1150734
Total [baseline] (9.532 s) : 0, 9531636
Agent [candidate] (1.152 s) : 0, 1151730
Total [candidate] (9.574 s) : 0, 9574014
section iast
Agent [baseline] (1.166 s) : 0, 1166486
Total [baseline] (9.546 s) : 0, 9545751
Agent [candidate] (1.162 s) : 0, 1162378
Total [candidate] (9.549 s) : 0, 9548755
section profiling
Agent [baseline] (1.244 s) : 0, 1243666
Total [baseline] (9.639 s) : 0, 9639212
Agent [candidate] (1.265 s) : 0, 1265145
Total [candidate] (9.604 s) : 0, 9604253

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.05 s	-
Agent	appsec	1.151 s	100.743 ms (9.6%)
Agent	iast	1.166 s	116.495 ms (11.1%)
Agent	profiling	1.244 s	193.674 ms (18.4%)
Total	tracing	9.349 s	-
Total	appsec	9.532 s	182.562 ms (2.0%)
Total	iast	9.546 s	196.678 ms (2.1%)
Total	profiling	9.639 s	290.139 ms (3.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.152 s	103.291 ms (9.9%)
Agent	iast	1.162 s	113.939 ms (10.9%)
Agent	profiling	1.265 s	216.706 ms (20.7%)
Total	tracing	9.326 s	-
Total	appsec	9.574 s	247.661 ms (2.7%)
Total	iast	9.549 s	222.402 ms (2.4%)
Total	profiling	9.604 s	277.9 ms (3.0%)

gantt
    title petclinic - break down per module: candidate=1.26.0-SNAPSHOT~3129c2c396, baseline=1.26.0-SNAPSHOT~6bb350544f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (648.751 ms) : 0, 648751
BytebuddyAgent [candidate] (648.06 ms) : 0, 648060
GlobalTracer [baseline] (307.747 ms) : 0, 307747
GlobalTracer [candidate] (306.995 ms) : 0, 306995
AppSec [baseline] (51.224 ms) : 0, 51224
AppSec [candidate] (51.289 ms) : 0, 51289
Remote Config [baseline] (673.645 µs) : 0, 674
Remote Config [candidate] (676.363 µs) : 0, 676
Telemetry [baseline] (7.266 ms) : 0, 7266
Telemetry [candidate] (7.167 ms) : 0, 7167
section appsec
BytebuddyAgent [baseline] (651.626 ms) : 0, 651626
BytebuddyAgent [candidate] (651.928 ms) : 0, 651928
GlobalTracer [baseline] (307.698 ms) : 0, 307698
GlobalTracer [candidate] (308.231 ms) : 0, 308231
AppSec [baseline] (149.424 ms) : 0, 149424
AppSec [candidate] (149.457 ms) : 0, 149457
Remote Config [baseline] (648.876 µs) : 0, 649
Remote Config [candidate] (646.777 µs) : 0, 647
Telemetry [baseline] (6.899 ms) : 0, 6899
Telemetry [candidate] (6.9 ms) : 0, 6900
section iast
BytebuddyAgent [baseline] (769.226 ms) : 0, 769226
BytebuddyAgent [candidate] (766.122 ms) : 0, 766122
GlobalTracer [baseline] (285.288 ms) : 0, 285288
GlobalTracer [candidate] (284.922 ms) : 0, 284922
AppSec [baseline] (49.123 ms) : 0, 49123
AppSec [candidate] (48.889 ms) : 0, 48889
Remote Config [baseline] (615.62 µs) : 0, 616
Remote Config [candidate] (612.346 µs) : 0, 612
Telemetry [baseline] (6.516 ms) : 0, 6516
Telemetry [candidate] (7.185 ms) : 0, 7185
IAST [baseline] (21.34 ms) : 0, 21340
IAST [candidate] (20.542 ms) : 0, 20542
section profiling
ProfilingAgent [baseline] (94.253 ms) : 0, 94253
ProfilingAgent [candidate] (95.738 ms) : 0, 95738
BytebuddyAgent [baseline] (658.332 ms) : 0, 658332
BytebuddyAgent [candidate] (671.656 ms) : 0, 671656
GlobalTracer [baseline] (377.168 ms) : 0, 377168
GlobalTracer [candidate] (381.921 ms) : 0, 381921
AppSec [baseline] (51.538 ms) : 0, 51538
AppSec [candidate] (52.082 ms) : 0, 52082
Remote Config [baseline] (672.708 µs) : 0, 673
Remote Config [candidate] (675.354 µs) : 0, 675
Telemetry [baseline] (7.486 ms) : 0, 7486
Telemetry [candidate] (7.572 ms) : 0, 7572
Profiling [baseline] (94.279 ms) : 0, 94279
Profiling [candidate] (95.763 ms) : 0, 95763

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2023-12-18T12:56:29	2023-12-18T13:13:05
git_branch	master	mcculls/tracer-flare-triage-report
git_commit_date	1702895354	1702902832
git_commit_sha	`6bb3505`	`3129c2c`
release_version	1.26.0-SNAPSHOT~6bb350544f	1.26.0-SNAPSHOT~3129c2c396
start_time	2023-12-18T12:56:16	2023-12-18T13:12:51

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702905439	1702905439
ci_job_id	391778374	391778374
ci_pipeline_id	25236002	25236002
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~3129c2c396, baseline=1.26.0-SNAPSHOT~6bb350544f
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.074 µs) : 346, 385
.   : milestone, 365,
iast (482.669 µs) : 462, 504
.   : milestone, 483,
iast_FULL (539.195 µs) : 519, 560
.   : milestone, 539,
iast_INACTIVE (445.061 µs) : 425, 466
.   : milestone, 445,
iast_TELEMETRY_OFF (472.642 µs) : 452, 494
.   : milestone, 473,
tracing (442.33 µs) : 422, 463
.   : milestone, 442,
section candidate
no_agent (370.718 µs) : 351, 391
.   : milestone, 371,
iast (478.872 µs) : 458, 500
.   : milestone, 479,
iast_FULL (539.955 µs) : 520, 560
.   : milestone, 540,
iast_INACTIVE (453.228 µs) : 432, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (470.768 µs) : 450, 491
.   : milestone, 471,
tracing (446.403 µs) : 425, 467
.   : milestone, 446,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	365.074 µs [345.602 µs, 384.547 µs]	-
iast	482.669 µs [461.565 µs, 503.773 µs]	117.595 µs (32.2%)
iast_FULL	539.195 µs [518.773 µs, 559.618 µs]	174.121 µs (47.7%)
iast_INACTIVE	445.061 µs [424.556 µs, 465.565 µs]	79.986 µs (21.9%)
iast_TELEMETRY_OFF	472.642 µs [451.781 µs, 493.502 µs]	107.567 µs (29.5%)
tracing	442.33 µs [421.608 µs, 463.051 µs]	77.255 µs (21.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	370.718 µs [350.688 µs, 390.749 µs]	-
iast	478.872 µs [457.965 µs, 499.779 µs]	108.153 µs (29.2%)
iast_FULL	539.955 µs [519.516 µs, 560.395 µs]	169.237 µs (45.7%)
iast_INACTIVE	453.228 µs [432.018 µs, 474.437 µs]	82.509 µs (22.3%)
iast_TELEMETRY_OFF	470.768 µs [450.438 µs, 491.097 µs]	100.05 µs (27.0%)
tracing	446.403 µs [425.45 µs, 467.357 µs]	75.685 µs (20.4%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~3129c2c396, baseline=1.26.0-SNAPSHOT~6bb350544f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.372 ms) : 1353, 1392
.   : milestone, 1372,
appsec (1.759 ms) : 1732, 1785
.   : milestone, 1759,
iast (1.54 ms) : 1516, 1564
.   : milestone, 1540,
profiling (1.494 ms) : 1469, 1520
.   : milestone, 1494,
tracing (1.499 ms) : 1475, 1523
.   : milestone, 1499,
section candidate
no_agent (1.368 ms) : 1349, 1388
.   : milestone, 1368,
appsec (1.757 ms) : 1731, 1783
.   : milestone, 1757,
iast (1.542 ms) : 1517, 1566
.   : milestone, 1542,
profiling (1.518 ms) : 1492, 1545
.   : milestone, 1518,
tracing (1.533 ms) : 1509, 1558
.   : milestone, 1533,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.372 ms [1.353 ms, 1.392 ms]	-
appsec	1.759 ms [1.732 ms, 1.785 ms]	386.181 µs (28.1%)
iast	1.54 ms [1.516 ms, 1.564 ms]	167.589 µs (12.2%)
profiling	1.494 ms [1.469 ms, 1.52 ms]	121.914 µs (8.9%)
tracing	1.499 ms [1.475 ms, 1.523 ms]	126.634 µs (9.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.368 ms [1.349 ms, 1.388 ms]	-
appsec	1.757 ms [1.731 ms, 1.783 ms]	388.693 µs (28.4%)
iast	1.542 ms [1.517 ms, 1.566 ms]	173.237 µs (12.7%)
profiling	1.518 ms [1.492 ms, 1.545 ms]	150.039 µs (11.0%)
tracing	1.533 ms [1.509 ms, 1.558 ms]	164.833 µs (12.0%)

mcculls added tag: do not merge Do not merge changes tag: diagnostics Diagnostics related changes labels Dec 13, 2023

mcculls force-pushed the mcculls/tracer-flare-triage-report branch 11 times, most recently from be03e3d to 6840666 Compare December 18, 2023 09:45

Replace placeholder tracer-flare zip with live content

3129c2c

mcculls force-pushed the mcculls/tracer-flare-triage-report branch from 6840666 to 3129c2c Compare December 18, 2023 12:34

mcculls changed the title ~~[WIP] Replace placeholder tracer-flare zip with real content~~ Populate tracer-flare zip with configuration details Dec 18, 2023

mcculls removed the tag: do not merge Do not merge changes label Dec 18, 2023

mcculls marked this pull request as ready for review December 18, 2023 13:10

mcculls requested a review from a team as a code owner December 18, 2023 13:10

mcculls requested review from am312 and nayeem-kamal December 18, 2023 13:10

am312 approved these changes Dec 18, 2023

View reviewed changes

mcculls merged commit 94794cc into master Dec 18, 2023

mcculls deleted the mcculls/tracer-flare-triage-report branch December 18, 2023 15:42

github-actions Bot added this to the 1.26.0 milestone Dec 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Populate tracer-flare zip with configuration details#6359

Populate tracer-flare zip with configuration details#6359
mcculls merged 1 commit into
masterfrom
mcculls/tracer-flare-triage-report

mcculls commented Dec 13, 2023 •

edited by atlassian Bot

Loading

Uh oh!

pr-commenter Bot commented Dec 13, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

mcculls commented Dec 13, 2023 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Dec 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mcculls commented Dec 13, 2023 •

edited by atlassian Bot

Loading

pr-commenter Bot commented Dec 13, 2023 •

edited

Loading