Mark runEnabledWithDatadogAgent @Flaky by jpbempel · Pull Request #9948 · DataDog/dd-trace-java

jpbempel · 2025-11-13T05:54:49Z

very flaky

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-11-13T05:54:59Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

pr-commenter · 2025-11-13T06:15:01Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1763025132	1763025467
end_time	2025-11-13T09:13:36	2025-11-13T09:19:10
git_branch	master	jpbempel/mark-flaky
git_commit_sha	`b99070a`	`32d72a1`
start_time	2025-11-13T09:12:13	2025-11-13T09:17:48

See matching parameters

	Baseline	Candidate
ci_job_id	1231952391	1231952391
ci_pipeline_id	82210317	82210317
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1763024500	1763024500

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 7 unstable metrics.

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-25772.004ns; +27113.577ns] or [-8.794%; +9.252%]	unstable [-34.898µs; +43.293µs] or [-10.531%; +13.065%]	unstable [-46.031µs; +58.753µs] or [-13.318%; +16.999%]	unstable [-129.489µs; +157.961µs] or [-12.749%; +15.552%]	same
scenario:basic	same	same	same	unstable [-152.987µs; +16.018µs] or [-17.546%; +1.837%]	unstable [-130.158op/s; +130.158op/s] or [-5.076%; +5.076%]
scenario:loop	same	same	same	unstable [-621.312µs; +772.772µs] or [-7.024%; +8.736%]	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (331.364 µs) : 300, 363
.   : milestone, 331,
basic (292.422 µs) : 285, 300
.   : milestone, 292,
loop (8.425 ms) : 8363, 8488
.   : milestone, 8425,
section candidate
noprobe (335.562 µs) : 294, 377
.   : milestone, 336,
basic (293.948 µs) : 286, 301
.   : milestone, 294,
loop (8.434 ms) : 8373, 8495
.   : milestone, 8434,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	331.364 µs [299.596 µs, 363.132 µs]
basic	292.422 µs [285.297 µs, 299.548 µs]
loop	8.425 ms [8.363 ms, 8.488 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	335.562 µs [293.643 µs, 377.481 µs]
basic	293.948 µs [286.438 µs, 301.458 µs]
loop	8.434 ms [8.373 ms, 8.495 ms]

datadog-datadog-prod-us1 · 2025-11-13T06:47:26Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 81.32% (+21.71%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 32d72a1 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

pr-commenter · 2025-11-13T07:15:59Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/mark-flaky
git_commit_date	1762997139	1763024500
git_commit_sha	`b99070a`	`32d72a1`
release_version	1.56.0-SNAPSHOT~b99070a00c	1.56.0-SNAPSHOT~32d72a1369

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763026417	1763026417
ci_job_id	1231952382	1231952382
ci_pipeline_id	82210317	82210317
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-wasz5ine 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-wasz5ine 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 8 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.11 s) : 0, 1110357
Total [baseline] (10.793 s) : 0, 10792561
Agent [candidate] (1.103 s) : 0, 1102831
Total [candidate] (10.685 s) : 0, 10685192
section appsec
Agent [baseline] (1.279 s) : 0, 1278541
Total [baseline] (10.98 s) : 0, 10979596
Agent [candidate] (1.288 s) : 0, 1288262
Total [candidate] (11.116 s) : 0, 11116297
section iast
Agent [baseline] (1.244 s) : 0, 1243954
Total [baseline] (11.225 s) : 0, 11225477
Agent [candidate] (1.243 s) : 0, 1242708
Total [candidate] (11.199 s) : 0, 11199407
section profiling
Agent [baseline] (1.237 s) : 0, 1236654
Total [baseline] (10.996 s) : 0, 10996104
Agent [candidate] (1.232 s) : 0, 1232368
Total [candidate] (11.095 s) : 0, 11095048

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.11 s	-
Agent	appsec	1.279 s	168.185 ms (15.1%)
Agent	iast	1.244 s	133.597 ms (12.0%)
Agent	profiling	1.237 s	126.297 ms (11.4%)
Total	tracing	10.793 s	-
Total	appsec	10.98 s	187.035 ms (1.7%)
Total	iast	11.225 s	432.916 ms (4.0%)
Total	profiling	10.996 s	203.544 ms (1.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.103 s	-
Agent	appsec	1.288 s	185.431 ms (16.8%)
Agent	iast	1.243 s	139.877 ms (12.7%)
Agent	profiling	1.232 s	129.536 ms (11.7%)
Total	tracing	10.685 s	-
Total	appsec	11.116 s	431.105 ms (4.0%)
Total	iast	11.199 s	514.215 ms (4.8%)
Total	profiling	11.095 s	409.856 ms (3.8%)

gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.456 ms) : 0, 1456
BytebuddyAgent [baseline] (710.399 ms) : 0, 710399
BytebuddyAgent [candidate] (704.994 ms) : 0, 704994
GlobalTracer [baseline] (249.432 ms) : 0, 249432
GlobalTracer [candidate] (248.268 ms) : 0, 248268
AppSec [baseline] (32.335 ms) : 0, 32335
AppSec [candidate] (32.106 ms) : 0, 32106
Debugger [baseline] (69.293 ms) : 0, 69293
Debugger [candidate] (68.807 ms) : 0, 68807
Remote Config [baseline] (656.167 µs) : 0, 656
Remote Config [candidate] (636.43 µs) : 0, 636
Telemetry [baseline] (8.21 ms) : 0, 8210
Telemetry [candidate] (8.176 ms) : 0, 8176
Flare Poller [baseline] (3.717 ms) : 0, 3717
Flare Poller [candidate] (3.761 ms) : 0, 3761
section appsec
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.479 ms) : 0, 1479
BytebuddyAgent [baseline] (728.647 ms) : 0, 728647
BytebuddyAgent [candidate] (735.39 ms) : 0, 735390
GlobalTracer [baseline] (240.707 ms) : 0, 240707
GlobalTracer [candidate] (242.704 ms) : 0, 242704
AppSec [baseline] (174.112 ms) : 0, 174112
AppSec [candidate] (174.41 ms) : 0, 174410
Debugger [baseline] (61.142 ms) : 0, 61142
Debugger [candidate] (61.097 ms) : 0, 61097
Remote Config [baseline] (657.398 µs) : 0, 657
Remote Config [candidate] (655.337 µs) : 0, 655
Telemetry [baseline] (8.386 ms) : 0, 8386
Telemetry [candidate] (8.368 ms) : 0, 8368
Flare Poller [baseline] (3.847 ms) : 0, 3847
Flare Poller [candidate] (3.972 ms) : 0, 3972
IAST [baseline] (24.657 ms) : 0, 24657
IAST [candidate] (25.299 ms) : 0, 25299
section iast
crashtracking [baseline] (1.454 ms) : 0, 1454
crashtracking [candidate] (1.456 ms) : 0, 1456
BytebuddyAgent [baseline] (830.488 ms) : 0, 830488
BytebuddyAgent [candidate] (829.648 ms) : 0, 829648
GlobalTracer [baseline] (238.206 ms) : 0, 238206
GlobalTracer [candidate] (238.635 ms) : 0, 238635
AppSec [baseline] (29.584 ms) : 0, 29584
AppSec [candidate] (31.146 ms) : 0, 31146
Debugger [baseline] (66.095 ms) : 0, 66095
Debugger [candidate] (65.713 ms) : 0, 65713
Remote Config [baseline] (553.166 µs) : 0, 553
Remote Config [candidate] (549.495 µs) : 0, 549
Telemetry [baseline] (7.696 ms) : 0, 7696
Telemetry [candidate] (7.73 ms) : 0, 7730
Flare Poller [baseline] (3.602 ms) : 0, 3602
Flare Poller [candidate] (3.549 ms) : 0, 3549
IAST [baseline] (31.679 ms) : 0, 31679
IAST [candidate] (29.588 ms) : 0, 29588
section profiling
ProfilingAgent [baseline] (96.522 ms) : 0, 96522
ProfilingAgent [candidate] (96.63 ms) : 0, 96630
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.444 ms) : 0, 1444
BytebuddyAgent [baseline] (733.474 ms) : 0, 733474
BytebuddyAgent [candidate] (728.169 ms) : 0, 728169
GlobalTracer [baseline] (221.934 ms) : 0, 221934
GlobalTracer [candidate] (222.759 ms) : 0, 222759
AppSec [baseline] (32.453 ms) : 0, 32453
AppSec [candidate] (32.429 ms) : 0, 32429
Debugger [baseline] (68.26 ms) : 0, 68260
Debugger [candidate] (68.456 ms) : 0, 68456
Remote Config [baseline] (638.812 µs) : 0, 639
Remote Config [candidate] (644.389 µs) : 0, 644
Telemetry [baseline] (8.055 ms) : 0, 8055
Telemetry [candidate] (8.028 ms) : 0, 8028
Flare Poller [baseline] (3.832 ms) : 0, 3832
Flare Poller [candidate] (3.873 ms) : 0, 3873
Profiling [baseline] (97.093 ms) : 0, 97093
Profiling [candidate] (97.196 ms) : 0, 97196

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.116 s) : 0, 1116201
Total [baseline] (8.819 s) : 0, 8819499
Agent [candidate] (1.105 s) : 0, 1104632
Total [candidate] (8.867 s) : 0, 8867456
section iast
Agent [baseline] (1.248 s) : 0, 1248284
Total [baseline] (9.539 s) : 0, 9538825
Agent [candidate] (1.241 s) : 0, 1240665
Total [candidate] (9.537 s) : 0, 9536745

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.116 s	-
Agent	iast	1.248 s	132.083 ms (11.8%)
Total	tracing	8.819 s	-
Total	iast	9.539 s	719.326 ms (8.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.105 s	-
Agent	iast	1.241 s	136.032 ms (12.3%)
Total	tracing	8.867 s	-
Total	iast	9.537 s	669.289 ms (7.5%)

gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.493 ms) : 0, 1493
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (715.191 ms) : 0, 715191
BytebuddyAgent [candidate] (707.24 ms) : 0, 707240
GlobalTracer [baseline] (251.296 ms) : 0, 251296
GlobalTracer [candidate] (248.543 ms) : 0, 248543
AppSec [baseline] (32.538 ms) : 0, 32538
AppSec [candidate] (32.248 ms) : 0, 32248
Debugger [baseline] (68.437 ms) : 0, 68437
Debugger [candidate] (68.08 ms) : 0, 68080
Remote Config [baseline] (639.875 µs) : 0, 640
Remote Config [candidate] (627.65 µs) : 0, 628
Telemetry [baseline] (8.087 ms) : 0, 8087
Telemetry [candidate] (8.162 ms) : 0, 8162
Flare Poller [baseline] (3.688 ms) : 0, 3688
Flare Poller [candidate] (3.673 ms) : 0, 3673
section iast
crashtracking [baseline] (1.494 ms) : 0, 1494
crashtracking [candidate] (1.444 ms) : 0, 1444
BytebuddyAgent [baseline] (834.882 ms) : 0, 834882
BytebuddyAgent [candidate] (828.45 ms) : 0, 828450
GlobalTracer [baseline] (238.22 ms) : 0, 238220
GlobalTracer [candidate] (237.44 ms) : 0, 237440
IAST [baseline] (31.591 ms) : 0, 31591
IAST [candidate] (27.544 ms) : 0, 27544
AppSec [baseline] (30.33 ms) : 0, 30330
AppSec [candidate] (34.183 ms) : 0, 34183
Debugger [baseline] (65.164 ms) : 0, 65164
Debugger [candidate] (65.037 ms) : 0, 65037
Remote Config [baseline] (552.036 µs) : 0, 552
Remote Config [candidate] (563.269 µs) : 0, 563
Telemetry [baseline] (7.728 ms) : 0, 7728
Telemetry [candidate] (7.716 ms) : 0, 7716
Flare Poller [baseline] (3.568 ms) : 0, 3568
Flare Poller [candidate] (3.56 ms) : 0, 3560

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/mark-flaky
git_commit_date	1762997139	1763024500
git_commit_sha	`b99070a`	`32d72a1`
release_version	1.56.0-SNAPSHOT~b99070a00c	1.56.0-SNAPSHOT~32d72a1369

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763026823	1763026823
ci_job_id	1231952383	1231952383
ci_pipeline_id	82210317	82210317
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-6ijvnl9o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-6ijvnl9o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 19 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.568 ms) : 17392, 17744
.   : milestone, 17568,
appsec (18.782 ms) : 18590, 18975
.   : milestone, 18782,
code_origins (18.116 ms) : 17935, 18298
.   : milestone, 18116,
iast (17.787 ms) : 17609, 17965
.   : milestone, 17787,
profiling (18.976 ms) : 18785, 19168
.   : milestone, 18976,
tracing (17.585 ms) : 17415, 17755
.   : milestone, 17585,
section candidate
no_agent (18.229 ms) : 18041, 18417
.   : milestone, 18229,
appsec (18.62 ms) : 18431, 18810
.   : milestone, 18620,
code_origins (17.812 ms) : 17633, 17991
.   : milestone, 17812,
iast (17.67 ms) : 17494, 17846
.   : milestone, 17670,
profiling (18.336 ms) : 18156, 18515
.   : milestone, 18336,
tracing (18.038 ms) : 17859, 18218
.   : milestone, 18038,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.568 ms [17.392 ms, 17.744 ms]	-
appsec	18.782 ms [18.59 ms, 18.975 ms]	1.215 ms (6.9%)
code_origins	18.116 ms [17.935 ms, 18.298 ms]	548.539 µs (3.1%)
iast	17.787 ms [17.609 ms, 17.965 ms]	219.419 µs (1.2%)
profiling	18.976 ms [18.785 ms, 19.168 ms]	1.408 ms (8.0%)
tracing	17.585 ms [17.415 ms, 17.755 ms]	17.076 µs (0.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.229 ms [18.041 ms, 18.417 ms]	-
appsec	18.62 ms [18.431 ms, 18.81 ms]	391.675 µs (2.1%)
code_origins	17.812 ms [17.633 ms, 17.991 ms]	-416.726 µs (-2.3%)
iast	17.67 ms [17.494 ms, 17.846 ms]	-558.498 µs (-3.1%)
profiling	18.336 ms [18.156 ms, 18.515 ms]	106.79 µs (0.6%)
tracing	18.038 ms [17.859 ms, 18.218 ms]	-190.507 µs (-1.0%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.197 ms) : 1185, 1208
.   : milestone, 1197,
iast (3.194 ms) : 3151, 3237
.   : milestone, 3194,
iast_FULL (5.534 ms) : 5479, 5589
.   : milestone, 5534,
iast_GLOBAL (3.575 ms) : 3468, 3681
.   : milestone, 3575,
profiling (2.098 ms) : 2080, 2117
.   : milestone, 2098,
tracing (1.804 ms) : 1789, 1819
.   : milestone, 1804,
section candidate
no_agent (1.191 ms) : 1179, 1202
.   : milestone, 1191,
iast (3.205 ms) : 3106, 3304
.   : milestone, 3205,
iast_FULL (5.717 ms) : 5661, 5774
.   : milestone, 5717,
iast_GLOBAL (3.588 ms) : 3537, 3640
.   : milestone, 3588,
profiling (2.175 ms) : 2154, 2197
.   : milestone, 2175,
tracing (1.797 ms) : 1783, 1812
.   : milestone, 1797,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.197 ms [1.185 ms, 1.208 ms]	-
iast	3.194 ms [3.151 ms, 3.237 ms]	1.997 ms (166.9%)
iast_FULL	5.534 ms [5.479 ms, 5.589 ms]	4.338 ms (362.4%)
iast_GLOBAL	3.575 ms [3.468 ms, 3.681 ms]	2.378 ms (198.7%)
profiling	2.098 ms [2.08 ms, 2.117 ms]	901.658 µs (75.3%)
tracing	1.804 ms [1.789 ms, 1.819 ms]	607.561 µs (50.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.191 ms [1.179 ms, 1.202 ms]	-
iast	3.205 ms [3.106 ms, 3.304 ms]	2.014 ms (169.2%)
iast_FULL	5.717 ms [5.661 ms, 5.774 ms]	4.527 ms (380.2%)
iast_GLOBAL	3.588 ms [3.537 ms, 3.64 ms]	2.397 ms (201.4%)
profiling	2.175 ms [2.154 ms, 2.197 ms]	984.863 µs (82.7%)
tracing	1.797 ms [1.783 ms, 1.812 ms]	606.789 µs (51.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/mark-flaky
git_commit_date	1762997139	1763024500
git_commit_sha	`b99070a`	`32d72a1`
release_version	1.56.0-SNAPSHOT~b99070a00c	1.56.0-SNAPSHOT~32d72a1369

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1763026594	1763026594
ci_job_id	1231952385	1231952385
ci_pipeline_id	82210317	82210317
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-ue0w4bqo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-ue0w4bqo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.485 ms) : 1473, 1496
.   : milestone, 1485,
appsec (3.659 ms) : 3446, 3872
.   : milestone, 3659,
iast (2.216 ms) : 2152, 2280
.   : milestone, 2216,
iast_GLOBAL (2.27 ms) : 2206, 2335
.   : milestone, 2270,
profiling (2.066 ms) : 2015, 2117
.   : milestone, 2066,
tracing (2.036 ms) : 1986, 2085
.   : milestone, 2036,
section candidate
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (3.725 ms) : 3507, 3943
.   : milestone, 3725,
iast (2.213 ms) : 2149, 2276
.   : milestone, 2213,
iast_GLOBAL (2.265 ms) : 2201, 2329
.   : milestone, 2265,
profiling (2.081 ms) : 2028, 2134
.   : milestone, 2081,
tracing (2.027 ms) : 1978, 2077
.   : milestone, 2027,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.485 ms [1.473 ms, 1.496 ms]	-
appsec	3.659 ms [3.446 ms, 3.872 ms]	2.174 ms (146.5%)
iast	2.216 ms [2.152 ms, 2.28 ms]	731.515 µs (49.3%)
iast_GLOBAL	2.27 ms [2.206 ms, 2.335 ms]	785.791 µs (52.9%)
profiling	2.066 ms [2.015 ms, 2.117 ms]	581.534 µs (39.2%)
tracing	2.036 ms [1.986 ms, 2.085 ms]	550.924 µs (37.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.469 ms, 1.492 ms]	-
appsec	3.725 ms [3.507 ms, 3.943 ms]	2.245 ms (151.7%)
iast	2.213 ms [2.149 ms, 2.276 ms]	732.642 µs (49.5%)
iast_GLOBAL	2.265 ms [2.201 ms, 2.329 ms]	784.75 µs (53.0%)
profiling	2.081 ms [2.028 ms, 2.134 ms]	600.984 µs (40.6%)
tracing	2.027 ms [1.978 ms, 2.077 ms]	547.221 µs (37.0%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~32d72a1369, baseline=1.56.0-SNAPSHOT~b99070a00c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.411 s) : 15411000, 15411000
.   : milestone, 15411000,
appsec (14.603 s) : 14603000, 14603000
.   : milestone, 14603000,
iast (18.418 s) : 18418000, 18418000
.   : milestone, 18418000,
iast_GLOBAL (18.002 s) : 18002000, 18002000
.   : milestone, 18002000,
profiling (14.987 s) : 14987000, 14987000
.   : milestone, 14987000,
tracing (14.805 s) : 14805000, 14805000
.   : milestone, 14805000,
section candidate
no_agent (15.517 s) : 15517000, 15517000
.   : milestone, 15517000,
appsec (14.579 s) : 14579000, 14579000
.   : milestone, 14579000,
iast (18.078 s) : 18078000, 18078000
.   : milestone, 18078000,
iast_GLOBAL (17.906 s) : 17906000, 17906000
.   : milestone, 17906000,
profiling (15.129 s) : 15129000, 15129000
.   : milestone, 15129000,
tracing (14.947 s) : 14947000, 14947000
.   : milestone, 14947000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.411 s [15.411 s, 15.411 s]	-
appsec	14.603 s [14.603 s, 14.603 s]	-808.0 ms (-5.2%)
iast	18.418 s [18.418 s, 18.418 s]	3.007 s (19.5%)
iast_GLOBAL	18.002 s [18.002 s, 18.002 s]	2.591 s (16.8%)
profiling	14.987 s [14.987 s, 14.987 s]	-424.0 ms (-2.8%)
tracing	14.805 s [14.805 s, 14.805 s]	-606.0 ms (-3.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.517 s [15.517 s, 15.517 s]	-
appsec	14.579 s [14.579 s, 14.579 s]	-938.0 ms (-6.0%)
iast	18.078 s [18.078 s, 18.078 s]	2.561 s (16.5%)
iast_GLOBAL	17.906 s [17.906 s, 17.906 s]	2.389 s (15.4%)
profiling	15.129 s [15.129 s, 15.129 s]	-388.0 ms (-2.5%)
tracing	14.947 s [14.947 s, 14.947 s]	-570.0 ms (-3.7%)

very flaky

PerfectSlayer

Feel free to reference a JIRA card if you have one to follow up on the fix

jpbempel requested a review from a team as a code owner November 13, 2025 05:54

jpbempel requested review from shatzi and removed request for a team November 13, 2025 05:54

jpbempel added comp: debugger Dynamic Instrumentation type: bug Bug report and fix tag: no release notes Changes to exclude from release notes comp: testing Testing labels Nov 13, 2025

jpbempel force-pushed the jpbempel/mark-flaky branch from 98f3a2c to afff070 Compare November 13, 2025 06:28

Mark runEnabledWithDatadogAgent @flaky

32d72a1

very flaky

jpbempel force-pushed the jpbempel/mark-flaky branch from afff070 to 32d72a1 Compare November 13, 2025 09:01

PerfectSlayer added tag: flaky test Flaky tests tag: flaky test/disabled Disabled flaky tests and removed comp: testing Testing labels Nov 13, 2025

PerfectSlayer approved these changes Nov 13, 2025

View reviewed changes

jpbempel merged commit b112f74 into master Nov 13, 2025
551 of 552 checks passed

jpbempel deleted the jpbempel/mark-flaky branch November 13, 2025 11:04

github-actions Bot added this to the 1.56.0 milestone Nov 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mark runEnabledWithDatadogAgent @Flaky#9948

Mark runEnabledWithDatadogAgent @Flaky#9948
jpbempel merged 1 commit intomasterfrom
jpbempel/mark-flaky

jpbempel commented Nov 13, 2025

Uh oh!

github-actions Bot commented Nov 13, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Nov 13, 2025 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 Bot commented Nov 13, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Nov 13, 2025 •

edited

Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jpbempel commented Nov 13, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions Bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

datadog-datadog-prod-us1 Bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Nov 13, 2025 •

edited

Loading

pr-commenter Bot commented Nov 13, 2025 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Nov 13, 2025 •

edited

Loading

pr-commenter Bot commented Nov 13, 2025 •

edited

Loading