Add support for skip-EFD tagging by nikita-tkachenko-datadog · Pull Request #8487 · DataDog/dd-trace-java

nikita-tkachenko-datadog · 2025-03-03T16:52:26Z

What Does This Do

Implements possibility to disable EFD for a test case or a suite by marking them with datadog_skip_efd tag.
Marking should be done using the mechanism provided by the testing framework - similar to how it works for datadog_itr_unskippable tag.

Motivation

This is requested by DD Intellij Plugin that is dogfooding EFD.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-03-03T17:35:39Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/ignore-efd-tag
git_commit_date	1741082792	1741103696
git_commit_sha	`8f945b0`	`a617ea0`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~a617ea012f

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741106257	1741106257
ci_job_id	832074463	832074463
ci_pipeline_id	57662380	57662380
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-yzph2l9l-project-304-concurrent-1-88qlh9oj 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-yzph2l9l-project-304-concurrent-1-88qlh9oj 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.137 s) : 0, 1136792
Total [baseline] (11.443 s) : 0, 11442999
Agent [candidate] (1.137 s) : 0, 1136778
Total [candidate] (11.492 s) : 0, 11492314
section appsec
Agent [baseline] (1.274 s) : 0, 1273881
Total [baseline] (11.683 s) : 0, 11682769
Agent [candidate] (1.275 s) : 0, 1274962
Total [candidate] (11.646 s) : 0, 11646365
section iast
Agent [baseline] (1.258 s) : 0, 1257647
Total [baseline] (11.964 s) : 0, 11964270
Agent [candidate] (1.266 s) : 0, 1266380
Total [candidate] (12.01 s) : 0, 12010100
section profiling
Agent [baseline] (1.379 s) : 0, 1379360
Total [baseline] (11.772 s) : 0, 11771873
Agent [candidate] (1.387 s) : 0, 1387288
Total [candidate] (11.776 s) : 0, 11776019

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.137 s	-
Agent	appsec	1.274 s	137.089 ms (12.1%)
Agent	iast	1.258 s	120.856 ms (10.6%)
Agent	profiling	1.379 s	242.568 ms (21.3%)
Total	tracing	11.443 s	-
Total	appsec	11.683 s	239.771 ms (2.1%)
Total	iast	11.964 s	521.272 ms (4.6%)
Total	profiling	11.772 s	328.874 ms (2.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.137 s	-
Agent	appsec	1.275 s	138.184 ms (12.2%)
Agent	iast	1.266 s	129.601 ms (11.4%)
Agent	profiling	1.387 s	250.509 ms (22.0%)
Total	tracing	11.492 s	-
Total	appsec	11.646 s	154.051 ms (1.3%)
Total	iast	12.01 s	517.786 ms (4.5%)
Total	profiling	11.776 s	283.705 ms (2.5%)

gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.564 ms) : 0, 717564
BytebuddyAgent [candidate] (717.674 ms) : 0, 717674
GlobalTracer [baseline] (239.673 ms) : 0, 239673
GlobalTracer [candidate] (239.885 ms) : 0, 239885
AppSec [baseline] (55.641 ms) : 0, 55641
AppSec [candidate] (55.882 ms) : 0, 55882
Debugger [baseline] (95.812 ms) : 0, 95812
Debugger [candidate] (95.314 ms) : 0, 95314
Remote Config [baseline] (569.668 µs) : 0, 570
Remote Config [candidate] (583.585 µs) : 0, 584
Telemetry [baseline] (12.561 ms) : 0, 12561
Telemetry [candidate] (12.491 ms) : 0, 12491
section appsec
BytebuddyAgent [baseline] (736.428 ms) : 0, 736428
BytebuddyAgent [candidate] (736.555 ms) : 0, 736555
GlobalTracer [baseline] (237.053 ms) : 0, 237053
GlobalTracer [candidate] (237.202 ms) : 0, 237202
IAST [baseline] (21.357 ms) : 0, 21357
IAST [candidate] (21.452 ms) : 0, 21452
AppSec [baseline] (177.43 ms) : 0, 177430
AppSec [candidate] (177.571 ms) : 0, 177571
Debugger [baseline] (84.385 ms) : 0, 84385
Debugger [candidate] (84.893 ms) : 0, 84893
Remote Config [baseline] (638.845 µs) : 0, 639
Remote Config [candidate] (634.211 µs) : 0, 634
Telemetry [baseline] (12.276 ms) : 0, 12276
Telemetry [candidate] (12.4 ms) : 0, 12400
section iast
BytebuddyAgent [baseline] (835.61 ms) : 0, 835610
BytebuddyAgent [candidate] (841.098 ms) : 0, 841098
GlobalTracer [baseline] (230.482 ms) : 0, 230482
GlobalTracer [candidate] (231.554 ms) : 0, 231554
IAST [baseline] (22.858 ms) : 0, 22858
IAST [candidate] (23.068 ms) : 0, 23068
AppSec [baseline] (56.933 ms) : 0, 56933
AppSec [candidate] (57.194 ms) : 0, 57194
Debugger [baseline] (83.35 ms) : 0, 83350
Debugger [candidate] (83.652 ms) : 0, 83652
Remote Config [baseline] (517.737 µs) : 0, 518
Remote Config [candidate] (509.289 µs) : 0, 509
Telemetry [baseline] (12.952 ms) : 0, 12952
Telemetry [candidate] (14.2 ms) : 0, 14200
section profiling
BytebuddyAgent [baseline] (708.447 ms) : 0, 708447
BytebuddyAgent [candidate] (712.818 ms) : 0, 712818
GlobalTracer [baseline] (350.143 ms) : 0, 350143
GlobalTracer [candidate] (353.007 ms) : 0, 353007
AppSec [baseline] (55.696 ms) : 0, 55696
AppSec [candidate] (54.852 ms) : 0, 54852
Debugger [baseline] (88.636 ms) : 0, 88636
Debugger [candidate] (88.696 ms) : 0, 88696
Remote Config [baseline] (614.626 µs) : 0, 615
Remote Config [candidate] (613.856 µs) : 0, 614
Telemetry [baseline] (12.589 ms) : 0, 12589
Telemetry [candidate] (12.567 ms) : 0, 12567
ProfilingAgent [baseline] (122.976 ms) : 0, 122976
ProfilingAgent [candidate] (124.08 ms) : 0, 124080
Profiling [baseline] (122.997 ms) : 0, 122997
Profiling [candidate] (124.101 ms) : 0, 124101

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.138 s) : 0, 1137701
Total [baseline] (9.334 s) : 0, 9334051
Agent [candidate] (1.141 s) : 0, 1141415
Total [candidate] (9.259 s) : 0, 9258754
section iast
Agent [baseline] (1.26 s) : 0, 1259870
Total [baseline] (9.89 s) : 0, 9890373
Agent [candidate] (1.268 s) : 0, 1267942
Total [candidate] (9.872 s) : 0, 9871642
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.258 s) : 0, 1257566
Total [baseline] (9.819 s) : 0, 9818937
Agent [candidate] (1.267 s) : 0, 1267127
Total [candidate] (9.876 s) : 0, 9875826
section iast_TELEMETRY_OFF
Agent [baseline] (1.255 s) : 0, 1255410
Total [baseline] (9.866 s) : 0, 9866384
Agent [candidate] (1.264 s) : 0, 1263852
Total [candidate] (9.872 s) : 0, 9872243

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.138 s	-
Agent	iast	1.26 s	122.169 ms (10.7%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.258 s	119.864 ms (10.5%)
Agent	iast_TELEMETRY_OFF	1.255 s	117.709 ms (10.3%)
Total	tracing	9.334 s	-
Total	iast	9.89 s	556.322 ms (6.0%)
Total	iast_HARDCODED_SECRET_DISABLED	9.819 s	484.886 ms (5.2%)
Total	iast_TELEMETRY_OFF	9.866 s	532.334 ms (5.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.141 s	-
Agent	iast	1.268 s	126.527 ms (11.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.267 s	125.713 ms (11.0%)
Agent	iast_TELEMETRY_OFF	1.264 s	122.437 ms (10.7%)
Total	tracing	9.259 s	-
Total	iast	9.872 s	612.887 ms (6.6%)
Total	iast_HARDCODED_SECRET_DISABLED	9.876 s	617.071 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.872 s	613.489 ms (6.6%)

gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.939 ms) : 0, 717939
BytebuddyAgent [candidate] (721.231 ms) : 0, 721231
GlobalTracer [baseline] (240.112 ms) : 0, 240112
GlobalTracer [candidate] (240.824 ms) : 0, 240824
AppSec [baseline] (56.018 ms) : 0, 56018
AppSec [candidate] (55.507 ms) : 0, 55507
Debugger [baseline] (95.499 ms) : 0, 95499
Debugger [candidate] (95.734 ms) : 0, 95734
Remote Config [baseline] (596.572 µs) : 0, 597
Remote Config [candidate] (605.012 µs) : 0, 605
Telemetry [baseline] (12.588 ms) : 0, 12588
Telemetry [candidate] (12.417 ms) : 0, 12417
section iast
BytebuddyAgent [baseline] (836.136 ms) : 0, 836136
BytebuddyAgent [candidate] (842.224 ms) : 0, 842224
GlobalTracer [baseline] (230.381 ms) : 0, 230381
GlobalTracer [candidate] (231.568 ms) : 0, 231568
IAST [baseline] (22.801 ms) : 0, 22801
IAST [candidate] (23.019 ms) : 0, 23019
AppSec [baseline] (57.426 ms) : 0, 57426
AppSec [candidate] (57.548 ms) : 0, 57548
Debugger [baseline] (83.512 ms) : 0, 83512
Debugger [candidate] (83.733 ms) : 0, 83733
Remote Config [baseline] (506.423 µs) : 0, 506
Remote Config [candidate] (516.127 µs) : 0, 516
Telemetry [baseline] (14.123 ms) : 0, 14123
Telemetry [candidate] (14.259 ms) : 0, 14259
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (835.461 ms) : 0, 835461
BytebuddyAgent [candidate] (841.571 ms) : 0, 841571
GlobalTracer [baseline] (230.508 ms) : 0, 230508
GlobalTracer [candidate] (232.276 ms) : 0, 232276
IAST [baseline] (22.854 ms) : 0, 22854
IAST [candidate] (23.086 ms) : 0, 23086
AppSec [baseline] (56.879 ms) : 0, 56879
AppSec [candidate] (57.326 ms) : 0, 57326
Debugger [baseline] (83.549 ms) : 0, 83549
Debugger [candidate] (84.192 ms) : 0, 84192
Remote Config [baseline] (520.081 µs) : 0, 520
Remote Config [candidate] (538.102 µs) : 0, 538
Telemetry [baseline] (12.803 ms) : 0, 12803
Telemetry [candidate] (12.894 ms) : 0, 12894
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (834.698 ms) : 0, 834698
BytebuddyAgent [candidate] (839.914 ms) : 0, 839914
GlobalTracer [baseline] (229.879 ms) : 0, 229879
GlobalTracer [candidate] (231.979 ms) : 0, 231979
IAST [baseline] (22.926 ms) : 0, 22926
IAST [candidate] (24.115 ms) : 0, 24115
AppSec [baseline] (56.534 ms) : 0, 56534
AppSec [candidate] (55.823 ms) : 0, 55823
Debugger [baseline] (83.716 ms) : 0, 83716
Debugger [candidate] (84.221 ms) : 0, 84221
Remote Config [baseline] (527.786 µs) : 0, 528
Remote Config [candidate] (531.545 µs) : 0, 532
Telemetry [baseline] (12.134 ms) : 0, 12134
Telemetry [candidate] (12.269 ms) : 0, 12269

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-04T16:05:35	2025-03-04T16:13:31
git_branch	master	nikita-tkachenko/ignore-efd-tag
git_commit_date	1741082792	1741103696
git_commit_sha	`8f945b0`	`a617ea0`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~a617ea012f
start_time	2025-03-04T16:05:20	2025-03-04T16:13:17

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741105219	1741105219
ci_job_id	832074464	832074464
ci_pipeline_id	57662380	57662380
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-yzph2l9l-project-304-concurrent-2-yt3vbxbi 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-yzph2l9l-project-304-concurrent-2-yt3vbxbi 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (381.013 µs) : 361, 401
.   : milestone, 381,
iast (516.787 µs) : 495, 539
.   : milestone, 517,
iast_FULL (729.993 µs) : 708, 752
.   : milestone, 730,
iast_GLOBAL (570.349 µs) : 547, 593
.   : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (521.183 µs) : 499, 543
.   : milestone, 521,
iast_INACTIVE (466.763 µs) : 444, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (498.008 µs) : 476, 520
.   : milestone, 498,
tracing (459.739 µs) : 438, 481
.   : milestone, 460,
section candidate
no_agent (381.261 µs) : 361, 402
.   : milestone, 381,
iast (511.348 µs) : 489, 534
.   : milestone, 511,
iast_FULL (727.406 µs) : 705, 750
.   : milestone, 727,
iast_GLOBAL (563.43 µs) : 541, 586
.   : milestone, 563,
iast_HARDCODED_SECRET_DISABLED (517.697 µs) : 496, 539
.   : milestone, 518,
iast_INACTIVE (465.877 µs) : 444, 487
.   : milestone, 466,
iast_TELEMETRY_OFF (499.387 µs) : 478, 521
.   : milestone, 499,
tracing (455.575 µs) : 435, 477
.   : milestone, 456,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.013 µs [360.755 µs, 401.272 µs]	-
iast	516.787 µs [494.523 µs, 539.051 µs]	135.774 µs (35.6%)
iast_FULL	729.993 µs [707.783 µs, 752.204 µs]	348.98 µs (91.6%)
iast_GLOBAL	570.349 µs [547.236 µs, 593.463 µs]	189.336 µs (49.7%)
iast_HARDCODED_SECRET_DISABLED	521.183 µs [498.928 µs, 543.439 µs]	140.17 µs (36.8%)
iast_INACTIVE	466.763 µs [444.455 µs, 489.071 µs]	85.749 µs (22.5%)
iast_TELEMETRY_OFF	498.008 µs [476.135 µs, 519.882 µs]	116.995 µs (30.7%)
tracing	459.739 µs [438.309 µs, 481.169 µs]	78.725 µs (20.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.261 µs [360.503 µs, 402.02 µs]	-
iast	511.348 µs [489.053 µs, 533.643 µs]	130.087 µs (34.1%)
iast_FULL	727.406 µs [705.289 µs, 749.524 µs]	346.145 µs (90.8%)
iast_GLOBAL	563.43 µs [540.636 µs, 586.225 µs]	182.169 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED	517.697 µs [496.035 µs, 539.359 µs]	136.436 µs (35.8%)
iast_INACTIVE	465.877 µs [444.315 µs, 487.439 µs]	84.616 µs (22.2%)
iast_TELEMETRY_OFF	499.387 µs [477.657 µs, 521.118 µs]	118.126 µs (31.0%)
tracing	455.575 µs [434.568 µs, 476.581 µs]	74.313 µs (19.5%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.363 ms) : 1343, 1382
.   : milestone, 1363,
appsec (1.738 ms) : 1714, 1762
.   : milestone, 1738,
appsec_no_iast (1.737 ms) : 1713, 1761
.   : milestone, 1737,
code_origins (1.677 ms) : 1642, 1711
.   : milestone, 1677,
iast (1.527 ms) : 1502, 1552
.   : milestone, 1527,
profiling (1.576 ms) : 1551, 1602
.   : milestone, 1576,
tracing (1.48 ms) : 1456, 1504
.   : milestone, 1480,
section candidate
no_agent (1.349 ms) : 1330, 1368
.   : milestone, 1349,
appsec (1.743 ms) : 1719, 1767
.   : milestone, 1743,
appsec_no_iast (1.744 ms) : 1720, 1769
.   : milestone, 1744,
code_origins (1.701 ms) : 1667, 1734
.   : milestone, 1701,
iast (1.519 ms) : 1494, 1543
.   : milestone, 1519,
profiling (1.557 ms) : 1531, 1582
.   : milestone, 1557,
tracing (1.498 ms) : 1474, 1521
.   : milestone, 1498,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.363 ms [1.343 ms, 1.382 ms]	-
appsec	1.738 ms [1.714 ms, 1.762 ms]	375.066 µs (27.5%)
appsec_no_iast	1.737 ms [1.713 ms, 1.761 ms]	374.506 µs (27.5%)
code_origins	1.677 ms [1.642 ms, 1.711 ms]	313.901 µs (23.0%)
iast	1.527 ms [1.502 ms, 1.552 ms]	164.139 µs (12.0%)
profiling	1.576 ms [1.551 ms, 1.602 ms]	213.707 µs (15.7%)
tracing	1.48 ms [1.456 ms, 1.504 ms]	117.372 µs (8.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.349 ms [1.33 ms, 1.368 ms]	-
appsec	1.743 ms [1.719 ms, 1.767 ms]	393.792 µs (29.2%)
appsec_no_iast	1.744 ms [1.72 ms, 1.769 ms]	395.253 µs (29.3%)
code_origins	1.701 ms [1.667 ms, 1.734 ms]	351.554 µs (26.1%)
iast	1.519 ms [1.494 ms, 1.543 ms]	169.462 µs (12.6%)
profiling	1.557 ms [1.531 ms, 1.582 ms]	207.679 µs (15.4%)
tracing	1.498 ms [1.474 ms, 1.521 ms]	148.414 µs (11.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/ignore-efd-tag
git_commit_date	1741082792	1741103696
git_commit_sha	`8f945b0`	`a617ea0`
release_version	1.47.0-SNAPSHOT~8f945b02a7	1.47.0-SNAPSHOT~a617ea012f

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1741105771	1741105771
ci_job_id	832074467	832074467
ci_pipeline_id	57662380	57662380
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-yzph2l9l-project-304-concurrent-3-dww3hb5p 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-yzph2l9l-project-304-concurrent-3-dww3hb5p 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.299 s) : 15299000, 15299000
.   : milestone, 15299000,
appsec (15.171 s) : 15171000, 15171000
.   : milestone, 15171000,
iast (18.825 s) : 18825000, 18825000
.   : milestone, 18825000,
iast_GLOBAL (17.905 s) : 17905000, 17905000
.   : milestone, 17905000,
profiling (14.947 s) : 14947000, 14947000
.   : milestone, 14947000,
tracing (14.621 s) : 14621000, 14621000
.   : milestone, 14621000,
section candidate
no_agent (14.989 s) : 14989000, 14989000
.   : milestone, 14989000,
appsec (15.121 s) : 15121000, 15121000
.   : milestone, 15121000,
iast (18.901 s) : 18901000, 18901000
.   : milestone, 18901000,
iast_GLOBAL (18.193 s) : 18193000, 18193000
.   : milestone, 18193000,
profiling (14.965 s) : 14965000, 14965000
.   : milestone, 14965000,
tracing (14.77 s) : 14770000, 14770000
.   : milestone, 14770000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.299 s [15.299 s, 15.299 s]	-
appsec	15.171 s [15.171 s, 15.171 s]	-128.0 ms (-0.8%)
iast	18.825 s [18.825 s, 18.825 s]	3.526 s (23.0%)
iast_GLOBAL	17.905 s [17.905 s, 17.905 s]	2.606 s (17.0%)
profiling	14.947 s [14.947 s, 14.947 s]	-352.0 ms (-2.3%)
tracing	14.621 s [14.621 s, 14.621 s]	-678.0 ms (-4.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.989 s [14.989 s, 14.989 s]	-
appsec	15.121 s [15.121 s, 15.121 s]	132.0 ms (0.9%)
iast	18.901 s [18.901 s, 18.901 s]	3.912 s (26.1%)
iast_GLOBAL	18.193 s [18.193 s, 18.193 s]	3.204 s (21.4%)
profiling	14.965 s [14.965 s, 14.965 s]	-24.0 ms (-0.2%)
tracing	14.77 s [14.77 s, 14.77 s]	-219.0 ms (-1.5%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~a617ea012f, baseline=1.47.0-SNAPSHOT~8f945b02a7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.36 ms) : 2316, 2404
.   : milestone, 2360,
iast (2.121 ms) : 2066, 2177
.   : milestone, 2121,
iast_GLOBAL (2.167 ms) : 2111, 2223
.   : milestone, 2167,
profiling (1.986 ms) : 1942, 2030
.   : milestone, 1986,
tracing (1.966 ms) : 1923, 2009
.   : milestone, 1966,
section candidate
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (2.339 ms) : 2295, 2383
.   : milestone, 2339,
iast (2.125 ms) : 2069, 2181
.   : milestone, 2125,
iast_GLOBAL (2.169 ms) : 2113, 2225
.   : milestone, 2169,
profiling (1.983 ms) : 1938, 2027
.   : milestone, 1983,
tracing (1.969 ms) : 1926, 2012
.   : milestone, 1969,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.469 ms, 1.492 ms]	-
appsec	2.36 ms [2.316 ms, 2.404 ms]	879.473 µs (59.4%)
iast	2.121 ms [2.066 ms, 2.177 ms]	640.966 µs (43.3%)
iast_GLOBAL	2.167 ms [2.111 ms, 2.223 ms]	686.83 µs (46.4%)
profiling	1.986 ms [1.942 ms, 2.03 ms]	505.866 µs (34.2%)
tracing	1.966 ms [1.923 ms, 2.009 ms]	486.154 µs (32.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.465 ms, 1.488 ms]	-
appsec	2.339 ms [2.295 ms, 2.383 ms]	862.554 µs (58.4%)
iast	2.125 ms [2.069 ms, 2.181 ms]	648.629 µs (43.9%)
iast_GLOBAL	2.169 ms [2.113 ms, 2.225 ms]	692.112 µs (46.9%)
profiling	1.983 ms [1.938 ms, 2.027 ms]	505.884 µs (34.3%)
tracing	1.969 ms [1.926 ms, 2.012 ms]	492.316 µs (33.3%)

daniel-mohedano · 2025-03-04T08:22:13Z

Seems much simpler to generate the fixtures like this and we avoid pushing the uncommented code by mistake 👍 Thanks for fixing the issue the latestDepTest when using fixtures from test, just spotted it while generating fixtures myself.

Just to check, doesn't this cause issues with the following assert in datadog.trace.test.util.DDSpecification#setupSpec?

assert System.getenv().findAll { it.key.startsWith("DD_") }.isEmpty()

Good catch! Removed the prefix

pr-commenter · 2025-03-04T15:15:32Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1741104228	1741104598
end_time	2025-03-04T16:05:05	2025-03-04T16:11:15
git_branch	master	nikita-tkachenko/ignore-efd-tag
git_commit_sha	`8f945b0`	40cf744264e9f58669ddad77cf0eda4780ed7e44
start_time	2025-03-04T16:03:49	2025-03-04T16:09:59

See matching parameters

	Baseline	Candidate
ci_job_id	832069051	832069051
ci_pipeline_id	57662038	57662038
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1741103407	1741103407

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:loop	worse [+675.441µs; +695.774µs] or [+6.771%; +6.975%]	worse [+673.169µs; +752.297µs] or [+6.645%; +7.426%]	worse [+636.850µs; +749.901µs] or [+6.225%; +7.330%]	worse [+358.917µs; +1109.258µs] or [+3.380%; +10.447%]	worse [-7.686op/s; -4.671op/s] or [-7.916%; -4.811%]

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-30792.657ns; +30522.579ns] or [-11.277%; +11.178%]	unstable [-42744.435ns; +43469.825ns] or [-13.641%; +13.872%]	unstable [-55650.446ns; +55541.764ns] or [-16.884%; +16.851%]	unstable [-114.765µs; +117.163µs] or [-19.036%; +19.434%]	same
scenario:basic	same	same	same	unstable [-32.175µs; +67.694µs] or [-6.153%; +12.946%]	unstable [-123.478op/s; +251.683op/s] or [-4.939%; +10.067%]

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (313.361 µs) : 273, 353
.   : milestone, 313,
basic (308.735 µs) : 298, 320
.   : milestone, 309,
loop (10.13 ms) : 10088, 10173
.   : milestone, 10130,
section candidate
noprobe (313.724 µs) : 272, 356
.   : milestone, 314,
basic (305.328 µs) : 294, 317
.   : milestone, 305,
loop (10.843 ms) : 10813, 10874
.   : milestone, 10843,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	313.361 µs [273.43 µs, 353.292 µs]
basic	308.735 µs [297.709 µs, 319.761 µs]
loop	10.13 ms [10.088 ms, 10.173 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	313.724 µs [271.507 µs, 355.94 µs]
basic	305.328 µs [294.074 µs, 316.583 µs]
loop	10.843 ms [10.813 ms, 10.874 ms]

nikita-tkachenko-datadog added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels Mar 3, 2025

nikita-tkachenko-datadog marked this pull request as ready for review March 3, 2025 16:59

nikita-tkachenko-datadog requested a review from a team as a code owner March 3, 2025 16:59

nikita-tkachenko-datadog requested review from anmarchenko and daniel-mohedano March 3, 2025 16:59

anmarchenko approved these changes Mar 4, 2025

View reviewed changes

daniel-mohedano approved these changes Mar 4, 2025

View reviewed changes

nikita-tkachenko-datadog requested review from a team as code owners March 4, 2025 14:53

nikita-tkachenko-datadog requested review from PerfectSlayer and shatzi and removed request for a team March 4, 2025 14:53

nikita-tkachenko-datadog added 5 commits March 4, 2025 16:54

Add support for skip-EFD tagging

7e1c476

Fix test

8ba4d22

Remove unused imports

a06d082

Rename env var used for generating test fixtures

a52fe5c

Rename skip EFD tag

a617ea0

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/ignore-efd-tag branch from 40cf744 to a617ea0 Compare March 4, 2025 15:55

nikita-tkachenko-datadog removed request for a team, PerfectSlayer and shatzi March 4, 2025 15:55

Increase allowed tracer size

c398faa

nikita-tkachenko-datadog enabled auto-merge (squash) March 4, 2025 16:27

nikita-tkachenko-datadog merged commit 9966995 into master Mar 4, 2025

nikita-tkachenko-datadog deleted the nikita-tkachenko/ignore-efd-tag branch March 4, 2025 17:06

github-actions Bot added this to the 1.47.0 milestone Mar 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for skip-EFD tagging#8487

Add support for skip-EFD tagging#8487
nikita-tkachenko-datadog merged 6 commits intomasterfrom
nikita-tkachenko/ignore-efd-tag

nikita-tkachenko-datadog commented Mar 3, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Mar 3, 2025 •

edited

Loading

Uh oh!

daniel-mohedano Mar 4, 2025 •

edited

Loading

Uh oh!

daniel-mohedano Mar 4, 2025

Uh oh!

nikita-tkachenko-datadog Mar 4, 2025

Uh oh!

pr-commenter Bot commented Mar 4, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

nikita-tkachenko-datadog commented Mar 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Contributor Checklist

Uh oh!

pr-commenter Bot commented Mar 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

daniel-mohedano Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniel-mohedano Mar 4, 2025

Choose a reason for hiding this comment

Uh oh!

nikita-tkachenko-datadog Mar 4, 2025

Choose a reason for hiding this comment

Uh oh!

pr-commenter Bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nikita-tkachenko-datadog commented Mar 3, 2025 •

edited

Loading

pr-commenter Bot commented Mar 3, 2025 •

edited

Loading

daniel-mohedano Mar 4, 2025 •

edited

Loading

pr-commenter Bot commented Mar 4, 2025 •

edited

Loading