Don't use strict trace writing for vertx sql tests by amarziali · Pull Request #8662 · DataDog/dd-trace-java

amarziali · 2025-04-02T13:02:43Z

What Does This Do

Removes strict trace buffering since, when the root span is missing, the handler span (the result callback) is not guaranteed to finish before the sql call span (there can be wait when the CPU is busy especially in CI)

Relates to test #type without parent from VertxSqlClientForkedTest flakiness

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-04-02T13:49:59Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/flaky
git_commit_date	1743592817	1743598818
git_commit_sha	`32046a3`	`228f670`
release_version	1.48.0-SNAPSHOT~32046a39ed	1.48.0-SNAPSHOT~228f670d50

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1743601630	1743601630
ci_job_id	876993267	876993267
ci_pipeline_id	60795471	60795471
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-fsgpech5-project-304-concurrent-0-o4phy6o2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-fsgpech5-project-304-concurrent-0-o4phy6o2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058997
Total [baseline] (10.493 s) : 0, 10493368
Agent [candidate] (1.052 s) : 0, 1051833
Total [candidate] (10.572 s) : 0, 10571936
section appsec
Agent [baseline] (1.192 s) : 0, 1192463
Total [baseline] (10.743 s) : 0, 10743023
Agent [candidate] (1.192 s) : 0, 1191954
Total [candidate] (10.768 s) : 0, 10768226
section iast
Agent [baseline] (1.183 s) : 0, 1182624
Total [baseline] (11.045 s) : 0, 11045314
Agent [candidate] (1.185 s) : 0, 1184811
Total [candidate] (11.017 s) : 0, 11017133
section profiling
Agent [baseline] (1.283 s) : 0, 1283041
Total [baseline] (10.94 s) : 0, 10939569
Agent [candidate] (1.273 s) : 0, 1272587
Total [candidate] (10.84 s) : 0, 10839948

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.192 s	133.466 ms (12.6%)
Agent	iast	1.183 s	123.627 ms (11.7%)
Agent	profiling	1.283 s	224.044 ms (21.2%)
Total	tracing	10.493 s	-
Total	appsec	10.743 s	249.654 ms (2.4%)
Total	iast	11.045 s	551.946 ms (5.3%)
Total	profiling	10.94 s	446.201 ms (4.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	appsec	1.192 s	140.121 ms (13.3%)
Agent	iast	1.185 s	132.978 ms (12.6%)
Agent	profiling	1.273 s	220.754 ms (21.0%)
Total	tracing	10.572 s	-
Total	appsec	10.768 s	196.291 ms (1.9%)
Total	iast	11.017 s	445.197 ms (4.2%)
Total	profiling	10.84 s	268.012 ms (2.5%)

gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (722.303 ms) : 0, 722303
BytebuddyAgent [candidate] (718.617 ms) : 0, 718617
GlobalTracer [baseline] (240.489 ms) : 0, 240489
GlobalTracer [candidate] (239.195 ms) : 0, 239195
AppSec [baseline] (54.569 ms) : 0, 54569
AppSec [candidate] (54.164 ms) : 0, 54164
Debugger [baseline] (5.185 ms) : 0, 5185
Debugger [candidate] (5.131 ms) : 0, 5131
Remote Config [baseline] (702.846 µs) : 0, 703
Remote Config [candidate] (673.438 µs) : 0, 673
Telemetry [baseline] (15.111 ms) : 0, 15111
Telemetry [candidate] (13.504 ms) : 0, 13504
section appsec
BytebuddyAgent [baseline] (736.593 ms) : 0, 736593
BytebuddyAgent [candidate] (736.802 ms) : 0, 736802
GlobalTracer [baseline] (235.715 ms) : 0, 235715
GlobalTracer [candidate] (235.397 ms) : 0, 235397
AppSec [baseline] (175.509 ms) : 0, 175509
AppSec [candidate] (175.147 ms) : 0, 175147
Debugger [baseline] (4.284 ms) : 0, 4284
Debugger [candidate] (4.262 ms) : 0, 4262
Remote Config [baseline] (635.378 µs) : 0, 635
Remote Config [candidate] (647.028 µs) : 0, 647
Telemetry [baseline] (8.605 ms) : 0, 8605
Telemetry [candidate] (8.537 ms) : 0, 8537
IAST [baseline] (21.396 ms) : 0, 21396
IAST [candidate] (21.466 ms) : 0, 21466
section iast
BytebuddyAgent [baseline] (839.508 ms) : 0, 839508
BytebuddyAgent [candidate] (840.252 ms) : 0, 840252
GlobalTracer [baseline] (230.078 ms) : 0, 230078
GlobalTracer [candidate] (231.524 ms) : 0, 231524
AppSec [baseline] (56.166 ms) : 0, 56166
AppSec [candidate] (56.25 ms) : 0, 56250
Debugger [baseline] (4.173 ms) : 0, 4173
Debugger [candidate] (4.166 ms) : 0, 4166
Remote Config [baseline] (619.115 µs) : 0, 619
Remote Config [candidate] (613.098 µs) : 0, 613
Telemetry [baseline] (8.765 ms) : 0, 8765
Telemetry [candidate] (8.768 ms) : 0, 8768
IAST [baseline] (22.817 ms) : 0, 22817
IAST [candidate] (22.807 ms) : 0, 22807
section profiling
ProfilingAgent [baseline] (103.087 ms) : 0, 103087
ProfilingAgent [candidate] (103.007 ms) : 0, 103007
BytebuddyAgent [baseline] (714.591 ms) : 0, 714591
BytebuddyAgent [candidate] (708.438 ms) : 0, 708438
GlobalTracer [baseline] (350.88 ms) : 0, 350880
GlobalTracer [candidate] (348.331 ms) : 0, 348331
AppSec [baseline] (54.416 ms) : 0, 54416
AppSec [candidate] (53.176 ms) : 0, 53176
Debugger [baseline] (4.323 ms) : 0, 4323
Debugger [candidate] (4.308 ms) : 0, 4308
Remote Config [baseline] (713.938 µs) : 0, 714
Remote Config [candidate] (715.255 µs) : 0, 715
Telemetry [baseline] (9.055 ms) : 0, 9055
Telemetry [candidate] (8.992 ms) : 0, 8992
Profiling [baseline] (103.112 ms) : 0, 103112
Profiling [candidate] (103.032 ms) : 0, 103032

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058205
Total [baseline] (8.645 s) : 0, 8644787
Agent [candidate] (1.058 s) : 0, 1058271
Total [candidate] (8.659 s) : 0, 8659482
section iast
Agent [baseline] (1.179 s) : 0, 1179205
Total [baseline] (9.233 s) : 0, 9232983
Agent [candidate] (1.179 s) : 0, 1179396
Total [candidate] (9.243 s) : 0, 9243346
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.186 s) : 0, 1186263
Total [baseline] (9.191 s) : 0, 9191475
Agent [candidate] (1.18 s) : 0, 1179839
Total [candidate] (9.201 s) : 0, 9200745
section iast_TELEMETRY_OFF
Agent [baseline] (1.184 s) : 0, 1183540
Total [baseline] (9.227 s) : 0, 9226900
Agent [candidate] (1.2 s) : 0, 1199752
Total [candidate] (9.298 s) : 0, 9298490

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.179 s	121.0 ms (11.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.186 s	128.058 ms (12.1%)
Agent	iast_TELEMETRY_OFF	1.184 s	125.336 ms (11.8%)
Total	tracing	8.645 s	-
Total	iast	9.233 s	588.196 ms (6.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.191 s	546.688 ms (6.3%)
Total	iast_TELEMETRY_OFF	9.227 s	582.113 ms (6.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.179 s	121.125 ms (11.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.18 s	121.568 ms (11.5%)
Agent	iast_TELEMETRY_OFF	1.2 s	141.482 ms (13.4%)
Total	tracing	8.659 s	-
Total	iast	9.243 s	583.864 ms (6.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.201 s	541.263 ms (6.3%)
Total	iast_TELEMETRY_OFF	9.298 s	639.007 ms (7.4%)

gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (722.681 ms) : 0, 722681
BytebuddyAgent [candidate] (723.755 ms) : 0, 723755
GlobalTracer [baseline] (239.961 ms) : 0, 239961
GlobalTracer [candidate] (241.024 ms) : 0, 241024
AppSec [baseline] (56.949 ms) : 0, 56949
AppSec [candidate] (55.586 ms) : 0, 55586
Debugger [baseline] (4.474 ms) : 0, 4474
Debugger [candidate] (4.46 ms) : 0, 4460
Remote Config [baseline] (699.848 µs) : 0, 700
Remote Config [candidate] (684.415 µs) : 0, 684
Telemetry [baseline] (12.922 ms) : 0, 12922
Telemetry [candidate] (12.252 ms) : 0, 12252
section iast
BytebuddyAgent [baseline] (837.359 ms) : 0, 837359
BytebuddyAgent [candidate] (837.257 ms) : 0, 837257
GlobalTracer [baseline] (229.333 ms) : 0, 229333
GlobalTracer [candidate] (229.27 ms) : 0, 229270
IAST [baseline] (22.737 ms) : 0, 22737
IAST [candidate] (22.836 ms) : 0, 22836
AppSec [baseline] (55.792 ms) : 0, 55792
AppSec [candidate] (56.085 ms) : 0, 56085
Debugger [baseline] (4.155 ms) : 0, 4155
Debugger [candidate] (4.176 ms) : 0, 4176
Remote Config [baseline] (622.888 µs) : 0, 623
Remote Config [candidate] (614.607 µs) : 0, 615
Telemetry [baseline] (8.757 ms) : 0, 8757
Telemetry [candidate] (8.717 ms) : 0, 8717
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (843.036 ms) : 0, 843036
BytebuddyAgent [candidate] (837.336 ms) : 0, 837336
GlobalTracer [baseline] (230.322 ms) : 0, 230322
GlobalTracer [candidate] (229.687 ms) : 0, 229687
IAST [baseline] (22.875 ms) : 0, 22875
IAST [candidate] (22.736 ms) : 0, 22736
AppSec [baseline] (55.858 ms) : 0, 55858
AppSec [candidate] (56.002 ms) : 0, 56002
Debugger [baseline] (4.167 ms) : 0, 4167
Debugger [candidate] (4.154 ms) : 0, 4154
Remote Config [baseline] (603.67 µs) : 0, 604
Remote Config [candidate] (612.28 µs) : 0, 612
Telemetry [baseline] (8.775 ms) : 0, 8775
Telemetry [candidate] (8.814 ms) : 0, 8814
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (840.013 ms) : 0, 840013
BytebuddyAgent [candidate] (852.718 ms) : 0, 852718
GlobalTracer [baseline] (230.221 ms) : 0, 230221
GlobalTracer [candidate] (232.857 ms) : 0, 232857
IAST [baseline] (22.481 ms) : 0, 22481
IAST [candidate] (22.817 ms) : 0, 22817
AppSec [baseline] (56.642 ms) : 0, 56642
AppSec [candidate] (56.93 ms) : 0, 56930
Debugger [baseline] (4.231 ms) : 0, 4231
Debugger [candidate] (4.214 ms) : 0, 4214
Remote Config [baseline] (642.314 µs) : 0, 642
Remote Config [candidate] (630.205 µs) : 0, 630
Telemetry [baseline] (8.751 ms) : 0, 8751
Telemetry [candidate] (8.793 ms) : 0, 8793

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-04-02T13:20:48	2025-04-02T13:28:35
git_branch	master	andrea.marziali/flaky
git_commit_date	1743592817	1743598818
git_commit_sha	`32046a3`	`228f670`
release_version	1.48.0-SNAPSHOT~32046a39ed	1.48.0-SNAPSHOT~228f670d50
start_time	2025-04-02T13:20:34	2025-04-02T13:28:21

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1743600914	1743600914
ci_job_id	876993269	876993269
ci_pipeline_id	60795471	60795471
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-fsgpech5-project-304-concurrent-1-nwyo9318 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-fsgpech5-project-304-concurrent-1-nwyo9318 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (391.604 µs) : 370, 413
.   : milestone, 392,
iast (520.251 µs) : 498, 542
.   : milestone, 520,
iast_FULL (734.064 µs) : 712, 756
.   : milestone, 734,
iast_GLOBAL (566.02 µs) : 544, 588
.   : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (513.915 µs) : 492, 535
.   : milestone, 514,
iast_INACTIVE (469.971 µs) : 448, 491
.   : milestone, 470,
iast_TELEMETRY_OFF (510.778 µs) : 489, 532
.   : milestone, 511,
tracing (466.766 µs) : 446, 488
.   : milestone, 467,
section candidate
no_agent (389.04 µs) : 370, 409
.   : milestone, 389,
iast (519.259 µs) : 497, 541
.   : milestone, 519,
iast_FULL (735.22 µs) : 713, 757
.   : milestone, 735,
iast_GLOBAL (567.44 µs) : 545, 590
.   : milestone, 567,
iast_HARDCODED_SECRET_DISABLED (521.87 µs) : 500, 543
.   : milestone, 522,
iast_INACTIVE (468.844 µs) : 447, 491
.   : milestone, 469,
iast_TELEMETRY_OFF (504.717 µs) : 483, 526
.   : milestone, 505,
tracing (463.856 µs) : 443, 484
.   : milestone, 464,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	391.604 µs [370.431 µs, 412.778 µs]	-
iast	520.251 µs [498.362 µs, 542.141 µs]	128.647 µs (32.9%)
iast_FULL	734.064 µs [712.01 µs, 756.118 µs]	342.46 µs (87.5%)
iast_GLOBAL	566.02 µs [543.548 µs, 588.492 µs]	174.416 µs (44.5%)
iast_HARDCODED_SECRET_DISABLED	513.915 µs [492.402 µs, 535.428 µs]	122.311 µs (31.2%)
iast_INACTIVE	469.971 µs [448.443 µs, 491.498 µs]	78.366 µs (20.0%)
iast_TELEMETRY_OFF	510.778 µs [489.108 µs, 532.448 µs]	119.174 µs (30.4%)
tracing	466.766 µs [445.862 µs, 487.67 µs]	75.162 µs (19.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	389.04 µs [369.533 µs, 408.547 µs]	-
iast	519.259 µs [497.484 µs, 541.034 µs]	130.219 µs (33.5%)
iast_FULL	735.22 µs [713.058 µs, 757.382 µs]	346.18 µs (89.0%)
iast_GLOBAL	567.44 µs [545.243 µs, 589.638 µs]	178.4 µs (45.9%)
iast_HARDCODED_SECRET_DISABLED	521.87 µs [500.415 µs, 543.325 µs]	132.83 µs (34.1%)
iast_INACTIVE	468.844 µs [447.086 µs, 490.603 µs]	79.804 µs (20.5%)
iast_TELEMETRY_OFF	504.717 µs [483.066 µs, 526.368 µs]	115.677 µs (29.7%)
tracing	463.856 µs [443.285 µs, 484.428 µs]	74.817 µs (19.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.354 ms) : 1335, 1374
.   : milestone, 1354,
appsec (1.744 ms) : 1721, 1767
.   : milestone, 1744,
appsec_no_iast (1.741 ms) : 1718, 1764
.   : milestone, 1741,
code_origins (1.704 ms) : 1678, 1731
.   : milestone, 1704,
iast (1.522 ms) : 1498, 1546
.   : milestone, 1522,
profiling (1.512 ms) : 1489, 1535
.   : milestone, 1512,
tracing (1.5 ms) : 1475, 1525
.   : milestone, 1500,
section candidate
no_agent (1.369 ms) : 1350, 1389
.   : milestone, 1369,
appsec (1.713 ms) : 1689, 1737
.   : milestone, 1713,
appsec_no_iast (1.739 ms) : 1715, 1762
.   : milestone, 1739,
code_origins (1.71 ms) : 1683, 1736
.   : milestone, 1710,
iast (1.535 ms) : 1510, 1559
.   : milestone, 1535,
profiling (1.508 ms) : 1484, 1532
.   : milestone, 1508,
tracing (1.52 ms) : 1495, 1544
.   : milestone, 1520,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.354 ms [1.335 ms, 1.374 ms]	-
appsec	1.744 ms [1.721 ms, 1.767 ms]	389.917 µs (28.8%)
appsec_no_iast	1.741 ms [1.718 ms, 1.764 ms]	386.51 µs (28.5%)
code_origins	1.704 ms [1.678 ms, 1.731 ms]	350.293 µs (25.9%)
iast	1.522 ms [1.498 ms, 1.546 ms]	167.869 µs (12.4%)
profiling	1.512 ms [1.489 ms, 1.535 ms]	157.975 µs (11.7%)
tracing	1.5 ms [1.475 ms, 1.525 ms]	145.431 µs (10.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.369 ms [1.35 ms, 1.389 ms]	-
appsec	1.713 ms [1.689 ms, 1.737 ms]	343.989 µs (25.1%)
appsec_no_iast	1.739 ms [1.715 ms, 1.762 ms]	369.453 µs (27.0%)
code_origins	1.71 ms [1.683 ms, 1.736 ms]	340.491 µs (24.9%)
iast	1.535 ms [1.51 ms, 1.559 ms]	165.695 µs (12.1%)
profiling	1.508 ms [1.484 ms, 1.532 ms]	139.051 µs (10.2%)
tracing	1.52 ms [1.495 ms, 1.544 ms]	150.788 µs (11.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/flaky
git_commit_date	1743592817	1743598818
git_commit_sha	`32046a3`	`228f670`
release_version	1.48.0-SNAPSHOT~32046a39ed	1.48.0-SNAPSHOT~228f670d50

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1743601367	1743601367
ci_job_id	876993270	876993270
ci_pipeline_id	60795471	60795471
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-m1cw8fbu-project-304-concurrent-0-123unqfb 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-m1cw8fbu-project-304-concurrent-0-123unqfb 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.681 s) : 14681000, 14681000
.   : milestone, 14681000,
appsec (14.759 s) : 14759000, 14759000
.   : milestone, 14759000,
iast (18.807 s) : 18807000, 18807000
.   : milestone, 18807000,
iast_GLOBAL (17.924 s) : 17924000, 17924000
.   : milestone, 17924000,
profiling (14.9 s) : 14900000, 14900000
.   : milestone, 14900000,
tracing (14.843 s) : 14843000, 14843000
.   : milestone, 14843000,
section candidate
no_agent (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
appsec (15.015 s) : 15015000, 15015000
.   : milestone, 15015000,
iast (19.186 s) : 19186000, 19186000
.   : milestone, 19186000,
iast_GLOBAL (18.101 s) : 18101000, 18101000
.   : milestone, 18101000,
profiling (15.065 s) : 15065000, 15065000
.   : milestone, 15065000,
tracing (14.761 s) : 14761000, 14761000
.   : milestone, 14761000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.681 s [14.681 s, 14.681 s]	-
appsec	14.759 s [14.759 s, 14.759 s]	78.0 ms (0.5%)
iast	18.807 s [18.807 s, 18.807 s]	4.126 s (28.1%)
iast_GLOBAL	17.924 s [17.924 s, 17.924 s]	3.243 s (22.1%)
profiling	14.9 s [14.9 s, 14.9 s]	219.0 ms (1.5%)
tracing	14.843 s [14.843 s, 14.843 s]	162.0 ms (1.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.962 s [14.962 s, 14.962 s]	-
appsec	15.015 s [15.015 s, 15.015 s]	53.0 ms (0.4%)
iast	19.186 s [19.186 s, 19.186 s]	4.224 s (28.2%)
iast_GLOBAL	18.101 s [18.101 s, 18.101 s]	3.139 s (21.0%)
profiling	15.065 s [15.065 s, 15.065 s]	103.0 ms (0.7%)
tracing	14.761 s [14.761 s, 14.761 s]	-201.0 ms (-1.3%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~228f670d50, baseline=1.48.0-SNAPSHOT~32046a39ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.369 ms) : 2323, 2415
.   : milestone, 2369,
iast (2.151 ms) : 2092, 2209
.   : milestone, 2151,
iast_GLOBAL (2.197 ms) : 2138, 2256
.   : milestone, 2197,
profiling (1.996 ms) : 1950, 2042
.   : milestone, 1996,
tracing (1.976 ms) : 1931, 2021
.   : milestone, 1976,
section candidate
no_agent (1.479 ms) : 1467, 1490
.   : milestone, 1479,
appsec (2.368 ms) : 2322, 2414
.   : milestone, 2368,
iast (2.162 ms) : 2103, 2222
.   : milestone, 2162,
iast_GLOBAL (2.198 ms) : 2139, 2257
.   : milestone, 2198,
profiling (1.993 ms) : 1946, 2039
.   : milestone, 1993,
tracing (1.986 ms) : 1941, 2032
.   : milestone, 1986,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	2.369 ms [2.323 ms, 2.415 ms]	892.88 µs (60.5%)
iast	2.151 ms [2.092 ms, 2.209 ms]	674.861 µs (45.7%)
iast_GLOBAL	2.197 ms [2.138 ms, 2.256 ms]	721.159 µs (48.9%)
profiling	1.996 ms [1.95 ms, 2.042 ms]	520.241 µs (35.2%)
tracing	1.976 ms [1.931 ms, 2.021 ms]	500.364 µs (33.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.467 ms, 1.49 ms]	-
appsec	2.368 ms [2.322 ms, 2.414 ms]	889.021 µs (60.1%)
iast	2.162 ms [2.103 ms, 2.222 ms]	683.57 µs (46.2%)
iast_GLOBAL	2.198 ms [2.139 ms, 2.257 ms]	719.143 µs (48.6%)
profiling	1.993 ms [1.946 ms, 2.039 ms]	514.128 µs (34.8%)
tracing	1.986 ms [1.941 ms, 2.032 ms]	507.541 µs (34.3%)

Don't use strict trace writing for vertx sql tests

228f670

amarziali added comp: testing Testing tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests labels Apr 2, 2025

amarziali requested a review from a team as a code owner April 2, 2025 13:02

mcculls approved these changes Apr 2, 2025

View reviewed changes

amarziali merged commit 063a99c into master Apr 3, 2025
248 of 250 checks passed

amarziali deleted the andrea.marziali/flaky branch April 3, 2025 06:47

github-actions Bot added this to the 1.48.0 milestone Apr 3, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't use strict trace writing for vertx sql tests#8662

Don't use strict trace writing for vertx sql tests#8662
amarziali merged 1 commit intomasterfrom
andrea.marziali/flaky

amarziali commented Apr 2, 2025

Uh oh!

pr-commenter Bot commented Apr 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

amarziali commented Apr 2, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter Bot commented Apr 2, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants