Move base and peer service adder before span is published #9408

amarziali · 2025-08-22T09:05:48Z

What Does This Do

base service and peer service are very important foundations for the inferred services stats calcualtion.
When enabling the client stats, the base service information is lost (and it's required to be part of the bucket aggregation) because that tag is calculated only later on when the traces are written.

This PR aims to do changes as minimal as possible to the code base while keeping this calculation earlier. The rest of the post processor are still applied later on since they are not needed to be calculated early and may be skipped if the trace get dropped

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

datadog-official · 2025-08-22T09:26:25Z

🎯 Code Coverage
• Patch Coverage: 84.21%
• Total Coverage: 57.59% (-0.01%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 4bc78b1 | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-08-22T09:39:00Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/move-span-postprocessor-early
git_commit_date	1756304629	1756304801
git_commit_sha	`6ce5559`	`4bc78b1`
release_version	1.53.0-SNAPSHOT~6ce5559f8d	1.53.0-SNAPSHOT~4bc78b1d0a

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756306664	1756306664
ci_job_id	1101393343	1101393343
ci_pipeline_id	74862295	74862295
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-jqvpc751 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-jqvpc751 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1047806
Total [baseline] (10.763 s) : 0, 10763090
Agent [candidate] (1.056 s) : 0, 1055500
Total [candidate] (10.709 s) : 0, 10708749
section appsec
Agent [baseline] (1.224 s) : 0, 1223986
Total [baseline] (10.779 s) : 0, 10778907
Agent [candidate] (1.226 s) : 0, 1226118
Total [candidate] (10.788 s) : 0, 10787742
section iast
Agent [baseline] (1.189 s) : 0, 1189215
Total [baseline] (10.923 s) : 0, 10923045
Agent [candidate] (1.181 s) : 0, 1181459
Total [candidate] (10.954 s) : 0, 10953520
section profiling
Agent [baseline] (1.199 s) : 0, 1199078
Total [baseline] (10.958 s) : 0, 10957651
Agent [candidate] (1.195 s) : 0, 1195298
Total [candidate] (10.939 s) : 0, 10939229

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.224 s	176.18 ms (16.8%)
Agent	iast	1.189 s	141.41 ms (13.5%)
Agent	profiling	1.199 s	151.272 ms (14.4%)
Total	tracing	10.763 s	-
Total	appsec	10.779 s	15.817 ms (0.1%)
Total	iast	10.923 s	159.956 ms (1.5%)
Total	profiling	10.958 s	194.561 ms (1.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.226 s	170.618 ms (16.2%)
Agent	iast	1.181 s	125.959 ms (11.9%)
Agent	profiling	1.195 s	139.798 ms (13.2%)
Total	tracing	10.709 s	-
Total	appsec	10.788 s	78.993 ms (0.7%)
Total	iast	10.954 s	244.771 ms (2.3%)
Total	profiling	10.939 s	230.48 ms (2.2%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (732.984 ms) : 0, 732984
BytebuddyAgent [candidate] (737.336 ms) : 0, 737336
GlobalTracer [baseline] (242.498 ms) : 0, 242498
GlobalTracer [candidate] (244.432 ms) : 0, 244432
AppSec [baseline] (30.178 ms) : 0, 30178
AppSec [candidate] (30.453 ms) : 0, 30453
Debugger [baseline] (6.061 ms) : 0, 6061
Debugger [candidate] (6.064 ms) : 0, 6064
Remote Config [baseline] (671.811 µs) : 0, 672
Remote Config [candidate] (669.241 µs) : 0, 669
Telemetry [baseline] (12.878 ms) : 0, 12878
Telemetry [candidate] (13.855 ms) : 0, 13855
section appsec
crashtracking [baseline] (1.452 ms) : 0, 1452
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (755.719 ms) : 0, 755719
BytebuddyAgent [candidate] (757.126 ms) : 0, 757126
GlobalTracer [baseline] (235.586 ms) : 0, 235586
GlobalTracer [candidate] (235.807 ms) : 0, 235807
AppSec [baseline] (170.705 ms) : 0, 170705
AppSec [candidate] (169.581 ms) : 0, 169581
Debugger [baseline] (5.88 ms) : 0, 5880
Debugger [candidate] (6.66 ms) : 0, 6660
Remote Config [baseline] (634.75 µs) : 0, 635
Remote Config [candidate] (647.572 µs) : 0, 648
Telemetry [baseline] (9.331 ms) : 0, 9331
Telemetry [candidate] (9.95 ms) : 0, 9950
IAST [baseline] (23.518 ms) : 0, 23518
IAST [candidate] (23.69 ms) : 0, 23690
section iast
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.437 ms) : 0, 1437
BytebuddyAgent [baseline] (858.827 ms) : 0, 858827
BytebuddyAgent [candidate] (852.934 ms) : 0, 852934
GlobalTracer [baseline] (234.65 ms) : 0, 234650
GlobalTracer [candidate] (233.375 ms) : 0, 233375
AppSec [baseline] (25.054 ms) : 0, 25054
AppSec [candidate] (26.829 ms) : 0, 26829
Debugger [baseline] (6.65 ms) : 0, 6650
Debugger [candidate] (5.736 ms) : 0, 5736
Remote Config [baseline] (607.803 µs) : 0, 608
Remote Config [candidate] (598.561 µs) : 0, 599
Telemetry [baseline] (8.357 ms) : 0, 8357
Telemetry [candidate] (8.314 ms) : 0, 8314
IAST [baseline] (32.38 ms) : 0, 32380
IAST [candidate] (31.192 ms) : 0, 31192
section profiling
ProfilingAgent [baseline] (108.043 ms) : 0, 108043
ProfilingAgent [candidate] (108.633 ms) : 0, 108633
crashtracking [baseline] (1.444 ms) : 0, 1444
crashtracking [candidate] (1.444 ms) : 0, 1444
BytebuddyAgent [baseline] (763.407 ms) : 0, 763407
BytebuddyAgent [candidate] (760.729 ms) : 0, 760729
GlobalTracer [baseline] (222.559 ms) : 0, 222559
GlobalTracer [candidate] (221.812 ms) : 0, 221812
AppSec [baseline] (30.169 ms) : 0, 30169
AppSec [candidate] (29.749 ms) : 0, 29749
Debugger [baseline] (6.327 ms) : 0, 6327
Debugger [candidate] (6.992 ms) : 0, 6992
Remote Config [baseline] (745.702 µs) : 0, 746
Remote Config [candidate] (695.821 µs) : 0, 696
Telemetry [baseline] (16.512 ms) : 0, 16512
Telemetry [candidate] (15.705 ms) : 0, 15705
Profiling [baseline] (108.701 ms) : 0, 108701
Profiling [candidate] (109.285 ms) : 0, 109285

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.069 s) : 0, 1068936
Total [baseline] (8.623 s) : 0, 8623222
Agent [candidate] (1.052 s) : 0, 1052373
Total [candidate] (8.626 s) : 0, 8626285
section iast
Agent [baseline] (1.178 s) : 0, 1178217
Total [baseline] (9.314 s) : 0, 9314423
Agent [candidate] (1.183 s) : 0, 1182726
Total [candidate] (9.368 s) : 0, 9368350

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.069 s	-
Agent	iast	1.178 s	109.281 ms (10.2%)
Total	tracing	8.623 s	-
Total	iast	9.314 s	691.201 ms (8.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.183 s	130.352 ms (12.4%)
Total	tracing	8.626 s	-
Total	iast	9.368 s	742.065 ms (8.6%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.488 ms) : 0, 1488
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (747.316 ms) : 0, 747316
BytebuddyAgent [candidate] (737.452 ms) : 0, 737452
GlobalTracer [baseline] (247.049 ms) : 0, 247049
GlobalTracer [candidate] (243.89 ms) : 0, 243890
AppSec [baseline] (30.856 ms) : 0, 30856
AppSec [candidate] (30.29 ms) : 0, 30290
Debugger [baseline] (6.217 ms) : 0, 6217
Debugger [candidate] (6.084 ms) : 0, 6084
Remote Config [baseline] (687.283 µs) : 0, 687
Remote Config [candidate] (672.011 µs) : 0, 672
Telemetry [baseline] (14.016 ms) : 0, 14016
Telemetry [candidate] (11.512 ms) : 0, 11512
section iast
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (850.529 ms) : 0, 850529
BytebuddyAgent [candidate] (853.665 ms) : 0, 853665
GlobalTracer [baseline] (232.683 ms) : 0, 232683
GlobalTracer [candidate] (233.854 ms) : 0, 233854
AppSec [baseline] (28.443 ms) : 0, 28443
AppSec [candidate] (26.83 ms) : 0, 26830
Debugger [baseline] (5.722 ms) : 0, 5722
Debugger [candidate] (5.787 ms) : 0, 5787
Remote Config [baseline] (600.84 µs) : 0, 601
Remote Config [candidate] (594.771 µs) : 0, 595
Telemetry [baseline] (8.285 ms) : 0, 8285
Telemetry [candidate] (8.273 ms) : 0, 8273
IAST [baseline] (29.505 ms) : 0, 29505
IAST [candidate] (31.233 ms) : 0, 31233

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/move-span-postprocessor-early
git_commit_date	1756304629	1756304801
git_commit_sha	`6ce5559`	`4bc78b1`
release_version	1.53.0-SNAPSHOT~6ce5559f8d	1.53.0-SNAPSHOT~4bc78b1d0a

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756306319	1756306319
ci_job_id	1101393345	1101393345
ci_pipeline_id	74862295	74862295
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-wldrloqk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-wldrloqk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:appsec:high_load	worse [+1.718ms; +2.619ms] or [+3.800%; +5.795%]	unstable [-12.239op/s; +2.739op/s] or [-11.824%; +2.646%]	47.361ms	98.763op/s	45.193ms	103.513op/s
scenario:load:petclinic:tracing:high_load	better [-2.645ms; -1.800ms] or [-5.715%; -3.890%]	unstable [-3.339op/s; +10.955op/s] or [-3.262%; +10.700%]	44.054ms	106.188op/s	46.276ms	102.380op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.35 ms) : 4301, 4399
.   : milestone, 4350,
iast (9.26 ms) : 9109, 9410
.   : milestone, 9260,
iast_FULL (13.995 ms) : 13717, 14273
.   : milestone, 13995,
iast_GLOBAL (10.323 ms) : 10145, 10502
.   : milestone, 10323,
profiling (8.922 ms) : 8782, 9062
.   : milestone, 8922,
tracing (7.646 ms) : 7537, 7756
.   : milestone, 7646,
section candidate
no_agent (4.318 ms) : 4269, 4367
.   : milestone, 4318,
iast (9.071 ms) : 8921, 9221
.   : milestone, 9071,
iast_FULL (13.725 ms) : 13460, 13991
.   : milestone, 13725,
iast_GLOBAL (10.149 ms) : 9964, 10335
.   : milestone, 10149,
profiling (8.628 ms) : 8496, 8760
.   : milestone, 8628,
tracing (7.727 ms) : 7604, 7851
.   : milestone, 7727,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.35 ms [4.301 ms, 4.399 ms]	-
iast	9.26 ms [9.109 ms, 9.41 ms]	4.91 ms (112.9%)
iast_FULL	13.995 ms [13.717 ms, 14.273 ms]	9.646 ms (221.7%)
iast_GLOBAL	10.323 ms [10.145 ms, 10.502 ms]	5.974 ms (137.3%)
profiling	8.922 ms [8.782 ms, 9.062 ms]	4.572 ms (105.1%)
tracing	7.646 ms [7.537 ms, 7.756 ms]	3.297 ms (75.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.318 ms [4.269 ms, 4.367 ms]	-
iast	9.071 ms [8.921 ms, 9.221 ms]	4.754 ms (110.1%)
iast_FULL	13.725 ms [13.46 ms, 13.991 ms]	9.408 ms (217.9%)
iast_GLOBAL	10.149 ms [9.964 ms, 10.335 ms]	5.832 ms (135.1%)
profiling	8.628 ms [8.496 ms, 8.76 ms]	4.31 ms (99.8%)
tracing	7.727 ms [7.604 ms, 7.851 ms]	3.409 ms (79.0%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.007 ms) : 36698, 37315
.   : milestone, 37007,
appsec (45.193 ms) : 44790, 45595
.   : milestone, 45193,
code_origins (44.104 ms) : 43729, 44480
.   : milestone, 44104,
iast (43.509 ms) : 43122, 43896
.   : milestone, 43509,
profiling (46.962 ms) : 46536, 47387
.   : milestone, 46962,
tracing (46.276 ms) : 45878, 46675
.   : milestone, 46276,
section candidate
no_agent (36.51 ms) : 36215, 36804
.   : milestone, 36510,
appsec (47.361 ms) : 46926, 47796
.   : milestone, 47361,
code_origins (44.589 ms) : 44205, 44972
.   : milestone, 44589,
iast (44.592 ms) : 44201, 44983
.   : milestone, 44592,
profiling (47.204 ms) : 46745, 47663
.   : milestone, 47204,
tracing (44.054 ms) : 43668, 44440
.   : milestone, 44054,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.007 ms [36.698 ms, 37.315 ms]	-
appsec	45.193 ms [44.79 ms, 45.595 ms]	8.186 ms (22.1%)
code_origins	44.104 ms [43.729 ms, 44.48 ms]	7.098 ms (19.2%)
iast	43.509 ms [43.122 ms, 43.896 ms]	6.503 ms (17.6%)
profiling	46.962 ms [46.536 ms, 47.387 ms]	9.955 ms (26.9%)
tracing	46.276 ms [45.878 ms, 46.675 ms]	9.27 ms (25.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.51 ms [36.215 ms, 36.804 ms]	-
appsec	47.361 ms [46.926 ms, 47.796 ms]	10.852 ms (29.7%)
code_origins	44.589 ms [44.205 ms, 44.972 ms]	8.079 ms (22.1%)
iast	44.592 ms [44.201 ms, 44.983 ms]	8.083 ms (22.1%)
profiling	47.204 ms [46.745 ms, 47.663 ms]	10.695 ms (29.3%)
tracing	44.054 ms [43.668 ms, 44.44 ms]	7.545 ms (20.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/move-span-postprocessor-early
git_commit_date	1756304629	1756304801
git_commit_sha	`6ce5559`	`4bc78b1`
release_version	1.53.0-SNAPSHOT~6ce5559f8d	1.53.0-SNAPSHOT~4bc78b1d0a

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1756306769	1756306769
ci_job_id	1101393347	1101393347
ci_pipeline_id	74862295	74862295
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-4t2ack3b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-4t2ack3b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1462, 1486
.   : milestone, 1474,
appsec (3.654 ms) : 3435, 3873
.   : milestone, 3654,
iast (2.197 ms) : 2134, 2259
.   : milestone, 2197,
iast_GLOBAL (2.242 ms) : 2179, 2306
.   : milestone, 2242,
profiling (2.067 ms) : 2015, 2119
.   : milestone, 2067,
tracing (2.014 ms) : 1965, 2063
.   : milestone, 2014,
section candidate
no_agent (1.476 ms) : 1465, 1488
.   : milestone, 1476,
appsec (3.66 ms) : 3441, 3878
.   : milestone, 3660,
iast (2.196 ms) : 2133, 2259
.   : milestone, 2196,
iast_GLOBAL (2.256 ms) : 2193, 2319
.   : milestone, 2256,
profiling (2.066 ms) : 2014, 2118
.   : milestone, 2066,
tracing (2.031 ms) : 1982, 2080
.   : milestone, 2031,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.474 ms [1.462 ms, 1.486 ms]	-
appsec	3.654 ms [3.435 ms, 3.873 ms]	2.18 ms (147.9%)
iast	2.197 ms [2.134 ms, 2.259 ms]	722.756 µs (49.0%)
iast_GLOBAL	2.242 ms [2.179 ms, 2.306 ms]	768.28 µs (52.1%)
profiling	2.067 ms [2.015 ms, 2.119 ms]	592.777 µs (40.2%)
tracing	2.014 ms [1.965 ms, 2.063 ms]	540.08 µs (36.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.465 ms, 1.488 ms]	-
appsec	3.66 ms [3.441 ms, 3.878 ms]	2.183 ms (147.9%)
iast	2.196 ms [2.133 ms, 2.259 ms]	719.775 µs (48.8%)
iast_GLOBAL	2.256 ms [2.193 ms, 2.319 ms]	779.825 µs (52.8%)
profiling	2.066 ms [2.014 ms, 2.118 ms]	589.787 µs (40.0%)
tracing	2.031 ms [1.982 ms, 2.08 ms]	554.656 µs (37.6%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~4bc78b1d0a, baseline=1.53.0-SNAPSHOT~6ce5559f8d
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.571 s) : 15571000, 15571000
.   : milestone, 15571000,
appsec (14.919 s) : 14919000, 14919000
.   : milestone, 14919000,
iast (18.847 s) : 18847000, 18847000
.   : milestone, 18847000,
iast_GLOBAL (18.183 s) : 18183000, 18183000
.   : milestone, 18183000,
profiling (15.29 s) : 15290000, 15290000
.   : milestone, 15290000,
tracing (15.019 s) : 15019000, 15019000
.   : milestone, 15019000,
section candidate
no_agent (15.202 s) : 15202000, 15202000
.   : milestone, 15202000,
appsec (14.881 s) : 14881000, 14881000
.   : milestone, 14881000,
iast (18.399 s) : 18399000, 18399000
.   : milestone, 18399000,
iast_GLOBAL (18.356 s) : 18356000, 18356000
.   : milestone, 18356000,
profiling (15.873 s) : 15873000, 15873000
.   : milestone, 15873000,
tracing (14.847 s) : 14847000, 14847000
.   : milestone, 14847000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.571 s [15.571 s, 15.571 s]	-
appsec	14.919 s [14.919 s, 14.919 s]	-652.0 ms (-4.2%)
iast	18.847 s [18.847 s, 18.847 s]	3.276 s (21.0%)
iast_GLOBAL	18.183 s [18.183 s, 18.183 s]	2.612 s (16.8%)
profiling	15.29 s [15.29 s, 15.29 s]	-281.0 ms (-1.8%)
tracing	15.019 s [15.019 s, 15.019 s]	-552.0 ms (-3.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.202 s [15.202 s, 15.202 s]	-
appsec	14.881 s [14.881 s, 14.881 s]	-321.0 ms (-2.1%)
iast	18.399 s [18.399 s, 18.399 s]	3.197 s (21.0%)
iast_GLOBAL	18.356 s [18.356 s, 18.356 s]	3.154 s (20.7%)
profiling	15.873 s [15.873 s, 15.873 s]	671.0 ms (4.4%)
tracing	14.847 s [14.847 s, 14.847 s]	-355.0 ms (-2.3%)

PerfectSlayer

💭 thought: It looks like opening / exposing span and span context internals… I would like to think about a way to keep it private but would need more time to do the review.‏

amarziali · 2025-08-22T12:46:42Z

💭 thought: It looks like opening / exposing span and span context internals… I would like to think about a way to keep it private but would need more time to do the review.‏

@PerfectSlayer could we log a ticket about that debt and merge it? I'm worried about not having this change as part of the next release

mcculls

This looks acceptable as a short-term fix, but we should revisit the overall processing pipeline

PerfectSlayer · 2025-08-26T09:09:44Z

I will come up with design change later Today.

PerfectSlayer

I introduced eager / lazy processor chain, still lazily built, to have a distinction between the two phases of tag processors.
This keep the design very close to what already exists. We can review later when the eager processing should happen (we might be able to hide it inside one of the span / span context existing method) but I don’t have enough context to change it now.

@mcculls Can you have a look at it too? I would rather avoid being the only one approving my own changes. Thanks 🙏

mcculls · 2025-08-26T12:16:51Z

dd-trace-core/src/main/java/datadog/trace/core/CoreSpan.java

+   * service computation. Such tags are needed before span serialization so they can’t be processed
+   * lazily as part of the {@link #processTagsAndBaggage(MetadataConsumer)} API.
+   */
+  void earlyProcessTags();


Just wondering if we should make this a bit more concrete and call it something like processServiceTags?

That would fit better alongside the existing processTagsAndBaggage - and would also make clear that this early processing is focused on the service.

processEarlyTags or processEagerTags would also be options...

I'm good with processServiceTags. I will rename it and we can revise the processing of both early tags and (late) tags & baggage later then. This will fix client side computed metrics in the meantime.

mcculls

LGTM, just a small naming suggestion - thanks for doing the redesign!

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.52.1` -> `1.53.0` | --- ### Release Notes <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.53.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.53.0): 1.53.0 ### Components #### Application Security Management (WAF) - 🐛 Fix NullPointerException log in AppSec ([#9355](DataDog/dd-trace-java#9355) - [@jandro996](https://github.com/jandro996)) - ✨ Add Forwarded header as a source to client IP resolution ([#9310](DataDog/dd-trace-java#9310) - [@jandro996](https://github.com/jandro996)) #### Build & Tooling - 📖 Update CODEOWNERS ([#9294](DataDog/dd-trace-java#9294) - [@mhlidd](https://github.com/mhlidd)) - ✨ Add dependency on Maven Central deployment to OCI publish jobs ([#9204](DataDog/dd-trace-java#9204) - [@sarahchen6](https://github.com/sarahchen6)) #### Continuous Integration Visibility - ✨ Upload code coverage reports to Datadog ([#9425](DataDog/dd-trace-java#9425) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Do not follow symlinks by default when building repository index ([#9318](DataDog/dd-trace-java#9318) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Improve tag propagation between test event levels ([#9278](DataDog/dd-trace-java#9278) - [@daniel-mohedano](https://github.com/daniel-mohedano)) #### Data Streams Monitoring - ✨ Create container hash tags propagation ([#9282](DataDog/dd-trace-java#9282) - [@ygree](https://github.com/ygree)) #### Database Monitoring - 🐛 Prevent crash in SQL Server's JDBC when tracing execute methods with generated keys ([#9321](DataDog/dd-trace-java#9321) - [@na-ji](https://github.com/na-ji)) - ✨ Create container hash tags propagation ([#9282](DataDog/dd-trace-java#9282) - [@ygree](https://github.com/ygree)) #### GraalVM native-image - 🐛 Update GraalVM config to reflect TempLocationManager's new package ([#9337](DataDog/dd-trace-java#9337) - [@luneo7](https://github.com/luneo7) - thanks for the contribution!) #### Metrics - 🐛 Add reflect config for CSS and graalvm ([#9449](DataDog/dd-trace-java#9449) - [@amarziali](https://github.com/amarziali)) - ✨ Align CSS health metrics ([#9444](DataDog/dd-trace-java#9444) - [@amarziali](https://github.com/amarziali)) - ✨ Aggregate peer tags for consumer span kind ([#9442](DataDog/dd-trace-java#9442) - [@amarziali](https://github.com/amarziali)) - ✨ Implement health metrics for client stats ([#9377](DataDog/dd-trace-java#9377) - [@amarziali](https://github.com/amarziali)) - ✨ Add peer tags, span kind and trace root flag to MetricKey bucket ([#9178](DataDog/dd-trace-java#9178) - [@bric3](https://github.com/bric3)) #### Profiling - ✨ Create a profile flare reporter (requires tracing or CI vis to be enabled) ([#9433](DataDog/dd-trace-java#9433) - [@MattAlp](https://github.com/MattAlp)) - 🐛 Properly fall-back on UDS profiling URL ([#9401](DataDog/dd-trace-java#9401) - [@jbachorik](https://github.com/jbachorik)) - 🐛 Properly handle trace agent IPv6 URL in profiling ([#9334](DataDog/dd-trace-java#9334) - [@jbachorik](https://github.com/jbachorik)) #### Realtime User Monitoring - 🐛 Avoid multiple injections on dispatch on jboss ([#9392](DataDog/dd-trace-java#9392) - [@amarziali](https://github.com/amarziali)) - ✨ Support async servlet for RUM injection ([#9333](DataDog/dd-trace-java#9333) - [@amarziali](https://github.com/amarziali)) - 🐛 Improve RUM injection matching and avoid truncating responses ([#9323](DataDog/dd-trace-java#9323) - [@amarziali](https://github.com/amarziali)) - 💡 Add telemetry for the RUM injector ([#9267](DataDog/dd-trace-java#9267) - [@sarahchen6](https://github.com/sarahchen6)) - ✨ Make rum injector stream/writer more resilient to errors ([#9184](DataDog/dd-trace-java#9184) - [@amarziali](https://github.com/amarziali)) #### Telemetry - 🐛 Remediate error logs ([#9459](DataDog/dd-trace-java#9459) - [@ygree](https://github.com/ygree)) - ✨ Report config\_id for Hands Off Config files ([#9299](DataDog/dd-trace-java#9299) - [@mtoffl01](https://github.com/mtoffl01)) - ✨ Adding Config Inversion Telemetry component ([#9244](DataDog/dd-trace-java#9244) - [@mhlidd](https://github.com/mhlidd)) #### Trace context propagation - 🧹 Migrate instrumentations to Context API ([#9384](DataDog/dd-trace-java#9384), [#9378](DataDog/dd-trace-java#9378), [#9358](DataDog/dd-trace-java#9358) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🔍 Add baggage propagation telemetry ([#9289](DataDog/dd-trace-java#9289) - [@rachelyangdog](https://github.com/rachelyangdog)) - 🧹 Improve Instrumenter API to use Context instead of Span ([#9211](DataDog/dd-trace-java#9211) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Tracer core - 💡:test\_tube: Enable process level tags for java 21 applications ([#9458](DataDog/dd-trace-java#9458) - [@amarziali](https://github.com/amarziali)) - ✨ Move base and peer service adder before span is published ([#9408](DataDog/dd-trace-java#9408) - [@amarziali](https://github.com/amarziali)) - ✨ Add ErrorPriorities level for manual instrumentation ([#9387](DataDog/dd-trace-java#9387) - [@cecile75](https://github.com/cecile75)) - ✨ Throw InvalidBooleanValueException in ConfigConverter.booleanValueOf ([#9370](DataDog/dd-trace-java#9370) - [@mtoffl01](https://github.com/mtoffl01)) - 🐛 Avoid NPE on featureDiscovery creation ([#9353](DataDog/dd-trace-java#9353) - [@amarziali](https://github.com/amarziali)) - ✨ Stable Config improvements ([#9259](DataDog/dd-trace-java#9259) - [@mtoffl01](https://github.com/mtoffl01)) ### Instrumentations #### gRPC instrumentation - 🐛 Changing GRPC instrumentation codes from `status.code` to `grpc.status.code` ([#9367](DataDog/dd-trace-java#9367) - [@mhlidd](https://github.com/mhlidd)) #### JDBC instrumentation - 🐛 Prevent crash in SQL Server's JDBC when tracing execute methods with generated keys ([#9321](DataDog/dd-trace-java#9321) - [@na-ji](https://github.com/na-ji)) #### Kotlin instrumentation - 🐛 Take defensive copy of original scope stack when restoring on different thread ([#9403](DataDog/dd-trace-java#9403) - [@mcculls](https://github.com/mcculls)) #### OpenTelemetry instrumentation - 🐛 Fix using SpanAttribute annotation on multiple parameters ([#9412](DataDog/dd-trace-java#9412) - [@cecile75](https://github.com/cecile75)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: e23ec866cef91afc6b5226d5f0dc7da538d947e8

Move base and peer service adder before span is published

8358bca

amarziali requested a review from a team as a code owner August 22, 2025 09:05

amarziali requested a review from manuel-alvarez-alvarez August 22, 2025 09:05

amarziali added type: enhancement Enhancements and improvements comp: core Tracer core labels Aug 22, 2025

Change method signature to fix ugly groovy issues

0c0dd50

amarziali force-pushed the andrea.marziali/move-span-postprocessor-early branch from d6368e0 to 0c0dd50 Compare August 22, 2025 09:59

PerfectSlayer reviewed Aug 22, 2025

View reviewed changes

mcculls reviewed Aug 22, 2025

View reviewed changes

amarziali enabled auto-merge (squash) August 22, 2025 13:35

PerfectSlayer force-pushed the andrea.marziali/move-span-postprocessor-early branch from 73eb92e to 542da10 Compare August 26, 2025 09:41

PerfectSlayer disabled auto-merge August 26, 2025 11:35

PerfectSlayer approved these changes Aug 26, 2025

View reviewed changes

PerfectSlayer requested review from PerfectSlayer and mcculls August 26, 2025 11:35

PerfectSlayer approved these changes Aug 26, 2025

View reviewed changes

mcculls reviewed Aug 26, 2025

View reviewed changes

mcculls approved these changes Aug 26, 2025

View reviewed changes

feat(tracer): Introduce eager span tag processors

4bc78b1

PerfectSlayer force-pushed the andrea.marziali/move-span-postprocessor-early branch from 542da10 to 4bc78b1 Compare August 27, 2025 14:27

PerfectSlayer enabled auto-merge (squash) August 27, 2025 14:58

PerfectSlayer merged commit 02a3250 into master Aug 27, 2025
501 checks passed

PerfectSlayer deleted the andrea.marziali/move-span-postprocessor-early branch August 27, 2025 15:13

github-actions bot added this to the 1.53.0 milestone Aug 27, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Move base and peer service adder before span is published #9408

Move base and peer service adder before span is published #9408

Uh oh!

amarziali commented Aug 22, 2025

Uh oh!

datadog-official bot commented Aug 22, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 22, 2025 •

edited

Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

amarziali commented Aug 22, 2025

Uh oh!

mcculls left a comment

Uh oh!

PerfectSlayer commented Aug 26, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

mcculls Aug 26, 2025

Uh oh!

PerfectSlayer Aug 27, 2025

Uh oh!

mcculls left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Move base and peer service adder before span is published #9408

Move base and peer service adder before span is published #9408

Uh oh!

Conversation

amarziali commented Aug 22, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-official bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

amarziali commented Aug 22, 2025

Uh oh!

mcculls left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer commented Aug 26, 2025

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

mcculls Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer Aug 27, 2025

Choose a reason for hiding this comment

Uh oh!

mcculls left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

datadog-official bot commented Aug 22, 2025 •

edited

Loading

pr-commenter bot commented Aug 22, 2025 •

edited

Loading