Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented #9872

aboitreaud · 2025-10-28T09:30:59Z

What Does This Do

Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented

Motivation

Remediation to instrumentation blocked by Muzzle checks that we found out with the following DEBUG logs:

around missing MicroBatchExecution class:

[dd.trace 2025-10-28 08:29:52:563 +0000] [main] DEBUG datadog.trace.agent.tooling.InstrumenterState - Instrumentation blocked - instrumentation.names=[spark,apache-spark] instrumentation.class=datadog.trace.instrumentation.spark.Spark213Instrumentation instrumentation.target.classloader=jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6
[dd.trace 2025-10-28 08:29:52:589 +0000] [main] DEBUG datadog.trace.agent.tooling.muzzle.MuzzleCheck - Muzzled - instrumentation.names=[spark,apache-spark] instrumentation.class=datadog.trace.instrumentation.spark.Spark213Instrumentation instrumentation.target.classloader=jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6
[dd.trace 2025-10-28 08:29:52:591 +0000] [main] DEBUG datadog.trace.agent.tooling.muzzle.MuzzleCheck - Muzzled mismatch - instrumentation.names=[spark,apache-spark] instrumentation.class=datadog.trace.instrumentation.spark.Spark213Instrumentation instrumentation.target.classloader=jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6 muzzle.mismatch="datadog.trace.instrumentation.spark.AbstractDatadogSparkListener:1239 Missing class org.apache.spark.sql.execution.streaming.MicroBatchExecution"

caused by SparkPlanInfo Consructor being overloaded on Databricks' Spark custom version for both Spark212 and Spark213:

[dd.trace 2025-10-29 17:33:07:215 +0000] [main] DEBUG datadog.trace.agent.tooling.muzzle.MuzzleCheck - Muzzled - instrumentation.names=[spark,apache-spark] instrumentation.class=datadog.trace.instrumentation.spark.Spark213Instrumentation instrumentation.target.classloader=jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6
[dd.trace 2025-10-29 17:33:07:216 +0000] [main] DEBUG datadog.trace.agent.tooling.muzzle.MuzzleCheck - Muzzled mismatch - instrumentation.names=[spark,apache-spark] instrumentation.class=datadog.trace.instrumentation.spark.Spark213Instrumentation instrumentation.target.classloader=jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6 muzzle.mismatch="datadog.trace.instrumentation.spark.Spark213Instrumentation$SparkPlanInfoAdvice:111 Missing method org.apache.spark.sql.execution.SparkPlanInfo#<init>(Ljava/lang/String;Ljava/lang/String;Lscala/collection/immutable/Seq;Lscala/collection/immutable/Map;Lscala/collection/immutable/Seq;)V"

(1) is fixed with directly passing the harcoded property key from MicroBatchExecution BatchID key: https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/runtime/MicroBatchExecution.scala#L1331-L1333
(2) is fixed by looking up the various constructors of SparkPlanInfo and using the one with the expected number of arguments, 5, so that we're not affected by the overloaded Databricks constructors.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-10-28T09:31:11Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Remove the tag from the pull request title

If you need help, please check our contributing guidelines.

datadog-official · 2025-10-28T09:52:29Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 59.68% (-0.02%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 91f2995 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

…r bus matchers

pr-commenter · 2025-10-28T11:01:51Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	adrien.boitreaud/fix-muzzle-mismatch
git_commit_date	1761835849	1761900801
git_commit_sha	`9b91826`	`91f2995`
release_version	1.55.0-SNAPSHOT~9b91826873	1.55.0-SNAPSHOT~91f2995198

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1761902606	1761902606
ci_job_id	1208573711	1208573711
ci_pipeline_id	80848226	80848226
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-7453w05i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-7453w05i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 12 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.031 s) : 0, 1031402
Total [baseline] (8.692 s) : 0, 8691501
Agent [candidate] (1.036 s) : 0, 1035552
Total [candidate] (8.665 s) : 0, 8665394
section iast
Agent [baseline] (1.164 s) : 0, 1163820
Total [baseline] (9.38 s) : 0, 9380039
Agent [candidate] (1.163 s) : 0, 1163453
Total [candidate] (9.395 s) : 0, 9394657

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.031 s	-
Agent	iast	1.164 s	132.417 ms (12.8%)
Total	tracing	8.692 s	-
Total	iast	9.38 s	688.538 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.036 s	-
Agent	iast	1.163 s	127.901 ms (12.4%)
Total	tracing	8.665 s	-
Total	iast	9.395 s	729.263 ms (8.4%)

gantt
    title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.473 ms) : 0, 1473
crashtracking [candidate] (1.475 ms) : 0, 1475
BytebuddyAgent [baseline] (703.181 ms) : 0, 703181
BytebuddyAgent [candidate] (707.27 ms) : 0, 707270
GlobalTracer [baseline] (245.092 ms) : 0, 245092
GlobalTracer [candidate] (245.656 ms) : 0, 245656
AppSec [baseline] (32.373 ms) : 0, 32373
AppSec [candidate] (32.17 ms) : 0, 32170
Debugger [baseline] (6.399 ms) : 0, 6399
Debugger [candidate] (6.355 ms) : 0, 6355
Remote Config [baseline] (690.109 µs) : 0, 690
Remote Config [candidate] (668.486 µs) : 0, 668
Telemetry [baseline] (15.951 ms) : 0, 15951
Telemetry [candidate] (15.716 ms) : 0, 15716
Flare Poller [baseline] (4.973 ms) : 0, 4973
Flare Poller [candidate] (4.922 ms) : 0, 4922
section iast
crashtracking [baseline] (1.476 ms) : 0, 1476
crashtracking [candidate] (1.469 ms) : 0, 1469
BytebuddyAgent [baseline] (825.325 ms) : 0, 825325
BytebuddyAgent [candidate] (825.794 ms) : 0, 825794
GlobalTracer [baseline] (234.227 ms) : 0, 234227
GlobalTracer [candidate] (234.117 ms) : 0, 234117
AppSec [baseline] (30.002 ms) : 0, 30002
AppSec [candidate] (29.476 ms) : 0, 29476
Debugger [baseline] (6.106 ms) : 0, 6106
Debugger [candidate] (6.102 ms) : 0, 6102
Remote Config [baseline] (622.368 µs) : 0, 622
Remote Config [candidate] (598.172 µs) : 0, 598
Telemetry [baseline] (8.635 ms) : 0, 8635
Telemetry [candidate] (8.487 ms) : 0, 8487
Flare Poller [baseline] (4.182 ms) : 0, 4182
Flare Poller [candidate] (4.18 ms) : 0, 4180
IAST [baseline] (31.94 ms) : 0, 31940
IAST [candidate] (31.913 ms) : 0, 31913

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.031 s) : 0, 1031304
Total [baseline] (10.755 s) : 0, 10755403
Agent [candidate] (1.028 s) : 0, 1028411
Total [candidate] (10.836 s) : 0, 10836022
section appsec
Agent [baseline] (1.206 s) : 0, 1206151
Total [baseline] (10.956 s) : 0, 10955766
Agent [candidate] (1.205 s) : 0, 1205480
Total [candidate] (11.044 s) : 0, 11044143
section iast
Agent [baseline] (1.166 s) : 0, 1166126
Total [baseline] (11.131 s) : 0, 11131137
Agent [candidate] (1.167 s) : 0, 1166539
Total [candidate] (11.225 s) : 0, 11224720
section profiling
Agent [baseline] (1.179 s) : 0, 1178836
Total [baseline] (10.936 s) : 0, 10936413
Agent [candidate] (1.19 s) : 0, 1189993
Total [candidate] (10.935 s) : 0, 10934624

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.031 s	-
Agent	appsec	1.206 s	174.846 ms (17.0%)
Agent	iast	1.166 s	134.822 ms (13.1%)
Agent	profiling	1.179 s	147.532 ms (14.3%)
Total	tracing	10.755 s	-
Total	appsec	10.956 s	200.363 ms (1.9%)
Total	iast	11.131 s	375.734 ms (3.5%)
Total	profiling	10.936 s	181.011 ms (1.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.028 s	-
Agent	appsec	1.205 s	177.069 ms (17.2%)
Agent	iast	1.167 s	138.128 ms (13.4%)
Agent	profiling	1.19 s	161.581 ms (15.7%)
Total	tracing	10.836 s	-
Total	appsec	11.044 s	208.12 ms (1.9%)
Total	iast	11.225 s	388.698 ms (3.6%)
Total	profiling	10.935 s	98.601 ms (0.9%)

gantt
    title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (703.411 ms) : 0, 703411
BytebuddyAgent [candidate] (701.903 ms) : 0, 701903
GlobalTracer [baseline] (245.088 ms) : 0, 245088
GlobalTracer [candidate] (244.757 ms) : 0, 244757
AppSec [baseline] (32.28 ms) : 0, 32280
AppSec [candidate] (31.996 ms) : 0, 31996
Debugger [baseline] (6.354 ms) : 0, 6354
Debugger [candidate] (6.316 ms) : 0, 6316
Remote Config [baseline] (683.274 µs) : 0, 683
Remote Config [candidate] (677.693 µs) : 0, 678
Telemetry [baseline] (14.36 ms) : 0, 14360
Telemetry [candidate] (14.408 ms) : 0, 14408
Flare Poller [baseline] (6.395 ms) : 0, 6395
Flare Poller [candidate] (5.623 ms) : 0, 5623
section appsec
crashtracking [baseline] (1.469 ms) : 0, 1469
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (728.602 ms) : 0, 728602
BytebuddyAgent [candidate] (727.909 ms) : 0, 727909
GlobalTracer [baseline] (236.588 ms) : 0, 236588
GlobalTracer [candidate] (236.432 ms) : 0, 236432
IAST [baseline] (24.949 ms) : 0, 24949
IAST [candidate] (25.079 ms) : 0, 25079
AppSec [baseline] (174.494 ms) : 0, 174494
AppSec [candidate] (174.4 ms) : 0, 174400
Debugger [baseline] (5.932 ms) : 0, 5932
Debugger [candidate] (5.922 ms) : 0, 5922
Remote Config [baseline] (628.62 µs) : 0, 629
Remote Config [candidate] (642.327 µs) : 0, 642
Telemetry [baseline] (8.429 ms) : 0, 8429
Telemetry [candidate] (8.449 ms) : 0, 8449
Flare Poller [baseline] (3.887 ms) : 0, 3887
Flare Poller [candidate] (3.915 ms) : 0, 3915
section iast
crashtracking [baseline] (1.472 ms) : 0, 1472
crashtracking [candidate] (1.488 ms) : 0, 1488
BytebuddyAgent [baseline] (827.755 ms) : 0, 827755
BytebuddyAgent [candidate] (827.205 ms) : 0, 827205
GlobalTracer [baseline] (234.683 ms) : 0, 234683
GlobalTracer [candidate] (235.223 ms) : 0, 235223
IAST [baseline] (33.45 ms) : 0, 33450
IAST [candidate] (30.842 ms) : 0, 30842
AppSec [baseline] (28.121 ms) : 0, 28121
AppSec [candidate] (30.823 ms) : 0, 30823
Debugger [baseline] (6.089 ms) : 0, 6089
Debugger [candidate] (6.142 ms) : 0, 6142
Remote Config [baseline] (595.71 µs) : 0, 596
Remote Config [candidate] (602.854 µs) : 0, 603
Telemetry [baseline] (8.406 ms) : 0, 8406
Telemetry [candidate] (8.572 ms) : 0, 8572
Flare Poller [baseline] (4.159 ms) : 0, 4159
Flare Poller [candidate] (4.18 ms) : 0, 4180
section profiling
crashtracking [baseline] (1.478 ms) : 0, 1478
crashtracking [candidate] (1.48 ms) : 0, 1480
BytebuddyAgent [baseline] (731.357 ms) : 0, 731357
BytebuddyAgent [candidate] (737.095 ms) : 0, 737095
GlobalTracer [baseline] (220.732 ms) : 0, 220732
GlobalTracer [candidate] (223.263 ms) : 0, 223263
AppSec [baseline] (32.21 ms) : 0, 32210
AppSec [candidate] (32.991 ms) : 0, 32991
Debugger [baseline] (9.204 ms) : 0, 9204
Debugger [candidate] (10.982 ms) : 0, 10982
Remote Config [baseline] (717.388 µs) : 0, 717
Remote Config [candidate] (2.228 ms) : 0, 2228
Telemetry [baseline] (13.753 ms) : 0, 13753
Telemetry [candidate] (10.651 ms) : 0, 10651
Flare Poller [baseline] (4.177 ms) : 0, 4177
Flare Poller [candidate] (4.196 ms) : 0, 4196
ProfilingAgent [baseline] (109.357 ms) : 0, 109357
ProfilingAgent [candidate] (110.735 ms) : 0, 110735
Profiling [baseline] (110.017 ms) : 0, 110017
Profiling [candidate] (111.381 ms) : 0, 111381

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	adrien.boitreaud/fix-muzzle-mismatch
git_commit_date	1761835849	1761900801
git_commit_sha	`9b91826`	`91f2995`
release_version	1.55.0-SNAPSHOT~9b91826873	1.55.0-SNAPSHOT~91f2995198

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1761903096	1761903096
ci_job_id	1208573712	1208573712
ci_pipeline_id	80848226	80848226
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-gs39yusk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-gs39yusk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load	better [-244.838µs; -124.558µs] or [-4.260%; -2.167%]	unstable [-50.486op/s; +102.798op/s] or [-6.277%; +12.781%]	5.563ms	830.438op/s	5.748ms	804.281op/s
scenario:load:petclinic:tracing:high_load	worse [+582.552µs; +968.890µs] or [+3.323%; +5.527%]	unstable [-38.640op/s; +16.140op/s] or [-14.520%; +6.065%]	18.306ms	254.875op/s	17.531ms	266.125op/s
scenario:load:petclinic:code_origins:high_load	worse [+640.958µs; +1053.500µs] or [+3.435%; +5.646%]	unstable [-38.841op/s; +1.081op/s] or [-15.045%; +0.419%]	19.508ms	239.281op/s	18.661ms	258.161op/s
scenario:load:petclinic:profiling:high_load	better [-1.670ms; -1.228ms] or [-7.989%; -5.874%]	unstable [-8.039op/s; +41.101op/s] or [-3.599%; +18.403%]	19.460ms	239.875op/s	20.909ms	223.344op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.211 ms) : 1200, 1223
.   : milestone, 1211,
iast (3.285 ms) : 3237, 3334
.   : milestone, 3285,
iast_FULL (5.748 ms) : 5691, 5805
.   : milestone, 5748,
iast_GLOBAL (3.629 ms) : 3565, 3693
.   : milestone, 3629,
profiling (2.098 ms) : 2079, 2116
.   : milestone, 2098,
tracing (1.766 ms) : 1753, 1780
.   : milestone, 1766,
section candidate
no_agent (1.192 ms) : 1181, 1204
.   : milestone, 1192,
iast (3.338 ms) : 3297, 3380
.   : milestone, 3338,
iast_FULL (5.563 ms) : 5508, 5618
.   : milestone, 5563,
iast_GLOBAL (3.629 ms) : 3573, 3684
.   : milestone, 3629,
profiling (2.044 ms) : 2026, 2062
.   : milestone, 2044,
tracing (1.773 ms) : 1759, 1787
.   : milestone, 1773,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.211 ms [1.2 ms, 1.223 ms]	-
iast	3.285 ms [3.237 ms, 3.334 ms]	2.074 ms (171.2%)
iast_FULL	5.748 ms [5.691 ms, 5.805 ms]	4.536 ms (374.4%)
iast_GLOBAL	3.629 ms [3.565 ms, 3.693 ms]	2.418 ms (199.6%)
profiling	2.098 ms [2.079 ms, 2.116 ms]	886.257 µs (73.2%)
tracing	1.766 ms [1.753 ms, 1.78 ms]	554.829 µs (45.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.192 ms [1.181 ms, 1.204 ms]	-
iast	3.338 ms [3.297 ms, 3.38 ms]	2.146 ms (180.0%)
iast_FULL	5.563 ms [5.508 ms, 5.618 ms]	4.371 ms (366.5%)
iast_GLOBAL	3.629 ms [3.573 ms, 3.684 ms]	2.436 ms (204.3%)
profiling	2.044 ms [2.026 ms, 2.062 ms]	851.114 µs (71.4%)
tracing	1.773 ms [1.759 ms, 1.787 ms]	580.618 µs (48.7%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.017 ms) : 17833, 18200
.   : milestone, 18017,
appsec (20.711 ms) : 20498, 20923
.   : milestone, 20711,
code_origins (18.661 ms) : 18475, 18847
.   : milestone, 18661,
iast (18.548 ms) : 18360, 18736
.   : milestone, 18548,
profiling (20.909 ms) : 20698, 21121
.   : milestone, 20909,
tracing (17.531 ms) : 17358, 17703
.   : milestone, 17531,
section candidate
no_agent (18.257 ms) : 18067, 18447
.   : milestone, 18257,
appsec (20.879 ms) : 20670, 21087
.   : milestone, 20879,
code_origins (19.508 ms) : 19311, 19705
.   : milestone, 19508,
iast (18.458 ms) : 18270, 18645
.   : milestone, 18458,
profiling (19.46 ms) : 19261, 19660
.   : milestone, 19460,
tracing (18.306 ms) : 18120, 18492
.   : milestone, 18306,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.017 ms [17.833 ms, 18.2 ms]	-
appsec	20.711 ms [20.498 ms, 20.923 ms]	2.694 ms (15.0%)
code_origins	18.661 ms [18.475 ms, 18.847 ms]	643.67 µs (3.6%)
iast	18.548 ms [18.36 ms, 18.736 ms]	530.771 µs (2.9%)
profiling	20.909 ms [20.698 ms, 21.121 ms]	2.893 ms (16.1%)
tracing	17.531 ms [17.358 ms, 17.703 ms]	-486.373 µs (-2.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.257 ms [18.067 ms, 18.447 ms]	-
appsec	20.879 ms [20.67 ms, 21.087 ms]	2.621 ms (14.4%)
code_origins	19.508 ms [19.311 ms, 19.705 ms]	1.251 ms (6.8%)
iast	18.458 ms [18.27 ms, 18.645 ms]	200.272 µs (1.1%)
profiling	19.46 ms [19.261 ms, 19.66 ms]	1.203 ms (6.6%)
tracing	18.306 ms [18.12 ms, 18.492 ms]	48.979 µs (0.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	adrien.boitreaud/fix-muzzle-mismatch
git_commit_date	1761835849	1761900801
git_commit_sha	`9b91826`	`91f2995`
release_version	1.55.0-SNAPSHOT~9b91826873	1.55.0-SNAPSHOT~91f2995198

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1761902781	1761902781
ci_job_id	1208573713	1208573713
ci_pipeline_id	80848226	80848226
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-euwibpwy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-euwibpwy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.489 ms) : 1478, 1501
.   : milestone, 1489,
appsec (3.721 ms) : 3503, 3940
.   : milestone, 3721,
iast (2.219 ms) : 2155, 2282
.   : milestone, 2219,
iast_GLOBAL (2.273 ms) : 2209, 2337
.   : milestone, 2273,
profiling (2.066 ms) : 2014, 2117
.   : milestone, 2066,
tracing (2.048 ms) : 1999, 2098
.   : milestone, 2048,
section candidate
no_agent (1.488 ms) : 1477, 1500
.   : milestone, 1488,
appsec (3.665 ms) : 3448, 3882
.   : milestone, 3665,
iast (2.226 ms) : 2163, 2290
.   : milestone, 2226,
iast_GLOBAL (2.277 ms) : 2212, 2341
.   : milestone, 2277,
profiling (2.056 ms) : 2004, 2107
.   : milestone, 2056,
tracing (2.046 ms) : 1996, 2096
.   : milestone, 2046,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.489 ms [1.478 ms, 1.501 ms]	-
appsec	3.721 ms [3.503 ms, 3.94 ms]	2.232 ms (149.9%)
iast	2.219 ms [2.155 ms, 2.282 ms]	729.278 µs (49.0%)
iast_GLOBAL	2.273 ms [2.209 ms, 2.337 ms]	783.527 µs (52.6%)
profiling	2.066 ms [2.014 ms, 2.117 ms]	576.197 µs (38.7%)
tracing	2.048 ms [1.999 ms, 2.098 ms]	559.102 µs (37.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.488 ms [1.477 ms, 1.5 ms]	-
appsec	3.665 ms [3.448 ms, 3.882 ms]	2.176 ms (146.2%)
iast	2.226 ms [2.163 ms, 2.29 ms]	738.012 µs (49.6%)
iast_GLOBAL	2.277 ms [2.212 ms, 2.341 ms]	788.123 µs (53.0%)
profiling	2.056 ms [2.004 ms, 2.107 ms]	567.103 µs (38.1%)
tracing	2.046 ms [1.996 ms, 2.096 ms]	557.535 µs (37.5%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~91f2995198, baseline=1.55.0-SNAPSHOT~9b91826873
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.245 s) : 15245000, 15245000
.   : milestone, 15245000,
appsec (15.014 s) : 15014000, 15014000
.   : milestone, 15014000,
iast (18.782 s) : 18782000, 18782000
.   : milestone, 18782000,
iast_GLOBAL (18.22 s) : 18220000, 18220000
.   : milestone, 18220000,
profiling (15.28 s) : 15280000, 15280000
.   : milestone, 15280000,
tracing (14.915 s) : 14915000, 14915000
.   : milestone, 14915000,
section candidate
no_agent (15.403 s) : 15403000, 15403000
.   : milestone, 15403000,
appsec (15.008 s) : 15008000, 15008000
.   : milestone, 15008000,
iast (18.571 s) : 18571000, 18571000
.   : milestone, 18571000,
iast_GLOBAL (18.124 s) : 18124000, 18124000
.   : milestone, 18124000,
profiling (15.689 s) : 15689000, 15689000
.   : milestone, 15689000,
tracing (15.405 s) : 15405000, 15405000
.   : milestone, 15405000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.245 s [15.245 s, 15.245 s]	-
appsec	15.014 s [15.014 s, 15.014 s]	-231.0 ms (-1.5%)
iast	18.782 s [18.782 s, 18.782 s]	3.537 s (23.2%)
iast_GLOBAL	18.22 s [18.22 s, 18.22 s]	2.975 s (19.5%)
profiling	15.28 s [15.28 s, 15.28 s]	35.0 ms (0.2%)
tracing	14.915 s [14.915 s, 14.915 s]	-330.0 ms (-2.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.403 s [15.403 s, 15.403 s]	-
appsec	15.008 s [15.008 s, 15.008 s]	-395.0 ms (-2.6%)
iast	18.571 s [18.571 s, 18.571 s]	3.168 s (20.6%)
iast_GLOBAL	18.124 s [18.124 s, 18.124 s]	2.721 s (17.7%)
profiling	15.689 s [15.689 s, 15.689 s]	286.0 ms (1.9%)
tracing	15.405 s [15.405 s, 15.405 s]	2.0 ms (0.0%)

...on/spark/src/main/java/datadog/trace/instrumentation/spark/AbstractDatadogSparkListener.java

...on/spark/src/main/java/datadog/trace/instrumentation/spark/AbstractSparkInstrumentation.java

pawel-big-lebowski

Looks good to me.

…ameter classes (#9886)

aboitreaud added 2 commits October 28, 2025 10:19

relax muzzle check

869fbd6

relax more

06ee1a2

aboitreaud requested a review from a team as a code owner October 28, 2025 09:31

spotlessApply

b1efd24

fix muzzle block by avoiding streaming hard deps and relaxing listene…

fdcf61a

…r bus matchers

pawel-big-lebowski reviewed Oct 28, 2025

View reviewed changes

...on/spark/src/main/java/datadog/trace/instrumentation/spark/AbstractDatadogSparkListener.java Outdated Show resolved Hide resolved

aboitreaud force-pushed the adrien.boitreaud/fix-muzzle-mismatch branch from 0b0a74f to b3de67a Compare October 28, 2025 12:27

remove unnecessary defense on StreamExecution

2dbcfec

aboitreaud force-pushed the adrien.boitreaud/fix-muzzle-mismatch branch from b3de67a to 2dbcfec Compare October 28, 2025 12:28

aboitreaud changed the title ~~Adrien.boitreaud/fix muzzle mismatch~~ [Spark instrumentation] Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented Oct 28, 2025

PerfectSlayer added type: bug Bug report and fix inst: apache spark Apache Spark instrumentation labels Oct 28, 2025

PerfectSlayer changed the title ~~[Spark instrumentation] Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented~~ Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented Oct 28, 2025

pawel-big-lebowski reviewed Oct 28, 2025

View reviewed changes

...on/spark/src/main/java/datadog/trace/instrumentation/spark/AbstractSparkInstrumentation.java Outdated Show resolved Hide resolved

revert changes on OL listener

bcc5bde

aboitreaud force-pushed the adrien.boitreaud/fix-muzzle-mismatch branch from 6ec48b4 to bcc5bde Compare October 28, 2025 16:25

aboitreaud marked this pull request as draft October 28, 2025 16:39

aboitreaud added 2 commits October 28, 2025 18:26

Merge branch 'master' into adrien.boitreaud/fix-muzzle-mismatch

382658f

improve getMicroBatchExecutionBatchIdKey

6e95cce

aboitreaud force-pushed the adrien.boitreaud/fix-muzzle-mismatch branch from 4ee15d0 to 6e95cce Compare October 29, 2025 10:56

aboitreaud added 3 commits October 29, 2025 12:02

import InvocationTargetException

ad4e077

Merge branch 'master' into adrien.boitreaud/fix-muzzle-mismatch

6e71170

Update AbstractDatadogSparkListener.java

d54d86c

aboitreaud force-pushed the adrien.boitreaud/fix-muzzle-mismatch branch from b57ba96 to d54d86c Compare October 29, 2025 13:00

no need reflection since key is known

74db1a2

aboitreaud force-pushed the adrien.boitreaud/fix-muzzle-mismatch branch from 53e0799 to 74db1a2 Compare October 29, 2025 14:27

Update AbstractSparkInstrumentation.java

73c3ac0

aboitreaud added 3 commits October 29, 2025 19:38

reflection against SparkPlanInfoAdvice

0cae20c

spotless

2f70c92

Merge branch 'master' into adrien.boitreaud/fix-muzzle-mismatch

1934a3f

aboitreaud marked this pull request as ready for review October 30, 2025 09:08

pawel-big-lebowski approved these changes Oct 30, 2025

View reviewed changes

[DJM-974] Use reflection for constructor in Scala 2.12, lookup by par…

91f2995

…ameter classes (#9886)

aboitreaud merged commit feb33de into master Oct 31, 2025
541 checks passed

aboitreaud deleted the adrien.boitreaud/fix-muzzle-mismatch branch October 31, 2025 14:35

github-actions bot added this to the 1.55.0 milestone Oct 31, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented #9872

Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented #9872

Uh oh!

aboitreaud commented Oct 28, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Oct 28, 2025 •

edited

Loading

Uh oh!

datadog-official bot commented Oct 28, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading

Uh oh!

pr-commenter bot commented Oct 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

pawel-big-lebowski left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented #9872

Fix muzzle mismatch causing Spark jobs on Databricks 17.3 to not be instrumented #9872

Uh oh!

Conversation

aboitreaud commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-official bot commented Oct 28, 2025 • edited by datadog-datadog-prod-us1 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

pawel-big-lebowski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

aboitreaud commented Oct 28, 2025 •

edited

Loading

github-actions bot commented Oct 28, 2025 •

edited

Loading

datadog-official bot commented Oct 28, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading

pr-commenter bot commented Oct 28, 2025 •

edited

Loading