Skip to content

Reenable kafka 3.8 by default#8007

Merged
mcculls merged 2 commits intomasterfrom
nayeem-kamal/enable-kafka-3
Nov 25, 2024
Merged

Reenable kafka 3.8 by default#8007
mcculls merged 2 commits intomasterfrom
nayeem-kamal/enable-kafka-3

Conversation

@nayeem-kamal
Copy link
Copy Markdown
Contributor

@nayeem-kamal nayeem-kamal commented Nov 25, 2024 •
edited by atlassian Bot
Loading

What Does This Do

Re-enables kafka 3.8 because the double instrumentation issue has been resolved in #8006

Motivation

Additional Notes

Contributor Checklist

Jira ticket: AIDM-477

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 25, 2024 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_date 1732526948 1732529492
git_commit_sha b36006c 39229df
See matching parameters
Baseline Candidate
ci_job_date 1732530881 1732530881
ci_job_id 718553101 718553101
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce unsure
[-74856.414op/s; -14018.330op/s] or [-4.074%; -0.763%]
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@nayeem-kamal nayeem-kamal marked this pull request as ready for review November 25, 2024 08:10
@nayeem-kamal nayeem-kamal requested review from a team as code owners November 25, 2024 08:10
@nayeem-kamal nayeem-kamal requested a review from dougqh November 25, 2024 08:10
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 25, 2024 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_date 1732526948 1732529492
git_commit_sha b36006c 39229df
See matching parameters
Baseline Candidate
ci_job_date 1732530616 1732530616
ci_job_id 718529103 718529103
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

Base automatically changed from mcculls/kafka-client-muzzle-fix to master November 25, 2024 09:29
@mcculls mcculls force-pushed the nayeem-kamal/enable-kafka-3 branch from c21dcfd to 8c09a3d Compare November 25, 2024 09:30
amarziali
amarziali reviewed Nov 25, 2024
View reviewed changes
Comment thread ...n/java/datadog/trace/instrumentation/kafka_clients38/ConsumerCoordinatorInstrumentation.java Outdated
@@ -20,11 +19,6 @@ public ConsumerCoordinatorInstrumentation() {
super("kafka");
Copy link
Copy Markdown
Contributor

@amarziali amarziali Nov 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may we also have instrumentation names by version? i.e. ["kafka", "kafka-0.11"] for the old and ["kafka", "kafka-3.8"] for the new? it will give the possibility to selectivelt disable one of them if other issues will arise

Copy link
Copy Markdown
Contributor

@mcculls mcculls Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

39229df

I didn't add an alias to KafkaDeserializerInstrumentation because it is an IAST instrumentation that works across all known Kafka versions

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 25, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_sha b055376 39229df
release_version 1.43.0-SNAPSHOT~b0553764bc 1.43.0-SNAPSHOT~39229df6d9
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1732531837 1732531837
ci_job_id 718529098 718529098
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1732529492 1732529492
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 8 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.09 s) : 0, 1089870
Total [baseline] (10.459 s) : 0, 10458959
Agent [candidate] (1.092 s) : 0, 1092019
Total [candidate] (10.471 s) : 0, 10471268
section appsec
Agent [baseline] (1.223 s) : 0, 1222823
Total [baseline] (10.734 s) : 0, 10733672
Agent [candidate] (1.228 s) : 0, 1228206
Total [candidate] (10.739 s) : 0, 10738829
section iast
Agent [baseline] (1.216 s) : 0, 1216003
Total [baseline] (10.912 s) : 0, 10912136
Agent [candidate] (1.216 s) : 0, 1215910
Total [candidate] (10.911 s) : 0, 10910890
section profiling
Agent [baseline] (1.325 s) : 0, 1324802
Total [baseline] (10.876 s) : 0, 10876431
Agent [candidate] (1.317 s) : 0, 1317066
Total [candidate] (10.849 s) : 0, 10848939
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.09 s -
Agent appsec 1.223 s 132.953 ms (12.2%)
Agent iast 1.216 s 126.134 ms (11.6%)
Agent profiling 1.325 s 234.932 ms (21.6%)
Total tracing 10.459 s -
Total appsec 10.734 s 274.713 ms (2.6%)
Total iast 10.912 s 453.178 ms (4.3%)
Total profiling 10.876 s 417.472 ms (4.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent appsec 1.228 s 136.187 ms (12.5%)
Agent iast 1.216 s 123.891 ms (11.3%)
Agent profiling 1.317 s 225.047 ms (20.6%)
Total tracing 10.471 s -
Total appsec 10.739 s 267.561 ms (2.6%)
Total iast 10.911 s 439.622 ms (4.2%)
Total profiling 10.849 s 377.671 ms (3.6%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.047 ms) : 0, 693047
BytebuddyAgent [candidate] (694.563 ms) : 0, 694563
GlobalTracer [baseline] (317.617 ms) : 0, 317617
GlobalTracer [candidate] (319.107 ms) : 0, 319107
AppSec [baseline] (54.886 ms) : 0, 54886
AppSec [candidate] (54.646 ms) : 0, 54646
Remote Config [baseline] (693.039 µs) : 0, 693
Remote Config [candidate] (684.682 µs) : 0, 685
Telemetry [baseline] (9.931 ms) : 0, 9931
Telemetry [candidate] (9.299 ms) : 0, 9299
section appsec
BytebuddyAgent [baseline] (709.015 ms) : 0, 709015
BytebuddyAgent [candidate] (713.287 ms) : 0, 713287
GlobalTracer [baseline] (314.157 ms) : 0, 314157
GlobalTracer [candidate] (315.745 ms) : 0, 315745
AppSec [baseline] (168.048 ms) : 0, 168048
AppSec [candidate] (167.359 ms) : 0, 167359
Remote Config [baseline] (650.333 µs) : 0, 650
Remote Config [candidate] (653.91 µs) : 0, 654
Telemetry [baseline] (7.779 ms) : 0, 7779
Telemetry [candidate] (7.88 ms) : 0, 7880
IAST [baseline] (19.039 ms) : 0, 19039
IAST [candidate] (19.063 ms) : 0, 19063
section iast
BytebuddyAgent [baseline] (809.797 ms) : 0, 809797
BytebuddyAgent [candidate] (809.131 ms) : 0, 809131
GlobalTracer [baseline] (306.113 ms) : 0, 306113
GlobalTracer [candidate] (305.834 ms) : 0, 305834
AppSec [baseline] (57.799 ms) : 0, 57799
AppSec [candidate] (58.169 ms) : 0, 58169
Remote Config [baseline] (614.771 µs) : 0, 615
Remote Config [candidate] (613.646 µs) : 0, 614
Telemetry [baseline] (7.403 ms) : 0, 7403
Telemetry [candidate] (7.546 ms) : 0, 7546
IAST [baseline] (20.602 ms) : 0, 20602
IAST [candidate] (20.881 ms) : 0, 20881
section profiling
BytebuddyAgent [baseline] (691.462 ms) : 0, 691462
BytebuddyAgent [candidate] (688.515 ms) : 0, 688515
GlobalTracer [baseline] (438.043 ms) : 0, 438043
GlobalTracer [candidate] (433.688 ms) : 0, 433688
AppSec [baseline] (54.026 ms) : 0, 54026
AppSec [candidate] (53.838 ms) : 0, 53838
Remote Config [baseline] (662.132 µs) : 0, 662
Remote Config [candidate] (668.869 µs) : 0, 669
Telemetry [baseline] (7.785 ms) : 0, 7785
Telemetry [candidate] (7.715 ms) : 0, 7715
ProfilingAgent [baseline] (93.578 ms) : 0, 93578
ProfilingAgent [candidate] (93.581 ms) : 0, 93581
Profiling [baseline] (93.602 ms) : 0, 93602
Profiling [candidate] (93.604 ms) : 0, 93604
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.098 s) : 0, 1097969
Total [baseline] (8.682 s) : 0, 8682243
Agent [candidate] (1.092 s) : 0, 1091686
Total [candidate] (8.7 s) : 0, 8699962
section iast
Agent [baseline] (1.218 s) : 0, 1217833
Total [baseline] (9.17 s) : 0, 9170420
Agent [candidate] (1.219 s) : 0, 1219249
Total [candidate] (9.175 s) : 0, 9175256
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.221 s) : 0, 1220879
Total [baseline] (9.204 s) : 0, 9203858
Agent [candidate] (1.223 s) : 0, 1222914
Total [candidate] (9.182 s) : 0, 9181890
section iast_TELEMETRY_OFF
Agent [baseline] (1.22 s) : 0, 1219812
Total [baseline] (9.216 s) : 0, 9215553
Agent [candidate] (1.216 s) : 0, 1215760
Total [candidate] (9.18 s) : 0, 9180221
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.098 s -
Agent iast 1.218 s 119.864 ms (10.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.221 s 122.909 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.22 s 121.842 ms (11.1%)
Total tracing 8.682 s -
Total iast 9.17 s 488.178 ms (5.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.204 s 521.616 ms (6.0%)
Total iast_TELEMETRY_OFF 9.216 s 533.311 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent iast 1.219 s 127.564 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.223 s 131.229 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.216 s 124.074 ms (11.4%)
Total tracing 8.7 s -
Total iast 9.175 s 475.294 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.182 s 481.928 ms (5.5%)
Total iast_TELEMETRY_OFF 9.18 s 480.26 ms (5.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.519 ms) : 0, 698519
BytebuddyAgent [candidate] (694.579 ms) : 0, 694579
GlobalTracer [baseline] (319.38 ms) : 0, 319380
GlobalTracer [candidate] (319.377 ms) : 0, 319377
AppSec [baseline] (54.852 ms) : 0, 54852
AppSec [candidate] (54.743 ms) : 0, 54743
Remote Config [baseline] (685.244 µs) : 0, 685
Remote Config [candidate] (680.692 µs) : 0, 681
Telemetry [baseline] (10.71 ms) : 0, 10710
Telemetry [candidate] (8.541 ms) : 0, 8541
section iast
BytebuddyAgent [baseline] (811.013 ms) : 0, 811013
BytebuddyAgent [candidate] (811.876 ms) : 0, 811876
GlobalTracer [baseline] (306.241 ms) : 0, 306241
GlobalTracer [candidate] (306.266 ms) : 0, 306266
AppSec [baseline] (57.187 ms) : 0, 57187
AppSec [candidate] (58.172 ms) : 0, 58172
Remote Config [baseline] (650.899 µs) : 0, 651
Remote Config [candidate] (627.512 µs) : 0, 628
Telemetry [baseline] (7.479 ms) : 0, 7479
Telemetry [candidate] (7.563 ms) : 0, 7563
IAST [baseline] (21.568 ms) : 0, 21568
IAST [candidate] (20.911 ms) : 0, 20911
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (812.539 ms) : 0, 812539
BytebuddyAgent [candidate] (815.297 ms) : 0, 815297
GlobalTracer [baseline] (307.408 ms) : 0, 307408
GlobalTracer [candidate] (306.758 ms) : 0, 306758
AppSec [baseline] (56.555 ms) : 0, 56555
AppSec [candidate] (56.466 ms) : 0, 56466
Remote Config [baseline] (632.38 µs) : 0, 632
Remote Config [candidate] (613.393 µs) : 0, 613
Telemetry [baseline] (7.492 ms) : 0, 7492
Telemetry [candidate] (7.466 ms) : 0, 7466
IAST [baseline] (22.542 ms) : 0, 22542
IAST [candidate] (22.489 ms) : 0, 22489
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (812.08 ms) : 0, 812080
BytebuddyAgent [candidate] (808.199 ms) : 0, 808199
GlobalTracer [baseline] (307.197 ms) : 0, 307197
GlobalTracer [candidate] (306.48 ms) : 0, 306480
AppSec [baseline] (57.493 ms) : 0, 57493
AppSec [candidate] (57.938 ms) : 0, 57938
Remote Config [baseline] (645.532 µs) : 0, 646
Remote Config [candidate] (628.511 µs) : 0, 629
Telemetry [baseline] (7.402 ms) : 0, 7402
Telemetry [candidate] (8.253 ms) : 0, 8253
IAST [baseline] (21.225 ms) : 0, 21225
IAST [candidate] (20.554 ms) : 0, 20554
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-25T10:20:45 2024-11-25T10:27:45
git_branch master nayeem-kamal/enable-kafka-3
git_commit_sha b055376 39229df
release_version 1.43.0-SNAPSHOT~b0553764bc 1.43.0-SNAPSHOT~39229df6d9
start_time 2024-11-25T10:20:31 2024-11-25T10:27:32
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1732530818 1732530818
ci_job_id 718529099 718529099
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1732529492 1732529492
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 15 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-90.730µs; -39.410µs] or [-5.833%; -2.534%]		 unstable
[-460.534op/s; +688.454op/s] or [-15.543%; +23.235%]		 1.490ms 3076.923op/s 1.555ms 2962.963op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1333, 1374
.   : milestone, 1353,
appsec (1.772 ms) : 1748, 1797
.   : milestone, 1772,
appsec_no_iast (1.777 ms) : 1753, 1801
.   : milestone, 1777,
iast (1.514 ms) : 1491, 1537
.   : milestone, 1514,
profiling (1.555 ms) : 1531, 1579
.   : milestone, 1555,
tracing (1.482 ms) : 1457, 1506
.   : milestone, 1482,
section candidate
no_agent (1.359 ms) : 1339, 1379
.   : milestone, 1359,
appsec (1.756 ms) : 1731, 1781
.   : milestone, 1756,
appsec_no_iast (1.771 ms) : 1746, 1796
.   : milestone, 1771,
iast (1.497 ms) : 1474, 1520
.   : milestone, 1497,
profiling (1.49 ms) : 1467, 1514
.   : milestone, 1490,
tracing (1.481 ms) : 1456, 1507
.   : milestone, 1481,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.333 ms, 1.374 ms] -
appsec 1.772 ms [1.748 ms, 1.797 ms] 418.946 µs (31.0%)
appsec_no_iast 1.777 ms [1.753 ms, 1.801 ms] 423.632 µs (31.3%)
iast 1.514 ms [1.491 ms, 1.537 ms] 160.648 µs (11.9%)
profiling 1.555 ms [1.531 ms, 1.579 ms] 202.073 µs (14.9%)
tracing 1.482 ms [1.457 ms, 1.506 ms] 128.31 µs (9.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.359 ms [1.339 ms, 1.379 ms] -
appsec 1.756 ms [1.731 ms, 1.781 ms] 397.305 µs (29.2%)
appsec_no_iast 1.771 ms [1.746 ms, 1.796 ms] 411.727 µs (30.3%)
iast 1.497 ms [1.474 ms, 1.52 ms] 137.947 µs (10.2%)
profiling 1.49 ms [1.467 ms, 1.514 ms] 131.485 µs (9.7%)
tracing 1.481 ms [1.456 ms, 1.507 ms] 122.591 µs (9.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.019 µs) : 358, 398
.   : milestone, 378,
iast (491.513 µs) : 470, 513
.   : milestone, 492,
iast_FULL (655.907 µs) : 634, 678
.   : milestone, 656,
iast_GLOBAL (525.269 µs) : 504, 547
.   : milestone, 525,
iast_HARDCODED_SECRET_DISABLED (493.166 µs) : 472, 514
.   : milestone, 493,
iast_INACTIVE (450.204 µs) : 429, 471
.   : milestone, 450,
iast_TELEMETRY_OFF (486.252 µs) : 464, 508
.   : milestone, 486,
tracing (448.597 µs) : 428, 469
.   : milestone, 449,
section candidate
no_agent (379.839 µs) : 359, 400
.   : milestone, 380,
iast (496.872 µs) : 476, 518
.   : milestone, 497,
iast_FULL (659.88 µs) : 638, 681
.   : milestone, 660,
iast_GLOBAL (530.225 µs) : 508, 552
.   : milestone, 530,
iast_HARDCODED_SECRET_DISABLED (490.802 µs) : 469, 513
.   : milestone, 491,
iast_INACTIVE (448.027 µs) : 427, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (478.56 µs) : 457, 500
.   : milestone, 479,
tracing (454.741 µs) : 434, 476
.   : milestone, 455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.019 µs [358.072 µs, 397.965 µs] -
iast 491.513 µs [470.017 µs, 513.009 µs] 113.494 µs (30.0%)
iast_FULL 655.907 µs [633.888 µs, 677.925 µs] 277.888 µs (73.5%)
iast_GLOBAL 525.269 µs [503.939 µs, 546.598 µs] 147.25 µs (39.0%)
iast_HARDCODED_SECRET_DISABLED 493.166 µs [471.947 µs, 514.386 µs] 115.148 µs (30.5%)
iast_INACTIVE 450.204 µs [429.49 µs, 470.917 µs] 72.185 µs (19.1%)
iast_TELEMETRY_OFF 486.252 µs [464.19 µs, 508.314 µs] 108.233 µs (28.6%)
tracing 448.597 µs [428.04 µs, 469.154 µs] 70.578 µs (18.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.839 µs [359.485 µs, 400.193 µs] -
iast 496.872 µs [475.577 µs, 518.167 µs] 117.033 µs (30.8%)
iast_FULL 659.88 µs [638.392 µs, 681.368 µs] 280.041 µs (73.7%)
iast_GLOBAL 530.225 µs [508.053 µs, 552.397 µs] 150.386 µs (39.6%)
iast_HARDCODED_SECRET_DISABLED 490.802 µs [468.824 µs, 512.78 µs] 110.964 µs (29.2%)
iast_INACTIVE 448.027 µs [427.417 µs, 468.637 µs] 68.189 µs (18.0%)
iast_TELEMETRY_OFF 478.56 µs [457.455 µs, 499.665 µs] 98.721 µs (26.0%)
tracing 454.741 µs [433.844 µs, 475.638 µs] 74.902 µs (19.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_sha b055376 39229df
release_version 1.43.0-SNAPSHOT~b0553764bc 1.43.0-SNAPSHOT~39229df6d9
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1732531476 1732531476
ci_job_id 718529100 718529100
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1732529492 1732529492
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.621 s) : 15621000, 15621000
.   : milestone, 15621000,
appsec (15.024 s) : 15024000, 15024000
.   : milestone, 15024000,
iast (19.658 s) : 19658000, 19658000
.   : milestone, 19658000,
iast_GLOBAL (17.798 s) : 17798000, 17798000
.   : milestone, 17798000,
profiling (15.103 s) : 15103000, 15103000
.   : milestone, 15103000,
tracing (15.131 s) : 15131000, 15131000
.   : milestone, 15131000,
section candidate
no_agent (15.269 s) : 15269000, 15269000
.   : milestone, 15269000,
appsec (15.018 s) : 15018000, 15018000
.   : milestone, 15018000,
iast (18.55 s) : 18550000, 18550000
.   : milestone, 18550000,
iast_GLOBAL (17.917 s) : 17917000, 17917000
.   : milestone, 17917000,
profiling (14.953 s) : 14953000, 14953000
.   : milestone, 14953000,
tracing (14.938 s) : 14938000, 14938000
.   : milestone, 14938000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.621 s [15.621 s, 15.621 s] -
appsec 15.024 s [15.024 s, 15.024 s] -597.0 ms (-3.8%)
iast 19.658 s [19.658 s, 19.658 s] 4.037 s (25.8%)
iast_GLOBAL 17.798 s [17.798 s, 17.798 s] 2.177 s (13.9%)
profiling 15.103 s [15.103 s, 15.103 s] -518.0 ms (-3.3%)
tracing 15.131 s [15.131 s, 15.131 s] -490.0 ms (-3.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.269 s [15.269 s, 15.269 s] -
appsec 15.018 s [15.018 s, 15.018 s] -251.0 ms (-1.6%)
iast 18.55 s [18.55 s, 18.55 s] 3.281 s (21.5%)
iast_GLOBAL 17.917 s [17.917 s, 17.917 s] 2.648 s (17.3%)
profiling 14.953 s [14.953 s, 14.953 s] -316.0 ms (-2.1%)
tracing 14.938 s [14.938 s, 14.938 s] -331.0 ms (-2.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1452, 1475
.   : milestone, 1463,
appsec (2.311 ms) : 2270, 2352
.   : milestone, 2311,
iast (2.078 ms) : 2025, 2130
.   : milestone, 2078,
iast_GLOBAL (2.101 ms) : 2048, 2153
.   : milestone, 2101,
profiling (1.959 ms) : 1916, 2002
.   : milestone, 1959,
tracing (1.921 ms) : 1881, 1962
.   : milestone, 1921,
section candidate
no_agent (1.461 ms) : 1449, 1472
.   : milestone, 1461,
appsec (2.318 ms) : 2277, 2359
.   : milestone, 2318,
iast (2.068 ms) : 2015, 2121
.   : milestone, 2068,
iast_GLOBAL (2.118 ms) : 2065, 2171
.   : milestone, 2118,
profiling (1.947 ms) : 1904, 1990
.   : milestone, 1947,
tracing (1.914 ms) : 1874, 1953
.   : milestone, 1914,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.463 ms [1.452 ms, 1.475 ms] -
appsec 2.311 ms [2.27 ms, 2.352 ms] 847.493 µs (57.9%)
iast 2.078 ms [2.025 ms, 2.13 ms] 614.064 µs (42.0%)
iast_GLOBAL 2.101 ms [2.048 ms, 2.153 ms] 637.171 µs (43.5%)
profiling 1.959 ms [1.916 ms, 2.002 ms] 495.649 µs (33.9%)
tracing 1.921 ms [1.881 ms, 1.962 ms] 457.651 µs (31.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.461 ms [1.449 ms, 1.472 ms] -
appsec 2.318 ms [2.277 ms, 2.359 ms] 857.298 µs (58.7%)
iast 2.068 ms [2.015 ms, 2.121 ms] 607.306 µs (41.6%)
iast_GLOBAL 2.118 ms [2.065 ms, 2.171 ms] 657.669 µs (45.0%)
profiling 1.947 ms [1.904 ms, 1.99 ms] 486.374 µs (33.3%)
tracing 1.914 ms [1.874 ms, 1.953 ms] 452.764 µs (31.0%)

@mcculls mcculls merged commit f2d21ae into master Nov 25, 2024
@mcculls mcculls deleted the nayeem-kamal/enable-kafka-3 branch November 25, 2024 11:04
@github-actions github-actions Bot added this to the 1.43.0 milestone Nov 25, 2024
@mcculls mcculls added type: bug Bug report and fix inst: kafka Kafka instrumentation labels Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls left review comments

@amarziali amarziali amarziali approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

inst: kafka Kafka instrumentation type: bug Bug report and fix

Projects

None yet

Milestone

1.43.0

Development

Successfully merging this pull request may close these issues.

3 participants

@nayeem-kamal @mcculls @amarziali