Skip to content

Test support for reactor-kafka#7886

Merged
amarziali merged 5 commits intomasterfrom
andrea.marziali/kafka-reactor
Nov 7, 2024
Merged

Test support for reactor-kafka#7886
amarziali merged 5 commits intomasterfrom
andrea.marziali/kafka-reactor

Conversation

@amarziali
Copy link
Copy Markdown
Contributor

@amarziali amarziali commented Nov 5, 2024

What Does This Do

This PR adds testing for reactor-kafka. There is no need for an instrumentation. The kafka one is already handling the observability also for the reactive version (since it uses Flux.fromIterable(...))

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@amarziali amarziali requested review from a team as code owners November 5, 2024 16:22
@amarziali amarziali requested a review from dougqh November 5, 2024 16:22
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 5, 2024

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@amarziali amarziali added comp: testing Testing inst: kafka Kafka instrumentation labels Nov 5, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 5, 2024

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 5, 2024

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@amarziali amarziali added the type: enhancement Enhancements and improvements label Nov 5, 2024
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 5, 2024 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/kafka-reactor
git_commit_date 1730886991 1730904209
git_commit_sha 8233fc5 eaa5815
See matching parameters
Baseline Candidate
ci_job_date 1730905307 1730905307
ci_job_id 697382706 697382706
ci_pipeline_id 48351882 48351882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.21 11.0.21
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.21+9-post-Ubuntu-0ubuntu122.04 11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 5, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/kafka-reactor
git_commit_date 1730886991 1730904209
git_commit_sha 8233fc5 eaa5815
release_version 1.43.0-SNAPSHOT~8233fc5ff8 1.43.0-SNAPSHOT~eaa58153a9
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1730906559 1730906559
ci_job_id 697382702 697382702
ci_pipeline_id 48351882 48351882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 9 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.088 s) : 0, 1087664
Total [baseline] (8.566 s) : 0, 8566180
Agent [candidate] (1.091 s) : 0, 1091005
Total [candidate] (8.583 s) : 0, 8582913
section iast
Agent [baseline] (1.209 s) : 0, 1208652
Total [baseline] (9.144 s) : 0, 9144422
Agent [candidate] (1.219 s) : 0, 1218967
Total [candidate] (9.144 s) : 0, 9144113
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.21 s) : 0, 1210304
Total [baseline] (9.09 s) : 0, 9090339
Agent [candidate] (1.218 s) : 0, 1217701
Total [candidate] (9.097 s) : 0, 9096680
section iast_TELEMETRY_OFF
Agent [baseline] (1.21 s) : 0, 1210486
Total [baseline] (9.176 s) : 0, 9176063
Agent [candidate] (1.209 s) : 0, 1208824
Total [candidate] (9.143 s) : 0, 9143357
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent iast 1.209 s 120.987 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.21 s 122.639 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.21 s 122.822 ms (11.3%)
Total tracing 8.566 s -
Total iast 9.144 s 578.242 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.09 s 524.158 ms (6.1%)
Total iast_TELEMETRY_OFF 9.176 s 609.883 ms (7.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.091 s -
Agent iast 1.219 s 127.963 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.218 s 126.696 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.209 s 117.819 ms (10.8%)
Total tracing 8.583 s -
Total iast 9.144 s 561.201 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.097 s 513.768 ms (6.0%)
Total iast_TELEMETRY_OFF 9.143 s 560.444 ms (6.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (690.972 ms) : 0, 690972
BytebuddyAgent [candidate] (693.93 ms) : 0, 693930
GlobalTracer [baseline] (317.351 ms) : 0, 317351
GlobalTracer [candidate] (318.662 ms) : 0, 318662
AppSec [baseline] (54.431 ms) : 0, 54431
AppSec [candidate] (54.544 ms) : 0, 54544
Remote Config [baseline] (676.932 µs) : 0, 677
Remote Config [candidate] (692.065 µs) : 0, 692
Telemetry [baseline] (10.456 ms) : 0, 10456
Telemetry [candidate] (9.304 ms) : 0, 9304
section iast
BytebuddyAgent [baseline] (802.895 ms) : 0, 802895
BytebuddyAgent [candidate] (810.624 ms) : 0, 810624
GlobalTracer [baseline] (304.811 ms) : 0, 304811
GlobalTracer [candidate] (307.159 ms) : 0, 307159
AppSec [baseline] (57.352 ms) : 0, 57352
AppSec [candidate] (56.511 ms) : 0, 56511
IAST [baseline] (21.587 ms) : 0, 21587
IAST [candidate] (22.607 ms) : 0, 22607
Remote Config [baseline] (629.381 µs) : 0, 629
Remote Config [candidate] (627.752 µs) : 0, 628
Telemetry [baseline] (7.607 ms) : 0, 7607
Telemetry [candidate] (7.572 ms) : 0, 7572
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (805.076 ms) : 0, 805076
BytebuddyAgent [candidate] (809.613 ms) : 0, 809613
GlobalTracer [baseline] (305.181 ms) : 0, 305181
GlobalTracer [candidate] (307.122 ms) : 0, 307122
AppSec [baseline] (56.947 ms) : 0, 56947
AppSec [candidate] (57.285 ms) : 0, 57285
IAST [baseline] (21.263 ms) : 0, 21263
IAST [candidate] (21.601 ms) : 0, 21601
Remote Config [baseline] (604.494 µs) : 0, 604
Remote Config [candidate] (620.644 µs) : 0, 621
Telemetry [baseline] (7.465 ms) : 0, 7465
Telemetry [candidate] (7.555 ms) : 0, 7555
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (804.676 ms) : 0, 804676
BytebuddyAgent [candidate] (802.72 ms) : 0, 802720
GlobalTracer [baseline] (304.616 ms) : 0, 304616
GlobalTracer [candidate] (305.238 ms) : 0, 305238
AppSec [baseline] (58.414 ms) : 0, 58414
AppSec [candidate] (58.285 ms) : 0, 58285
IAST [baseline] (20.744 ms) : 0, 20744
IAST [candidate] (20.665 ms) : 0, 20665
Remote Config [baseline] (632.122 µs) : 0, 632
Remote Config [candidate] (618.412 µs) : 0, 618
Telemetry [baseline] (7.554 ms) : 0, 7554
Telemetry [candidate] (7.484 ms) : 0, 7484
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1084027
Total [baseline] (10.345 s) : 0, 10344862
Agent [candidate] (1.081 s) : 0, 1081425
Total [candidate] (10.392 s) : 0, 10391921
section appsec
Agent [baseline] (1.225 s) : 0, 1225227
Total [baseline] (10.7 s) : 0, 10700122
Agent [candidate] (1.223 s) : 0, 1222651
Total [candidate] (10.662 s) : 0, 10661747
section iast
Agent [baseline] (1.21 s) : 0, 1209550
Total [baseline] (10.905 s) : 0, 10905353
Agent [candidate] (1.211 s) : 0, 1211430
Total [candidate] (10.933 s) : 0, 10932804
section profiling
Agent [baseline] (1.282 s) : 0, 1282204
Total [baseline] (10.694 s) : 0, 10694385
Agent [candidate] (1.282 s) : 0, 1281607
Total [candidate] (10.724 s) : 0, 10723683
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.084 s -
Agent appsec 1.225 s 141.201 ms (13.0%)
Agent iast 1.21 s 125.524 ms (11.6%)
Agent profiling 1.282 s 198.177 ms (18.3%)
Total tracing 10.345 s -
Total appsec 10.7 s 355.26 ms (3.4%)
Total iast 10.905 s 560.491 ms (5.4%)
Total profiling 10.694 s 349.523 ms (3.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent appsec 1.223 s 141.226 ms (13.1%)
Agent iast 1.211 s 130.005 ms (12.0%)
Agent profiling 1.282 s 200.182 ms (18.5%)
Total tracing 10.392 s -
Total appsec 10.662 s 269.827 ms (2.6%)
Total iast 10.933 s 540.883 ms (5.2%)
Total profiling 10.724 s 331.762 ms (3.2%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.917 ms) : 0, 687917
BytebuddyAgent [candidate] (687.196 ms) : 0, 687196
GlobalTracer [baseline] (315.858 ms) : 0, 315858
GlobalTracer [candidate] (316.234 ms) : 0, 316234
AppSec [baseline] (54.51 ms) : 0, 54510
AppSec [candidate] (54.312 ms) : 0, 54312
Remote Config [baseline] (681.287 µs) : 0, 681
Remote Config [candidate] (671.891 µs) : 0, 672
Telemetry [baseline] (11.338 ms) : 0, 11338
Telemetry [candidate] (9.279 ms) : 0, 9279
section appsec
BytebuddyAgent [baseline] (710.308 ms) : 0, 710308
BytebuddyAgent [candidate] (708.334 ms) : 0, 708334
GlobalTracer [baseline] (315.631 ms) : 0, 315631
GlobalTracer [candidate] (315.32 ms) : 0, 315320
AppSec [baseline] (166.789 ms) : 0, 166789
AppSec [candidate] (166.362 ms) : 0, 166362
IAST [baseline] (20.391 ms) : 0, 20391
IAST [candidate] (19.429 ms) : 0, 19429
Remote Config [baseline] (651.826 µs) : 0, 652
Remote Config [candidate] (642.534 µs) : 0, 643
Telemetry [baseline] (7.852 ms) : 0, 7852
Telemetry [candidate] (8.512 ms) : 0, 8512
section iast
BytebuddyAgent [baseline] (804.903 ms) : 0, 804903
BytebuddyAgent [candidate] (805.05 ms) : 0, 805050
GlobalTracer [baseline] (304.502 ms) : 0, 304502
GlobalTracer [candidate] (305.535 ms) : 0, 305535
AppSec [baseline] (56.722 ms) : 0, 56722
AppSec [candidate] (57.707 ms) : 0, 57707
IAST [baseline] (21.535 ms) : 0, 21535
IAST [candidate] (21.097 ms) : 0, 21097
Remote Config [baseline] (624.29 µs) : 0, 624
Remote Config [candidate] (640.531 µs) : 0, 641
Telemetry [baseline] (7.463 ms) : 0, 7463
Telemetry [candidate] (7.649 ms) : 0, 7649
section profiling
BytebuddyAgent [baseline] (684.244 ms) : 0, 684244
BytebuddyAgent [candidate] (683.682 ms) : 0, 683682
GlobalTracer [baseline] (398.672 ms) : 0, 398672
GlobalTracer [candidate] (398.742 ms) : 0, 398742
AppSec [baseline] (54.526 ms) : 0, 54526
AppSec [candidate] (54.665 ms) : 0, 54665
Remote Config [baseline] (709.047 µs) : 0, 709
Remote Config [candidate] (683.074 µs) : 0, 683
Telemetry [baseline] (12.092 ms) : 0, 12092
Telemetry [candidate] (12.176 ms) : 0, 12176
ProfilingAgent [baseline] (92.884 ms) : 0, 92884
ProfilingAgent [candidate] (92.649 ms) : 0, 92649
Profiling [baseline] (92.909 ms) : 0, 92909
Profiling [candidate] (92.673 ms) : 0, 92673
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-06T14:55:02 2024-11-06T15:01:57
git_branch master andrea.marziali/kafka-reactor
git_commit_date 1730886991 1730904209
git_commit_sha 8233fc5 eaa5815
release_version 1.43.0-SNAPSHOT~8233fc5ff8 1.43.0-SNAPSHOT~eaa58153a9
start_time 2024-11-06T14:54:48 2024-11-06T15:01:43
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1730905666 1730905666
ci_job_id 697382703 697382703
ci_pipeline_id 48351882 48351882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.344 ms) : 1324, 1363
.   : milestone, 1344,
appsec (1.744 ms) : 1719, 1770
.   : milestone, 1744,
appsec_no_iast (1.731 ms) : 1707, 1756
.   : milestone, 1731,
iast (1.485 ms) : 1463, 1508
.   : milestone, 1485,
profiling (1.522 ms) : 1499, 1546
.   : milestone, 1522,
tracing (1.478 ms) : 1453, 1502
.   : milestone, 1478,
section candidate
no_agent (1.339 ms) : 1319, 1359
.   : milestone, 1339,
appsec (1.732 ms) : 1707, 1758
.   : milestone, 1732,
appsec_no_iast (1.726 ms) : 1700, 1751
.   : milestone, 1726,
iast (1.485 ms) : 1462, 1507
.   : milestone, 1485,
profiling (1.488 ms) : 1464, 1512
.   : milestone, 1488,
tracing (1.476 ms) : 1452, 1500
.   : milestone, 1476,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.324 ms, 1.363 ms] -
appsec 1.744 ms [1.719 ms, 1.77 ms] 400.907 µs (29.8%)
appsec_no_iast 1.731 ms [1.707 ms, 1.756 ms] 387.936 µs (28.9%)
iast 1.485 ms [1.463 ms, 1.508 ms] 141.782 µs (10.6%)
profiling 1.522 ms [1.499 ms, 1.546 ms] 178.63 µs (13.3%)
tracing 1.478 ms [1.453 ms, 1.502 ms] 134.138 µs (10.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.339 ms [1.319 ms, 1.359 ms] -
appsec 1.732 ms [1.707 ms, 1.758 ms] 393.533 µs (29.4%)
appsec_no_iast 1.726 ms [1.7 ms, 1.751 ms] 386.988 µs (28.9%)
iast 1.485 ms [1.462 ms, 1.507 ms] 145.905 µs (10.9%)
profiling 1.488 ms [1.464 ms, 1.512 ms] 149.457 µs (11.2%)
tracing 1.476 ms [1.452 ms, 1.5 ms] 137.519 µs (10.3%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.145 µs) : 346, 385
.   : milestone, 365,
iast (482.26 µs) : 461, 503
.   : milestone, 482,
iast_FULL (636.441 µs) : 615, 658
.   : milestone, 636,
iast_GLOBAL (504.797 µs) : 484, 526
.   : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (486.353 µs) : 465, 507
.   : milestone, 486,
iast_INACTIVE (442.964 µs) : 422, 464
.   : milestone, 443,
iast_TELEMETRY_OFF (475.546 µs) : 454, 497
.   : milestone, 476,
tracing (449.213 µs) : 428, 470
.   : milestone, 449,
section candidate
no_agent (372.715 µs) : 353, 392
.   : milestone, 373,
iast (485.125 µs) : 463, 507
.   : milestone, 485,
iast_FULL (641.948 µs) : 621, 663
.   : milestone, 642,
iast_GLOBAL (506.865 µs) : 486, 528
.   : milestone, 507,
iast_HARDCODED_SECRET_DISABLED (482.928 µs) : 462, 504
.   : milestone, 483,
iast_INACTIVE (444.376 µs) : 423, 466
.   : milestone, 444,
iast_TELEMETRY_OFF (470.123 µs) : 449, 491
.   : milestone, 470,
tracing (438.48 µs) : 418, 459
.   : milestone, 438,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.145 µs [345.673 µs, 384.617 µs] -
iast 482.26 µs [461.024 µs, 503.496 µs] 117.115 µs (32.1%)
iast_FULL 636.441 µs [615.174 µs, 657.707 µs] 271.296 µs (74.3%)
iast_GLOBAL 504.797 µs [483.725 µs, 525.868 µs] 139.652 µs (38.2%)
iast_HARDCODED_SECRET_DISABLED 486.353 µs [465.208 µs, 507.497 µs] 121.208 µs (33.2%)
iast_INACTIVE 442.964 µs [422.274 µs, 463.653 µs] 77.819 µs (21.3%)
iast_TELEMETRY_OFF 475.546 µs [454.256 µs, 496.836 µs] 110.401 µs (30.2%)
tracing 449.213 µs [428.416 µs, 470.01 µs] 84.068 µs (23.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.715 µs [353.268 µs, 392.163 µs] -
iast 485.125 µs [463.39 µs, 506.859 µs] 112.409 µs (30.2%)
iast_FULL 641.948 µs [620.686 µs, 663.21 µs] 269.232 µs (72.2%)
iast_GLOBAL 506.865 µs [485.667 µs, 528.064 µs] 134.15 µs (36.0%)
iast_HARDCODED_SECRET_DISABLED 482.928 µs [461.762 µs, 504.095 µs] 110.213 µs (29.6%)
iast_INACTIVE 444.376 µs [423.091 µs, 465.661 µs] 71.661 µs (19.2%)
iast_TELEMETRY_OFF 470.123 µs [448.781 µs, 491.464 µs] 97.408 µs (26.1%)
tracing 438.48 µs [418.079 µs, 458.881 µs] 65.765 µs (17.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/kafka-reactor
git_commit_date 1730886991 1730904209
git_commit_sha 8233fc5 eaa5815
release_version 1.43.0-SNAPSHOT~8233fc5ff8 1.43.0-SNAPSHOT~eaa58153a9
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1730906182 1730906182
ci_job_id 697382704 697382704
ci_pipeline_id 48351882 48351882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.462 ms) : 1450, 1473
.   : milestone, 1462,
appsec (2.324 ms) : 2283, 2365
.   : milestone, 2324,
iast (2.076 ms) : 2024, 2128
.   : milestone, 2076,
iast_GLOBAL (2.132 ms) : 2079, 2185
.   : milestone, 2132,
profiling (1.946 ms) : 1905, 1988
.   : milestone, 1946,
tracing (1.921 ms) : 1880, 1961
.   : milestone, 1921,
section candidate
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.343 ms) : 2301, 2384
.   : milestone, 2343,
iast (2.07 ms) : 2018, 2123
.   : milestone, 2070,
iast_GLOBAL (2.112 ms) : 2060, 2163
.   : milestone, 2112,
profiling (1.944 ms) : 1903, 1986
.   : milestone, 1944,
tracing (1.933 ms) : 1893, 1973
.   : milestone, 1933,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.462 ms [1.45 ms, 1.473 ms] -
appsec 2.324 ms [2.283 ms, 2.365 ms] 862.185 µs (59.0%)
iast 2.076 ms [2.024 ms, 2.128 ms] 614.272 µs (42.0%)
iast_GLOBAL 2.132 ms [2.079 ms, 2.185 ms] 670.225 µs (45.9%)
profiling 1.946 ms [1.905 ms, 1.988 ms] 484.533 µs (33.1%)
tracing 1.921 ms [1.88 ms, 1.961 ms] 459.072 µs (31.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.455 ms, 1.478 ms] -
appsec 2.343 ms [2.301 ms, 2.384 ms] 876.138 µs (59.7%)
iast 2.07 ms [2.018 ms, 2.123 ms] 603.456 µs (41.1%)
iast_GLOBAL 2.112 ms [2.06 ms, 2.163 ms] 644.957 µs (44.0%)
profiling 1.944 ms [1.903 ms, 1.986 ms] 477.504 µs (32.6%)
tracing 1.933 ms [1.893 ms, 1.973 ms] 466.459 µs (31.8%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~eaa58153a9, baseline=1.43.0-SNAPSHOT~8233fc5ff8
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.101 s) : 15101000, 15101000
.   : milestone, 15101000,
appsec (15.134 s) : 15134000, 15134000
.   : milestone, 15134000,
iast (18.357 s) : 18357000, 18357000
.   : milestone, 18357000,
iast_GLOBAL (18.092 s) : 18092000, 18092000
.   : milestone, 18092000,
profiling (15.265 s) : 15265000, 15265000
.   : milestone, 15265000,
tracing (15.02 s) : 15020000, 15020000
.   : milestone, 15020000,
section candidate
no_agent (15.124 s) : 15124000, 15124000
.   : milestone, 15124000,
appsec (15.298 s) : 15298000, 15298000
.   : milestone, 15298000,
iast (18.69 s) : 18690000, 18690000
.   : milestone, 18690000,
iast_GLOBAL (18.296 s) : 18296000, 18296000
.   : milestone, 18296000,
profiling (15.284 s) : 15284000, 15284000
.   : milestone, 15284000,
tracing (15.442 s) : 15442000, 15442000
.   : milestone, 15442000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.101 s [15.101 s, 15.101 s] -
appsec 15.134 s [15.134 s, 15.134 s] 33.0 ms (0.2%)
iast 18.357 s [18.357 s, 18.357 s] 3.256 s (21.6%)
iast_GLOBAL 18.092 s [18.092 s, 18.092 s] 2.991 s (19.8%)
profiling 15.265 s [15.265 s, 15.265 s] 164.0 ms (1.1%)
tracing 15.02 s [15.02 s, 15.02 s] -81.0 ms (-0.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.124 s [15.124 s, 15.124 s] -
appsec 15.298 s [15.298 s, 15.298 s] 174.0 ms (1.2%)
iast 18.69 s [18.69 s, 18.69 s] 3.566 s (23.6%)
iast_GLOBAL 18.296 s [18.296 s, 18.296 s] 3.172 s (21.0%)
profiling 15.284 s [15.284 s, 15.284 s] 160.0 ms (1.1%)
tracing 15.442 s [15.442 s, 15.442 s] 318.0 ms (2.1%)

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 5, 2024 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/kafka-reactor
git_commit_date 1730886991 1730904209
git_commit_sha 8233fc5 eaa5815
See matching parameters
Baseline Candidate
ci_job_date 1730905348 1730905348
ci_job_id 697382707 697382707
ci_pipeline_id 48351882 48351882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.21 11.0.21
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.21+9-post-Ubuntu-0ubuntu122.04 11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@amarziali amarziali force-pushed the andrea.marziali/kafka-reactor branch from e98b448 to eaa5815 Compare November 6, 2024 14:44
ygree
ygree approved these changes Nov 6, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@ygree ygree left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've noticed that the KafkaReactorForkedTests are identical in the latestDepTest and the test modules. It might be worth looking into why that is. Otherwise, it looks good to me.

@amarziali
Copy link
Copy Markdown
Contributor Author

amarziali commented Nov 7, 2024

I've noticed that the KafkaReactorForkedTests are identical in the latestDepTest and the test modules. It might be worth looking into why that is. Otherwise, it looks good to me.

Yes @ygree you are right. The latestDep are duplicated in another folder because of kafka client api being different on the latest version comparing to 0.11. This is not the same for the reactive api that are the same. however, to have the latest version, the test class has to be on latestDep hence the reason why is duplicated

@amarziali amarziali merged commit 8e1ab2b into master Nov 7, 2024
@amarziali amarziali deleted the andrea.marziali/kafka-reactor branch November 7, 2024 07:40
@github-actions github-actions Bot added this to the 1.43.0 milestone Nov 7, 2024
@mcculls mcculls added tag: no release notes Changes to exclude from release notes and removed tag: no release notes Changes to exclude from release notes labels Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ygree ygree ygree approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh was automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: testing Testing inst: kafka Kafka instrumentation type: enhancement Enhancements and improvements

Projects

None yet

Milestone

1.43.0

Development

Successfully merging this pull request may close these issues.

3 participants

@amarziali @ygree @mcculls