Skip to content

Check async propagation flag via agent tracer#8397

Merged
mcculls merged 2 commits intomasterfrom
mcculls/check-async-propagation-flag-via-agenttracer
Feb 18, 2025
Merged

Check async propagation flag via agent tracer#8397
mcculls merged 2 commits intomasterfrom
mcculls/check-async-propagation-flag-via-agenttracer

Conversation

@mcculls
Copy link
Copy Markdown
Contributor

@mcculls mcculls commented Feb 16, 2025 •
edited by atlassian Bot
Loading

What Does This Do

Migrates tests and helpers to check async-propagation flag via AgentTracer

Motivation

Removes use of deprecated scope method in our code.

Contributor Checklist

Jira ticket: APMAPI-954

@mcculls mcculls added comp: core Tracer core tag: no release notes Changes to exclude from release notes type: refactoring labels Feb 16, 2025
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 16, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/check-async-propagation-flag-via-agenttracer
git_commit_date 1739899344 1739902841
git_commit_sha 919d607 9bc1a1e
See matching parameters
Baseline Candidate
ci_job_date 1739904148 1739904148
ci_job_id 812621219 812621219
ci_pipeline_id 56195189 56195189
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 16, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/check-async-propagation-flag-via-agenttracer
git_commit_date 1739899344 1739902841
git_commit_sha 919d607 9bc1a1e
release_version 1.47.0-SNAPSHOT~919d6073eb 1.47.0-SNAPSHOT~9bc1a1e4a7
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1739905417 1739905417
ci_job_id 812621215 812621215
ci_pipeline_id 56195189 56195189
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zgkpwvox-project-304-concurrent-0-x3gfwa6l 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zgkpwvox-project-304-concurrent-0-x3gfwa6l 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 6 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1043366
Total [baseline] (10.467 s) : 0, 10466651
Agent [candidate] (1.042 s) : 0, 1042429
Total [candidate] (10.444 s) : 0, 10443501
section appsec
Agent [baseline] (1.185 s) : 0, 1185287
Total [baseline] (10.754 s) : 0, 10754483
Agent [candidate] (1.186 s) : 0, 1186120
Total [candidate] (10.799 s) : 0, 10798653
section iast
Agent [baseline] (1.172 s) : 0, 1171718
Total [baseline] (10.975 s) : 0, 10975434
Agent [candidate] (1.196 s) : 0, 1195809
Total [candidate] (11.186 s) : 0, 11186046
section profiling
Agent [baseline] (1.267 s) : 0, 1267290
Total [baseline] (10.911 s) : 0, 10910887
Agent [candidate] (1.265 s) : 0, 1264504
Total [candidate] (10.926 s) : 0, 10926466
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent appsec 1.185 s 141.921 ms (13.6%)
Agent iast 1.172 s 128.352 ms (12.3%)
Agent profiling 1.267 s 223.924 ms (21.5%)
Total tracing 10.467 s -
Total appsec 10.754 s 287.832 ms (2.7%)
Total iast 10.975 s 508.783 ms (4.9%)
Total profiling 10.911 s 444.236 ms (4.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.042 s -
Agent appsec 1.186 s 143.691 ms (13.8%)
Agent iast 1.196 s 153.381 ms (14.7%)
Agent profiling 1.265 s 222.076 ms (21.3%)
Total tracing 10.444 s -
Total appsec 10.799 s 355.151 ms (3.4%)
Total iast 11.186 s 742.544 ms (7.1%)
Total profiling 10.926 s 482.965 ms (4.6%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.942 ms) : 0, 718942
BytebuddyAgent [candidate] (717.112 ms) : 0, 717112
GlobalTracer [baseline] (240.631 ms) : 0, 240631
GlobalTracer [candidate] (239.505 ms) : 0, 239505
AppSec [baseline] (55.646 ms) : 0, 55646
AppSec [candidate] (55.452 ms) : 0, 55452
Remote Config [baseline] (699.489 µs) : 0, 699
Remote Config [candidate] (698.423 µs) : 0, 698
Telemetry [baseline] (12.169 ms) : 0, 12169
Telemetry [candidate] (14.313 ms) : 0, 14313
section appsec
BytebuddyAgent [baseline] (736.634 ms) : 0, 736634
BytebuddyAgent [candidate] (736.724 ms) : 0, 736724
GlobalTracer [baseline] (236.783 ms) : 0, 236783
GlobalTracer [candidate] (237.349 ms) : 0, 237349
IAST [baseline] (21.708 ms) : 0, 21708
IAST [candidate] (21.476 ms) : 0, 21476
AppSec [baseline] (176.828 ms) : 0, 176828
AppSec [candidate] (177.045 ms) : 0, 177045
Remote Config [baseline] (665.659 µs) : 0, 666
Remote Config [candidate] (676.57 µs) : 0, 677
Telemetry [baseline] (8.269 ms) : 0, 8269
Telemetry [candidate] (8.288 ms) : 0, 8288
section iast
BytebuddyAgent [baseline] (836.478 ms) : 0, 836478
BytebuddyAgent [candidate] (854.429 ms) : 0, 854429
GlobalTracer [baseline] (230.634 ms) : 0, 230634
GlobalTracer [candidate] (234.798 ms) : 0, 234798
IAST [baseline] (22.779 ms) : 0, 22779
IAST [candidate] (23.55 ms) : 0, 23550
AppSec [baseline] (57.322 ms) : 0, 57322
AppSec [candidate] (58.057 ms) : 0, 58057
Remote Config [baseline] (613.83 µs) : 0, 614
Remote Config [candidate] (626.058 µs) : 0, 626
Telemetry [baseline] (8.645 ms) : 0, 8645
Telemetry [candidate] (8.931 ms) : 0, 8931
section profiling
BytebuddyAgent [baseline] (712.446 ms) : 0, 712446
BytebuddyAgent [candidate] (711.317 ms) : 0, 711317
GlobalTracer [baseline] (350.659 ms) : 0, 350659
GlobalTracer [candidate] (351.527 ms) : 0, 351527
AppSec [baseline] (55.734 ms) : 0, 55734
AppSec [candidate] (54.461 ms) : 0, 54461
Remote Config [baseline] (668.732 µs) : 0, 669
Remote Config [candidate] (671.031 µs) : 0, 671
Telemetry [baseline] (8.894 ms) : 0, 8894
Telemetry [candidate] (8.888 ms) : 0, 8888
ProfilingAgent [baseline] (96.277 ms) : 0, 96277
ProfilingAgent [candidate] (95.005 ms) : 0, 95005
Profiling [baseline] (96.302 ms) : 0, 96302
Profiling [candidate] (95.03 ms) : 0, 95030
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042505
Total [baseline] (8.642 s) : 0, 8641534
Agent [candidate] (1.041 s) : 0, 1040673
Total [candidate] (8.638 s) : 0, 8637676
section iast
Agent [baseline] (1.168 s) : 0, 1167757
Total [baseline] (9.244 s) : 0, 9243873
Agent [candidate] (1.168 s) : 0, 1168487
Total [candidate] (9.285 s) : 0, 9284974
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.169 s) : 0, 1168707
Total [baseline] (9.23 s) : 0, 9230283
Agent [candidate] (1.173 s) : 0, 1172646
Total [candidate] (9.22 s) : 0, 9220119
section iast_TELEMETRY_OFF
Agent [baseline] (1.165 s) : 0, 1165491
Total [baseline] (9.249 s) : 0, 9249435
Agent [candidate] (1.171 s) : 0, 1170529
Total [candidate] (9.247 s) : 0, 9246894
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent iast 1.168 s 125.252 ms (12.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.169 s 126.202 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.165 s 122.986 ms (11.8%)
Total tracing 8.642 s -
Total iast 9.244 s 602.339 ms (7.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.23 s 588.749 ms (6.8%)
Total iast_TELEMETRY_OFF 9.249 s 607.901 ms (7.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent iast 1.168 s 127.814 ms (12.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.173 s 131.973 ms (12.7%)
Agent iast_TELEMETRY_OFF 1.171 s 129.856 ms (12.5%)
Total tracing 8.638 s -
Total iast 9.285 s 647.299 ms (7.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.22 s 582.444 ms (6.7%)
Total iast_TELEMETRY_OFF 9.247 s 609.219 ms (7.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.105 ms) : 0, 718105
BytebuddyAgent [candidate] (717.866 ms) : 0, 717866
GlobalTracer [baseline] (240.008 ms) : 0, 240008
GlobalTracer [candidate] (239.674 ms) : 0, 239674
AppSec [baseline] (55.443 ms) : 0, 55443
AppSec [candidate] (55.285 ms) : 0, 55285
Remote Config [baseline] (694.579 µs) : 0, 695
Remote Config [candidate] (696.091 µs) : 0, 696
Telemetry [baseline] (12.944 ms) : 0, 12944
Telemetry [candidate] (11.988 ms) : 0, 11988
section iast
BytebuddyAgent [baseline] (833.639 ms) : 0, 833639
BytebuddyAgent [candidate] (834.241 ms) : 0, 834241
GlobalTracer [baseline] (230.052 ms) : 0, 230052
GlobalTracer [candidate] (230.009 ms) : 0, 230009
IAST [baseline] (22.729 ms) : 0, 22729
IAST [candidate] (22.767 ms) : 0, 22767
AppSec [baseline] (56.856 ms) : 0, 56856
AppSec [candidate] (56.927 ms) : 0, 56927
Remote Config [baseline] (618.614 µs) : 0, 619
Remote Config [candidate] (603.482 µs) : 0, 603
Telemetry [baseline] (8.645 ms) : 0, 8645
Telemetry [candidate] (8.669 ms) : 0, 8669
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (834.051 ms) : 0, 834051
BytebuddyAgent [candidate] (836.938 ms) : 0, 836938
GlobalTracer [baseline] (229.667 ms) : 0, 229667
GlobalTracer [candidate] (231.013 ms) : 0, 231013
IAST [baseline] (22.673 ms) : 0, 22673
IAST [candidate] (22.884 ms) : 0, 22884
AppSec [baseline] (57.619 ms) : 0, 57619
AppSec [candidate] (57.175 ms) : 0, 57175
Remote Config [baseline] (626.035 µs) : 0, 626
Remote Config [candidate] (597.253 µs) : 0, 597
Telemetry [baseline] (8.754 ms) : 0, 8754
Telemetry [candidate] (8.7 ms) : 0, 8700
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (832.388 ms) : 0, 832388
BytebuddyAgent [candidate] (833.948 ms) : 0, 833948
GlobalTracer [baseline] (229.866 ms) : 0, 229866
GlobalTracer [candidate] (232.112 ms) : 0, 232112
IAST [baseline] (24.628 ms) : 0, 24628
IAST [candidate] (26.369 ms) : 0, 26369
AppSec [baseline] (54.306 ms) : 0, 54306
AppSec [candidate] (53.512 ms) : 0, 53512
Remote Config [baseline] (611.288 µs) : 0, 611
Remote Config [candidate] (623.484 µs) : 0, 623
Telemetry [baseline] (8.484 ms) : 0, 8484
Telemetry [candidate] (8.633 ms) : 0, 8633
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-02-18T18:33:30 2025-02-18T18:40:37
git_branch master mcculls/check-async-propagation-flag-via-agenttracer
git_commit_date 1739899344 1739902841
git_commit_sha 919d607 9bc1a1e
release_version 1.47.0-SNAPSHOT~919d6073eb 1.47.0-SNAPSHOT~9bc1a1e4a7
start_time 2025-02-18T18:33:16 2025-02-18T18:40:23
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1739904397 1739904397
ci_job_id 812621216 812621216
ci_pipeline_id 56195189 56195189
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zgkpwvox-project-304-concurrent-1-jzpzv89l 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zgkpwvox-project-304-concurrent-1-jzpzv89l 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (385.314 µs) : 366, 405
.   : milestone, 385,
iast (517.853 µs) : 496, 540
.   : milestone, 518,
iast_FULL (726.538 µs) : 704, 749
.   : milestone, 727,
iast_GLOBAL (566.866 µs) : 545, 589
.   : milestone, 567,
iast_HARDCODED_SECRET_DISABLED (521.929 µs) : 500, 544
.   : milestone, 522,
iast_INACTIVE (465.782 µs) : 445, 487
.   : milestone, 466,
iast_TELEMETRY_OFF (508.581 µs) : 485, 532
.   : milestone, 509,
tracing (463.015 µs) : 442, 484
.   : milestone, 463,
section candidate
no_agent (390.596 µs) : 370, 411
.   : milestone, 391,
iast (517.88 µs) : 496, 540
.   : milestone, 518,
iast_FULL (733.764 µs) : 712, 756
.   : milestone, 734,
iast_GLOBAL (566.058 µs) : 544, 588
.   : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (516.106 µs) : 494, 538
.   : milestone, 516,
iast_INACTIVE (474.356 µs) : 452, 497
.   : milestone, 474,
iast_TELEMETRY_OFF (506.183 µs) : 483, 530
.   : milestone, 506,
tracing (458.21 µs) : 437, 479
.   : milestone, 458,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.314 µs [365.719 µs, 404.909 µs] -
iast 517.853 µs [495.891 µs, 539.816 µs] 132.54 µs (34.4%)
iast_FULL 726.538 µs [704.475 µs, 748.601 µs] 341.224 µs (88.6%)
iast_GLOBAL 566.866 µs [545.127 µs, 588.605 µs] 181.552 µs (47.1%)
iast_HARDCODED_SECRET_DISABLED 521.929 µs [499.567 µs, 544.292 µs] 136.616 µs (35.5%)
iast_INACTIVE 465.782 µs [444.584 µs, 486.981 µs] 80.469 µs (20.9%)
iast_TELEMETRY_OFF 508.581 µs [484.897 µs, 532.266 µs] 123.268 µs (32.0%)
tracing 463.015 µs [442.035 µs, 483.995 µs] 77.702 µs (20.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 390.596 µs [370.209 µs, 410.982 µs] -
iast 517.88 µs [495.693 µs, 540.067 µs] 127.284 µs (32.6%)
iast_FULL 733.764 µs [711.888 µs, 755.639 µs] 343.168 µs (87.9%)
iast_GLOBAL 566.058 µs [543.826 µs, 588.29 µs] 175.463 µs (44.9%)
iast_HARDCODED_SECRET_DISABLED 516.106 µs [494.217 µs, 537.995 µs] 125.511 µs (32.1%)
iast_INACTIVE 474.356 µs [451.916 µs, 496.796 µs] 83.76 µs (21.4%)
iast_TELEMETRY_OFF 506.183 µs [482.644 µs, 529.723 µs] 115.588 µs (29.6%)
tracing 458.21 µs [437.145 µs, 479.275 µs] 67.615 µs (17.3%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.374 ms) : 1354, 1393
.   : milestone, 1374,
appsec (1.761 ms) : 1737, 1785
.   : milestone, 1761,
appsec_no_iast (1.753 ms) : 1728, 1779
.   : milestone, 1753,
iast (1.525 ms) : 1501, 1548
.   : milestone, 1525,
profiling (1.522 ms) : 1498, 1546
.   : milestone, 1522,
tracing (1.528 ms) : 1502, 1553
.   : milestone, 1528,
section candidate
no_agent (1.378 ms) : 1358, 1399
.   : milestone, 1378,
appsec (1.743 ms) : 1719, 1766
.   : milestone, 1743,
appsec_no_iast (1.771 ms) : 1749, 1794
.   : milestone, 1771,
iast (1.517 ms) : 1493, 1541
.   : milestone, 1517,
profiling (1.506 ms) : 1483, 1529
.   : milestone, 1506,
tracing (1.504 ms) : 1479, 1529
.   : milestone, 1504,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.374 ms [1.354 ms, 1.393 ms] -
appsec 1.761 ms [1.737 ms, 1.785 ms] 387.263 µs (28.2%)
appsec_no_iast 1.753 ms [1.728 ms, 1.779 ms] 379.655 µs (27.6%)
iast 1.525 ms [1.501 ms, 1.548 ms] 150.732 µs (11.0%)
profiling 1.522 ms [1.498 ms, 1.546 ms] 148.011 µs (10.8%)
tracing 1.528 ms [1.502 ms, 1.553 ms] 153.976 µs (11.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.378 ms [1.358 ms, 1.399 ms] -
appsec 1.743 ms [1.719 ms, 1.766 ms] 364.097 µs (26.4%)
appsec_no_iast 1.771 ms [1.749 ms, 1.794 ms] 393.031 µs (28.5%)
iast 1.517 ms [1.493 ms, 1.541 ms] 138.666 µs (10.1%)
profiling 1.506 ms [1.483 ms, 1.529 ms] 127.294 µs (9.2%)
tracing 1.504 ms [1.479 ms, 1.529 ms] 125.821 µs (9.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/check-async-propagation-flag-via-agenttracer
git_commit_date 1739899344 1739902841
git_commit_sha 919d607 9bc1a1e
release_version 1.47.0-SNAPSHOT~919d6073eb 1.47.0-SNAPSHOT~9bc1a1e4a7
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1739904994 1739904994
ci_job_id 812621217 812621217
ci_pipeline_id 56195189 56195189
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kkoh8pz-project-304-concurrent-0-ke7784h3 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kkoh8pz-project-304-concurrent-0-ke7784h3 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.72 s) : 14720000, 14720000
.   : milestone, 14720000,
appsec (14.912 s) : 14912000, 14912000
.   : milestone, 14912000,
iast (18.636 s) : 18636000, 18636000
.   : milestone, 18636000,
iast_GLOBAL (18.154 s) : 18154000, 18154000
.   : milestone, 18154000,
profiling (15.168 s) : 15168000, 15168000
.   : milestone, 15168000,
tracing (14.793 s) : 14793000, 14793000
.   : milestone, 14793000,
section candidate
no_agent (14.989 s) : 14989000, 14989000
.   : milestone, 14989000,
appsec (14.831 s) : 14831000, 14831000
.   : milestone, 14831000,
iast (18.662 s) : 18662000, 18662000
.   : milestone, 18662000,
iast_GLOBAL (17.659 s) : 17659000, 17659000
.   : milestone, 17659000,
profiling (15.321 s) : 15321000, 15321000
.   : milestone, 15321000,
tracing (14.884 s) : 14884000, 14884000
.   : milestone, 14884000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.72 s [14.72 s, 14.72 s] -
appsec 14.912 s [14.912 s, 14.912 s] 192.0 ms (1.3%)
iast 18.636 s [18.636 s, 18.636 s] 3.916 s (26.6%)
iast_GLOBAL 18.154 s [18.154 s, 18.154 s] 3.434 s (23.3%)
profiling 15.168 s [15.168 s, 15.168 s] 448.0 ms (3.0%)
tracing 14.793 s [14.793 s, 14.793 s] 73.0 ms (0.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.989 s [14.989 s, 14.989 s] -
appsec 14.831 s [14.831 s, 14.831 s] -158.0 ms (-1.1%)
iast 18.662 s [18.662 s, 18.662 s] 3.673 s (24.5%)
iast_GLOBAL 17.659 s [17.659 s, 17.659 s] 2.67 s (17.8%)
profiling 15.321 s [15.321 s, 15.321 s] 332.0 ms (2.2%)
tracing 14.884 s [14.884 s, 14.884 s] -105.0 ms (-0.7%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~9bc1a1e4a7, baseline=1.47.0-SNAPSHOT~919d6073eb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (2.369 ms) : 2326, 2413
.   : milestone, 2369,
iast (2.102 ms) : 2047, 2157
.   : milestone, 2102,
iast_GLOBAL (2.15 ms) : 2095, 2205
.   : milestone, 2150,
profiling (1.972 ms) : 1928, 2016
.   : milestone, 1972,
tracing (1.95 ms) : 1908, 1992
.   : milestone, 1950,
section candidate
no_agent (1.474 ms) : 1462, 1485
.   : milestone, 1474,
appsec (2.364 ms) : 2320, 2408
.   : milestone, 2364,
iast (2.114 ms) : 2059, 2169
.   : milestone, 2114,
iast_GLOBAL (2.153 ms) : 2098, 2208
.   : milestone, 2153,
profiling (2.002 ms) : 1957, 2047
.   : milestone, 2002,
tracing (1.951 ms) : 1909, 1993
.   : milestone, 1951,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.461 ms, 1.484 ms] -
appsec 2.369 ms [2.326 ms, 2.413 ms] 896.761 µs (60.9%)
iast 2.102 ms [2.047 ms, 2.157 ms] 629.408 µs (42.7%)
iast_GLOBAL 2.15 ms [2.095 ms, 2.205 ms] 677.477 µs (46.0%)
profiling 1.972 ms [1.928 ms, 2.016 ms] 499.501 µs (33.9%)
tracing 1.95 ms [1.908 ms, 1.992 ms] 477.714 µs (32.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.462 ms, 1.485 ms] -
appsec 2.364 ms [2.32 ms, 2.408 ms] 890.13 µs (60.4%)
iast 2.114 ms [2.059 ms, 2.169 ms] 640.059 µs (43.4%)
iast_GLOBAL 2.153 ms [2.098 ms, 2.208 ms] 678.985 µs (46.1%)
profiling 2.002 ms [1.957 ms, 2.047 ms] 527.973 µs (35.8%)
tracing 1.951 ms [1.909 ms, 1.993 ms] 477.31 µs (32.4%)

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 16, 2025 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/check-async-propagation-flag-via-agenttracer
git_commit_date 1739899344 1739902841
git_commit_sha 919d607 9bc1a1e
See matching parameters
Baseline Candidate
ci_job_date 1739904162 1739904162
ci_job_id 812621220 812621220
ci_pipeline_id 56195189 56195189
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume unsure
[-11702.133op/s; -1007.591op/s] or [-3.754%; -0.323%]
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@mcculls mcculls force-pushed the mcculls/check-async-propagation-flag-via-agenttracer branch from a3eeff8 to b9995a5 Compare February 16, 2025 21:43
@mcculls mcculls marked this pull request as ready for review February 16, 2025 22:05
PerfectSlayer
PerfectSlayer reviewed Feb 17, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor questions about checking active span first (it can simplify the code but might feel implicit behavior) and about scope async propagation (is it still scope?)

Comment thread ...va/datadog/trace/bootstrap/instrumentation/java/concurrent/ExecutorInstrumentationUtils.java Outdated
Comment thread ...mentation/scala-promise/src/main/java/datadog/trace/instrumentation/scala/PromiseHelper.java Outdated
Comment thread dd-java-agent/instrumentation/vertx-web-3.9/src/test/java/server/IastVertx39TestVerticle.java Outdated
Comment thread dd-java-agent/instrumentation/vertx-web-4.0/src/latestDepTest/java/server/VertxTestServer.java Outdated
Comment thread dd-java-agent/instrumentation/vertx-web-4.0/src/test/java/server/VertxTestServer.java Outdated
Comment thread dd-java-agent/instrumentation/vertx-web-5.0/src/test/java/server/VertxTestServer.java Outdated
Comment thread dd-java-agent/testing/src/main/groovy/datadog/trace/agent/test/base/HttpServerTest.groovy Outdated
@mcculls mcculls force-pushed the mcculls/capture-continuations-via-agenttracer branch from 745a7ba to 419f799 Compare February 17, 2025 10:24
@mcculls mcculls force-pushed the mcculls/check-async-propagation-flag-via-agenttracer branch from b9995a5 to 0640b39 Compare February 17, 2025 11:08
Base automatically changed from mcculls/capture-continuations-via-agenttracer to master February 17, 2025 11:36
@mcculls mcculls force-pushed the mcculls/check-async-propagation-flag-via-agenttracer branch 2 times, most recently from 2e446f6 to c9dbf04 Compare February 17, 2025 21:28
PerfectSlayer
PerfectSlayer approved these changes Feb 18, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So let's merge this one now and revisit the activeSpan() / isAsyncPropagationEnabled() later 👍

@mcculls mcculls force-pushed the mcculls/check-async-propagation-flag-via-agenttracer branch 2 times, most recently from b4f611d to cc4c5a3 Compare February 18, 2025 11:34
@mcculls mcculls force-pushed the mcculls/check-async-propagation-flag-via-agenttracer branch from cc4c5a3 to 9bc1a1e Compare February 18, 2025 18:20
@mcculls mcculls merged commit 5b7780b into master Feb 18, 2025
@mcculls mcculls deleted the mcculls/check-async-propagation-flag-via-agenttracer branch February 18, 2025 19:43
@github-actions github-actions Bot added this to the 1.47.0 milestone Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/asm-java

@ValentinZakharov ValentinZakharov Awaiting requested review from ValentinZakharov ValentinZakharov is a code owner automatically assigned from DataDog/asm-java

Assignees

No one assigned

Labels

comp: core Tracer core tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Milestone

1.47.0

Development

Successfully merging this pull request may close these issues.

2 participants

@mcculls @PerfectSlayer