Skip to content

Make sure budgets apply to line probes, too.#8378

Merged
evanchooly merged 2 commits intomasterfrom
lineProbeBudgets
Feb 18, 2025
Merged

Make sure budgets apply to line probes, too.#8378
evanchooly merged 2 commits intomasterfrom
lineProbeBudgets

Conversation

@evanchooly
Copy link
Copy Markdown
Contributor

@evanchooly evanchooly commented Feb 12, 2025 •
edited by atlassian Bot
Loading

What Does This Do

Make sure budgets apply to line probes, too.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: DEBUG-3496

@evanchooly evanchooly requested a review from a team as a code owner February 12, 2025 20:24
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 12, 2025 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

jpbempel
jpbempel approved these changes Feb 13, 2025
View reviewed changes
Comment thread dd-java-agent/agent-debugger/src/test/java/com/datadog/debugger/probe/LogProbeTest.java Outdated
Comment thread dd-java-agent/agent-debugger/src/test/java/com/datadog/debugger/probe/LogProbeTest.java Outdated
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 18, 2025

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1739912540 1739912911
end_time 2025-02-18T21:03:37 2025-02-18T21:09:47
git_branch master lineProbeBudgets
git_commit_sha 5b7780b 098b029
start_time 2025-02-18T21:02:21 2025-02-18T21:08:32
See matching parameters
Baseline Candidate
ci_job_id 812930666 812930666
ci_pipeline_id 56214974 56214974
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1739911928 1739911928

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+1.311ms; +1.338ms] or [+13.132%; +13.403%]		 worse
[+1.322ms; +1.392ms] or [+13.038%; +13.727%]		 worse
[+1.268ms; +1.383ms] or [+12.380%; +13.502%]		 worse
[+1.194ms; +1.643ms] or [+11.331%; +15.586%]		 worse
[-12.037op/s; -9.724op/s] or [-12.398%; -10.016%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-52.202µs; +21.896µs] or [-18.558%; +7.784%]		 unstable
[-69.686µs; +29.226µs] or [-21.496%; +9.015%]		 unstable
[-85.834µs; +36.743µs] or [-25.124%; +10.755%]		 unstable
[-795.786µs; +254.519µs] or [-93.963%; +30.052%]		 same
scenario:basic same same same unstable
[-56.345µs; +38.665µs] or [-10.601%; +7.275%]		 unstable
[-109.004op/s; +237.209op/s] or [-4.360%; +9.488%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (324.184 µs) : 261, 387
.   : milestone, 324,
basic (307.923 µs) : 292, 323
.   : milestone, 308,
loop (10.138 ms) : 10102, 10174
.   : milestone, 10138,
section candidate
noprobe (303.954 µs) : 283, 325
.   : milestone, 304,
basic (310.478 µs) : 301, 320
.   : milestone, 310,
loop (11.495 ms) : 11466, 11523
.   : milestone, 11495,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 324.184 µs [261.027 µs, 387.342 µs]
basic 307.923 µs [292.451 µs, 323.396 µs]
loop 10.138 ms [10.102 ms, 10.174 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 303.954 µs [282.601 µs, 325.308 µs]
basic 310.478 µs [301.434 µs, 319.523 µs]
loop 11.495 ms [11.466 ms, 11.523 ms]

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 18, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master lineProbeBudgets
git_commit_date 1739907811 1739911928
git_commit_sha 5b7780b 098b029
release_version 1.47.0-SNAPSHOT~5b7780bc05 1.47.0-SNAPSHOT~098b0298d6
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1739914287 1739914287
ci_job_id 812930660 812930660
ci_pipeline_id 56214974 56214974
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kpnzgrdt-project-304-concurrent-1-xmuwdfr8 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kpnzgrdt-project-304-concurrent-1-xmuwdfr8 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1045287
Total [baseline] (8.661 s) : 0, 8660613
Agent [candidate] (1.041 s) : 0, 1041244
Total [candidate] (8.63 s) : 0, 8630193
section iast
Agent [baseline] (1.175 s) : 0, 1174754
Total [baseline] (9.207 s) : 0, 9207048
Agent [candidate] (1.172 s) : 0, 1172432
Total [candidate] (9.222 s) : 0, 9221867
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.172 s) : 0, 1172088
Total [baseline] (9.197 s) : 0, 9197178
Agent [candidate] (1.172 s) : 0, 1171687
Total [candidate] (9.205 s) : 0, 9205006
section iast_TELEMETRY_OFF
Agent [baseline] (1.166 s) : 0, 1166208
Total [baseline] (9.205 s) : 0, 9204859
Agent [candidate] (1.166 s) : 0, 1166186
Total [candidate] (9.213 s) : 0, 9212813
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.045 s -
Agent iast 1.175 s 129.467 ms (12.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 126.802 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.166 s 120.922 ms (11.6%)
Total tracing 8.661 s -
Total iast 9.207 s 546.435 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.197 s 536.565 ms (6.2%)
Total iast_TELEMETRY_OFF 9.205 s 544.246 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent iast 1.172 s 131.188 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 130.443 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.166 s 124.942 ms (12.0%)
Total tracing 8.63 s -
Total iast 9.222 s 591.674 ms (6.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.205 s 574.813 ms (6.7%)
Total iast_TELEMETRY_OFF 9.213 s 582.62 ms (6.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.445 ms) : 0, 720445
BytebuddyAgent [candidate] (716.491 ms) : 0, 716491
GlobalTracer [baseline] (240.029 ms) : 0, 240029
GlobalTracer [candidate] (240.468 ms) : 0, 240468
AppSec [baseline] (55.661 ms) : 0, 55661
AppSec [candidate] (55.452 ms) : 0, 55452
Remote Config [baseline] (699.235 µs) : 0, 699
Remote Config [candidate] (713.732 µs) : 0, 714
Telemetry [baseline] (13.157 ms) : 0, 13157
Telemetry [candidate] (12.934 ms) : 0, 12934
section iast
BytebuddyAgent [baseline] (838.933 ms) : 0, 838933
BytebuddyAgent [candidate] (836.72 ms) : 0, 836720
GlobalTracer [baseline] (231.234 ms) : 0, 231234
GlobalTracer [candidate] (231.314 ms) : 0, 231314
IAST [baseline] (22.941 ms) : 0, 22941
IAST [candidate] (22.764 ms) : 0, 22764
AppSec [baseline] (57.125 ms) : 0, 57125
AppSec [candidate] (57.028 ms) : 0, 57028
Remote Config [baseline] (614.206 µs) : 0, 614
Remote Config [candidate] (611.606 µs) : 0, 612
Telemetry [baseline] (8.699 ms) : 0, 8699
Telemetry [candidate] (8.757 ms) : 0, 8757
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (836.984 ms) : 0, 836984
BytebuddyAgent [candidate] (835.5 ms) : 0, 835500
GlobalTracer [baseline] (230.723 ms) : 0, 230723
GlobalTracer [candidate] (231.388 ms) : 0, 231388
IAST [baseline] (22.671 ms) : 0, 22671
IAST [candidate] (22.732 ms) : 0, 22732
AppSec [baseline] (57.201 ms) : 0, 57201
AppSec [candidate] (57.497 ms) : 0, 57497
Remote Config [baseline] (598.579 µs) : 0, 599
Remote Config [candidate] (626.146 µs) : 0, 626
Telemetry [baseline] (8.644 ms) : 0, 8644
Telemetry [candidate] (8.71 ms) : 0, 8710
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (832.815 ms) : 0, 832815
BytebuddyAgent [candidate] (831.871 ms) : 0, 831871
GlobalTracer [baseline] (230.13 ms) : 0, 230130
GlobalTracer [candidate] (230.879 ms) : 0, 230879
IAST [baseline] (26.837 ms) : 0, 26837
IAST [candidate] (26.221 ms) : 0, 26221
AppSec [baseline] (52.063 ms) : 0, 52063
AppSec [candidate] (52.89 ms) : 0, 52890
Remote Config [baseline] (614.116 µs) : 0, 614
Remote Config [candidate] (603.207 µs) : 0, 603
Telemetry [baseline] (8.504 ms) : 0, 8504
Telemetry [candidate] (8.558 ms) : 0, 8558
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1047100
Total [baseline] (10.477 s) : 0, 10477185
Agent [candidate] (1.041 s) : 0, 1041168
Total [candidate] (10.527 s) : 0, 10526737
section appsec
Agent [baseline] (1.182 s) : 0, 1182405
Total [baseline] (10.735 s) : 0, 10735223
Agent [candidate] (1.19 s) : 0, 1189978
Total [candidate] (10.71 s) : 0, 10710202
section iast
Agent [baseline] (1.171 s) : 0, 1170625
Total [baseline] (10.937 s) : 0, 10936969
Agent [candidate] (1.17 s) : 0, 1169663
Total [candidate] (10.936 s) : 0, 10936309
section profiling
Agent [baseline] (1.259 s) : 0, 1258515
Total [baseline] (10.85 s) : 0, 10849501
Agent [candidate] (1.263 s) : 0, 1263230
Total [candidate] (10.86 s) : 0, 10860184
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent appsec 1.182 s 135.305 ms (12.9%)
Agent iast 1.171 s 123.524 ms (11.8%)
Agent profiling 1.259 s 211.414 ms (20.2%)
Total tracing 10.477 s -
Total appsec 10.735 s 258.038 ms (2.5%)
Total iast 10.937 s 459.784 ms (4.4%)
Total profiling 10.85 s 372.316 ms (3.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent appsec 1.19 s 148.81 ms (14.3%)
Agent iast 1.17 s 128.495 ms (12.3%)
Agent profiling 1.263 s 222.062 ms (21.3%)
Total tracing 10.527 s -
Total appsec 10.71 s 183.464 ms (1.7%)
Total iast 10.936 s 409.572 ms (3.9%)
Total profiling 10.86 s 333.447 ms (3.2%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (721.737 ms) : 0, 721737
BytebuddyAgent [candidate] (717.263 ms) : 0, 717263
GlobalTracer [baseline] (241.01 ms) : 0, 241010
GlobalTracer [candidate] (241.014 ms) : 0, 241014
AppSec [baseline] (55.446 ms) : 0, 55446
AppSec [candidate] (55.557 ms) : 0, 55557
Remote Config [baseline] (704.967 µs) : 0, 705
Remote Config [candidate] (712.974 µs) : 0, 713
Telemetry [baseline] (12.983 ms) : 0, 12983
Telemetry [candidate] (11.362 ms) : 0, 11362
section appsec
BytebuddyAgent [baseline] (734.363 ms) : 0, 734363
BytebuddyAgent [candidate] (738.515 ms) : 0, 738515
GlobalTracer [baseline] (236.304 ms) : 0, 236304
GlobalTracer [candidate] (238.863 ms) : 0, 238863
AppSec [baseline] (176.796 ms) : 0, 176796
AppSec [candidate] (177.419 ms) : 0, 177419
Remote Config [baseline] (670.046 µs) : 0, 670
Remote Config [candidate] (663.975 µs) : 0, 664
Telemetry [baseline] (8.323 ms) : 0, 8323
Telemetry [candidate] (8.281 ms) : 0, 8281
IAST [baseline] (21.49 ms) : 0, 21490
IAST [candidate] (21.969 ms) : 0, 21969
section iast
BytebuddyAgent [baseline] (834.257 ms) : 0, 834257
BytebuddyAgent [candidate] (834.56 ms) : 0, 834560
GlobalTracer [baseline] (231.132 ms) : 0, 231132
GlobalTracer [candidate] (231.098 ms) : 0, 231098
AppSec [baseline] (57.698 ms) : 0, 57698
AppSec [candidate] (56.784 ms) : 0, 56784
Remote Config [baseline] (624.832 µs) : 0, 625
Remote Config [candidate] (605.009 µs) : 0, 605
Telemetry [baseline] (8.738 ms) : 0, 8738
Telemetry [candidate] (8.654 ms) : 0, 8654
IAST [baseline] (22.912 ms) : 0, 22912
IAST [candidate] (22.717 ms) : 0, 22717
section profiling
BytebuddyAgent [baseline] (706.296 ms) : 0, 706296
BytebuddyAgent [candidate] (708.418 ms) : 0, 708418
GlobalTracer [baseline] (349.352 ms) : 0, 349352
GlobalTracer [candidate] (352.291 ms) : 0, 352291
AppSec [baseline] (55.155 ms) : 0, 55155
AppSec [candidate] (54.701 ms) : 0, 54701
Remote Config [baseline] (673.746 µs) : 0, 674
Remote Config [candidate] (702.998 µs) : 0, 703
Telemetry [baseline] (8.994 ms) : 0, 8994
Telemetry [candidate] (8.851 ms) : 0, 8851
ProfilingAgent [baseline] (95.942 ms) : 0, 95942
ProfilingAgent [candidate] (95.959 ms) : 0, 95959
Profiling [baseline] (95.967 ms) : 0, 95967
Profiling [candidate] (95.985 ms) : 0, 95985
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-02-18T21:01:27 2025-02-18T21:08:32
git_branch master lineProbeBudgets
git_commit_date 1739907811 1739911928
git_commit_sha 5b7780b 098b029
release_version 1.47.0-SNAPSHOT~5b7780bc05 1.47.0-SNAPSHOT~098b0298d6
start_time 2025-02-18T21:01:13 2025-02-18T21:08:18
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1739913271 1739913271
ci_job_id 812930661 812930661
ci_pipeline_id 56214974 56214974
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-dsbfzdx3-project-304-concurrent-0-7fa4w9aj 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-dsbfzdx3-project-304-concurrent-0-7fa4w9aj 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.363 ms) : 1342, 1383
.   : milestone, 1363,
appsec (1.762 ms) : 1739, 1785
.   : milestone, 1762,
appsec_no_iast (1.741 ms) : 1718, 1764
.   : milestone, 1741,
iast (1.516 ms) : 1492, 1541
.   : milestone, 1516,
profiling (1.538 ms) : 1513, 1564
.   : milestone, 1538,
tracing (1.485 ms) : 1460, 1510
.   : milestone, 1485,
section candidate
no_agent (1.369 ms) : 1348, 1391
.   : milestone, 1369,
appsec (1.761 ms) : 1738, 1783
.   : milestone, 1761,
appsec_no_iast (1.757 ms) : 1733, 1782
.   : milestone, 1757,
iast (1.524 ms) : 1500, 1547
.   : milestone, 1524,
profiling (1.504 ms) : 1480, 1528
.   : milestone, 1504,
tracing (1.489 ms) : 1463, 1514
.   : milestone, 1489,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.342 ms, 1.383 ms] -
appsec 1.762 ms [1.739 ms, 1.785 ms] 399.466 µs (29.3%)
appsec_no_iast 1.741 ms [1.718 ms, 1.764 ms] 378.286 µs (27.8%)
iast 1.516 ms [1.492 ms, 1.541 ms] 153.825 µs (11.3%)
profiling 1.538 ms [1.513 ms, 1.564 ms] 175.902 µs (12.9%)
tracing 1.485 ms [1.46 ms, 1.51 ms] 122.131 µs (9.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.369 ms [1.348 ms, 1.391 ms] -
appsec 1.761 ms [1.738 ms, 1.783 ms] 391.356 µs (28.6%)
appsec_no_iast 1.757 ms [1.733 ms, 1.782 ms] 388.197 µs (28.4%)
iast 1.524 ms [1.5 ms, 1.547 ms] 154.418 µs (11.3%)
profiling 1.504 ms [1.48 ms, 1.528 ms] 134.702 µs (9.8%)
tracing 1.489 ms [1.463 ms, 1.514 ms] 119.562 µs (8.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05
    dateFormat X
    axisFormat %s
section baseline
no_agent (381.243 µs) : 362, 401
.   : milestone, 381,
iast (506.92 µs) : 485, 528
.   : milestone, 507,
iast_FULL (726.368 µs) : 705, 748
.   : milestone, 726,
iast_GLOBAL (559.167 µs) : 537, 581
.   : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (516.286 µs) : 493, 540
.   : milestone, 516,
iast_INACTIVE (458.698 µs) : 438, 480
.   : milestone, 459,
iast_TELEMETRY_OFF (504.659 µs) : 481, 528
.   : milestone, 505,
tracing (462.48 µs) : 441, 484
.   : milestone, 462,
section candidate
no_agent (379.24 µs) : 359, 399
.   : milestone, 379,
iast (516.559 µs) : 495, 538
.   : milestone, 517,
iast_FULL (739.577 µs) : 718, 762
.   : milestone, 740,
iast_GLOBAL (564.7 µs) : 542, 587
.   : milestone, 565,
iast_HARDCODED_SECRET_DISABLED (511.613 µs) : 490, 533
.   : milestone, 512,
iast_INACTIVE (462.364 µs) : 440, 485
.   : milestone, 462,
iast_TELEMETRY_OFF (495.046 µs) : 472, 518
.   : milestone, 495,
tracing (451.987 µs) : 431, 473
.   : milestone, 452,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.243 µs [361.678 µs, 400.808 µs] -
iast 506.92 µs [485.423 µs, 528.417 µs] 125.677 µs (33.0%)
iast_FULL 726.368 µs [704.509 µs, 748.226 µs] 345.125 µs (90.5%)
iast_GLOBAL 559.167 µs [536.918 µs, 581.416 µs] 177.924 µs (46.7%)
iast_HARDCODED_SECRET_DISABLED 516.286 µs [492.988 µs, 539.585 µs] 135.044 µs (35.4%)
iast_INACTIVE 458.698 µs [437.78 µs, 479.616 µs] 77.456 µs (20.3%)
iast_TELEMETRY_OFF 504.659 µs [480.886 µs, 528.433 µs] 123.417 µs (32.4%)
tracing 462.48 µs [440.972 µs, 483.988 µs] 81.238 µs (21.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.24 µs [359.452 µs, 399.028 µs] -
iast 516.559 µs [494.622 µs, 538.496 µs] 137.319 µs (36.2%)
iast_FULL 739.577 µs [717.598 µs, 761.556 µs] 360.337 µs (95.0%)
iast_GLOBAL 564.7 µs [542.213 µs, 587.186 µs] 185.46 µs (48.9%)
iast_HARDCODED_SECRET_DISABLED 511.613 µs [489.765 µs, 533.462 µs] 132.373 µs (34.9%)
iast_INACTIVE 462.364 µs [439.818 µs, 484.909 µs] 83.124 µs (21.9%)
iast_TELEMETRY_OFF 495.046 µs [472.023 µs, 518.068 µs] 115.806 µs (30.5%)
tracing 451.987 µs [430.971 µs, 473.004 µs] 72.747 µs (19.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master lineProbeBudgets
git_commit_date 1739907811 1739911928
git_commit_sha 5b7780b 098b029
release_version 1.47.0-SNAPSHOT~5b7780bc05 1.47.0-SNAPSHOT~098b0298d6
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1739913813 1739913813
ci_job_id 812930662 812930662
ci_pipeline_id 56214974 56214974
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kpnzgrdt-project-304-concurrent-2-6k1gw4wc 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kpnzgrdt-project-304-concurrent-2-6k1gw4wc 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.489 s) : 15489000, 15489000
.   : milestone, 15489000,
appsec (15.006 s) : 15006000, 15006000
.   : milestone, 15006000,
iast (18.386 s) : 18386000, 18386000
.   : milestone, 18386000,
iast_GLOBAL (17.286 s) : 17286000, 17286000
.   : milestone, 17286000,
profiling (15.163 s) : 15163000, 15163000
.   : milestone, 15163000,
tracing (15.236 s) : 15236000, 15236000
.   : milestone, 15236000,
section candidate
no_agent (15.328 s) : 15328000, 15328000
.   : milestone, 15328000,
appsec (15.312 s) : 15312000, 15312000
.   : milestone, 15312000,
iast (18.797 s) : 18797000, 18797000
.   : milestone, 18797000,
iast_GLOBAL (18.033 s) : 18033000, 18033000
.   : milestone, 18033000,
profiling (15.686 s) : 15686000, 15686000
.   : milestone, 15686000,
tracing (14.854 s) : 14854000, 14854000
.   : milestone, 14854000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.489 s [15.489 s, 15.489 s] -
appsec 15.006 s [15.006 s, 15.006 s] -483.0 ms (-3.1%)
iast 18.386 s [18.386 s, 18.386 s] 2.897 s (18.7%)
iast_GLOBAL 17.286 s [17.286 s, 17.286 s] 1.797 s (11.6%)
profiling 15.163 s [15.163 s, 15.163 s] -326.0 ms (-2.1%)
tracing 15.236 s [15.236 s, 15.236 s] -253.0 ms (-1.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.328 s [15.328 s, 15.328 s] -
appsec 15.312 s [15.312 s, 15.312 s] -16.0 ms (-0.1%)
iast 18.797 s [18.797 s, 18.797 s] 3.469 s (22.6%)
iast_GLOBAL 18.033 s [18.033 s, 18.033 s] 2.705 s (17.6%)
profiling 15.686 s [15.686 s, 15.686 s] 358.0 ms (2.3%)
tracing 14.854 s [14.854 s, 14.854 s] -474.0 ms (-3.1%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~098b0298d6, baseline=1.47.0-SNAPSHOT~5b7780bc05
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.465 ms) : 1453, 1476
.   : milestone, 1465,
appsec (2.358 ms) : 2315, 2402
.   : milestone, 2358,
iast (2.106 ms) : 2051, 2161
.   : milestone, 2106,
iast_GLOBAL (2.159 ms) : 2103, 2215
.   : milestone, 2159,
profiling (1.963 ms) : 1920, 2007
.   : milestone, 1963,
tracing (1.947 ms) : 1905, 1989
.   : milestone, 1947,
section candidate
no_agent (1.467 ms) : 1456, 1479
.   : milestone, 1467,
appsec (2.345 ms) : 2302, 2389
.   : milestone, 2345,
iast (2.105 ms) : 2050, 2160
.   : milestone, 2105,
iast_GLOBAL (2.156 ms) : 2101, 2211
.   : milestone, 2156,
profiling (1.967 ms) : 1922, 2011
.   : milestone, 1967,
tracing (1.933 ms) : 1892, 1975
.   : milestone, 1933,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.453 ms, 1.476 ms] -
appsec 2.358 ms [2.315 ms, 2.402 ms] 893.935 µs (61.0%)
iast 2.106 ms [2.051 ms, 2.161 ms] 641.53 µs (43.8%)
iast_GLOBAL 2.159 ms [2.103 ms, 2.215 ms] 694.305 µs (47.4%)
profiling 1.963 ms [1.92 ms, 2.007 ms] 498.815 µs (34.1%)
tracing 1.947 ms [1.905 ms, 1.989 ms] 482.451 µs (32.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.456 ms, 1.479 ms] -
appsec 2.345 ms [2.302 ms, 2.389 ms] 878.122 µs (59.8%)
iast 2.105 ms [2.05 ms, 2.16 ms] 637.653 µs (43.5%)
iast_GLOBAL 2.156 ms [2.101 ms, 2.211 ms] 688.851 µs (46.9%)
profiling 1.967 ms [1.922 ms, 2.011 ms] 499.319 µs (34.0%)
tracing 1.933 ms [1.892 ms, 1.975 ms] 466.239 µs (31.8%)

@evanchooly evanchooly added comp: debugger Dynamic Instrumentation inst: java Core Java language instrumentation tag: no release notes Changes to exclude from release notes type: task and removed inst: java Core Java language instrumentation labels Feb 18, 2025
@evanchooly evanchooly merged commit 17f4653 into master Feb 18, 2025
@evanchooly evanchooly deleted the lineProbeBudgets branch February 18, 2025 22:09
@github-actions github-actions Bot added this to the 1.47.0 milestone Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jpbempel jpbempel jpbempel approved these changes

@shatzi shatzi Awaiting requested review from shatzi

Assignees

No one assigned

Labels

comp: debugger Dynamic Instrumentation inst: java Core Java language instrumentation tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.47.0

Development

Successfully merging this pull request may close these issues.

2 participants

@evanchooly @jpbempel