Skip to content

Fix debugger test for stable JVM#9233

Merged
jpbempel merged 1 commit intomasterfrom
jpbempel/fix-debugger-tests
Jul 25, 2025
Merged

Fix debugger test for stable JVM#9233
jpbempel merged 1 commit intomasterfrom
jpbempel/fix-debugger-tests

Conversation

@jpbempel
Copy link
Copy Markdown
Member

@jpbempel jpbempel commented Jul 24, 2025 •
edited
Loading

What Does This Do

Since JDK 24, lambdas are renumbered from the local scope and not
from the whole classfile

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Jul 24, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-debugger-tests
git_commit_date 1753362658 1753370306
git_commit_sha 4fde84f 9a86668
release_version 1.51.1-SNAPSHOT~4fde84fc67 1.51.1-SNAPSHOT~9a866682f0
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753373145 1753373145
ci_job_id 1046074581 1046074581
ci_pipeline_id 71603455 71603455
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-h0bw20z5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-h0bw20z5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042517
Total [baseline] (10.717 s) : 0, 10716983
Agent [candidate] (1.044 s) : 0, 1044493
Total [candidate] (10.728 s) : 0, 10727729
section appsec
Agent [baseline] (1.229 s) : 0, 1229331
Total [baseline] (10.858 s) : 0, 10858054
Agent [candidate] (1.219 s) : 0, 1219226
Total [candidate] (10.839 s) : 0, 10838902
section iast
Agent [baseline] (1.174 s) : 0, 1174068
Total [baseline] (10.881 s) : 0, 10881254
Agent [candidate] (1.175 s) : 0, 1174738
Total [candidate] (10.887 s) : 0, 10886853
section profiling
Agent [baseline] (1.189 s) : 0, 1189417
Total [baseline] (10.833 s) : 0, 10832837
Agent [candidate] (1.188 s) : 0, 1188489
Total [candidate] (10.772 s) : 0, 10771843
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent appsec 1.229 s 186.815 ms (17.9%)
Agent iast 1.174 s 131.551 ms (12.6%)
Agent profiling 1.189 s 146.9 ms (14.1%)
Total tracing 10.717 s -
Total appsec 10.858 s 141.071 ms (1.3%)
Total iast 10.881 s 164.271 ms (1.5%)
Total profiling 10.833 s 115.854 ms (1.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.044 s -
Agent appsec 1.219 s 174.732 ms (16.7%)
Agent iast 1.175 s 130.244 ms (12.5%)
Agent profiling 1.188 s 143.996 ms (13.8%)
Total tracing 10.728 s -
Total appsec 10.839 s 111.173 ms (1.0%)
Total iast 10.887 s 159.125 ms (1.5%)
Total profiling 10.772 s 44.114 ms (0.4%) 
gantt
    title petclinic - break down per module: candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.42 ms) : 0, 1420
crashtracking [candidate] (1.422 ms) : 0, 1422
BytebuddyAgent [baseline] (729.745 ms) : 0, 729745
BytebuddyAgent [candidate] (730.079 ms) : 0, 730079
GlobalTracer [baseline] (241.615 ms) : 0, 241615
GlobalTracer [candidate] (241.175 ms) : 0, 241175
AppSec [baseline] (30.321 ms) : 0, 30321
AppSec [candidate] (30.339 ms) : 0, 30339
Debugger [baseline] (5.955 ms) : 0, 5955
Debugger [candidate] (5.96 ms) : 0, 5960
Remote Config [baseline] (643.163 µs) : 0, 643
Remote Config [candidate] (654.613 µs) : 0, 655
Telemetry [baseline] (11.884 ms) : 0, 11884
Telemetry [candidate] (13.865 ms) : 0, 13865
section appsec
crashtracking [baseline] (1.448 ms) : 0, 1448
crashtracking [candidate] (1.427 ms) : 0, 1427
BytebuddyAgent [baseline] (758.383 ms) : 0, 758383
BytebuddyAgent [candidate] (752.021 ms) : 0, 752021
GlobalTracer [baseline] (237.63 ms) : 0, 237630
GlobalTracer [candidate] (234.806 ms) : 0, 234806
IAST [baseline] (24.051 ms) : 0, 24051
IAST [candidate] (23.571 ms) : 0, 23571
AppSec [baseline] (169.425 ms) : 0, 169425
AppSec [candidate] (171.392 ms) : 0, 171392
Debugger [baseline] (6.53 ms) : 0, 6530
Debugger [candidate] (5.712 ms) : 0, 5712
Remote Config [baseline] (630.684 µs) : 0, 631
Remote Config [candidate] (648.812 µs) : 0, 649
Telemetry [baseline] (9.837 ms) : 0, 9837
Telemetry [candidate] (8.321 ms) : 0, 8321
section iast
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (847.81 ms) : 0, 847810
BytebuddyAgent [candidate] (847.938 ms) : 0, 847938
GlobalTracer [baseline] (231.412 ms) : 0, 231412
GlobalTracer [candidate] (231.787 ms) : 0, 231787
IAST [baseline] (30.665 ms) : 0, 30665
IAST [candidate] (29.088 ms) : 0, 29088
AppSec [baseline] (24.752 ms) : 0, 24752
AppSec [candidate] (27.258 ms) : 0, 27258
Debugger [baseline] (8.456 ms) : 0, 8456
Debugger [candidate] (7.472 ms) : 0, 7472
Remote Config [baseline] (589.515 µs) : 0, 590
Remote Config [candidate] (595.726 µs) : 0, 596
Telemetry [baseline] (8.12 ms) : 0, 8120
Telemetry [candidate] (8.093 ms) : 0, 8093
section profiling
crashtracking [baseline] (1.39 ms) : 0, 1390
crashtracking [candidate] (1.393 ms) : 0, 1393
BytebuddyAgent [baseline] (758.853 ms) : 0, 758853
BytebuddyAgent [candidate] (758.228 ms) : 0, 758228
GlobalTracer [baseline] (220.124 ms) : 0, 220124
GlobalTracer [candidate] (220.326 ms) : 0, 220326
AppSec [baseline] (30.267 ms) : 0, 30267
AppSec [candidate] (30.217 ms) : 0, 30217
Debugger [baseline] (6.233 ms) : 0, 6233
Debugger [candidate] (6.168 ms) : 0, 6168
Remote Config [baseline] (703.113 µs) : 0, 703
Remote Config [candidate] (675.754 µs) : 0, 676
Telemetry [baseline] (15.896 ms) : 0, 15896
Telemetry [candidate] (15.851 ms) : 0, 15851
ProfilingAgent [baseline] (107.497 ms) : 0, 107497
ProfilingAgent [candidate] (107.124 ms) : 0, 107124
Profiling [baseline] (107.519 ms) : 0, 107519
Profiling [candidate] (107.148 ms) : 0, 107148
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1044302
Total [baseline] (8.588 s) : 0, 8588381
Agent [candidate] (1.043 s) : 0, 1042620
Total [candidate] (8.614 s) : 0, 8613922
section iast
Agent [baseline] (1.17 s) : 0, 1169770
Total [baseline] (9.274 s) : 0, 9274437
Agent [candidate] (1.177 s) : 0, 1177171
Total [candidate] (9.271 s) : 0, 9270546
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.044 s -
Agent iast 1.17 s 125.468 ms (12.0%)
Total tracing 8.588 s -
Total iast 9.274 s 686.056 ms (8.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent iast 1.177 s 134.551 ms (12.9%)
Total tracing 8.614 s -
Total iast 9.271 s 656.624 ms (7.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.424 ms) : 0, 1424
crashtracking [candidate] (1.435 ms) : 0, 1435
BytebuddyAgent [baseline] (730.711 ms) : 0, 730711
BytebuddyAgent [candidate] (730.911 ms) : 0, 730911
GlobalTracer [baseline] (241.549 ms) : 0, 241549
GlobalTracer [candidate] (242.317 ms) : 0, 242317
AppSec [baseline] (30.485 ms) : 0, 30485
AppSec [candidate] (30.506 ms) : 0, 30506
Debugger [baseline] (5.988 ms) : 0, 5988
Debugger [candidate] (5.977 ms) : 0, 5977
Remote Config [baseline] (652.701 µs) : 0, 653
Remote Config [candidate] (648.25 µs) : 0, 648
Telemetry [baseline] (12.577 ms) : 0, 12577
Telemetry [candidate] (9.907 ms) : 0, 9907
section iast
crashtracking [baseline] (1.424 ms) : 0, 1424
crashtracking [candidate] (1.436 ms) : 0, 1436
BytebuddyAgent [baseline] (844.321 ms) : 0, 844321
BytebuddyAgent [candidate] (851.003 ms) : 0, 851003
GlobalTracer [baseline] (230.475 ms) : 0, 230475
GlobalTracer [candidate] (231.431 ms) : 0, 231431
IAST [baseline] (28.263 ms) : 0, 28263
IAST [candidate] (29.707 ms) : 0, 29707
AppSec [baseline] (27.306 ms) : 0, 27306
AppSec [candidate] (26.472 ms) : 0, 26472
Debugger [baseline] (8.462 ms) : 0, 8462
Debugger [candidate] (7.486 ms) : 0, 7486
Remote Config [baseline] (581.831 µs) : 0, 582
Remote Config [candidate] (582.721 µs) : 0, 583
Telemetry [baseline] (8.088 ms) : 0, 8088
Telemetry [candidate] (8.118 ms) : 0, 8118
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-debugger-tests
git_commit_date 1753362658 1753370306
git_commit_sha 4fde84f 9a86668
release_version 1.51.1-SNAPSHOT~4fde84fc67 1.51.1-SNAPSHOT~9a866682f0
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753372822 1753372822
ci_job_id 1046074584 1046074584
ci_pipeline_id 71603455 71603455
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-qltuny31 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-qltuny31 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 11 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:no_agent:high_load worse
[+1.222ms; +1.852ms] or [+3.397%; +5.152%]		 unstable
[-13.737op/s; +3.112op/s] or [-10.564%; +2.393%]		 37.493ms 124.725op/s 35.956ms 130.037op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.331 ms) : 4282, 4381
.   : milestone, 4331,
iast (9.191 ms) : 9043, 9340
.   : milestone, 9191,
iast_FULL (14.253 ms) : 13972, 14534
.   : milestone, 14253,
iast_GLOBAL (9.999 ms) : 9826, 10172
.   : milestone, 9999,
profiling (8.867 ms) : 8730, 9004
.   : milestone, 8867,
tracing (7.216 ms) : 7117, 7315
.   : milestone, 7216,
section candidate
no_agent (4.351 ms) : 4303, 4400
.   : milestone, 4351,
iast (9.433 ms) : 9279, 9586
.   : milestone, 9433,
iast_FULL (14.61 ms) : 14323, 14896
.   : milestone, 14610,
iast_GLOBAL (10.138 ms) : 9948, 10329
.   : milestone, 10138,
profiling (8.79 ms) : 8656, 8924
.   : milestone, 8790,
tracing (7.434 ms) : 7328, 7539
.   : milestone, 7434,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.331 ms [4.282 ms, 4.381 ms] -
iast 9.191 ms [9.043 ms, 9.34 ms] 4.86 ms (112.2%)
iast_FULL 14.253 ms [13.972 ms, 14.534 ms] 9.922 ms (229.1%)
iast_GLOBAL 9.999 ms [9.826 ms, 10.172 ms] 5.668 ms (130.9%)
profiling 8.867 ms [8.73 ms, 9.004 ms] 4.536 ms (104.7%)
tracing 7.216 ms [7.117 ms, 7.315 ms] 2.885 ms (66.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.351 ms [4.303 ms, 4.4 ms] -
iast 9.433 ms [9.279 ms, 9.586 ms] 5.081 ms (116.8%)
iast_FULL 14.61 ms [14.323 ms, 14.896 ms] 10.258 ms (235.7%)
iast_GLOBAL 10.138 ms [9.948 ms, 10.329 ms] 5.787 ms (133.0%)
profiling 8.79 ms [8.656 ms, 8.924 ms] 4.439 ms (102.0%)
tracing 7.434 ms [7.328 ms, 7.539 ms] 3.082 ms (70.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67
    dateFormat X
    axisFormat %s
section baseline
no_agent (35.956 ms) : 35671, 36241
.   : milestone, 35956,
appsec (46.996 ms) : 46577, 47416
.   : milestone, 46996,
code_origins (44.986 ms) : 44588, 45385
.   : milestone, 44986,
iast (44.973 ms) : 44586, 45360
.   : milestone, 44973,
profiling (49.41 ms) : 48982, 49838
.   : milestone, 49410,
tracing (42.409 ms) : 42060, 42758
.   : milestone, 42409,
section candidate
no_agent (37.493 ms) : 37192, 37794
.   : milestone, 37493,
appsec (47.424 ms) : 47006, 47842
.   : milestone, 47424,
code_origins (46.051 ms) : 45655, 46447
.   : milestone, 46051,
iast (43.729 ms) : 43365, 44094
.   : milestone, 43729,
profiling (49.537 ms) : 49091, 49982
.   : milestone, 49537,
tracing (43.093 ms) : 42737, 43448
.   : milestone, 43093,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 35.956 ms [35.671 ms, 36.241 ms] -
appsec 46.996 ms [46.577 ms, 47.416 ms] 11.04 ms (30.7%)
code_origins 44.986 ms [44.588 ms, 45.385 ms] 9.03 ms (25.1%)
iast 44.973 ms [44.586 ms, 45.36 ms] 9.017 ms (25.1%)
profiling 49.41 ms [48.982 ms, 49.838 ms] 13.454 ms (37.4%)
tracing 42.409 ms [42.06 ms, 42.758 ms] 6.453 ms (17.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.493 ms [37.192 ms, 37.794 ms] -
appsec 47.424 ms [47.006 ms, 47.842 ms] 9.931 ms (26.5%)
code_origins 46.051 ms [45.655 ms, 46.447 ms] 8.558 ms (22.8%)
iast 43.729 ms [43.365 ms, 44.094 ms] 6.236 ms (16.6%)
profiling 49.537 ms [49.091 ms, 49.982 ms] 12.044 ms (32.1%)
tracing 43.093 ms [42.737 ms, 43.448 ms] 5.6 ms (14.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-debugger-tests
git_commit_date 1753362658 1753370306
git_commit_sha 4fde84f 9a86668
release_version 1.51.1-SNAPSHOT~4fde84fc67 1.51.1-SNAPSHOT~9a866682f0
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1753373281 1753373281
ci_job_id 1046074587 1046074587
ci_pipeline_id 71603455 71603455
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-1kzgbzxq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-1kzgbzxq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.458 ms) : 2406, 2510
.   : milestone, 2458,
iast (2.203 ms) : 2141, 2265
.   : milestone, 2203,
iast_GLOBAL (2.232 ms) : 2169, 2294
.   : milestone, 2232,
profiling (2.037 ms) : 1986, 2087
.   : milestone, 2037,
tracing (2.015 ms) : 1966, 2063
.   : milestone, 2015,
section candidate
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (3.655 ms) : 3439, 3871
.   : milestone, 3655,
iast (2.191 ms) : 2129, 2254
.   : milestone, 2191,
iast_GLOBAL (2.229 ms) : 2166, 2291
.   : milestone, 2229,
profiling (2.032 ms) : 1983, 2082
.   : milestone, 2032,
tracing (2.016 ms) : 1968, 2065
.   : milestone, 2016,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.461 ms, 1.484 ms] -
appsec 2.458 ms [2.406 ms, 2.51 ms] 985.754 µs (67.0%)
iast 2.203 ms [2.141 ms, 2.265 ms] 730.553 µs (49.6%)
iast_GLOBAL 2.232 ms [2.169 ms, 2.294 ms] 759.502 µs (51.6%)
profiling 2.037 ms [1.986 ms, 2.087 ms] 564.241 µs (38.3%)
tracing 2.015 ms [1.966 ms, 2.063 ms] 542.272 µs (36.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.459 ms, 1.482 ms] -
appsec 3.655 ms [3.439 ms, 3.871 ms] 2.184 ms (148.5%)
iast 2.191 ms [2.129 ms, 2.254 ms] 720.509 µs (49.0%)
iast_GLOBAL 2.229 ms [2.166 ms, 2.291 ms] 757.877 µs (51.5%)
profiling 2.032 ms [1.983 ms, 2.082 ms] 561.416 µs (38.2%)
tracing 2.016 ms [1.968 ms, 2.065 ms] 545.393 µs (37.1%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~9a866682f0, baseline=1.51.1-SNAPSHOT~4fde84fc67
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.348 s) : 15348000, 15348000
.   : milestone, 15348000,
appsec (14.844 s) : 14844000, 14844000
.   : milestone, 14844000,
iast (18.705 s) : 18705000, 18705000
.   : milestone, 18705000,
iast_GLOBAL (17.952 s) : 17952000, 17952000
.   : milestone, 17952000,
profiling (15.335 s) : 15335000, 15335000
.   : milestone, 15335000,
tracing (14.982 s) : 14982000, 14982000
.   : milestone, 14982000,
section candidate
no_agent (15.107 s) : 15107000, 15107000
.   : milestone, 15107000,
appsec (14.618 s) : 14618000, 14618000
.   : milestone, 14618000,
iast (18.314 s) : 18314000, 18314000
.   : milestone, 18314000,
iast_GLOBAL (18.191 s) : 18191000, 18191000
.   : milestone, 18191000,
profiling (15.275 s) : 15275000, 15275000
.   : milestone, 15275000,
tracing (15.067 s) : 15067000, 15067000
.   : milestone, 15067000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.348 s [15.348 s, 15.348 s] -
appsec 14.844 s [14.844 s, 14.844 s] -504.0 ms (-3.3%)
iast 18.705 s [18.705 s, 18.705 s] 3.357 s (21.9%)
iast_GLOBAL 17.952 s [17.952 s, 17.952 s] 2.604 s (17.0%)
profiling 15.335 s [15.335 s, 15.335 s] -13.0 ms (-0.1%)
tracing 14.982 s [14.982 s, 14.982 s] -366.0 ms (-2.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.107 s [15.107 s, 15.107 s] -
appsec 14.618 s [14.618 s, 14.618 s] -489.0 ms (-3.2%)
iast 18.314 s [18.314 s, 18.314 s] 3.207 s (21.2%)
iast_GLOBAL 18.191 s [18.191 s, 18.191 s] 3.084 s (20.4%)
profiling 15.275 s [15.275 s, 15.275 s] 168.0 ms (1.1%)
tracing 15.067 s [15.067 s, 15.067 s] -40.0 ms (-0.3%)

@jpbempel jpbempel force-pushed the jpbempel/fix-debugger-tests branch from afe6077 to e44fd6f Compare July 24, 2025 10:26
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Jul 24, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1753371886 1753372231
end_time 2025-07-24T15:46:07 2025-07-24T15:51:52
git_branch master jpbempel/fix-debugger-tests
git_commit_sha 4fde84f 9a86668
start_time 2025-07-24T15:44:47 2025-07-24T15:50:32
See matching parameters
Baseline Candidate
ci_job_id 1046074592 1046074592
ci_pipeline_id 71603455 71603455
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1753370306 1753370306

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-20.774µs; +32.091µs] or [-7.459%; +11.523%]		 unstable
[-31.108µs; +43.925µs] or [-9.754%; +13.773%]		 unstable
[-42.159µs; +55.789µs] or [-12.640%; +16.727%]		 unstable
[-83.991µs; +136.975µs] or [-9.054%; +14.765%]		 same
scenario:basic same same same unstable
[-58.293µs; +100.706µs] or [-7.935%; +13.709%]		 unstable
[-247.034op/s; +104.786op/s] or [-9.140%; +3.877%]
scenario:loop unsure
[+6.635µs; +13.075µs] or [+0.075%; +0.148%]		 unsure
[+7.058µs; +16.171µs] or [+0.079%; +0.181%]		 unsure
[+6.209µs; +16.257µs] or [+0.069%; +0.181%]		 same same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (318.922 µs) : 287, 351
.   : milestone, 319,
basic (277.173 µs) : 270, 284
.   : milestone, 277,
loop (8.956 ms) : 8951, 8960
.   : milestone, 8956,
section candidate
noprobe (325.331 µs) : 286, 364
.   : milestone, 325,
basic (276.64 µs) : 271, 283
.   : milestone, 277,
loop (8.967 ms) : 8963, 8971
.   : milestone, 8967,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 318.922 µs [286.657 µs, 351.187 µs]
basic 277.173 µs [270.233 µs, 284.113 µs]
loop 8.956 ms [8.951 ms, 8.96 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 325.331 µs [286.471 µs, 364.191 µs]
basic 276.64 µs [270.663 µs, 282.617 µs]
loop 8.967 ms [8.963 ms, 8.971 ms]

@jpbempel
fix debugger test for stable JVM
9a86668 
Since JDK 24, lambdas are renumbered from the local scope and not
from the whole classfile
@jpbempel jpbempel force-pushed the jpbempel/fix-debugger-tests branch from e44fd6f to 9a86668 Compare July 24, 2025 15:19
@jpbempel jpbempel changed the title Fix CI_SPLIT for base case test matrix @jpbempel fix debugger test for stable JVM Jul 24, 2025
@jpbempel jpbempel added comp: debugger Dynamic Instrumentation type: refactoring tag: no release notes Changes to exclude from release notes comp: testing Testing labels Jul 24, 2025
@jpbempel jpbempel marked this pull request as ready for review July 24, 2025 15:21
@jpbempel jpbempel requested a review from a team as a code owner July 24, 2025 15:21
@jpbempel jpbempel requested review from shatzi and removed request for a team July 24, 2025 15:21
@jpbempel jpbempel changed the title @jpbempel fix debugger test for stable JVM Fix debugger test for stable JVM Jul 24, 2025
@jpbempel jpbempel merged commit 6ae1a5c into master Jul 25, 2025
684 of 687 checks passed
@jpbempel jpbempel deleted the jpbempel/fix-debugger-tests branch July 25, 2025 08:31
@github-actions github-actions Bot added this to the 1.52.0 milestone Jul 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@evanchooly evanchooly evanchooly approved these changes

@shatzi shatzi Awaiting requested review from shatzi shatzi is a code owner automatically assigned from DataDog/debugger-java

Assignees

No one assigned

Labels

comp: debugger Dynamic Instrumentation comp: testing Testing tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Milestone

1.52.0

Development

Successfully merging this pull request may close these issues.

2 participants

@jpbempel @evanchooly