Skip to content

Update system-tests#8777

Merged
zarirhamza merged 6 commits intomasterfrom
zarir/update-system-tests
May 7, 2025
Merged

Update system-tests#8777
zarirhamza merged 6 commits intomasterfrom
zarir/update-system-tests

Conversation

@zarirhamza
Copy link
Copy Markdown
Contributor

@zarirhamza zarirhamza commented May 7, 2025

What Does This Do

Update system-tests to b0b2e1f212f8c483b52aa3adc6ffd4132b1ba9b8

Motivation

Latest system tests fix current failing tests

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@zarirhamza zarirhamza requested a review from a team as a code owner May 7, 2025 06:23
@zarirhamza zarirhamza requested a review from amarziali May 7, 2025 06:23
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2025 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@PerfectSlayer PerfectSlayer added tag: no release notes Changes to exclude from release notes and removed type: refactoring labels May 7, 2025
@zarirhamza zarirhamza enabled auto-merge (squash) May 7, 2025 06:51
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 7, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master zarir/update-system-tests
git_commit_date 1746628596 1746628657
git_commit_sha 746a7fd 95fa4b9
release_version 1.50.0-SNAPSHOT~746a7fd953 1.50.0-SNAPSHOT~95fa4b9701
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1746631630 1746631630
ci_job_id 928115531 928115531
ci_pipeline_id 64409244 64409244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-gmcb38w3-project-304-concurrent-1-8o0wffjp 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-gmcb38w3-project-304-concurrent-1-8o0wffjp 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.019 s) : 0, 1019141
Total [baseline] (10.49 s) : 0, 10490008
Agent [candidate] (1.031 s) : 0, 1030739
Total [candidate] (10.473 s) : 0, 10473363
section appsec
Agent [baseline] (1.165 s) : 0, 1165393
Total [baseline] (10.668 s) : 0, 10667796
Agent [candidate] (1.166 s) : 0, 1166016
Total [candidate] (10.71 s) : 0, 10709718
section iast
Agent [baseline] (1.149 s) : 0, 1149121
Total [baseline] (10.932 s) : 0, 10932383
Agent [candidate] (1.152 s) : 0, 1151581
Total [candidate] (10.888 s) : 0, 10887865
section profiling
Agent [baseline] (1.275 s) : 0, 1275438
Total [baseline] (10.813 s) : 0, 10812991
Agent [candidate] (1.277 s) : 0, 1277453
Total [candidate] (10.958 s) : 0, 10958408
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.019 s -
Agent appsec 1.165 s 146.252 ms (14.4%)
Agent iast 1.149 s 129.98 ms (12.8%)
Agent profiling 1.275 s 256.297 ms (25.1%)
Total tracing 10.49 s -
Total appsec 10.668 s 177.788 ms (1.7%)
Total iast 10.932 s 442.375 ms (4.2%)
Total profiling 10.813 s 322.983 ms (3.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.031 s -
Agent appsec 1.166 s 135.277 ms (13.1%)
Agent iast 1.152 s 120.842 ms (11.7%)
Agent profiling 1.277 s 246.714 ms (23.9%)
Total tracing 10.473 s -
Total appsec 10.71 s 236.355 ms (2.3%)
Total iast 10.888 s 414.502 ms (4.0%)
Total profiling 10.958 s 485.046 ms (4.6%) 
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.765 ms) : 0, 682765
BytebuddyAgent [candidate] (691.874 ms) : 0, 691874
GlobalTracer [baseline] (239.305 ms) : 0, 239305
GlobalTracer [candidate] (240.916 ms) : 0, 240916
AppSec [baseline] (55.778 ms) : 0, 55778
AppSec [candidate] (54.672 ms) : 0, 54672
Debugger [baseline] (8.921 ms) : 0, 8921
Debugger [candidate] (9.148 ms) : 0, 9148
Remote Config [baseline] (676.49 µs) : 0, 676
Remote Config [candidate] (693.98 µs) : 0, 694
Telemetry [baseline] (8.18 ms) : 0, 8180
Telemetry [candidate] (9.686 ms) : 0, 9686
section appsec
BytebuddyAgent [baseline] (702.976 ms) : 0, 702976
BytebuddyAgent [candidate] (703.342 ms) : 0, 703342
GlobalTracer [baseline] (237.38 ms) : 0, 237380
GlobalTracer [candidate] (237.962 ms) : 0, 237962
IAST [baseline] (21.795 ms) : 0, 21795
IAST [candidate] (21.733 ms) : 0, 21733
AppSec [baseline] (176.298 ms) : 0, 176298
AppSec [candidate] (176.414 ms) : 0, 176414
Debugger [baseline] (6.305 ms) : 0, 6305
Debugger [candidate] (5.931 ms) : 0, 5931
Remote Config [baseline] (626.88 µs) : 0, 627
Remote Config [candidate] (624.866 µs) : 0, 625
Telemetry [baseline] (7.384 ms) : 0, 7384
Telemetry [candidate] (7.42 ms) : 0, 7420
section iast
BytebuddyAgent [baseline] (801.789 ms) : 0, 801789
BytebuddyAgent [candidate] (803.988 ms) : 0, 803988
GlobalTracer [baseline] (230.906 ms) : 0, 230906
GlobalTracer [candidate] (231.263 ms) : 0, 231263
IAST [baseline] (25.834 ms) : 0, 25834
IAST [candidate] (25.867 ms) : 0, 25867
AppSec [baseline] (52.802 ms) : 0, 52802
AppSec [candidate] (51.831 ms) : 0, 51831
Debugger [baseline] (5.894 ms) : 0, 5894
Debugger [candidate] (5.941 ms) : 0, 5941
Remote Config [baseline] (583.207 µs) : 0, 583
Remote Config [candidate] (589.905 µs) : 0, 590
Telemetry [baseline] (7.846 ms) : 0, 7846
Telemetry [candidate] (7.94 ms) : 0, 7940
section profiling
BytebuddyAgent [baseline] (674.37 ms) : 0, 674370
BytebuddyAgent [candidate] (674.491 ms) : 0, 674491
GlobalTracer [baseline] (377.837 ms) : 0, 377837
GlobalTracer [candidate] (379.324 ms) : 0, 379324
AppSec [baseline] (54.732 ms) : 0, 54732
AppSec [candidate] (53.862 ms) : 0, 53862
Debugger [baseline] (6.139 ms) : 0, 6139
Debugger [candidate] (6.104 ms) : 0, 6104
Remote Config [baseline] (673.004 µs) : 0, 673
Remote Config [candidate] (652.764 µs) : 0, 653
Telemetry [baseline] (8.111 ms) : 0, 8111
Telemetry [candidate] (8.105 ms) : 0, 8105
ProfilingAgent [baseline] (102.992 ms) : 0, 102992
ProfilingAgent [candidate] (104.329 ms) : 0, 104329
Profiling [baseline] (103.017 ms) : 0, 103017
Profiling [candidate] (104.354 ms) : 0, 104354
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.023 s) : 0, 1023281
Total [baseline] (8.646 s) : 0, 8646103
Agent [candidate] (1.023 s) : 0, 1023078
Total [candidate] (8.66 s) : 0, 8660390
section iast
Agent [baseline] (1.157 s) : 0, 1157005
Total [baseline] (9.272 s) : 0, 9271527
Agent [candidate] (1.166 s) : 0, 1166152
Total [candidate] (9.282 s) : 0, 9281979
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.148 s) : 0, 1147889
Total [baseline] (9.216 s) : 0, 9215789
Agent [candidate] (1.16 s) : 0, 1160246
Total [candidate] (9.202 s) : 0, 9201500
section iast_TELEMETRY_OFF
Agent [baseline] (1.153 s) : 0, 1152954
Total [baseline] (9.284 s) : 0, 9283683
Agent [candidate] (1.145 s) : 0, 1144735
Total [candidate] (9.215 s) : 0, 9214964
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.023 s -
Agent iast 1.157 s 133.724 ms (13.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.148 s 124.608 ms (12.2%)
Agent iast_TELEMETRY_OFF 1.153 s 129.672 ms (12.7%)
Total tracing 8.646 s -
Total iast 9.272 s 625.424 ms (7.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.216 s 569.686 ms (6.6%)
Total iast_TELEMETRY_OFF 9.284 s 637.581 ms (7.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.023 s -
Agent iast 1.166 s 143.074 ms (14.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.16 s 137.168 ms (13.4%)
Agent iast_TELEMETRY_OFF 1.145 s 121.657 ms (11.9%)
Total tracing 8.66 s -
Total iast 9.282 s 621.588 ms (7.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.202 s 541.11 ms (6.2%)
Total iast_TELEMETRY_OFF 9.215 s 554.573 ms (6.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (683.489 ms) : 0, 683489
BytebuddyAgent [candidate] (683.456 ms) : 0, 683456
GlobalTracer [baseline] (240.892 ms) : 0, 240892
GlobalTracer [candidate] (241.094 ms) : 0, 241094
AppSec [baseline] (55.186 ms) : 0, 55186
AppSec [candidate] (54.473 ms) : 0, 54473
Debugger [baseline] (10.521 ms) : 0, 10521
Debugger [candidate] (9.152 ms) : 0, 9152
Remote Config [baseline] (676.628 µs) : 0, 677
Remote Config [candidate] (695.123 µs) : 0, 695
Telemetry [baseline] (9.0 ms) : 0, 9000
Telemetry [candidate] (10.732 ms) : 0, 10732
section iast
BytebuddyAgent [baseline] (807.357 ms) : 0, 807357
BytebuddyAgent [candidate] (814.79 ms) : 0, 814790
GlobalTracer [baseline] (232.131 ms) : 0, 232131
GlobalTracer [candidate] (233.561 ms) : 0, 233561
IAST [baseline] (30.173 ms) : 0, 30173
IAST [candidate] (29.006 ms) : 0, 29006
AppSec [baseline] (49.13 ms) : 0, 49130
AppSec [candidate] (49.746 ms) : 0, 49746
Debugger [baseline] (5.97 ms) : 0, 5970
Debugger [candidate] (5.969 ms) : 0, 5969
Remote Config [baseline] (604.522 µs) : 0, 605
Remote Config [candidate] (609.285 µs) : 0, 609
Telemetry [baseline] (8.045 ms) : 0, 8045
Telemetry [candidate] (7.97 ms) : 0, 7970
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.638 ms) : 0, 800638
BytebuddyAgent [candidate] (810.134 ms) : 0, 810134
GlobalTracer [baseline] (230.573 ms) : 0, 230573
GlobalTracer [candidate] (232.896 ms) : 0, 232896
IAST [baseline] (24.263 ms) : 0, 24263
IAST [candidate] (25.575 ms) : 0, 25575
AppSec [baseline] (51.494 ms) : 0, 51494
AppSec [candidate] (49.502 ms) : 0, 49502
Debugger [baseline] (5.918 ms) : 0, 5918
Debugger [candidate] (5.987 ms) : 0, 5987
Remote Config [baseline] (583.264 µs) : 0, 583
Remote Config [candidate] (580.893 µs) : 0, 581
Telemetry [baseline] (7.917 ms) : 0, 7917
Telemetry [candidate] (7.905 ms) : 0, 7905
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (803.991 ms) : 0, 803991
BytebuddyAgent [candidate] (798.443 ms) : 0, 798443
GlobalTracer [baseline] (232.293 ms) : 0, 232293
GlobalTracer [candidate] (230.77 ms) : 0, 230770
IAST [baseline] (24.96 ms) : 0, 24960
IAST [candidate] (22.279 ms) : 0, 22279
AppSec [baseline] (53.685 ms) : 0, 53685
AppSec [candidate] (55.522 ms) : 0, 55522
Debugger [baseline] (6.068 ms) : 0, 6068
Debugger [candidate] (5.927 ms) : 0, 5927
Remote Config [baseline] (618.153 µs) : 0, 618
Remote Config [candidate] (610.437 µs) : 0, 610
Telemetry [baseline] (7.863 ms) : 0, 7863
Telemetry [candidate] (7.805 ms) : 0, 7805
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-07T14:57:31 2025-05-07T15:05:14
git_branch master zarir/update-system-tests
git_commit_date 1746628596 1746628657
git_commit_sha 746a7fd 95fa4b9
release_version 1.50.0-SNAPSHOT~746a7fd953 1.50.0-SNAPSHOT~95fa4b9701
start_time 2025-05-07T14:57:17 2025-05-07T15:05:00
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1746630710 1746630710
ci_job_id 928115532 928115532
ci_pipeline_id 64409244 64409244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-brbdd7pm-project-304-concurrent-0-f2xbd08f 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-brbdd7pm-project-304-concurrent-0-f2xbd08f 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 18 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-96.222µs; -45.195µs] or [-6.162%; -2.894%]		 unstable
[-478.896op/s; +706.816op/s] or [-16.163%; +23.855%]		 1.491ms 3076.923op/s 1.562ms 2962.963op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953
    dateFormat X
    axisFormat %s
section baseline
no_agent (374.094 µs) : 354, 394
.   : milestone, 374,
iast (516.797 µs) : 494, 539
.   : milestone, 517,
iast_FULL (729.859 µs) : 708, 752
.   : milestone, 730,
iast_GLOBAL (555.602 µs) : 534, 577
.   : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (508.133 µs) : 486, 530
.   : milestone, 508,
iast_INACTIVE (460.068 µs) : 438, 482
.   : milestone, 460,
iast_TELEMETRY_OFF (504.056 µs) : 481, 527
.   : milestone, 504,
tracing (452.283 µs) : 430, 474
.   : milestone, 452,
section candidate
no_agent (379.845 µs) : 360, 399
.   : milestone, 380,
iast (512.4 µs) : 491, 534
.   : milestone, 512,
iast_FULL (731.97 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (568.007 µs) : 546, 590
.   : milestone, 568,
iast_HARDCODED_SECRET_DISABLED (507.399 µs) : 484, 531
.   : milestone, 507,
iast_INACTIVE (464.457 µs) : 442, 487
.   : milestone, 464,
iast_TELEMETRY_OFF (507.42 µs) : 484, 531
.   : milestone, 507,
tracing (458.814 µs) : 437, 481
.   : milestone, 459,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.094 µs [353.758 µs, 394.43 µs] -
iast 516.797 µs [494.184 µs, 539.41 µs] 142.703 µs (38.1%)
iast_FULL 729.859 µs [707.981 µs, 751.738 µs] 355.765 µs (95.1%)
iast_GLOBAL 555.602 µs [534.009 µs, 577.194 µs] 181.507 µs (48.5%)
iast_HARDCODED_SECRET_DISABLED 508.133 µs [486.253 µs, 530.014 µs] 134.039 µs (35.8%)
iast_INACTIVE 460.068 µs [437.81 µs, 482.327 µs] 85.974 µs (23.0%)
iast_TELEMETRY_OFF 504.056 µs [480.718 µs, 527.394 µs] 129.962 µs (34.7%)
tracing 452.283 µs [430.451 µs, 474.115 µs] 78.189 µs (20.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.845 µs [360.342 µs, 399.348 µs] -
iast 512.4 µs [490.546 µs, 534.254 µs] 132.555 µs (34.9%)
iast_FULL 731.97 µs [710.184 µs, 753.757 µs] 352.125 µs (92.7%)
iast_GLOBAL 568.007 µs [546.02 µs, 589.994 µs] 188.162 µs (49.5%)
iast_HARDCODED_SECRET_DISABLED 507.399 µs [484.249 µs, 530.55 µs] 127.554 µs (33.6%)
iast_INACTIVE 464.457 µs [442.231 µs, 486.683 µs] 84.612 µs (22.3%)
iast_TELEMETRY_OFF 507.42 µs [484.218 µs, 530.623 µs] 127.576 µs (33.6%)
tracing 458.814 µs [436.774 µs, 480.854 µs] 78.97 µs (20.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1334, 1372
.   : milestone, 1353,
appsec (1.715 ms) : 1692, 1738
.   : milestone, 1715,
appsec_no_iast (1.747 ms) : 1724, 1770
.   : milestone, 1747,
code_origins (1.663 ms) : 1634, 1693
.   : milestone, 1663,
iast (1.517 ms) : 1494, 1541
.   : milestone, 1517,
profiling (1.562 ms) : 1537, 1586
.   : milestone, 1562,
tracing (1.499 ms) : 1474, 1524
.   : milestone, 1499,
section candidate
no_agent (1.369 ms) : 1350, 1389
.   : milestone, 1369,
appsec (1.721 ms) : 1698, 1744
.   : milestone, 1721,
appsec_no_iast (1.72 ms) : 1696, 1743
.   : milestone, 1720,
code_origins (1.661 ms) : 1634, 1688
.   : milestone, 1661,
iast (1.507 ms) : 1482, 1531
.   : milestone, 1507,
profiling (1.491 ms) : 1468, 1514
.   : milestone, 1491,
tracing (1.482 ms) : 1458, 1506
.   : milestone, 1482,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.334 ms, 1.372 ms] -
appsec 1.715 ms [1.692 ms, 1.738 ms] 361.568 µs (26.7%)
appsec_no_iast 1.747 ms [1.724 ms, 1.77 ms] 394.17 µs (29.1%)
code_origins 1.663 ms [1.634 ms, 1.693 ms] 310.268 µs (22.9%)
iast 1.517 ms [1.494 ms, 1.541 ms] 164.259 µs (12.1%)
profiling 1.562 ms [1.537 ms, 1.586 ms] 208.434 µs (15.4%)
tracing 1.499 ms [1.474 ms, 1.524 ms] 146.087 µs (10.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.369 ms [1.35 ms, 1.389 ms] -
appsec 1.721 ms [1.698 ms, 1.744 ms] 351.471 µs (25.7%)
appsec_no_iast 1.72 ms [1.696 ms, 1.743 ms] 350.152 µs (25.6%)
code_origins 1.661 ms [1.634 ms, 1.688 ms] 291.624 µs (21.3%)
iast 1.507 ms [1.482 ms, 1.531 ms] 137.233 µs (10.0%)
profiling 1.491 ms [1.468 ms, 1.514 ms] 121.426 µs (8.9%)
tracing 1.482 ms [1.458 ms, 1.506 ms] 112.485 µs (8.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master zarir/update-system-tests
git_commit_date 1746628596 1746628657
git_commit_sha 746a7fd 95fa4b9
release_version 1.50.0-SNAPSHOT~746a7fd953 1.50.0-SNAPSHOT~95fa4b9701
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1746631206 1746631206
ci_job_id 928115533 928115533
ci_pipeline_id 64409244 64409244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-gmcb38w3-project-304-concurrent-2-kt776e2f 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-gmcb38w3-project-304-concurrent-2-kt776e2f 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.484 s) : 15484000, 15484000
.   : milestone, 15484000,
appsec (15.064 s) : 15064000, 15064000
.   : milestone, 15064000,
iast (18.672 s) : 18672000, 18672000
.   : milestone, 18672000,
iast_GLOBAL (18.002 s) : 18002000, 18002000
.   : milestone, 18002000,
profiling (14.915 s) : 14915000, 14915000
.   : milestone, 14915000,
tracing (15.06 s) : 15060000, 15060000
.   : milestone, 15060000,
section candidate
no_agent (15.655 s) : 15655000, 15655000
.   : milestone, 15655000,
appsec (14.97 s) : 14970000, 14970000
.   : milestone, 14970000,
iast (18.724 s) : 18724000, 18724000
.   : milestone, 18724000,
iast_GLOBAL (18.2 s) : 18200000, 18200000
.   : milestone, 18200000,
profiling (15.088 s) : 15088000, 15088000
.   : milestone, 15088000,
tracing (14.912 s) : 14912000, 14912000
.   : milestone, 14912000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.484 s [15.484 s, 15.484 s] -
appsec 15.064 s [15.064 s, 15.064 s] -420.0 ms (-2.7%)
iast 18.672 s [18.672 s, 18.672 s] 3.188 s (20.6%)
iast_GLOBAL 18.002 s [18.002 s, 18.002 s] 2.518 s (16.3%)
profiling 14.915 s [14.915 s, 14.915 s] -569.0 ms (-3.7%)
tracing 15.06 s [15.06 s, 15.06 s] -424.0 ms (-2.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.655 s [15.655 s, 15.655 s] -
appsec 14.97 s [14.97 s, 14.97 s] -685.0 ms (-4.4%)
iast 18.724 s [18.724 s, 18.724 s] 3.069 s (19.6%)
iast_GLOBAL 18.2 s [18.2 s, 18.2 s] 2.545 s (16.3%)
profiling 15.088 s [15.088 s, 15.088 s] -567.0 ms (-3.6%)
tracing 14.912 s [14.912 s, 14.912 s] -743.0 ms (-4.7%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~95fa4b9701, baseline=1.50.0-SNAPSHOT~746a7fd953
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (2.385 ms) : 2338, 2432
.   : milestone, 2385,
iast (2.169 ms) : 2110, 2228
.   : milestone, 2169,
iast_GLOBAL (2.208 ms) : 2148, 2268
.   : milestone, 2208,
profiling (2.034 ms) : 1985, 2083
.   : milestone, 2034,
tracing (1.99 ms) : 1944, 2036
.   : milestone, 1990,
section candidate
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.383 ms) : 2336, 2430
.   : milestone, 2383,
iast (2.151 ms) : 2091, 2210
.   : milestone, 2151,
iast_GLOBAL (2.205 ms) : 2145, 2265
.   : milestone, 2205,
profiling (2.464 ms) : 2287, 2642
.   : milestone, 2464,
tracing (1.995 ms) : 1949, 2041
.   : milestone, 1995,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.46 ms, 1.483 ms] -
appsec 2.385 ms [2.338 ms, 2.432 ms] 913.442 µs (62.1%)
iast 2.169 ms [2.11 ms, 2.228 ms] 697.757 µs (47.4%)
iast_GLOBAL 2.208 ms [2.148 ms, 2.268 ms] 736.85 µs (50.1%)
profiling 2.034 ms [1.985 ms, 2.083 ms] 563.059 µs (38.3%)
tracing 1.99 ms [1.944 ms, 2.036 ms] 518.293 µs (35.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 2.383 ms [2.336 ms, 2.43 ms] 907.734 µs (61.5%)
iast 2.151 ms [2.091 ms, 2.21 ms] 675.078 µs (45.7%)
iast_GLOBAL 2.205 ms [2.145 ms, 2.265 ms] 729.687 µs (49.5%)
profiling 2.464 ms [2.287 ms, 2.642 ms] 988.727 µs (67.0%)
tracing 1.995 ms [1.949 ms, 2.041 ms] 519.54 µs (35.2%)

@zarirhamza zarirhamza merged commit 5a5e4a5 into master May 7, 2025
454 of 455 checks passed
@zarirhamza zarirhamza deleted the zarir/update-system-tests branch May 7, 2025 16:18
@github-actions github-actions Bot added this to the 1.50.0 milestone May 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls approved these changes

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@amarziali amarziali Awaiting requested review from amarziali amarziali is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.50.0

Development

Successfully merging this pull request may close these issues.

3 participants

@zarirhamza @mcculls @PerfectSlayer