Skip to content

Skip benchmarks on release#8363

Merged
randomanderson merged 1 commit intomasterfrom
landerson/no-benchmarks-on-release
Feb 11, 2025
Merged

Skip benchmarks on release#8363
randomanderson merged 1 commit intomasterfrom
landerson/no-benchmarks-on-release

Conversation

@randomanderson
Copy link
Copy Markdown
Contributor

@randomanderson randomanderson commented Feb 10, 2025

What Does This Do

Makes benchmarks optional and manual during releases.

Motivation

Most of the benchmarks make incorrect assumptions about artifacts that are not true for releases. This leads to the benchmark jobs failing on the release

@randomanderson randomanderson added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Feb 10, 2025
@randomanderson randomanderson requested a review from a team as a code owner February 10, 2025 23:06
@randomanderson randomanderson enabled auto-merge (squash) February 10, 2025 23:31
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 10, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/no-benchmarks-on-release
git_commit_date 1739207934 1739228661
git_commit_sha added74 fd57e94
release_version 1.47.0-SNAPSHOT~added74bb5 1.47.0-SNAPSHOT~fd57e948ff
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1739231174 1739231174
ci_job_id 802151571 802151571
ci_pipeline_id 55477580 55477580
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zp9z32c-project-304-concurrent-0-2ctpg5da 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zp9z32c-project-304-concurrent-0-2ctpg5da 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1044620
Total [baseline] (10.437 s) : 0, 10437442
Agent [candidate] (1.046 s) : 0, 1046352
Total [candidate] (10.456 s) : 0, 10455807
section appsec
Agent [baseline] (1.184 s) : 0, 1184405
Total [baseline] (10.737 s) : 0, 10737295
Agent [candidate] (1.194 s) : 0, 1194130
Total [candidate] (10.792 s) : 0, 10792337
section iast
Agent [baseline] (1.172 s) : 0, 1171792
Total [baseline] (11.009 s) : 0, 11008832
Agent [candidate] (1.178 s) : 0, 1178451
Total [candidate] (11.031 s) : 0, 11030854
section profiling
Agent [baseline] (1.273 s) : 0, 1272816
Total [baseline] (10.956 s) : 0, 10955697
Agent [candidate] (1.265 s) : 0, 1265381
Total [candidate] (10.922 s) : 0, 10921613
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.045 s -
Agent appsec 1.184 s 139.785 ms (13.4%)
Agent iast 1.172 s 127.172 ms (12.2%)
Agent profiling 1.273 s 228.197 ms (21.8%)
Total tracing 10.437 s -
Total appsec 10.737 s 299.853 ms (2.9%)
Total iast 11.009 s 571.39 ms (5.5%)
Total profiling 10.956 s 518.255 ms (5.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent appsec 1.194 s 147.778 ms (14.1%)
Agent iast 1.178 s 132.098 ms (12.6%)
Agent profiling 1.265 s 219.029 ms (20.9%)
Total tracing 10.456 s -
Total appsec 10.792 s 336.53 ms (3.2%)
Total iast 11.031 s 575.048 ms (5.5%)
Total profiling 10.922 s 465.807 ms (4.5%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.649 ms) : 0, 717649
BytebuddyAgent [candidate] (717.961 ms) : 0, 717961
GlobalTracer [baseline] (243.429 ms) : 0, 243429
GlobalTracer [candidate] (244.018 ms) : 0, 244018
AppSec [baseline] (55.204 ms) : 0, 55204
AppSec [candidate] (55.399 ms) : 0, 55399
Remote Config [baseline] (722.849 µs) : 0, 723
Remote Config [candidate] (721.482 µs) : 0, 721
Telemetry [baseline] (12.295 ms) : 0, 12295
Telemetry [candidate] (13.03 ms) : 0, 13030
section appsec
BytebuddyAgent [baseline] (732.775 ms) : 0, 732775
BytebuddyAgent [candidate] (739.754 ms) : 0, 739754
GlobalTracer [baseline] (240.024 ms) : 0, 240024
GlobalTracer [candidate] (242.067 ms) : 0, 242067
AppSec [baseline] (176.646 ms) : 0, 176646
AppSec [candidate] (176.988 ms) : 0, 176988
Remote Config [baseline] (652.425 µs) : 0, 652
Remote Config [candidate] (663.502 µs) : 0, 664
Telemetry [baseline] (8.268 ms) : 0, 8268
Telemetry [candidate] (8.314 ms) : 0, 8314
IAST [baseline] (21.674 ms) : 0, 21674
IAST [candidate] (21.938 ms) : 0, 21938
section iast
BytebuddyAgent [baseline] (834.166 ms) : 0, 834166
BytebuddyAgent [candidate] (839.915 ms) : 0, 839915
GlobalTracer [baseline] (233.432 ms) : 0, 233432
GlobalTracer [candidate] (233.305 ms) : 0, 233305
AppSec [baseline] (56.828 ms) : 0, 56828
AppSec [candidate] (57.509 ms) : 0, 57509
Remote Config [baseline] (617.43 µs) : 0, 617
Remote Config [candidate] (624.08 µs) : 0, 624
Telemetry [baseline] (8.608 ms) : 0, 8608
Telemetry [candidate] (8.829 ms) : 0, 8829
IAST [baseline] (22.852 ms) : 0, 22852
IAST [candidate] (22.905 ms) : 0, 22905
section profiling
ProfilingAgent [baseline] (97.074 ms) : 0, 97074
ProfilingAgent [candidate] (95.304 ms) : 0, 95304
BytebuddyAgent [baseline] (712.531 ms) : 0, 712531
BytebuddyAgent [candidate] (709.098 ms) : 0, 709098
GlobalTracer [baseline] (355.177 ms) : 0, 355177
GlobalTracer [candidate] (354.706 ms) : 0, 354706
AppSec [baseline] (55.827 ms) : 0, 55827
AppSec [candidate] (54.175 ms) : 0, 54175
Remote Config [baseline] (732.035 µs) : 0, 732
Remote Config [candidate] (713.932 µs) : 0, 714
Telemetry [baseline] (8.999 ms) : 0, 8999
Telemetry [candidate] (8.933 ms) : 0, 8933
Profiling [baseline] (97.099 ms) : 0, 97099
Profiling [candidate] (95.328 ms) : 0, 95328
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046906
Total [baseline] (8.664 s) : 0, 8663901
Agent [candidate] (1.05 s) : 0, 1049923
Total [candidate] (8.664 s) : 0, 8663552
section iast
Agent [baseline] (1.176 s) : 0, 1176415
Total [baseline] (9.262 s) : 0, 9262218
Agent [candidate] (1.171 s) : 0, 1171124
Total [candidate] (9.202 s) : 0, 9201733
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.175 s) : 0, 1175165
Total [baseline] (9.237 s) : 0, 9237129
Agent [candidate] (1.174 s) : 0, 1173747
Total [candidate] (9.231 s) : 0, 9230943
section iast_TELEMETRY_OFF
Agent [baseline] (1.185 s) : 0, 1184883
Total [baseline] (9.292 s) : 0, 9292035
Agent [candidate] (1.189 s) : 0, 1188705
Total [candidate] (9.316 s) : 0, 9315511
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent iast 1.176 s 129.508 ms (12.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.175 s 128.258 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.185 s 137.976 ms (13.2%)
Total tracing 8.664 s -
Total iast 9.262 s 598.317 ms (6.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.237 s 573.228 ms (6.6%)
Total iast_TELEMETRY_OFF 9.292 s 628.134 ms (7.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.05 s -
Agent iast 1.171 s 121.202 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.174 s 123.825 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.189 s 138.782 ms (13.2%)
Total tracing 8.664 s -
Total iast 9.202 s 538.181 ms (6.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.231 s 567.391 ms (6.5%)
Total iast_TELEMETRY_OFF 9.316 s 651.959 ms (7.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (719.112 ms) : 0, 719112
BytebuddyAgent [candidate] (721.477 ms) : 0, 721477
GlobalTracer [baseline] (244.499 ms) : 0, 244499
GlobalTracer [candidate] (244.906 ms) : 0, 244906
AppSec [baseline] (55.895 ms) : 0, 55895
AppSec [candidate] (55.379 ms) : 0, 55379
Remote Config [baseline] (716.446 µs) : 0, 716
Remote Config [candidate] (719.216 µs) : 0, 719
Telemetry [baseline] (11.472 ms) : 0, 11472
Telemetry [candidate] (12.105 ms) : 0, 12105
section iast
BytebuddyAgent [baseline] (836.857 ms) : 0, 836857
BytebuddyAgent [candidate] (833.448 ms) : 0, 833448
GlobalTracer [baseline] (234.017 ms) : 0, 234017
GlobalTracer [candidate] (233.375 ms) : 0, 233375
IAST [baseline] (23.047 ms) : 0, 23047
IAST [candidate] (22.689 ms) : 0, 22689
AppSec [baseline] (57.768 ms) : 0, 57768
AppSec [candidate] (57.043 ms) : 0, 57043
Remote Config [baseline] (626.683 µs) : 0, 627
Remote Config [candidate] (624.133 µs) : 0, 624
Telemetry [baseline] (8.838 ms) : 0, 8838
Telemetry [candidate] (8.69 ms) : 0, 8690
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (836.009 ms) : 0, 836009
BytebuddyAgent [candidate] (834.997 ms) : 0, 834997
GlobalTracer [baseline] (234.122 ms) : 0, 234122
GlobalTracer [candidate] (233.841 ms) : 0, 233841
IAST [baseline] (22.916 ms) : 0, 22916
IAST [candidate] (22.918 ms) : 0, 22918
AppSec [baseline] (57.491 ms) : 0, 57491
AppSec [candidate] (57.342 ms) : 0, 57342
Remote Config [baseline] (631.786 µs) : 0, 632
Remote Config [candidate] (619.538 µs) : 0, 620
Telemetry [baseline] (8.733 ms) : 0, 8733
Telemetry [candidate] (8.721 ms) : 0, 8721
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (843.654 ms) : 0, 843654
BytebuddyAgent [candidate] (845.78 ms) : 0, 845780
GlobalTracer [baseline] (236.032 ms) : 0, 236032
GlobalTracer [candidate] (237.174 ms) : 0, 237174
IAST [baseline] (24.335 ms) : 0, 24335
IAST [candidate] (26.226 ms) : 0, 26226
AppSec [baseline] (55.969 ms) : 0, 55969
AppSec [candidate] (54.67 ms) : 0, 54670
Remote Config [baseline] (623.728 µs) : 0, 624
Remote Config [candidate] (637.808 µs) : 0, 638
Telemetry [baseline] (8.716 ms) : 0, 8716
Telemetry [candidate] (8.763 ms) : 0, 8763
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-02-10T23:16:03 2025-02-10T23:23:05
git_branch master landerson/no-benchmarks-on-release
git_commit_date 1739207934 1739228661
git_commit_sha added74 fd57e94
release_version 1.47.0-SNAPSHOT~added74bb5 1.47.0-SNAPSHOT~fd57e948ff
start_time 2025-02-10T23:15:49 2025-02-10T23:22:51
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1739230143 1739230143
ci_job_id 802151572 802151572
ci_pipeline_id 55477580 55477580
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zp9z32c-project-304-concurrent-1-g0gh6o55 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zp9z32c-project-304-concurrent-1-g0gh6o55 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1332, 1373
.   : milestone, 1352,
appsec (1.752 ms) : 1728, 1775
.   : milestone, 1752,
appsec_no_iast (1.737 ms) : 1711, 1763
.   : milestone, 1737,
iast (1.502 ms) : 1478, 1527
.   : milestone, 1502,
profiling (1.5 ms) : 1477, 1524
.   : milestone, 1500,
tracing (1.465 ms) : 1440, 1490
.   : milestone, 1465,
section candidate
no_agent (1.348 ms) : 1329, 1368
.   : milestone, 1348,
appsec (1.762 ms) : 1738, 1786
.   : milestone, 1762,
appsec_no_iast (1.734 ms) : 1711, 1758
.   : milestone, 1734,
iast (1.502 ms) : 1477, 1526
.   : milestone, 1502,
profiling (1.521 ms) : 1496, 1546
.   : milestone, 1521,
tracing (1.489 ms) : 1464, 1514
.   : milestone, 1489,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.332 ms, 1.373 ms] -
appsec 1.752 ms [1.728 ms, 1.775 ms] 399.447 µs (29.5%)
appsec_no_iast 1.737 ms [1.711 ms, 1.763 ms] 384.999 µs (28.5%)
iast 1.502 ms [1.478 ms, 1.527 ms] 150.051 µs (11.1%)
profiling 1.5 ms [1.477 ms, 1.524 ms] 148.365 µs (11.0%)
tracing 1.465 ms [1.44 ms, 1.49 ms] 113.062 µs (8.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.329 ms, 1.368 ms] -
appsec 1.762 ms [1.738 ms, 1.786 ms] 413.446 µs (30.7%)
appsec_no_iast 1.734 ms [1.711 ms, 1.758 ms] 385.795 µs (28.6%)
iast 1.502 ms [1.477 ms, 1.526 ms] 153.281 µs (11.4%)
profiling 1.521 ms [1.496 ms, 1.546 ms] 172.916 µs (12.8%)
tracing 1.489 ms [1.464 ms, 1.514 ms] 140.867 µs (10.4%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5
    dateFormat X
    axisFormat %s
section baseline
no_agent (373.739 µs) : 353, 394
.   : milestone, 374,
iast (503.438 µs) : 482, 525
.   : milestone, 503,
iast_FULL (735.94 µs) : 714, 758
.   : milestone, 736,
iast_GLOBAL (557.463 µs) : 535, 580
.   : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (512.74 µs) : 491, 535
.   : milestone, 513,
iast_INACTIVE (452.906 µs) : 432, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (497.928 µs) : 475, 521
.   : milestone, 498,
tracing (449.762 µs) : 429, 471
.   : milestone, 450,
section candidate
no_agent (378.704 µs) : 359, 399
.   : milestone, 379,
iast (509.308 µs) : 487, 531
.   : milestone, 509,
iast_FULL (740.97 µs) : 719, 763
.   : milestone, 741,
iast_GLOBAL (562.571 µs) : 539, 586
.   : milestone, 563,
iast_HARDCODED_SECRET_DISABLED (503.157 µs) : 482, 525
.   : milestone, 503,
iast_INACTIVE (455.428 µs) : 434, 477
.   : milestone, 455,
iast_TELEMETRY_OFF (498.015 µs) : 475, 521
.   : milestone, 498,
tracing (450.198 µs) : 429, 471
.   : milestone, 450,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 373.739 µs [353.036 µs, 394.441 µs] -
iast 503.438 µs [481.869 µs, 525.008 µs] 129.7 µs (34.7%)
iast_FULL 735.94 µs [714.175 µs, 757.705 µs] 362.201 µs (96.9%)
iast_GLOBAL 557.463 µs [534.691 µs, 580.235 µs] 183.725 µs (49.2%)
iast_HARDCODED_SECRET_DISABLED 512.74 µs [490.969 µs, 534.51 µs] 139.001 µs (37.2%)
iast_INACTIVE 452.906 µs [431.809 µs, 474.003 µs] 79.167 µs (21.2%)
iast_TELEMETRY_OFF 497.928 µs [474.855 µs, 521.001 µs] 124.189 µs (33.2%)
tracing 449.762 µs [428.614 µs, 470.91 µs] 76.023 µs (20.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.704 µs [358.776 µs, 398.631 µs] -
iast 509.308 µs [487.241 µs, 531.375 µs] 130.604 µs (34.5%)
iast_FULL 740.97 µs [718.927 µs, 763.012 µs] 362.266 µs (95.7%)
iast_GLOBAL 562.571 µs [539.006 µs, 586.135 µs] 183.867 µs (48.6%)
iast_HARDCODED_SECRET_DISABLED 503.157 µs [481.507 µs, 524.807 µs] 124.454 µs (32.9%)
iast_INACTIVE 455.428 µs [434.069 µs, 476.787 µs] 76.724 µs (20.3%)
iast_TELEMETRY_OFF 498.015 µs [474.62 µs, 521.41 µs] 119.311 µs (31.5%)
tracing 450.198 µs [429.377 µs, 471.02 µs] 71.494 µs (18.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/no-benchmarks-on-release
git_commit_date 1739207934 1739228661
git_commit_sha added74 fd57e94
release_version 1.47.0-SNAPSHOT~added74bb5 1.47.0-SNAPSHOT~fd57e948ff
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1739230703 1739230703
ci_job_id 802151573 802151573
ci_pipeline_id 55477580 55477580
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-t7btoxr-project-304-concurrent-0-mjcdvzsn 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-t7btoxr-project-304-concurrent-0-mjcdvzsn 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.238 s) : 15238000, 15238000
.   : milestone, 15238000,
appsec (15.24 s) : 15240000, 15240000
.   : milestone, 15240000,
iast (19.202 s) : 19202000, 19202000
.   : milestone, 19202000,
iast_GLOBAL (17.929 s) : 17929000, 17929000
.   : milestone, 17929000,
profiling (15.089 s) : 15089000, 15089000
.   : milestone, 15089000,
tracing (14.851 s) : 14851000, 14851000
.   : milestone, 14851000,
section candidate
no_agent (15.174 s) : 15174000, 15174000
.   : milestone, 15174000,
appsec (15.194 s) : 15194000, 15194000
.   : milestone, 15194000,
iast (18.342 s) : 18342000, 18342000
.   : milestone, 18342000,
iast_GLOBAL (17.725 s) : 17725000, 17725000
.   : milestone, 17725000,
profiling (15.081 s) : 15081000, 15081000
.   : milestone, 15081000,
tracing (14.877 s) : 14877000, 14877000
.   : milestone, 14877000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.238 s [15.238 s, 15.238 s] -
appsec 15.24 s [15.24 s, 15.24 s] 2.0 ms (0.0%)
iast 19.202 s [19.202 s, 19.202 s] 3.964 s (26.0%)
iast_GLOBAL 17.929 s [17.929 s, 17.929 s] 2.691 s (17.7%)
profiling 15.089 s [15.089 s, 15.089 s] -149.0 ms (-1.0%)
tracing 14.851 s [14.851 s, 14.851 s] -387.0 ms (-2.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.174 s [15.174 s, 15.174 s] -
appsec 15.194 s [15.194 s, 15.194 s] 20.0 ms (0.1%)
iast 18.342 s [18.342 s, 18.342 s] 3.168 s (20.9%)
iast_GLOBAL 17.725 s [17.725 s, 17.725 s] 2.551 s (16.8%)
profiling 15.081 s [15.081 s, 15.081 s] -93.0 ms (-0.6%)
tracing 14.877 s [14.877 s, 14.877 s] -297.0 ms (-2.0%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~fd57e948ff, baseline=1.47.0-SNAPSHOT~added74bb5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.359 ms) : 2316, 2403
.   : milestone, 2359,
iast (2.113 ms) : 2058, 2168
.   : milestone, 2113,
iast_GLOBAL (2.16 ms) : 2105, 2216
.   : milestone, 2160,
profiling (1.967 ms) : 1924, 2011
.   : milestone, 1967,
tracing (1.947 ms) : 1905, 1989
.   : milestone, 1947,
section candidate
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.35 ms) : 2307, 2393
.   : milestone, 2350,
iast (2.108 ms) : 2053, 2163
.   : milestone, 2108,
iast_GLOBAL (2.154 ms) : 2099, 2209
.   : milestone, 2154,
profiling (1.999 ms) : 1954, 2044
.   : milestone, 1999,
tracing (1.938 ms) : 1896, 1979
.   : milestone, 1938,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.359 ms [2.316 ms, 2.403 ms] 887.338 µs (60.3%)
iast 2.113 ms [2.058 ms, 2.168 ms] 641.219 µs (43.6%)
iast_GLOBAL 2.16 ms [2.105 ms, 2.216 ms] 688.383 µs (46.8%)
profiling 1.967 ms [1.924 ms, 2.011 ms] 495.54 µs (33.7%)
tracing 1.947 ms [1.905 ms, 1.989 ms] 475.275 µs (32.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.463 ms, 1.486 ms] -
appsec 2.35 ms [2.307 ms, 2.393 ms] 875.199 µs (59.3%)
iast 2.108 ms [2.053 ms, 2.163 ms] 633.126 µs (42.9%)
iast_GLOBAL 2.154 ms [2.099 ms, 2.209 ms] 679.19 µs (46.1%)
profiling 1.999 ms [1.954 ms, 2.044 ms] 524.686 µs (35.6%)
tracing 1.938 ms [1.896 ms, 1.979 ms] 462.782 µs (31.4%)

PerfectSlayer
PerfectSlayer approved these changes Feb 11, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix

@randomanderson randomanderson merged commit 652b465 into master Feb 11, 2025
@randomanderson randomanderson deleted the landerson/no-benchmarks-on-release branch February 11, 2025 12:19
@github-actions github-actions Bot added this to the 1.47.0 milestone Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.47.0

Development

Successfully merging this pull request may close these issues.

2 participants

@randomanderson @PerfectSlayer