Skip to content

Do not use strict write with latest lettuce#8670

Merged
amarziali merged 1 commit intomasterfrom
andrea.marziali/lettuce-latest-tests
Apr 4, 2025
Merged

Do not use strict write with latest lettuce#8670
amarziali merged 1 commit intomasterfrom
andrea.marziali/lettuce-latest-tests

Conversation

@amarziali
Copy link
Copy Markdown
Contributor

@amarziali amarziali commented Apr 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@amarziali amarziali requested a review from a team as a code owner April 4, 2025 13:16
@amarziali amarziali added comp: testing Testing tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests labels Apr 4, 2025
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 4, 2025

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@amarziali amarziali force-pushed the andrea.marziali/lettuce-latest-tests branch from 2b98d6c to 0c2e77f Compare April 4, 2025 13:19
@amarziali amarziali enabled auto-merge (squash) April 4, 2025 13:29
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Apr 4, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/lettuce-latest-tests
git_commit_date 1743764529 1743772385
git_commit_sha 2dfc8ee 2b98d6c877
release_version 1.48.0-SNAPSHOT~2dfc8eeba9 1.48.0-SNAPSHOT~2b98d6c877
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743775636 1743775636
ci_job_id 881416874 881416874
ci_pipeline_id 61100097 61100097
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-x-j3wh1g-project-304-concurrent-0-u23bah85 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-x-j3wh1g-project-304-concurrent-0-u23bah85 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 69 metrics, 2 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064141
Total [baseline] (8.685 s) : 0, 8685149
Agent [candidate] (1.058 s) : 0, 1058482
Total [candidate] (8.682 s) : 0, 8682030
section iast
Agent [baseline] (1.185 s) : 0, 1184949
Total [baseline] (9.247 s) : 0, 9246967
Agent [candidate] (1.185 s) : 0, 1184891
Total [candidate] (9.281 s) : 0, 9281319
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.183 s) : 0, 1182865
Total [baseline] (9.22 s) : 0, 9219965
Agent [candidate] (1.181 s) : 0, 1181285
Total [candidate] (9.201 s) : 0, 9200750
section iast_TELEMETRY_OFF
Agent [baseline] (1.185 s) : 0, 1184718
Total [baseline] (9.279 s) : 0, 9279109
Agent [candidate] (1.194 s) : 0, 1193505
Total [candidate] (9.263 s) : 0, 9262832
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.185 s 120.808 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.183 s 118.724 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.185 s 120.577 ms (11.3%)
Total tracing 8.685 s -
Total iast 9.247 s 561.818 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.22 s 534.816 ms (6.2%)
Total iast_TELEMETRY_OFF 9.279 s 593.96 ms (6.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent iast 1.185 s 126.409 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.181 s 122.803 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.194 s 135.022 ms (12.8%)
Total tracing 8.682 s -
Total iast 9.281 s 599.289 ms (6.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.201 s 518.72 ms (6.0%)
Total iast_TELEMETRY_OFF 9.263 s 580.802 ms (6.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (727.141 ms) : 0, 727141
BytebuddyAgent [candidate] (722.649 ms) : 0, 722649
GlobalTracer [baseline] (241.514 ms) : 0, 241514
GlobalTracer [candidate] (240.18 ms) : 0, 240180
AppSec [baseline] (54.978 ms) : 0, 54978
AppSec [candidate] (54.696 ms) : 0, 54696
Debugger [baseline] (4.444 ms) : 0, 4444
Debugger [candidate] (4.409 ms) : 0, 4409
Remote Config [baseline] (694.824 µs) : 0, 695
Remote Config [candidate] (694.386 µs) : 0, 694
Telemetry [baseline] (14.38 ms) : 0, 14380
Telemetry [candidate] (14.987 ms) : 0, 14987
section iast
BytebuddyAgent [baseline] (841.426 ms) : 0, 841426
BytebuddyAgent [candidate] (842.259 ms) : 0, 842259
GlobalTracer [baseline] (230.682 ms) : 0, 230682
GlobalTracer [candidate] (230.29 ms) : 0, 230290
IAST [baseline] (22.715 ms) : 0, 22715
IAST [candidate] (22.741 ms) : 0, 22741
AppSec [baseline] (55.879 ms) : 0, 55879
AppSec [candidate] (55.403 ms) : 0, 55403
Debugger [baseline] (4.124 ms) : 0, 4124
Debugger [candidate] (4.123 ms) : 0, 4123
Remote Config [baseline] (596.542 µs) : 0, 597
Remote Config [candidate] (589.115 µs) : 0, 589
Telemetry [baseline] (8.705 ms) : 0, 8705
Telemetry [candidate] (8.686 ms) : 0, 8686
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (840.011 ms) : 0, 840011
BytebuddyAgent [candidate] (839.082 ms) : 0, 839082
GlobalTracer [baseline] (230.003 ms) : 0, 230003
GlobalTracer [candidate] (229.702 ms) : 0, 229702
IAST [baseline] (22.85 ms) : 0, 22850
IAST [candidate] (22.732 ms) : 0, 22732
AppSec [baseline] (55.757 ms) : 0, 55757
AppSec [candidate] (55.597 ms) : 0, 55597
Debugger [baseline] (4.143 ms) : 0, 4143
Debugger [candidate] (4.127 ms) : 0, 4127
Remote Config [baseline] (582.808 µs) : 0, 583
Remote Config [candidate] (585.067 µs) : 0, 585
Telemetry [baseline] (8.673 ms) : 0, 8673
Telemetry [candidate] (8.659 ms) : 0, 8659
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (840.664 ms) : 0, 840664
BytebuddyAgent [candidate] (846.483 ms) : 0, 846483
GlobalTracer [baseline] (231.126 ms) : 0, 231126
GlobalTracer [candidate] (233.116 ms) : 0, 233116
IAST [baseline] (22.318 ms) : 0, 22318
IAST [candidate] (22.602 ms) : 0, 22602
AppSec [baseline] (56.251 ms) : 0, 56251
AppSec [candidate] (56.635 ms) : 0, 56635
Debugger [baseline] (4.175 ms) : 0, 4175
Debugger [candidate] (4.225 ms) : 0, 4225
Remote Config [baseline] (606.494 µs) : 0, 606
Remote Config [candidate] (607.721 µs) : 0, 608
Telemetry [baseline] (8.707 ms) : 0, 8707
Telemetry [candidate] (8.822 ms) : 0, 8822
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062614
Total [baseline] (10.463 s) : 0, 10463389
Agent [candidate] (1.059 s) : 0, 1058514
Total [candidate] (10.491 s) : 0, 10491237
section appsec
Agent [baseline] (1.199 s) : 0, 1199059
Total [baseline] (10.771 s) : 0, 10770538
Agent [candidate] (1.206 s) : 0, 1205952
Total [candidate] (10.802 s) : 0, 10802460
section iast
Agent [baseline] (1.186 s) : 0, 1185629
Total [baseline] (11.047 s) : 0, 11046954
Agent [candidate] (1.186 s) : 0, 1185724
Total [candidate] (11.053 s) : 0, 11053120
section profiling
Agent [baseline] (1.279 s) : 0, 1278596
Total [baseline] (10.882 s) : 0, 10881939
Agent [candidate] (1.294 s) : 0, 1294467
Total [candidate] (10.993 s) : 0, 10992935
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.199 s 136.445 ms (12.8%)
Agent iast 1.186 s 123.015 ms (11.6%)
Agent profiling 1.279 s 215.981 ms (20.3%)
Total tracing 10.463 s -
Total appsec 10.771 s 307.149 ms (2.9%)
Total iast 11.047 s 583.565 ms (5.6%)
Total profiling 10.882 s 418.55 ms (4.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent appsec 1.206 s 147.438 ms (13.9%)
Agent iast 1.186 s 127.21 ms (12.0%)
Agent profiling 1.294 s 235.953 ms (22.3%)
Total tracing 10.491 s -
Total appsec 10.802 s 311.224 ms (3.0%)
Total iast 11.053 s 561.883 ms (5.4%)
Total profiling 10.993 s 501.699 ms (4.8%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (725.969 ms) : 0, 725969
BytebuddyAgent [candidate] (721.854 ms) : 0, 721854
GlobalTracer [baseline] (241.418 ms) : 0, 241418
GlobalTracer [candidate] (240.248 ms) : 0, 240248
AppSec [baseline] (54.887 ms) : 0, 54887
AppSec [candidate] (54.585 ms) : 0, 54585
Debugger [baseline] (4.474 ms) : 0, 4474
Debugger [candidate] (4.433 ms) : 0, 4433
Remote Config [baseline] (697.008 µs) : 0, 697
Remote Config [candidate] (698.038 µs) : 0, 698
Telemetry [baseline] (14.175 ms) : 0, 14175
Telemetry [candidate] (15.827 ms) : 0, 15827
section appsec
BytebuddyAgent [baseline] (740.972 ms) : 0, 740972
BytebuddyAgent [candidate] (746.597 ms) : 0, 746597
GlobalTracer [baseline] (236.904 ms) : 0, 236904
GlobalTracer [candidate] (237.749 ms) : 0, 237749
IAST [baseline] (21.463 ms) : 0, 21463
IAST [candidate] (21.362 ms) : 0, 21362
AppSec [baseline] (176.452 ms) : 0, 176452
AppSec [candidate] (176.768 ms) : 0, 176768
Debugger [baseline] (4.293 ms) : 0, 4293
Debugger [candidate] (4.299 ms) : 0, 4299
Remote Config [baseline] (645.826 µs) : 0, 646
Remote Config [candidate] (645.466 µs) : 0, 645
Telemetry [baseline] (8.213 ms) : 0, 8213
Telemetry [candidate] (8.275 ms) : 0, 8275
section iast
BytebuddyAgent [baseline] (842.177 ms) : 0, 842177
BytebuddyAgent [candidate] (842.732 ms) : 0, 842732
GlobalTracer [baseline] (230.899 ms) : 0, 230899
GlobalTracer [candidate] (230.348 ms) : 0, 230348
IAST [baseline] (22.697 ms) : 0, 22697
IAST [candidate] (22.676 ms) : 0, 22676
AppSec [baseline] (55.469 ms) : 0, 55469
AppSec [candidate] (55.744 ms) : 0, 55744
Debugger [baseline] (4.127 ms) : 0, 4127
Debugger [candidate] (4.061 ms) : 0, 4061
Remote Config [baseline] (592.722 µs) : 0, 593
Remote Config [candidate] (590.195 µs) : 0, 590
Telemetry [baseline] (8.726 ms) : 0, 8726
Telemetry [candidate] (8.704 ms) : 0, 8704
section profiling
BytebuddyAgent [baseline] (711.336 ms) : 0, 711336
BytebuddyAgent [candidate] (720.064 ms) : 0, 720064
GlobalTracer [baseline] (349.818 ms) : 0, 349818
GlobalTracer [candidate] (353.409 ms) : 0, 353409
AppSec [baseline] (53.994 ms) : 0, 53994
AppSec [candidate] (53.974 ms) : 0, 53974
Debugger [baseline] (4.319 ms) : 0, 4319
Debugger [candidate] (4.378 ms) : 0, 4378
Remote Config [baseline] (684.346 µs) : 0, 684
Remote Config [candidate] (708.682 µs) : 0, 709
Telemetry [baseline] (8.946 ms) : 0, 8946
Telemetry [candidate] (9.062 ms) : 0, 9062
ProfilingAgent [baseline] (101.412 ms) : 0, 101412
ProfilingAgent [candidate] (104.379 ms) : 0, 104379
Profiling [baseline] (101.438 ms) : 0, 101438
Profiling [candidate] (104.406 ms) : 0, 104406
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-04T13:37:21 2025-04-04T13:45:07
git_branch master andrea.marziali/lettuce-latest-tests
git_commit_date 1743764529 1743772385
git_commit_sha 2dfc8ee 2b98d6c877
release_version 1.48.0-SNAPSHOT~2dfc8eeba9 1.48.0-SNAPSHOT~2b98d6c877
start_time 2025-04-04T13:37:07 2025-04-04T13:44:53
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743774707 1743774707
ci_job_id 881416877 881416877
ci_pipeline_id 61100097 61100097
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-x-j3wh1g-project-304-concurrent-1-8543r91e 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-x-j3wh1g-project-304-concurrent-1-8543r91e 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.655 µs) : 359, 399
.   : milestone, 379,
iast (526.773 µs) : 504, 549
.   : milestone, 527,
iast_FULL (733.882 µs) : 712, 756
.   : milestone, 734,
iast_GLOBAL (568.949 µs) : 547, 591
.   : milestone, 569,
iast_HARDCODED_SECRET_DISABLED (525.003 µs) : 503, 547
.   : milestone, 525,
iast_INACTIVE (470.366 µs) : 449, 492
.   : milestone, 470,
iast_TELEMETRY_OFF (500.486 µs) : 479, 522
.   : milestone, 500,
tracing (465.527 µs) : 445, 486
.   : milestone, 466,
section candidate
no_agent (383.112 µs) : 363, 403
.   : milestone, 383,
iast (513.403 µs) : 491, 536
.   : milestone, 513,
iast_FULL (732.704 µs) : 711, 755
.   : milestone, 733,
iast_GLOBAL (572.91 µs) : 550, 596
.   : milestone, 573,
iast_HARDCODED_SECRET_DISABLED (521.441 µs) : 500, 543
.   : milestone, 521,
iast_INACTIVE (469.652 µs) : 448, 491
.   : milestone, 470,
iast_TELEMETRY_OFF (507.508 µs) : 485, 530
.   : milestone, 508,
tracing (463.716 µs) : 443, 485
.   : milestone, 464,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.655 µs [358.792 µs, 398.518 µs] -
iast 526.773 µs [504.353 µs, 549.193 µs] 148.118 µs (39.1%)
iast_FULL 733.882 µs [711.77 µs, 755.995 µs] 355.227 µs (93.8%)
iast_GLOBAL 568.949 µs [547.286 µs, 590.612 µs] 190.294 µs (50.3%)
iast_HARDCODED_SECRET_DISABLED 525.003 µs [502.777 µs, 547.23 µs] 146.348 µs (38.6%)
iast_INACTIVE 470.366 µs [448.856 µs, 491.876 µs] 91.711 µs (24.2%)
iast_TELEMETRY_OFF 500.486 µs [478.911 µs, 522.061 µs] 121.831 µs (32.2%)
tracing 465.527 µs [444.58 µs, 486.475 µs] 86.872 µs (22.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 383.112 µs [362.831 µs, 403.393 µs] -
iast 513.403 µs [491.235 µs, 535.572 µs] 130.292 µs (34.0%)
iast_FULL 732.704 µs [710.516 µs, 754.892 µs] 349.592 µs (91.3%)
iast_GLOBAL 572.91 µs [549.917 µs, 595.903 µs] 189.798 µs (49.5%)
iast_HARDCODED_SECRET_DISABLED 521.441 µs [499.686 µs, 543.196 µs] 138.329 µs (36.1%)
iast_INACTIVE 469.652 µs [448.233 µs, 491.071 µs] 86.54 µs (22.6%)
iast_TELEMETRY_OFF 507.508 µs [485.178 µs, 529.837 µs] 124.396 µs (32.5%)
tracing 463.716 µs [442.6 µs, 484.832 µs] 80.605 µs (21.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.354 ms) : 1334, 1374
.   : milestone, 1354,
appsec (1.746 ms) : 1722, 1769
.   : milestone, 1746,
appsec_no_iast (1.751 ms) : 1726, 1775
.   : milestone, 1751,
code_origins (1.688 ms) : 1660, 1715
.   : milestone, 1688,
iast (1.5 ms) : 1476, 1524
.   : milestone, 1500,
profiling (1.516 ms) : 1492, 1540
.   : milestone, 1516,
tracing (1.519 ms) : 1494, 1543
.   : milestone, 1519,
section candidate
no_agent (1.388 ms) : 1369, 1408
.   : milestone, 1388,
appsec (1.737 ms) : 1714, 1761
.   : milestone, 1737,
appsec_no_iast (1.723 ms) : 1700, 1746
.   : milestone, 1723,
code_origins (1.707 ms) : 1681, 1734
.   : milestone, 1707,
iast (1.515 ms) : 1491, 1540
.   : milestone, 1515,
profiling (1.528 ms) : 1503, 1553
.   : milestone, 1528,
tracing (1.514 ms) : 1488, 1540
.   : milestone, 1514,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.354 ms [1.334 ms, 1.374 ms] -
appsec 1.746 ms [1.722 ms, 1.769 ms] 391.778 µs (28.9%)
appsec_no_iast 1.751 ms [1.726 ms, 1.775 ms] 396.701 µs (29.3%)
code_origins 1.688 ms [1.66 ms, 1.715 ms] 334.01 µs (24.7%)
iast 1.5 ms [1.476 ms, 1.524 ms] 146.157 µs (10.8%)
profiling 1.516 ms [1.492 ms, 1.54 ms] 162.111 µs (12.0%)
tracing 1.519 ms [1.494 ms, 1.543 ms] 164.921 µs (12.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.388 ms [1.369 ms, 1.408 ms] -
appsec 1.737 ms [1.714 ms, 1.761 ms] 348.73 µs (25.1%)
appsec_no_iast 1.723 ms [1.7 ms, 1.746 ms] 334.364 µs (24.1%)
code_origins 1.707 ms [1.681 ms, 1.734 ms] 318.917 µs (23.0%)
iast 1.515 ms [1.491 ms, 1.54 ms] 126.742 µs (9.1%)
profiling 1.528 ms [1.503 ms, 1.553 ms] 139.513 µs (10.0%)
tracing 1.514 ms [1.488 ms, 1.54 ms] 125.829 µs (9.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrea.marziali/lettuce-latest-tests
git_commit_date 1743764529 1743772385
git_commit_sha 2dfc8ee 2b98d6c877
release_version 1.48.0-SNAPSHOT~2dfc8eeba9 1.48.0-SNAPSHOT~2b98d6c877
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1743775214 1743775214
ci_job_id 881416881 881416881
ci_pipeline_id 61100097 61100097
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-duquzfu1-project-304-concurrent-0-yhraakhd 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-duquzfu1-project-304-concurrent-0-yhraakhd 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.382 ms) : 2335, 2428
.   : milestone, 2382,
iast (2.155 ms) : 2096, 2213
.   : milestone, 2155,
iast_GLOBAL (2.197 ms) : 2138, 2256
.   : milestone, 2197,
profiling (2.017 ms) : 1969, 2065
.   : milestone, 2017,
tracing (1.987 ms) : 1942, 2032
.   : milestone, 1987,
section candidate
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.373 ms) : 2326, 2419
.   : milestone, 2373,
iast (2.161 ms) : 2103, 2220
.   : milestone, 2161,
iast_GLOBAL (2.202 ms) : 2143, 2261
.   : milestone, 2202,
profiling (2.011 ms) : 1963, 2058
.   : milestone, 2011,
tracing (1.977 ms) : 1932, 2022
.   : milestone, 1977,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.382 ms [2.335 ms, 2.428 ms] 906.377 µs (61.4%)
iast 2.155 ms [2.096 ms, 2.213 ms] 679.701 µs (46.1%)
iast_GLOBAL 2.197 ms [2.138 ms, 2.256 ms] 721.51 µs (48.9%)
profiling 2.017 ms [1.969 ms, 2.065 ms] 541.805 µs (36.7%)
tracing 1.987 ms [1.942 ms, 2.032 ms] 511.888 µs (34.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.463 ms, 1.486 ms] -
appsec 2.373 ms [2.326 ms, 2.419 ms] 898.134 µs (60.9%)
iast 2.161 ms [2.103 ms, 2.22 ms] 686.81 µs (46.6%)
iast_GLOBAL 2.202 ms [2.143 ms, 2.261 ms] 727.631 µs (49.3%)
profiling 2.011 ms [1.963 ms, 2.058 ms] 536.195 µs (36.4%)
tracing 1.977 ms [1.932 ms, 2.022 ms] 502.395 µs (34.1%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~2b98d6c877, baseline=1.48.0-SNAPSHOT~2dfc8eeba9
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.409 s) : 15409000, 15409000
.   : milestone, 15409000,
appsec (14.972 s) : 14972000, 14972000
.   : milestone, 14972000,
iast (19.099 s) : 19099000, 19099000
.   : milestone, 19099000,
iast_GLOBAL (18.342 s) : 18342000, 18342000
.   : milestone, 18342000,
profiling (15.043 s) : 15043000, 15043000
.   : milestone, 15043000,
tracing (14.695 s) : 14695000, 14695000
.   : milestone, 14695000,
section candidate
no_agent (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
appsec (14.765 s) : 14765000, 14765000
.   : milestone, 14765000,
iast (18.811 s) : 18811000, 18811000
.   : milestone, 18811000,
iast_GLOBAL (18.243 s) : 18243000, 18243000
.   : milestone, 18243000,
profiling (15.264 s) : 15264000, 15264000
.   : milestone, 15264000,
tracing (14.629 s) : 14629000, 14629000
.   : milestone, 14629000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.409 s [15.409 s, 15.409 s] -
appsec 14.972 s [14.972 s, 14.972 s] -437.0 ms (-2.8%)
iast 19.099 s [19.099 s, 19.099 s] 3.69 s (23.9%)
iast_GLOBAL 18.342 s [18.342 s, 18.342 s] 2.933 s (19.0%)
profiling 15.043 s [15.043 s, 15.043 s] -366.0 ms (-2.4%)
tracing 14.695 s [14.695 s, 14.695 s] -714.0 ms (-4.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.893 s [14.893 s, 14.893 s] -
appsec 14.765 s [14.765 s, 14.765 s] -128.0 ms (-0.9%)
iast 18.811 s [18.811 s, 18.811 s] 3.918 s (26.3%)
iast_GLOBAL 18.243 s [18.243 s, 18.243 s] 3.35 s (22.5%)
profiling 15.264 s [15.264 s, 15.264 s] 371.0 ms (2.5%)
tracing 14.629 s [14.629 s, 14.629 s] -264.0 ms (-1.8%)

@amarziali amarziali merged commit c0aff90 into master Apr 4, 2025
248 of 250 checks passed
@amarziali amarziali deleted the andrea.marziali/lettuce-latest-tests branch April 4, 2025 14:10
@github-actions github-actions Bot added this to the 1.48.0 milestone Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls approved these changes

Assignees

No one assigned

Labels

comp: testing Testing tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.48.0

Development

Successfully merging this pull request may close these issues.

2 participants

@amarziali @mcculls