Skip to content

Update system-tests to 20b61e9be1e8c922804f8e5ddc80ba4a1f4cdd5b#7492

Merged
PerfectSlayer merged 1 commit intomasterfrom
bbujon/ci-system-tests
Aug 23, 2024
Merged

Update system-tests to 20b61e9be1e8c922804f8e5ddc80ba4a1f4cdd5b#7492
PerfectSlayer merged 1 commit intomasterfrom
bbujon/ci-system-tests

Conversation

@PerfectSlayer
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer commented Aug 22, 2024 •
edited
Loading

What Does This Do

This PR update the system-tests pinned version.
It also restore worker count, use bigger runner and update the timeout accordingly.

Motivation

Add latest scenarios and new parametric client implementation.

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@PerfectSlayer PerfectSlayer added tag: no release notes Changes to exclude from release notes tag: dependencies Dependencies related changes labels Aug 22, 2024
@PerfectSlayer PerfectSlayer requested a review from a team as a code owner August 22, 2024 14:03
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Aug 22, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/ci-system-tests
git_commit_date 1724336653 1724400939
git_commit_sha f6c87de 9cd4098
release_version 1.39.0-SNAPSHOT~f6c87de39a 1.39.0-SNAPSHOT~9cd4098716
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724403983 1724403983
ci_job_id 614891252 614891252
ci_pipeline_id 42658798 42658798
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 17 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.552 s) : 0, 1552413
Total [baseline] (11.814 s) : 0, 11813799
Agent [candidate] (1.566 s) : 0, 1565968
Total [candidate] (11.899 s) : 0, 11898786
section iast
Agent [baseline] (1.716 s) : 0, 1715733
Total [baseline] (12.539 s) : 0, 12538627
Agent [candidate] (1.717 s) : 0, 1716745
Total [candidate] (12.489 s) : 0, 12488662
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.716 s) : 0, 1716457
Total [baseline] (12.461 s) : 0, 12460836
Agent [candidate] (1.718 s) : 0, 1717970
Total [candidate] (12.541 s) : 0, 12541384
section iast_TELEMETRY_OFF
Agent [baseline] (1.71 s) : 0, 1709523
Total [baseline] (12.506 s) : 0, 12505909
Agent [candidate] (1.713 s) : 0, 1713029
Total [candidate] (12.492 s) : 0, 12491507
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.552 s -
Agent iast 1.716 s 163.32 ms (10.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.716 s 164.044 ms (10.6%)
Agent iast_TELEMETRY_OFF 1.71 s 157.11 ms (10.1%)
Total tracing 11.814 s -
Total iast 12.539 s 724.828 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 12.461 s 647.037 ms (5.5%)
Total iast_TELEMETRY_OFF 12.506 s 692.11 ms (5.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.566 s -
Agent iast 1.717 s 150.777 ms (9.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.718 s 152.002 ms (9.7%)
Agent iast_TELEMETRY_OFF 1.713 s 147.06 ms (9.4%)
Total tracing 11.899 s -
Total iast 12.489 s 589.876 ms (5.0%)
Total iast_HARDCODED_SECRET_DISABLED 12.541 s 642.598 ms (5.4%)
Total iast_TELEMETRY_OFF 12.492 s 592.721 ms (5.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (995.179 ms) : 0, 995179
BytebuddyAgent [candidate] (1.004 s) : 0, 1003772
GlobalTracer [baseline] (453.76 ms) : 0, 453760
GlobalTracer [candidate] (458.052 ms) : 0, 458052
AppSec [baseline] (72.594 ms) : 0, 72594
AppSec [candidate] (73.035 ms) : 0, 73035
Remote Config [baseline] (841.885 µs) : 0, 842
Remote Config [candidate] (837.38 µs) : 0, 837
Telemetry [baseline] (9.978 ms) : 0, 9978
Telemetry [candidate] (9.981 ms) : 0, 9981
section iast
BytebuddyAgent [baseline] (1.148 s) : 0, 1147681
BytebuddyAgent [candidate] (1.148 s) : 0, 1147942
GlobalTracer [baseline] (435.407 ms) : 0, 435407
GlobalTracer [candidate] (435.216 ms) : 0, 435216
AppSec [baseline] (71.772 ms) : 0, 71772
AppSec [candidate] (69.989 ms) : 0, 69989
IAST [baseline] (28.222 ms) : 0, 28222
IAST [candidate] (31.111 ms) : 0, 31111
Remote Config [baseline] (739.734 µs) : 0, 740
Remote Config [candidate] (759.386 µs) : 0, 759
Telemetry [baseline] (11.828 ms) : 0, 11828
Telemetry [candidate] (11.602 ms) : 0, 11602
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (1.147 s) : 0, 1146991
BytebuddyAgent [candidate] (1.148 s) : 0, 1148071
GlobalTracer [baseline] (435.422 ms) : 0, 435422
GlobalTracer [candidate] (436.209 ms) : 0, 436209
AppSec [baseline] (71.255 ms) : 0, 71255
AppSec [candidate] (73.925 ms) : 0, 73925
IAST [baseline] (30.166 ms) : 0, 30166
IAST [candidate] (28.165 ms) : 0, 28165
Remote Config [baseline] (738.838 µs) : 0, 739
Remote Config [candidate] (729.472 µs) : 0, 729
Telemetry [baseline] (11.844 ms) : 0, 11844
Telemetry [candidate] (10.724 ms) : 0, 10724
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (1.143 s) : 0, 1142915
BytebuddyAgent [candidate] (1.145 s) : 0, 1144500
GlobalTracer [baseline] (434.532 ms) : 0, 434532
GlobalTracer [candidate] (435.136 ms) : 0, 435136
AppSec [baseline] (70.913 ms) : 0, 70913
AppSec [candidate] (73.044 ms) : 0, 73044
IAST [baseline] (29.484 ms) : 0, 29484
IAST [candidate] (28.592 ms) : 0, 28592
Remote Config [baseline] (750.94 µs) : 0, 751
Remote Config [candidate] (736.299 µs) : 0, 736
Telemetry [baseline] (10.904 ms) : 0, 10904
Telemetry [candidate] (10.867 ms) : 0, 10867
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.549 s) : 0, 1549374
Total [baseline] (14.327 s) : 0, 14327232
Agent [candidate] (1.559 s) : 0, 1559250
Total [candidate] (14.334 s) : 0, 14333764
section appsec
Agent [baseline] (1.739 s) : 0, 1738517
Total [baseline] (14.43 s) : 0, 14430369
Agent [candidate] (1.734 s) : 0, 1733874
Total [candidate] (14.404 s) : 0, 14403590
section iast
Agent [baseline] (1.714 s) : 0, 1714228
Total [baseline] (14.819 s) : 0, 14818794
Agent [candidate] (1.723 s) : 0, 1722546
Total [candidate] (14.822 s) : 0, 14822173
section profiling
Agent [baseline] (1.857 s) : 0, 1857246
Total [baseline] (14.575 s) : 0, 14575450
Agent [candidate] (1.865 s) : 0, 1864976
Total [candidate] (14.638 s) : 0, 14638390
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.549 s -
Agent appsec 1.739 s 189.143 ms (12.2%)
Agent iast 1.714 s 164.854 ms (10.6%)
Agent profiling 1.857 s 307.872 ms (19.9%)
Total tracing 14.327 s -
Total appsec 14.43 s 103.137 ms (0.7%)
Total iast 14.819 s 491.562 ms (3.4%)
Total profiling 14.575 s 248.218 ms (1.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.559 s -
Agent appsec 1.734 s 174.625 ms (11.2%)
Agent iast 1.723 s 163.297 ms (10.5%)
Agent profiling 1.865 s 305.727 ms (19.6%)
Total tracing 14.334 s -
Total appsec 14.404 s 69.827 ms (0.5%)
Total iast 14.822 s 488.409 ms (3.4%)
Total profiling 14.638 s 304.626 ms (2.1%) 
gantt
    title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (992.813 ms) : 0, 992813
BytebuddyAgent [candidate] (999.442 ms) : 0, 999442
GlobalTracer [baseline] (453.769 ms) : 0, 453769
GlobalTracer [candidate] (456.207 ms) : 0, 456207
AppSec [baseline] (72.203 ms) : 0, 72203
AppSec [candidate] (72.728 ms) : 0, 72728
Remote Config [baseline] (820.368 µs) : 0, 820
Remote Config [candidate] (826.147 µs) : 0, 826
Telemetry [baseline] (9.75 ms) : 0, 9750
Telemetry [candidate] (9.832 ms) : 0, 9832
section appsec
BytebuddyAgent [baseline] (1.011 s) : 0, 1010978
BytebuddyAgent [candidate] (1.009 s) : 0, 1008505
GlobalTracer [baseline] (448.501 ms) : 0, 448501
GlobalTracer [candidate] (446.557 ms) : 0, 446557
AppSec [baseline] (235.563 ms) : 0, 235563
AppSec [candidate] (234.76 ms) : 0, 234760
IAST [baseline] (24.696 ms) : 0, 24696
IAST [candidate] (25.285 ms) : 0, 25285
Remote Config [baseline] (755.846 µs) : 0, 756
Remote Config [candidate] (747.818 µs) : 0, 748
Telemetry [baseline] (10.301 ms) : 0, 10301
Telemetry [candidate] (10.593 ms) : 0, 10593
section iast
BytebuddyAgent [baseline] (1.146 s) : 0, 1145876
BytebuddyAgent [candidate] (1.153 s) : 0, 1152586
GlobalTracer [baseline] (434.387 ms) : 0, 434387
GlobalTracer [candidate] (436.127 ms) : 0, 436127
AppSec [baseline] (72.102 ms) : 0, 72102
AppSec [candidate] (72.058 ms) : 0, 72058
IAST [baseline] (28.419 ms) : 0, 28419
IAST [candidate] (29.161 ms) : 0, 29161
Remote Config [baseline] (805.346 µs) : 0, 805
Remote Config [candidate] (742.637 µs) : 0, 743
Telemetry [baseline] (12.591 ms) : 0, 12591
Telemetry [candidate] (11.708 ms) : 0, 11708
section profiling
BytebuddyAgent [baseline] (985.204 ms) : 0, 985204
BytebuddyAgent [candidate] (987.038 ms) : 0, 987038
GlobalTracer [baseline] (582.747 ms) : 0, 582747
GlobalTracer [candidate] (584.855 ms) : 0, 584855
AppSec [baseline] (73.403 ms) : 0, 73403
AppSec [candidate] (73.598 ms) : 0, 73598
Remote Config [baseline] (880.664 µs) : 0, 881
Remote Config [candidate] (895.208 µs) : 0, 895
Telemetry [baseline] (9.512 ms) : 0, 9512
Telemetry [candidate] (9.456 ms) : 0, 9456
ProfilingAgent [baseline] (149.614 ms) : 0, 149614
ProfilingAgent [candidate] (153.171 ms) : 0, 153171
Profiling [baseline] (149.686 ms) : 0, 149686
Profiling [candidate] (153.244 ms) : 0, 153244
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-08-23T08:25:36 2024-08-23T08:32:23
git_branch master bbujon/ci-system-tests
git_commit_date 1724336653 1724400939
git_commit_sha f6c87de 9cd4098
release_version 1.39.0-SNAPSHOT~f6c87de39a 1.39.0-SNAPSHOT~9cd4098716
start_time 2024-08-23T08:25:23 2024-08-23T08:32:10
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724402285 1724402285
ci_job_id 614891253 614891253
ci_pipeline_id 42658798 42658798
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 18 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:tracing better
[-139.430µs; -95.936µs] or [-25.459%; -17.517%]		 unstable
[-814.539op/s; +4276.077op/s] or [-10.861%; +57.014%]		 429.981µs 9230.769op/s 547.664µs 7500.000op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.344 ms) : 1325, 1363
.   : milestone, 1344,
appsec (1.719 ms) : 1695, 1743
.   : milestone, 1719,
appsec_no_iast (1.716 ms) : 1692, 1740
.   : milestone, 1716,
iast (1.481 ms) : 1458, 1503
.   : milestone, 1481,
profiling (1.494 ms) : 1470, 1517
.   : milestone, 1494,
tracing (1.476 ms) : 1452, 1500
.   : milestone, 1476,
section candidate
no_agent (1.336 ms) : 1317, 1355
.   : milestone, 1336,
appsec (1.73 ms) : 1706, 1754
.   : milestone, 1730,
appsec_no_iast (1.729 ms) : 1705, 1752
.   : milestone, 1729,
iast (1.469 ms) : 1446, 1492
.   : milestone, 1469,
profiling (1.471 ms) : 1446, 1495
.   : milestone, 1471,
tracing (1.452 ms) : 1428, 1476
.   : milestone, 1452,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.325 ms, 1.363 ms] -
appsec 1.719 ms [1.695 ms, 1.743 ms] 374.384 µs (27.9%)
appsec_no_iast 1.716 ms [1.692 ms, 1.74 ms] 371.915 µs (27.7%)
iast 1.481 ms [1.458 ms, 1.503 ms] 136.658 µs (10.2%)
profiling 1.494 ms [1.47 ms, 1.517 ms] 149.506 µs (11.1%)
tracing 1.476 ms [1.452 ms, 1.5 ms] 131.605 µs (9.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.336 ms [1.317 ms, 1.355 ms] -
appsec 1.73 ms [1.706 ms, 1.754 ms] 394.267 µs (29.5%)
appsec_no_iast 1.729 ms [1.705 ms, 1.752 ms] 392.89 µs (29.4%)
iast 1.469 ms [1.446 ms, 1.492 ms] 133.218 µs (10.0%)
profiling 1.471 ms [1.446 ms, 1.495 ms] 134.654 µs (10.1%)
tracing 1.452 ms [1.428 ms, 1.476 ms] 115.973 µs (8.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a
    dateFormat X
    axisFormat %s
section baseline
no_agent (362.735 µs) : 342, 383
.   : milestone, 363,
iast (476.354 µs) : 454, 499
.   : milestone, 476,
iast_FULL (546.077 µs) : 524, 568
.   : milestone, 546,
iast_GLOBAL (497.496 µs) : 475, 519
.   : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (475.348 µs) : 453, 498
.   : milestone, 475,
iast_INACTIVE (438.912 µs) : 418, 460
.   : milestone, 439,
iast_TELEMETRY_OFF (467.997 µs) : 447, 489
.   : milestone, 468,
tracing (547.664 µs) : 527, 568
.   : milestone, 548,
section candidate
no_agent (361.062 µs) : 341, 381
.   : milestone, 361,
iast (478.68 µs) : 456, 502
.   : milestone, 479,
iast_FULL (543.533 µs) : 522, 565
.   : milestone, 544,
iast_GLOBAL (496.602 µs) : 474, 519
.   : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (478.402 µs) : 456, 501
.   : milestone, 478,
iast_INACTIVE (439.787 µs) : 419, 461
.   : milestone, 440,
iast_TELEMETRY_OFF (467.183 µs) : 445, 489
.   : milestone, 467,
tracing (429.981 µs) : 410, 450
.   : milestone, 430,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 362.735 µs [342.246 µs, 383.223 µs] -
iast 476.354 µs [453.829 µs, 498.878 µs] 113.619 µs (31.3%)
iast_FULL 546.077 µs [524.235 µs, 567.919 µs] 183.342 µs (50.5%)
iast_GLOBAL 497.496 µs [475.499 µs, 519.492 µs] 134.761 µs (37.2%)
iast_HARDCODED_SECRET_DISABLED 475.348 µs [452.541 µs, 498.155 µs] 112.613 µs (31.0%)
iast_INACTIVE 438.912 µs [418.039 µs, 459.784 µs] 76.177 µs (21.0%)
iast_TELEMETRY_OFF 467.997 µs [446.587 µs, 489.407 µs] 105.262 µs (29.0%)
tracing 547.664 µs [527.219 µs, 568.108 µs] 184.929 µs (51.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.062 µs [341.1 µs, 381.023 µs] -
iast 478.68 µs [455.71 µs, 501.65 µs] 117.618 µs (32.6%)
iast_FULL 543.533 µs [522.111 µs, 564.955 µs] 182.471 µs (50.5%)
iast_GLOBAL 496.602 µs [473.754 µs, 519.45 µs] 135.54 µs (37.5%)
iast_HARDCODED_SECRET_DISABLED 478.402 µs [456.257 µs, 500.547 µs] 117.34 µs (32.5%)
iast_INACTIVE 439.787 µs [418.575 µs, 461.0 µs] 78.725 µs (21.8%)
iast_TELEMETRY_OFF 467.183 µs [445.5 µs, 488.866 µs] 106.121 µs (29.4%)
tracing 429.981 µs [410.009 µs, 449.952 µs] 68.919 µs (19.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/ci-system-tests
git_commit_date 1724336653 1724400939
git_commit_sha f6c87de 9cd4098
release_version 1.39.0-SNAPSHOT~f6c87de39a 1.39.0-SNAPSHOT~9cd4098716
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1724402953 1724402953
ci_job_id 614891254 614891254
ci_pipeline_id 42658798 42658798
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.21 s) : 15210000, 15210000
.   : milestone, 15210000,
appsec (15.006 s) : 15006000, 15006000
.   : milestone, 15006000,
iast (18.747 s) : 18747000, 18747000
.   : milestone, 18747000,
iast_GLOBAL (17.821 s) : 17821000, 17821000
.   : milestone, 17821000,
profiling (15.51 s) : 15510000, 15510000
.   : milestone, 15510000,
tracing (15.024 s) : 15024000, 15024000
.   : milestone, 15024000,
section candidate
no_agent (15.004 s) : 15004000, 15004000
.   : milestone, 15004000,
appsec (14.862 s) : 14862000, 14862000
.   : milestone, 14862000,
iast (18.735 s) : 18735000, 18735000
.   : milestone, 18735000,
iast_GLOBAL (18.341 s) : 18341000, 18341000
.   : milestone, 18341000,
profiling (15.027 s) : 15027000, 15027000
.   : milestone, 15027000,
tracing (14.978 s) : 14978000, 14978000
.   : milestone, 14978000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.21 s [15.21 s, 15.21 s] -
appsec 15.006 s [15.006 s, 15.006 s] -204.0 ms (-1.3%)
iast 18.747 s [18.747 s, 18.747 s] 3.537 s (23.3%)
iast_GLOBAL 17.821 s [17.821 s, 17.821 s] 2.611 s (17.2%)
profiling 15.51 s [15.51 s, 15.51 s] 300.0 ms (2.0%)
tracing 15.024 s [15.024 s, 15.024 s] -186.0 ms (-1.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.004 s [15.004 s, 15.004 s] -
appsec 14.862 s [14.862 s, 14.862 s] -142.0 ms (-0.9%)
iast 18.735 s [18.735 s, 18.735 s] 3.731 s (24.9%)
iast_GLOBAL 18.341 s [18.341 s, 18.341 s] 3.337 s (22.2%)
profiling 15.027 s [15.027 s, 15.027 s] 23.0 ms (0.2%)
tracing 14.978 s [14.978 s, 14.978 s] -26.0 ms (-0.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~9cd4098716, baseline=1.39.0-SNAPSHOT~f6c87de39a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.236 ms) : 2201, 2271
.   : milestone, 2236,
iast (1.978 ms) : 1936, 2020
.   : milestone, 1978,
iast_GLOBAL (2.029 ms) : 1985, 2072
.   : milestone, 2029,
profiling (1.862 ms) : 1827, 1896
.   : milestone, 1862,
tracing (1.849 ms) : 1816, 1883
.   : milestone, 1849,
section candidate
no_agent (1.464 ms) : 1453, 1476
.   : milestone, 1464,
appsec (2.241 ms) : 2206, 2276
.   : milestone, 2241,
iast (1.974 ms) : 1932, 2016
.   : milestone, 1974,
iast_GLOBAL (2.018 ms) : 1975, 2061
.   : milestone, 2018,
profiling (2.284 ms) : 2111, 2457
.   : milestone, 2284,
tracing (1.846 ms) : 1813, 1878
.   : milestone, 1846,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.236 ms [2.201 ms, 2.271 ms] 764.464 µs (52.0%)
iast 1.978 ms [1.936 ms, 2.02 ms] 506.39 µs (34.4%)
iast_GLOBAL 2.029 ms [1.985 ms, 2.072 ms] 557.135 µs (37.9%)
profiling 1.862 ms [1.827 ms, 1.896 ms] 390.184 µs (26.5%)
tracing 1.849 ms [1.816 ms, 1.883 ms] 377.77 µs (25.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.464 ms [1.453 ms, 1.476 ms] -
appsec 2.241 ms [2.206 ms, 2.276 ms] 776.329 µs (53.0%)
iast 1.974 ms [1.932 ms, 2.016 ms] 509.773 µs (34.8%)
iast_GLOBAL 2.018 ms [1.975 ms, 2.061 ms] 553.727 µs (37.8%)
profiling 2.284 ms [2.111 ms, 2.457 ms] 819.632 µs (56.0%)
tracing 1.846 ms [1.813 ms, 1.878 ms] 381.348 µs (26.0%)

@PerfectSlayer PerfectSlayer force-pushed the bbujon/ci-system-tests branch from 322fd10 to d720fd3 Compare August 22, 2024 16:05
@PerfectSlayer
Update system-tests to 20b61e9be1e8c922804f8e5ddc80ba4a1f4cdd5b
9cd4098 
Upgrade resource class
Restore worker count
Update timeout
@PerfectSlayer PerfectSlayer force-pushed the bbujon/ci-system-tests branch from 973be68 to 9cd4098 Compare August 23, 2024 08:16
@PerfectSlayer PerfectSlayer mentioned this pull request Aug 23, 2024
Closed
5 tasks
@PerfectSlayer PerfectSlayer added comp: tooling Build & Tooling comp: testing Testing and removed comp: tooling Build & Tooling labels Aug 23, 2024
@PerfectSlayer PerfectSlayer enabled auto-merge (squash) August 23, 2024 08:53
@PerfectSlayer PerfectSlayer merged commit 1efda00 into master Aug 23, 2024
@PerfectSlayer PerfectSlayer deleted the bbujon/ci-system-tests branch August 23, 2024 09:21
@github-actions github-actions Bot added this to the 1.39.0 milestone Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@smola smola smola approved these changes

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: testing Testing tag: dependencies Dependencies related changes tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.39.0

Development

Successfully merging this pull request may close these issues.

2 participants

@PerfectSlayer @smola