Skip to content

Update system-tests to 1c9542783eedfed1b995d976963fbeb14ba772b9#7526

Merged
smola merged 2 commits intomasterfrom
smola/update-system-tests-20240828
Aug 29, 2024
Merged

Update system-tests to 1c9542783eedfed1b995d976963fbeb14ba772b9#7526
smola merged 2 commits intomasterfrom
smola/update-system-tests-20240828

Conversation

@smola
Copy link
Copy Markdown
Member

@smola smola commented Aug 28, 2024 •
edited
Loading

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Aug 28, 2024
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Aug 28, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/update-system-tests-20240828
git_commit_date 1724934482 1724934648
git_commit_sha a5af871 c0bed73
release_version 1.39.0-SNAPSHOT~a5af871b8e 1.39.0-SNAPSHOT~c0bed736bf
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724937134 1724937134
ci_job_id 621419843 621419843
ci_pipeline_id 43145414 43145414
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1046190
Total [baseline] (10.308 s) : 0, 10307502
Agent [candidate] (1.046 s) : 0, 1046100
Total [candidate] (10.414 s) : 0, 10413931
section appsec
Agent [baseline] (1.176 s) : 0, 1176185
Total [baseline] (10.579 s) : 0, 10578594
Agent [candidate] (1.18 s) : 0, 1179601
Total [candidate] (10.559 s) : 0, 10558719
section iast
Agent [baseline] (1.171 s) : 0, 1170713
Total [baseline] (10.845 s) : 0, 10845264
Agent [candidate] (1.186 s) : 0, 1186425
Total [candidate] (10.805 s) : 0, 10805453
section profiling
Agent [baseline] (1.254 s) : 0, 1253632
Total [baseline] (10.652 s) : 0, 10651909
Agent [candidate] (1.247 s) : 0, 1246790
Total [candidate] (10.608 s) : 0, 10608182
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent appsec 1.176 s 129.994 ms (12.4%)
Agent iast 1.171 s 124.522 ms (11.9%)
Agent profiling 1.254 s 207.442 ms (19.8%)
Total tracing 10.308 s -
Total appsec 10.579 s 271.092 ms (2.6%)
Total iast 10.845 s 537.762 ms (5.2%)
Total profiling 10.652 s 344.407 ms (3.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent appsec 1.18 s 133.501 ms (12.8%)
Agent iast 1.186 s 140.325 ms (13.4%)
Agent profiling 1.247 s 200.69 ms (19.2%)
Total tracing 10.414 s -
Total appsec 10.559 s 144.789 ms (1.4%)
Total iast 10.805 s 391.522 ms (3.8%)
Total profiling 10.608 s 194.252 ms (1.9%) 
gantt
    title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (667.258 ms) : 0, 667258
BytebuddyAgent [candidate] (667.466 ms) : 0, 667466
GlobalTracer [baseline] (305.771 ms) : 0, 305771
GlobalTracer [candidate] (305.637 ms) : 0, 305637
AppSec [baseline] (51.415 ms) : 0, 51415
AppSec [candidate] (51.388 ms) : 0, 51388
Remote Config [baseline] (677.909 µs) : 0, 678
Remote Config [candidate] (663.095 µs) : 0, 663
Telemetry [baseline] (7.534 ms) : 0, 7534
Telemetry [candidate] (7.41 ms) : 0, 7410
section appsec
BytebuddyAgent [baseline] (684.198 ms) : 0, 684198
BytebuddyAgent [candidate] (686.386 ms) : 0, 686386
GlobalTracer [baseline] (299.577 ms) : 0, 299577
GlobalTracer [candidate] (300.238 ms) : 0, 300238
AppSec [baseline] (158.886 ms) : 0, 158886
AppSec [candidate] (161.186 ms) : 0, 161186
Remote Config [baseline] (612.787 µs) : 0, 613
Remote Config [candidate] (610.835 µs) : 0, 611
Telemetry [baseline] (8.171 ms) : 0, 8171
Telemetry [candidate] (7.772 ms) : 0, 7772
IAST [baseline] (22.414 ms) : 0, 22414
IAST [candidate] (19.691 ms) : 0, 19691
section iast
BytebuddyAgent [baseline] (778.052 ms) : 0, 778052
BytebuddyAgent [candidate] (789.713 ms) : 0, 789713
GlobalTracer [baseline] (295.45 ms) : 0, 295450
GlobalTracer [candidate] (299.419 ms) : 0, 299419
AppSec [baseline] (49.2 ms) : 0, 49200
AppSec [candidate] (49.351 ms) : 0, 49351
Remote Config [baseline] (600.094 µs) : 0, 600
Remote Config [candidate] (596.519 µs) : 0, 597
Telemetry [baseline] (8.851 ms) : 0, 8851
Telemetry [candidate] (8.663 ms) : 0, 8663
IAST [baseline] (24.982 ms) : 0, 24982
IAST [candidate] (24.863 ms) : 0, 24863
section profiling
BytebuddyAgent [baseline] (668.259 ms) : 0, 668259
BytebuddyAgent [candidate] (663.305 ms) : 0, 663305
GlobalTracer [baseline] (390.712 ms) : 0, 390712
GlobalTracer [candidate] (389.618 ms) : 0, 389618
AppSec [baseline] (52.529 ms) : 0, 52529
AppSec [candidate] (52.305 ms) : 0, 52305
Remote Config [baseline] (685.219 µs) : 0, 685
Remote Config [candidate] (686.599 µs) : 0, 687
Telemetry [baseline] (7.398 ms) : 0, 7398
Telemetry [candidate] (7.331 ms) : 0, 7331
ProfilingAgent [baseline] (96.49 ms) : 0, 96490
ProfilingAgent [candidate] (96.242 ms) : 0, 96242
Profiling [baseline] (96.514 ms) : 0, 96514
Profiling [candidate] (96.266 ms) : 0, 96266
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1052555
Total [baseline] (8.499 s) : 0, 8498842
Agent [candidate] (1.054 s) : 0, 1054340
Total [candidate] (8.525 s) : 0, 8524560
section iast
Agent [baseline] (1.178 s) : 0, 1177585
Total [baseline] (8.983 s) : 0, 8983026
Agent [candidate] (1.171 s) : 0, 1171300
Total [candidate] (8.954 s) : 0, 8953887
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.181 s) : 0, 1180535
Total [baseline] (8.956 s) : 0, 8955752
Agent [candidate] (1.17 s) : 0, 1170231
Total [candidate] (8.962 s) : 0, 8962126
section iast_TELEMETRY_OFF
Agent [baseline] (1.173 s) : 0, 1173253
Total [baseline] (8.95 s) : 0, 8949965
Agent [candidate] (1.187 s) : 0, 1186883
Total [candidate] (8.98 s) : 0, 8979552
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent iast 1.178 s 125.03 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.181 s 127.98 ms (12.2%)
Agent iast_TELEMETRY_OFF 1.173 s 120.698 ms (11.5%)
Total tracing 8.499 s -
Total iast 8.983 s 484.184 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 8.956 s 456.91 ms (5.4%)
Total iast_TELEMETRY_OFF 8.95 s 451.123 ms (5.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent iast 1.171 s 116.96 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.17 s 115.891 ms (11.0%)
Agent iast_TELEMETRY_OFF 1.187 s 132.543 ms (12.6%)
Total tracing 8.525 s -
Total iast 8.954 s 429.327 ms (5.0%)
Total iast_HARDCODED_SECRET_DISABLED 8.962 s 437.566 ms (5.1%)
Total iast_TELEMETRY_OFF 8.98 s 454.992 ms (5.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (672.022 ms) : 0, 672022
BytebuddyAgent [candidate] (672.779 ms) : 0, 672779
GlobalTracer [baseline] (307.543 ms) : 0, 307543
GlobalTracer [candidate] (308.066 ms) : 0, 308066
AppSec [baseline] (51.207 ms) : 0, 51207
AppSec [candidate] (51.567 ms) : 0, 51567
Remote Config [baseline] (670.318 µs) : 0, 670
Remote Config [candidate] (677.958 µs) : 0, 678
Telemetry [baseline] (7.463 ms) : 0, 7463
Telemetry [candidate] (7.583 ms) : 0, 7583
section iast
BytebuddyAgent [baseline] (783.497 ms) : 0, 783497
BytebuddyAgent [candidate] (778.397 ms) : 0, 778397
GlobalTracer [baseline] (297.431 ms) : 0, 297431
GlobalTracer [candidate] (295.382 ms) : 0, 295382
AppSec [baseline] (49.832 ms) : 0, 49832
AppSec [candidate] (51.335 ms) : 0, 51335
Remote Config [baseline] (607.062 µs) : 0, 607
Remote Config [candidate] (585.094 µs) : 0, 585
Telemetry [baseline] (8.852 ms) : 0, 8852
Telemetry [candidate] (7.918 ms) : 0, 7918
IAST [baseline] (23.726 ms) : 0, 23726
IAST [candidate] (24.098 ms) : 0, 24098
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (783.733 ms) : 0, 783733
BytebuddyAgent [candidate] (776.34 ms) : 0, 776340
GlobalTracer [baseline] (298.217 ms) : 0, 298217
GlobalTracer [candidate] (295.652 ms) : 0, 295652
AppSec [baseline] (49.75 ms) : 0, 49750
AppSec [candidate] (51.685 ms) : 0, 51685
Remote Config [baseline] (583.055 µs) : 0, 583
Remote Config [candidate] (596.946 µs) : 0, 597
Telemetry [baseline] (7.822 ms) : 0, 7822
Telemetry [candidate] (9.486 ms) : 0, 9486
IAST [baseline] (26.745 ms) : 0, 26745
IAST [candidate] (22.918 ms) : 0, 22918
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.525 ms) : 0, 779525
BytebuddyAgent [candidate] (788.358 ms) : 0, 788358
GlobalTracer [baseline] (295.676 ms) : 0, 295676
GlobalTracer [candidate] (300.08 ms) : 0, 300080
AppSec [baseline] (50.602 ms) : 0, 50602
AppSec [candidate] (54.366 ms) : 0, 54366
Remote Config [baseline] (583.952 µs) : 0, 584
Remote Config [candidate] (604.387 µs) : 0, 604
Telemetry [baseline] (9.351 ms) : 0, 9351
Telemetry [candidate] (8.733 ms) : 0, 8733
IAST [baseline] (23.861 ms) : 0, 23861
IAST [candidate] (21.0 ms) : 0, 21000
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-08-29T12:42:44 2024-08-29T12:49:33
git_branch master smola/update-system-tests-20240828
git_commit_date 1724934482 1724934648
git_commit_sha a5af871 c0bed73
release_version 1.39.0-SNAPSHOT~a5af871b8e 1.39.0-SNAPSHOT~c0bed736bf
start_time 2024-08-29T12:42:31 2024-08-29T12:49:20
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724936115 1724936115
ci_job_id 621419844 621419844
ci_pipeline_id 43145414 43145414
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 17 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-89.360µs; -38.195µs] or [-5.834%; -2.493%]		 unstable
[-589.024op/s; +589.024op/s] or [-19.143%; +19.143%]		 1.468ms 3076.923op/s 1.532ms 3076.923op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e
    dateFormat X
    axisFormat %s
section baseline
no_agent (370.554 µs) : 349, 392
.   : milestone, 371,
iast (482.624 µs) : 460, 505
.   : milestone, 483,
iast_FULL (540.915 µs) : 520, 562
.   : milestone, 541,
iast_GLOBAL (492.625 µs) : 472, 514
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (474.662 µs) : 453, 497
.   : milestone, 475,
iast_INACTIVE (443.363 µs) : 422, 465
.   : milestone, 443,
iast_TELEMETRY_OFF (463.762 µs) : 442, 486
.   : milestone, 464,
tracing (436.836 µs) : 417, 457
.   : milestone, 437,
section candidate
no_agent (362.534 µs) : 343, 382
.   : milestone, 363,
iast (477.02 µs) : 455, 499
.   : milestone, 477,
iast_FULL (547.451 µs) : 526, 569
.   : milestone, 547,
iast_GLOBAL (493.935 µs) : 473, 515
.   : milestone, 494,
iast_HARDCODED_SECRET_DISABLED (474.858 µs) : 453, 497
.   : milestone, 475,
iast_INACTIVE (439.279 µs) : 419, 460
.   : milestone, 439,
iast_TELEMETRY_OFF (467.355 µs) : 445, 490
.   : milestone, 467,
tracing (433.166 µs) : 413, 453
.   : milestone, 433,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.554 µs [348.695 µs, 392.413 µs] -
iast 482.624 µs [460.316 µs, 504.933 µs] 112.07 µs (30.2%)
iast_FULL 540.915 µs [520.079 µs, 561.75 µs] 170.361 µs (46.0%)
iast_GLOBAL 492.625 µs [471.632 µs, 513.618 µs] 122.071 µs (32.9%)
iast_HARDCODED_SECRET_DISABLED 474.662 µs [452.534 µs, 496.789 µs] 104.108 µs (28.1%)
iast_INACTIVE 443.363 µs [421.913 µs, 464.813 µs] 72.809 µs (19.6%)
iast_TELEMETRY_OFF 463.762 µs [441.707 µs, 485.818 µs] 93.209 µs (25.2%)
tracing 436.836 µs [416.686 µs, 456.986 µs] 66.282 µs (17.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 362.534 µs [343.152 µs, 381.917 µs] -
iast 477.02 µs [454.541 µs, 499.499 µs] 114.486 µs (31.6%)
iast_FULL 547.451 µs [526.168 µs, 568.734 µs] 184.917 µs (51.0%)
iast_GLOBAL 493.935 µs [472.926 µs, 514.944 µs] 131.401 µs (36.2%)
iast_HARDCODED_SECRET_DISABLED 474.858 µs [452.881 µs, 496.834 µs] 112.323 µs (31.0%)
iast_INACTIVE 439.279 µs [418.501 µs, 460.057 µs] 76.745 µs (21.2%)
iast_TELEMETRY_OFF 467.355 µs [444.803 µs, 489.906 µs] 104.82 µs (28.9%)
tracing 433.166 µs [412.963 µs, 453.369 µs] 70.632 µs (19.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.331 ms) : 1313, 1350
.   : milestone, 1331,
appsec (1.728 ms) : 1705, 1752
.   : milestone, 1728,
appsec_no_iast (1.73 ms) : 1706, 1753
.   : milestone, 1730,
iast (1.471 ms) : 1449, 1493
.   : milestone, 1471,
profiling (1.532 ms) : 1507, 1556
.   : milestone, 1532,
tracing (1.443 ms) : 1418, 1467
.   : milestone, 1443,
section candidate
no_agent (1.319 ms) : 1300, 1338
.   : milestone, 1319,
appsec (1.71 ms) : 1686, 1735
.   : milestone, 1710,
appsec_no_iast (1.723 ms) : 1699, 1747
.   : milestone, 1723,
iast (1.458 ms) : 1436, 1480
.   : milestone, 1458,
profiling (1.468 ms) : 1445, 1491
.   : milestone, 1468,
tracing (1.454 ms) : 1430, 1478
.   : milestone, 1454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.331 ms [1.313 ms, 1.35 ms] -
appsec 1.728 ms [1.705 ms, 1.752 ms] 396.804 µs (29.8%)
appsec_no_iast 1.73 ms [1.706 ms, 1.753 ms] 398.348 µs (29.9%)
iast 1.471 ms [1.449 ms, 1.493 ms] 139.318 µs (10.5%)
profiling 1.532 ms [1.507 ms, 1.556 ms] 200.52 µs (15.1%)
tracing 1.443 ms [1.418 ms, 1.467 ms] 111.499 µs (8.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.319 ms [1.3 ms, 1.338 ms] -
appsec 1.71 ms [1.686 ms, 1.735 ms] 391.17 µs (29.7%)
appsec_no_iast 1.723 ms [1.699 ms, 1.747 ms] 404.099 µs (30.6%)
iast 1.458 ms [1.436 ms, 1.48 ms] 139.138 µs (10.5%)
profiling 1.468 ms [1.445 ms, 1.491 ms] 148.985 µs (11.3%)
tracing 1.454 ms [1.43 ms, 1.478 ms] 135.219 µs (10.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/update-system-tests-20240828
git_commit_date 1724934482 1724934648
git_commit_sha a5af871 c0bed73
release_version 1.39.0-SNAPSHOT~a5af871b8e 1.39.0-SNAPSHOT~c0bed736bf
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1724936675 1724936675
ci_job_id 621419845 621419845
ci_pipeline_id 43145414 43145414
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.458 ms) : 1447, 1470
.   : milestone, 1458,
appsec (2.224 ms) : 2188, 2259
.   : milestone, 2224,
iast (1.949 ms) : 1907, 1991
.   : milestone, 1949,
iast_GLOBAL (2.008 ms) : 1965, 2051
.   : milestone, 2008,
profiling (1.864 ms) : 1829, 1900
.   : milestone, 1864,
tracing (1.838 ms) : 1805, 1871
.   : milestone, 1838,
section candidate
no_agent (1.467 ms) : 1455, 1479
.   : milestone, 1467,
appsec (2.215 ms) : 2180, 2249
.   : milestone, 2215,
iast (1.973 ms) : 1930, 2015
.   : milestone, 1973,
iast_GLOBAL (2.02 ms) : 1976, 2065
.   : milestone, 2020,
profiling (1.852 ms) : 1819, 1886
.   : milestone, 1852,
tracing (1.843 ms) : 1810, 1876
.   : milestone, 1843,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.458 ms [1.447 ms, 1.47 ms] -
appsec 2.224 ms [2.188 ms, 2.259 ms] 765.329 µs (52.5%)
iast 1.949 ms [1.907 ms, 1.991 ms] 490.669 µs (33.6%)
iast_GLOBAL 2.008 ms [1.965 ms, 2.051 ms] 549.933 µs (37.7%)
profiling 1.864 ms [1.829 ms, 1.9 ms] 406.047 µs (27.8%)
tracing 1.838 ms [1.805 ms, 1.871 ms] 380.087 µs (26.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.455 ms, 1.479 ms] -
appsec 2.215 ms [2.18 ms, 2.249 ms] 747.889 µs (51.0%)
iast 1.973 ms [1.93 ms, 2.015 ms] 505.726 µs (34.5%)
iast_GLOBAL 2.02 ms [1.976 ms, 2.065 ms] 553.667 µs (37.7%)
profiling 1.852 ms [1.819 ms, 1.886 ms] 385.608 µs (26.3%)
tracing 1.843 ms [1.81 ms, 1.876 ms] 375.968 µs (25.6%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~c0bed736bf, baseline=1.39.0-SNAPSHOT~a5af871b8e
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.784 s) : 14784000, 14784000
.   : milestone, 14784000,
appsec (15.377 s) : 15377000, 15377000
.   : milestone, 15377000,
iast (18.635 s) : 18635000, 18635000
.   : milestone, 18635000,
iast_GLOBAL (17.893 s) : 17893000, 17893000
.   : milestone, 17893000,
profiling (15.603 s) : 15603000, 15603000
.   : milestone, 15603000,
tracing (15.26 s) : 15260000, 15260000
.   : milestone, 15260000,
section candidate
no_agent (14.908 s) : 14908000, 14908000
.   : milestone, 14908000,
appsec (15.358 s) : 15358000, 15358000
.   : milestone, 15358000,
iast (18.868 s) : 18868000, 18868000
.   : milestone, 18868000,
iast_GLOBAL (17.809 s) : 17809000, 17809000
.   : milestone, 17809000,
profiling (15.926 s) : 15926000, 15926000
.   : milestone, 15926000,
tracing (15.119 s) : 15119000, 15119000
.   : milestone, 15119000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.784 s [14.784 s, 14.784 s] -
appsec 15.377 s [15.377 s, 15.377 s] 593.0 ms (4.0%)
iast 18.635 s [18.635 s, 18.635 s] 3.851 s (26.0%)
iast_GLOBAL 17.893 s [17.893 s, 17.893 s] 3.109 s (21.0%)
profiling 15.603 s [15.603 s, 15.603 s] 819.0 ms (5.5%)
tracing 15.26 s [15.26 s, 15.26 s] 476.0 ms (3.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.908 s [14.908 s, 14.908 s] -
appsec 15.358 s [15.358 s, 15.358 s] 450.0 ms (3.0%)
iast 18.868 s [18.868 s, 18.868 s] 3.96 s (26.6%)
iast_GLOBAL 17.809 s [17.809 s, 17.809 s] 2.901 s (19.5%)
profiling 15.926 s [15.926 s, 15.926 s] 1.018 s (6.8%)
tracing 15.119 s [15.119 s, 15.119 s] 211.0 ms (1.4%)

@smola smola marked this pull request as ready for review August 28, 2024 14:34
@smola smola requested a review from a team as a code owner August 28, 2024 14:34
@smola smola requested a review from PerfectSlayer August 28, 2024 14:34
@smola smola enabled auto-merge (squash) August 29, 2024 07:24
@smola smola merged commit ea5731f into master Aug 29, 2024
@smola smola deleted the smola/update-system-tests-20240828 branch August 29, 2024 15:25
@github-actions github-actions Bot added this to the 1.39.0 milestone Aug 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ValentinZakharov ValentinZakharov ValentinZakharov approved these changes

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer PerfectSlayer is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.39.0

Development

Successfully merging this pull request may close these issues.

2 participants

@smola @ValentinZakharov