Skip to content

Fix TestNG tracing for parameterized tests that modify parameters#7226

Merged
nikita-tkachenko-datadog merged 2 commits into
masterfrom
nikita-tkachenko/test-ng-parameters-fix
Jun 26, 2024
Merged

Fix TestNG tracing for parameterized tests that modify parameters#7226
nikita-tkachenko-datadog merged 2 commits into
masterfrom
nikita-tkachenko/test-ng-parameters-fix

Conversation

@nikita-tkachenko-datadog

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Jun 20, 2024 •
edited by atlassian Bot
Loading

Copy link
Copy Markdown
Contributor

What Does This Do

Fixes tracing for parameterized TestNG test cases that modify their own test parameters.

Additional Notes

The reason tracing breaks is because of the way test cases are matched in test event listener.
When a "test started" event is received, a TestDescriptor instance is created that identifies corresponding DDTest instance.
When a "test finished" event is receved, another TestDescriptor instance is created, and is used to look up the DDTest instance created on test start.

The descriptor contains a stringified version of test parameters.
If a test case modifies its parameters during execution, the descriptor created afterwards will contain a different string for test parameters representation.
As the result, it will not match the descriptor created earlier, and the DDTest instance will not be found.

The fix is to use TestNG domain objects (ITestResult) as the test descriptors: even if the parameters are modified, the test result instance is the same on test start and on test finish.

Jira ticket: SDTEST-483

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug fix Bug fix comp: ci visibility Continuous Integration Visibility labels Jun 20, 2024
@pr-commenter

pr-commenter Bot commented Jun 20, 2024 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/test-ng-parameters-fix
git_commit_date 1719394739 1719398263
git_commit_sha 9f7d0c2 b181b7c
release_version 1.36.0-SNAPSHOT~9f7d0c27cd 1.36.0-SNAPSHOT~b181b7cee6
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1719411606 1719411606
ci_job_id 553617875 553617875
ci_pipeline_id 37583595 37583595
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 48 metrics, 14 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:profiling:Remote Config worse
[+59.561µs; +122.587µs] or [+8.808%; +18.128%]		 767.320µs 676.246µs
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1061326
Total [baseline] (8.542 s) : 0, 8542265
Agent [candidate] (1.063 s) : 0, 1063238
Total [candidate] (8.561 s) : 0, 8561291
section iast
Agent [baseline] (1.178 s) : 0, 1178226
Total [baseline] (9.04 s) : 0, 9039907
Agent [candidate] (1.171 s) : 0, 1170548
Total [candidate] (8.991 s) : 0, 8991000
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.178 s) : 0, 1177820
Total [baseline] (8.997 s) : 0, 8996835
Agent [candidate] (1.168 s) : 0, 1167660
Total [candidate] (8.979 s) : 0, 8978998
section iast_TELEMETRY_OFF
Agent [baseline] (1.176 s) : 0, 1176475
Total [baseline] (9.001 s) : 0, 9001231
Agent [candidate] (1.165 s) : 0, 1165224
Total [candidate] (8.984 s) : 0, 8984496
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.178 s 116.9 ms (11.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.178 s 116.494 ms (11.0%)
Agent iast_TELEMETRY_OFF 1.176 s 115.149 ms (10.8%)
Total tracing 8.542 s -
Total iast 9.04 s 497.643 ms (5.8%)
Total iast_HARDCODED_SECRET_DISABLED 8.997 s 454.57 ms (5.3%)
Total iast_TELEMETRY_OFF 9.001 s 458.967 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.171 s 107.31 ms (10.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.168 s 104.422 ms (9.8%)
Agent iast_TELEMETRY_OFF 1.165 s 101.986 ms (9.6%)
Total tracing 8.561 s -
Total iast 8.991 s 429.709 ms (5.0%)
Total iast_HARDCODED_SECRET_DISABLED 8.979 s 417.707 ms (4.9%)
Total iast_TELEMETRY_OFF 8.984 s 423.204 ms (4.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (664.659 ms) : 0, 664659
BytebuddyAgent [candidate] (666.403 ms) : 0, 666403
GlobalTracer [baseline] (304.348 ms) : 0, 304348
GlobalTracer [candidate] (303.876 ms) : 0, 303876
AppSec [baseline] (49.819 ms) : 0, 49819
AppSec [candidate] (50.276 ms) : 0, 50276
Remote Config [baseline] (697.96 µs) : 0, 698
Remote Config [candidate] (687.529 µs) : 0, 688
Telemetry [baseline] (7.603 ms) : 0, 7603
Telemetry [candidate] (7.568 ms) : 0, 7568
section iast
BytebuddyAgent [baseline] (785.816 ms) : 0, 785816
BytebuddyAgent [candidate] (780.57 ms) : 0, 780570
GlobalTracer [baseline] (295.97 ms) : 0, 295970
GlobalTracer [candidate] (293.305 ms) : 0, 293305
AppSec [baseline] (47.211 ms) : 0, 47211
AppSec [candidate] (47.032 ms) : 0, 47032
IAST [baseline] (28.245 ms) : 0, 28245
IAST [candidate] (27.294 ms) : 0, 27294
Remote Config [baseline] (614.906 µs) : 0, 615
Remote Config [candidate] (598.992 µs) : 0, 599
Telemetry [baseline] (7.013 ms) : 0, 7013
Telemetry [candidate] (8.453 ms) : 0, 8453
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (784.991 ms) : 0, 784991
BytebuddyAgent [candidate] (778.363 ms) : 0, 778363
GlobalTracer [baseline] (296.611 ms) : 0, 296611
GlobalTracer [candidate] (293.621 ms) : 0, 293621
AppSec [baseline] (47.474 ms) : 0, 47474
AppSec [candidate] (46.876 ms) : 0, 46876
IAST [baseline] (26.934 ms) : 0, 26934
IAST [candidate] (27.976 ms) : 0, 27976
Remote Config [baseline] (608.882 µs) : 0, 609
Remote Config [candidate] (632.63 µs) : 0, 633
Telemetry [baseline] (7.768 ms) : 0, 7768
Telemetry [candidate] (6.877 ms) : 0, 6877
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (783.236 ms) : 0, 783236
BytebuddyAgent [candidate] (776.777 ms) : 0, 776777
GlobalTracer [baseline] (296.034 ms) : 0, 296034
GlobalTracer [candidate] (293.174 ms) : 0, 293174
AppSec [baseline] (47.525 ms) : 0, 47525
AppSec [candidate] (46.987 ms) : 0, 46987
IAST [baseline] (26.558 ms) : 0, 26558
IAST [candidate] (25.269 ms) : 0, 25269
Remote Config [baseline] (610.317 µs) : 0, 610
Remote Config [candidate] (593.043 µs) : 0, 593
Telemetry [baseline] (9.076 ms) : 0, 9076
Telemetry [candidate] (9.095 ms) : 0, 9095
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.069 s) : 0, 1068681
Total [baseline] (10.371 s) : 0, 10371191
Agent [candidate] (1.062 s) : 0, 1061655
Total [candidate] (10.325 s) : 0, 10325226
section appsec
Agent [baseline] (1.19 s) : 0, 1190176
Total [baseline] (10.536 s) : 0, 10536024
Agent [candidate] (1.185 s) : 0, 1184753
Total [candidate] (10.45 s) : 0, 10449946
section iast
Agent [baseline] (1.178 s) : 0, 1178272
Total [baseline] (10.715 s) : 0, 10715067
Agent [candidate] (1.18 s) : 0, 1179910
Total [candidate] (10.679 s) : 0, 10679277
section profiling
Agent [baseline] (1.263 s) : 0, 1263010
Total [baseline] (10.622 s) : 0, 10622185
Agent [candidate] (1.276 s) : 0, 1275838
Total [candidate] (10.703 s) : 0, 10703307
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.069 s -
Agent appsec 1.19 s 121.495 ms (11.4%)
Agent iast 1.178 s 109.591 ms (10.3%)
Agent profiling 1.263 s 194.329 ms (18.2%)
Total tracing 10.371 s -
Total appsec 10.536 s 164.833 ms (1.6%)
Total iast 10.715 s 343.876 ms (3.3%)
Total profiling 10.622 s 250.993 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.185 s 123.098 ms (11.6%)
Agent iast 1.18 s 118.255 ms (11.1%)
Agent profiling 1.276 s 214.183 ms (20.2%)
Total tracing 10.325 s -
Total appsec 10.45 s 124.72 ms (1.2%)
Total iast 10.679 s 354.051 ms (3.4%)
Total profiling 10.703 s 378.081 ms (3.7%) 
gantt
    title petclinic - break down per module: candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.317 ms) : 0, 670317
BytebuddyAgent [candidate] (665.367 ms) : 0, 665367
GlobalTracer [baseline] (305.216 ms) : 0, 305216
GlobalTracer [candidate] (303.254 ms) : 0, 303254
AppSec [baseline] (50.077 ms) : 0, 50077
AppSec [candidate] (50.299 ms) : 0, 50299
Remote Config [baseline] (750.052 µs) : 0, 750
Remote Config [candidate] (695.093 µs) : 0, 695
Telemetry [baseline] (7.609 ms) : 0, 7609
Telemetry [candidate] (7.558 ms) : 0, 7558
section appsec
BytebuddyAgent [baseline] (682.139 ms) : 0, 682139
BytebuddyAgent [candidate] (677.343 ms) : 0, 677343
GlobalTracer [baseline] (299.858 ms) : 0, 299858
GlobalTracer [candidate] (298.324 ms) : 0, 298324
AppSec [baseline] (154.038 ms) : 0, 154038
AppSec [candidate] (153.952 ms) : 0, 153952
IAST [baseline] (19.9 ms) : 0, 19900
IAST [candidate] (21.751 ms) : 0, 21751
Remote Config [baseline] (652.632 µs) : 0, 653
Remote Config [candidate] (640.537 µs) : 0, 641
Telemetry [baseline] (8.814 ms) : 0, 8814
Telemetry [candidate] (9.066 ms) : 0, 9066
section iast
BytebuddyAgent [baseline] (786.277 ms) : 0, 786277
BytebuddyAgent [candidate] (787.258 ms) : 0, 787258
GlobalTracer [baseline] (296.048 ms) : 0, 296048
GlobalTracer [candidate] (295.88 ms) : 0, 295880
AppSec [baseline] (47.436 ms) : 0, 47436
AppSec [candidate] (47.445 ms) : 0, 47445
IAST [baseline] (27.441 ms) : 0, 27441
IAST [candidate] (27.471 ms) : 0, 27471
Remote Config [baseline] (641.343 µs) : 0, 641
Remote Config [candidate] (616.05 µs) : 0, 616
Telemetry [baseline] (7.077 ms) : 0, 7077
Telemetry [candidate] (7.786 ms) : 0, 7786
section profiling
BytebuddyAgent [baseline] (663.658 ms) : 0, 663658
BytebuddyAgent [candidate] (671.113 ms) : 0, 671113
GlobalTracer [baseline] (388.133 ms) : 0, 388133
GlobalTracer [candidate] (389.178 ms) : 0, 389178
AppSec [baseline] (51.6 ms) : 0, 51600
AppSec [candidate] (51.922 ms) : 0, 51922
Remote Config [baseline] (676.246 µs) : 0, 676
Remote Config [candidate] (767.32 µs) : 0, 767
Telemetry [baseline] (7.424 ms) : 0, 7424
Telemetry [candidate] (7.453 ms) : 0, 7453
ProfilingAgent [baseline] (94.46 ms) : 0, 94460
ProfilingAgent [candidate] (97.685 ms) : 0, 97685
Profiling [baseline] (94.484 ms) : 0, 94484
Profiling [candidate] (97.709 ms) : 0, 97709
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-06-26T13:50:49 2024-06-26T13:57:40
git_branch master nikita-tkachenko/test-ng-parameters-fix
git_commit_date 1719394739 1719398263
git_commit_sha 9f7d0c2 b181b7c
release_version 1.36.0-SNAPSHOT~9f7d0c27cd 1.36.0-SNAPSHOT~b181b7cee6
start_time 2024-06-26T13:50:36 2024-06-26T13:57:27
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1719410606 1719410606
ci_job_id 553617877 553617877
ci_pipeline_id 37583595 37583595
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.345 ms) : 1326, 1363
.   : milestone, 1345,
appsec (1.742 ms) : 1718, 1767
.   : milestone, 1742,
appsec_no_iast (1.718 ms) : 1693, 1742
.   : milestone, 1718,
iast (1.477 ms) : 1454, 1500
.   : milestone, 1477,
profiling (1.529 ms) : 1503, 1555
.   : milestone, 1529,
tracing (1.475 ms) : 1452, 1499
.   : milestone, 1475,
section candidate
no_agent (1.357 ms) : 1338, 1377
.   : milestone, 1357,
appsec (1.718 ms) : 1694, 1742
.   : milestone, 1718,
appsec_no_iast (1.733 ms) : 1708, 1757
.   : milestone, 1733,
iast (1.483 ms) : 1460, 1507
.   : milestone, 1483,
profiling (1.49 ms) : 1465, 1515
.   : milestone, 1490,
tracing (1.477 ms) : 1452, 1501
.   : milestone, 1477,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.363 ms] -
appsec 1.742 ms [1.718 ms, 1.767 ms] 397.923 µs (29.6%)
appsec_no_iast 1.718 ms [1.693 ms, 1.742 ms] 373.124 µs (27.8%)
iast 1.477 ms [1.454 ms, 1.5 ms] 132.468 µs (9.9%)
profiling 1.529 ms [1.503 ms, 1.555 ms] 184.8 µs (13.7%)
tracing 1.475 ms [1.452 ms, 1.499 ms] 130.855 µs (9.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.338 ms, 1.377 ms] -
appsec 1.718 ms [1.694 ms, 1.742 ms] 360.473 µs (26.6%)
appsec_no_iast 1.733 ms [1.708 ms, 1.757 ms] 375.2 µs (27.6%)
iast 1.483 ms [1.46 ms, 1.507 ms] 125.988 µs (9.3%)
profiling 1.49 ms [1.465 ms, 1.515 ms] 132.144 µs (9.7%)
tracing 1.477 ms [1.452 ms, 1.501 ms] 119.161 µs (8.8%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd
    dateFormat X
    axisFormat %s
section baseline
no_agent (370.53 µs) : 351, 390
.   : milestone, 371,
iast (483.733 µs) : 463, 505
.   : milestone, 484,
iast_FULL (551.632 µs) : 531, 573
.   : milestone, 552,
iast_GLOBAL (500.996 µs) : 480, 522
.   : milestone, 501,
iast_HARDCODED_SECRET_DISABLED (480.694 µs) : 460, 502
.   : milestone, 481,
iast_INACTIVE (459.248 µs) : 437, 481
.   : milestone, 459,
iast_TELEMETRY_OFF (475.699 µs) : 455, 497
.   : milestone, 476,
tracing (443.214 µs) : 422, 464
.   : milestone, 443,
section candidate
no_agent (369.318 µs) : 350, 389
.   : milestone, 369,
iast (485.777 µs) : 464, 507
.   : milestone, 486,
iast_FULL (549.882 µs) : 529, 571
.   : milestone, 550,
iast_GLOBAL (508.149 µs) : 487, 529
.   : milestone, 508,
iast_HARDCODED_SECRET_DISABLED (482.016 µs) : 461, 503
.   : milestone, 482,
iast_INACTIVE (451.412 µs) : 430, 472
.   : milestone, 451,
iast_TELEMETRY_OFF (472.924 µs) : 452, 494
.   : milestone, 473,
tracing (445.198 µs) : 425, 466
.   : milestone, 445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.53 µs [351.402 µs, 389.658 µs] -
iast 483.733 µs [462.794 µs, 504.672 µs] 113.203 µs (30.6%)
iast_FULL 551.632 µs [530.583 µs, 572.681 µs] 181.102 µs (48.9%)
iast_GLOBAL 500.996 µs [479.984 µs, 522.009 µs] 130.467 µs (35.2%)
iast_HARDCODED_SECRET_DISABLED 480.694 µs [459.513 µs, 501.875 µs] 110.164 µs (29.7%)
iast_INACTIVE 459.248 µs [437.142 µs, 481.353 µs] 88.718 µs (23.9%)
iast_TELEMETRY_OFF 475.699 µs [454.718 µs, 496.68 µs] 105.169 µs (28.4%)
tracing 443.214 µs [422.192 µs, 464.235 µs] 72.684 µs (19.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.318 µs [349.873 µs, 388.764 µs] -
iast 485.777 µs [464.361 µs, 507.194 µs] 116.459 µs (31.5%)
iast_FULL 549.882 µs [528.739 µs, 571.024 µs] 180.563 µs (48.9%)
iast_GLOBAL 508.149 µs [486.886 µs, 529.413 µs] 138.831 µs (37.6%)
iast_HARDCODED_SECRET_DISABLED 482.016 µs [460.953 µs, 503.079 µs] 112.698 µs (30.5%)
iast_INACTIVE 451.412 µs [430.436 µs, 472.389 µs] 82.094 µs (22.2%)
iast_TELEMETRY_OFF 472.924 µs [451.626 µs, 494.223 µs] 103.606 µs (28.1%)
tracing 445.198 µs [424.888 µs, 465.508 µs] 75.88 µs (20.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/test-ng-parameters-fix
git_commit_date 1719394739 1719398263
git_commit_sha 9f7d0c2 b181b7c
release_version 1.36.0-SNAPSHOT~9f7d0c27cd 1.36.0-SNAPSHOT~b181b7cee6
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1719410995 1719410995
ci_job_id 553617879 553617879
ci_pipeline_id 37583595 37583595
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.466 ms) : 1455, 1478
.   : milestone, 1466,
appsec (2.213 ms) : 2179, 2247
.   : milestone, 2213,
iast (1.979 ms) : 1937, 2021
.   : milestone, 1979,
iast_GLOBAL (2.022 ms) : 1980, 2064
.   : milestone, 2022,
profiling (1.857 ms) : 1823, 1891
.   : milestone, 1857,
tracing (1.839 ms) : 1807, 1872
.   : milestone, 1839,
section candidate
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (2.222 ms) : 2188, 2256
.   : milestone, 2222,
iast (1.975 ms) : 1934, 2015
.   : milestone, 1975,
iast_GLOBAL (2.017 ms) : 1976, 2058
.   : milestone, 2017,
profiling (1.869 ms) : 1835, 1902
.   : milestone, 1869,
tracing (1.835 ms) : 1803, 1868
.   : milestone, 1835,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.466 ms [1.455 ms, 1.478 ms] -
appsec 2.213 ms [2.179 ms, 2.247 ms] 746.947 µs (50.9%)
iast 1.979 ms [1.937 ms, 2.021 ms] 512.86 µs (35.0%)
iast_GLOBAL 2.022 ms [1.98 ms, 2.064 ms] 556.04 µs (37.9%)
profiling 1.857 ms [1.823 ms, 1.891 ms] 390.86 µs (26.7%)
tracing 1.839 ms [1.807 ms, 1.872 ms] 373.248 µs (25.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.468 ms [1.457 ms, 1.48 ms] -
appsec 2.222 ms [2.188 ms, 2.256 ms] 754.14 µs (51.4%)
iast 1.975 ms [1.934 ms, 2.015 ms] 506.451 µs (34.5%)
iast_GLOBAL 2.017 ms [1.976 ms, 2.058 ms] 548.707 µs (37.4%)
profiling 1.869 ms [1.835 ms, 1.902 ms] 400.539 µs (27.3%)
tracing 1.835 ms [1.803 ms, 1.868 ms] 367.173 µs (25.0%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.36.0-SNAPSHOT~b181b7cee6, baseline=1.36.0-SNAPSHOT~9f7d0c27cd
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.334 s) : 15334000, 15334000
.   : milestone, 15334000,
appsec (15.212 s) : 15212000, 15212000
.   : milestone, 15212000,
iast (18.602 s) : 18602000, 18602000
.   : milestone, 18602000,
iast_GLOBAL (17.856 s) : 17856000, 17856000
.   : milestone, 17856000,
profiling (15.691 s) : 15691000, 15691000
.   : milestone, 15691000,
tracing (14.973 s) : 14973000, 14973000
.   : milestone, 14973000,
section candidate
no_agent (15.014 s) : 15014000, 15014000
.   : milestone, 15014000,
appsec (15.018 s) : 15018000, 15018000
.   : milestone, 15018000,
iast (19.084 s) : 19084000, 19084000
.   : milestone, 19084000,
iast_GLOBAL (18.062 s) : 18062000, 18062000
.   : milestone, 18062000,
profiling (14.865 s) : 14865000, 14865000
.   : milestone, 14865000,
tracing (15.069 s) : 15069000, 15069000
.   : milestone, 15069000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.334 s [15.334 s, 15.334 s] -
appsec 15.212 s [15.212 s, 15.212 s] -122.0 ms (-0.8%)
iast 18.602 s [18.602 s, 18.602 s] 3.268 s (21.3%)
iast_GLOBAL 17.856 s [17.856 s, 17.856 s] 2.522 s (16.4%)
profiling 15.691 s [15.691 s, 15.691 s] 357.0 ms (2.3%)
tracing 14.973 s [14.973 s, 14.973 s] -361.0 ms (-2.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.014 s [15.014 s, 15.014 s] -
appsec 15.018 s [15.018 s, 15.018 s] 4.0 ms (0.0%)
iast 19.084 s [19.084 s, 19.084 s] 4.07 s (27.1%)
iast_GLOBAL 18.062 s [18.062 s, 18.062 s] 3.048 s (20.3%)
profiling 14.865 s [14.865 s, 14.865 s] -149.0 ms (-1.0%)
tracing 15.069 s [15.069 s, 15.069 s] 55.0 ms (0.4%)

@nikita-tkachenko-datadog
nikita-tkachenko-datadog marked this pull request as ready for review June 21, 2024 09:02
@nikita-tkachenko-datadog
nikita-tkachenko-datadog requested a review from a team as a code owner June 21, 2024 09:02
@nikita-tkachenko-datadog
nikita-tkachenko-datadog force-pushed the nikita-tkachenko/test-ng-parameters-fix branch from b181b7c to 94d24cb Compare June 26, 2024 12:04
@nikita-tkachenko-datadog
nikita-tkachenko-datadog merged commit b6b3c7f into master Jun 26, 2024
@nikita-tkachenko-datadog
nikita-tkachenko-datadog deleted the nikita-tkachenko/test-ng-parameters-fix branch June 26, 2024 15:39
@github-actions github-actions Bot added this to the 1.36.0 milestone Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@romainkomorn-exdatadog romainkomorn-exdatadog romainkomorn-exdatadog approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp: ci visibility Continuous Integration Visibility type: bug fix Bug fix

Projects

None yet

Milestone

1.36.0

Development

Successfully merging this pull request may close these issues.

2 participants

@nikita-tkachenko-datadog @romainkomorn-exdatadog