Skip to content

Fix NPE in logic that parses flaky tests list#8070

Merged
nikita-tkachenko-datadog merged 2 commits intomasterfrom
nikita-tkachenko/fix-npe-in-flaky-tests-list
Dec 9, 2024
Merged

Fix NPE in logic that parses flaky tests list#8070
nikita-tkachenko-datadog merged 2 commits intomasterfrom
nikita-tkachenko/fix-npe-in-flaky-tests-list

Conversation

@nikita-tkachenko-datadog
Copy link
Copy Markdown
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Dec 9, 2024

What Does This Do

Fixes NPE in logic that parses the list of flaky tests received from the backend.
The error only happens when a "partial build" is run for a single module.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug Bug report and fix tag: no release notes Changes to exclude from release notes comp: ci visibility Continuous Integration Visibility labels Dec 9, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review December 9, 2024 14:48
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner December 9, 2024 14:49
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Dec 9, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/fix-npe-in-flaky-tests-list
git_commit_date 1733749565 1733757733
git_commit_sha 88c9405 e90472b
release_version 1.44.0-SNAPSHOT~88c9405c96 1.44.0-SNAPSHOT~e90472b827
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1733760221 1733760221
ci_job_id 731094706 731094706
ci_pipeline_id 50582628 50582628
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.093 s) : 0, 1093109
Total [baseline] (8.687 s) : 0, 8687191
Agent [candidate] (1.102 s) : 0, 1101700
Total [candidate] (8.681 s) : 0, 8680504
section iast
Agent [baseline] (1.223 s) : 0, 1222824
Total [baseline] (9.218 s) : 0, 9218067
Agent [candidate] (1.229 s) : 0, 1229251
Total [candidate] (9.246 s) : 0, 9245670
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.22 s) : 0, 1220061
Total [baseline] (9.17 s) : 0, 9170003
Agent [candidate] (1.219 s) : 0, 1219066
Total [candidate] (9.161 s) : 0, 9161292
section iast_TELEMETRY_OFF
Agent [baseline] (1.222 s) : 0, 1221692
Total [baseline] (9.209 s) : 0, 9208523
Agent [candidate] (1.218 s) : 0, 1218353
Total [candidate] (9.181 s) : 0, 9180985
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.093 s -
Agent iast 1.223 s 129.716 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.22 s 126.952 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.222 s 128.583 ms (11.8%)
Total tracing 8.687 s -
Total iast 9.218 s 530.876 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.17 s 482.812 ms (5.6%)
Total iast_TELEMETRY_OFF 9.209 s 521.332 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.102 s -
Agent iast 1.229 s 127.552 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.219 s 117.366 ms (10.7%)
Agent iast_TELEMETRY_OFF 1.218 s 116.653 ms (10.6%)
Total tracing 8.681 s -
Total iast 9.246 s 565.166 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.161 s 480.787 ms (5.5%)
Total iast_TELEMETRY_OFF 9.181 s 500.481 ms (5.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (695.996 ms) : 0, 695996
BytebuddyAgent [candidate] (700.581 ms) : 0, 700581
GlobalTracer [baseline] (318.652 ms) : 0, 318652
GlobalTracer [candidate] (320.847 ms) : 0, 320847
AppSec [baseline] (54.741 ms) : 0, 54741
AppSec [candidate] (55.137 ms) : 0, 55137
Remote Config [baseline] (695.171 µs) : 0, 695
Remote Config [candidate] (699.096 µs) : 0, 699
Telemetry [baseline] (9.268 ms) : 0, 9268
Telemetry [candidate] (10.543 ms) : 0, 10543
section iast
BytebuddyAgent [baseline] (814.306 ms) : 0, 814306
BytebuddyAgent [candidate] (818.962 ms) : 0, 818962
GlobalTracer [baseline] (306.999 ms) : 0, 306999
GlobalTracer [candidate] (308.565 ms) : 0, 308565
AppSec [baseline] (56.576 ms) : 0, 56576
AppSec [candidate] (57.492 ms) : 0, 57492
IAST [baseline] (22.922 ms) : 0, 22922
IAST [candidate] (22.003 ms) : 0, 22003
Remote Config [baseline] (635.84 µs) : 0, 636
Remote Config [candidate] (643.765 µs) : 0, 644
Telemetry [baseline] (7.602 ms) : 0, 7602
Telemetry [candidate] (7.646 ms) : 0, 7646
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (812.659 ms) : 0, 812659
BytebuddyAgent [candidate] (811.916 ms) : 0, 811916
GlobalTracer [baseline] (306.556 ms) : 0, 306556
GlobalTracer [candidate] (306.47 ms) : 0, 306470
AppSec [baseline] (57.182 ms) : 0, 57182
AppSec [candidate] (56.874 ms) : 0, 56874
IAST [baseline] (21.688 ms) : 0, 21688
IAST [candidate] (21.833 ms) : 0, 21833
Remote Config [baseline] (634.74 µs) : 0, 635
Remote Config [candidate] (633.057 µs) : 0, 633
Telemetry [baseline] (7.557 ms) : 0, 7557
Telemetry [candidate] (7.544 ms) : 0, 7544
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (812.158 ms) : 0, 812158
BytebuddyAgent [candidate] (811.091 ms) : 0, 811091
GlobalTracer [baseline] (308.368 ms) : 0, 308368
GlobalTracer [candidate] (307.132 ms) : 0, 307132
AppSec [baseline] (57.547 ms) : 0, 57547
AppSec [candidate] (57.994 ms) : 0, 57994
IAST [baseline] (21.592 ms) : 0, 21592
IAST [candidate] (20.333 ms) : 0, 20333
Remote Config [baseline] (649.227 µs) : 0, 649
Remote Config [candidate] (618.949 µs) : 0, 619
Telemetry [baseline] (7.574 ms) : 0, 7574
Telemetry [candidate] (7.396 ms) : 0, 7396
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.092 s) : 0, 1091588
Total [baseline] (10.397 s) : 0, 10397103
Agent [candidate] (1.091 s) : 0, 1091044
Total [candidate] (10.437 s) : 0, 10436999
section appsec
Agent [baseline] (1.226 s) : 0, 1225578
Total [baseline] (10.696 s) : 0, 10695812
Agent [candidate] (1.232 s) : 0, 1231563
Total [candidate] (10.774 s) : 0, 10774261
section iast
Agent [baseline] (1.232 s) : 0, 1232195
Total [baseline] (10.964 s) : 0, 10963663
Agent [candidate] (1.222 s) : 0, 1221794
Total [candidate] (10.948 s) : 0, 10947888
section profiling
Agent [baseline] (1.327 s) : 0, 1327077
Total [baseline] (10.823 s) : 0, 10822644
Agent [candidate] (1.323 s) : 0, 1323491
Total [candidate] (10.842 s) : 0, 10842483
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent appsec 1.226 s 133.99 ms (12.3%)
Agent iast 1.232 s 140.607 ms (12.9%)
Agent profiling 1.327 s 235.489 ms (21.6%)
Total tracing 10.397 s -
Total appsec 10.696 s 298.71 ms (2.9%)
Total iast 10.964 s 566.56 ms (5.4%)
Total profiling 10.823 s 425.542 ms (4.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.091 s -
Agent appsec 1.232 s 140.519 ms (12.9%)
Agent iast 1.222 s 130.75 ms (12.0%)
Agent profiling 1.323 s 232.447 ms (21.3%)
Total tracing 10.437 s -
Total appsec 10.774 s 337.262 ms (3.2%)
Total iast 10.948 s 510.889 ms (4.9%)
Total profiling 10.842 s 405.484 ms (3.9%) 
gantt
    title petclinic - break down per module: candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.934 ms) : 0, 693934
BytebuddyAgent [candidate] (695.603 ms) : 0, 695603
GlobalTracer [baseline] (317.94 ms) : 0, 317940
GlobalTracer [candidate] (318.462 ms) : 0, 318462
AppSec [baseline] (54.66 ms) : 0, 54660
AppSec [candidate] (54.772 ms) : 0, 54772
Remote Config [baseline] (698.332 µs) : 0, 698
Remote Config [candidate] (687.279 µs) : 0, 687
Telemetry [baseline] (10.626 ms) : 0, 10626
Telemetry [candidate] (7.723 ms) : 0, 7723
section appsec
BytebuddyAgent [baseline] (711.715 ms) : 0, 711715
BytebuddyAgent [candidate] (714.883 ms) : 0, 714883
GlobalTracer [baseline] (314.447 ms) : 0, 314447
GlobalTracer [candidate] (316.787 ms) : 0, 316787
AppSec [baseline] (166.222 ms) : 0, 166222
AppSec [candidate] (168.087 ms) : 0, 168087
IAST [baseline] (21.274 ms) : 0, 21274
IAST [candidate] (19.08 ms) : 0, 19080
Remote Config [baseline] (682.414 µs) : 0, 682
Remote Config [candidate] (647.797 µs) : 0, 648
Telemetry [baseline] (8.113 ms) : 0, 8113
Telemetry [candidate] (7.784 ms) : 0, 7784
section iast
BytebuddyAgent [baseline] (821.25 ms) : 0, 821250
BytebuddyAgent [candidate] (813.945 ms) : 0, 813945
GlobalTracer [baseline] (308.824 ms) : 0, 308824
GlobalTracer [candidate] (306.703 ms) : 0, 306703
AppSec [baseline] (58.655 ms) : 0, 58655
AppSec [candidate] (57.24 ms) : 0, 57240
IAST [baseline] (21.242 ms) : 0, 21242
IAST [candidate] (21.892 ms) : 0, 21892
Remote Config [baseline] (653.731 µs) : 0, 654
Remote Config [candidate] (655.436 µs) : 0, 655
Telemetry [baseline] (7.641 ms) : 0, 7641
Telemetry [candidate] (7.575 ms) : 0, 7575
section profiling
BytebuddyAgent [baseline] (694.97 ms) : 0, 694970
BytebuddyAgent [candidate] (694.03 ms) : 0, 694030
GlobalTracer [baseline] (435.543 ms) : 0, 435543
GlobalTracer [candidate] (433.933 ms) : 0, 433933
AppSec [baseline] (53.885 ms) : 0, 53885
AppSec [candidate] (53.693 ms) : 0, 53693
Remote Config [baseline] (665.271 µs) : 0, 665
Remote Config [candidate] (665.813 µs) : 0, 666
Telemetry [baseline] (7.801 ms) : 0, 7801
Telemetry [candidate] (7.694 ms) : 0, 7694
ProfilingAgent [baseline] (94.699 ms) : 0, 94699
ProfilingAgent [candidate] (94.017 ms) : 0, 94017
Profiling [baseline] (94.723 ms) : 0, 94723
Profiling [candidate] (94.041 ms) : 0, 94041
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-12-09T15:33:49 2024-12-09T15:40:47
git_branch master nikita-tkachenko/fix-npe-in-flaky-tests-list
git_commit_date 1733749565 1733757733
git_commit_sha 88c9405 e90472b
release_version 1.44.0-SNAPSHOT~88c9405c96 1.44.0-SNAPSHOT~e90472b827
start_time 2024-12-09T15:33:36 2024-12-09T15:40:33
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1733759200 1733759200
ci_job_id 731094709 731094709
ci_pipeline_id 50582628 50582628
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.355 ms) : 1335, 1375
.   : milestone, 1355,
appsec (1.736 ms) : 1710, 1761
.   : milestone, 1736,
appsec_no_iast (1.743 ms) : 1718, 1768
.   : milestone, 1743,
iast (1.503 ms) : 1480, 1526
.   : milestone, 1503,
profiling (1.497 ms) : 1473, 1520
.   : milestone, 1497,
tracing (1.494 ms) : 1470, 1518
.   : milestone, 1494,
section candidate
no_agent (1.357 ms) : 1338, 1377
.   : milestone, 1357,
appsec (1.763 ms) : 1739, 1786
.   : milestone, 1763,
appsec_no_iast (1.721 ms) : 1696, 1746
.   : milestone, 1721,
iast (1.52 ms) : 1498, 1543
.   : milestone, 1520,
profiling (1.511 ms) : 1488, 1535
.   : milestone, 1511,
tracing (1.484 ms) : 1459, 1508
.   : milestone, 1484,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.335 ms, 1.375 ms] -
appsec 1.736 ms [1.71 ms, 1.761 ms] 380.731 µs (28.1%)
appsec_no_iast 1.743 ms [1.718 ms, 1.768 ms] 388.078 µs (28.6%)
iast 1.503 ms [1.48 ms, 1.526 ms] 148.178 µs (10.9%)
profiling 1.497 ms [1.473 ms, 1.52 ms] 141.773 µs (10.5%)
tracing 1.494 ms [1.47 ms, 1.518 ms] 139.315 µs (10.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.338 ms, 1.377 ms] -
appsec 1.763 ms [1.739 ms, 1.786 ms] 405.422 µs (29.9%)
appsec_no_iast 1.721 ms [1.696 ms, 1.746 ms] 363.955 µs (26.8%)
iast 1.52 ms [1.498 ms, 1.543 ms] 163.259 µs (12.0%)
profiling 1.511 ms [1.488 ms, 1.535 ms] 154.172 µs (11.4%)
tracing 1.484 ms [1.459 ms, 1.508 ms] 126.453 µs (9.3%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.078 µs) : 353, 392
.   : milestone, 372,
iast (489.689 µs) : 468, 511
.   : milestone, 490,
iast_FULL (652.876 µs) : 631, 674
.   : milestone, 653,
iast_GLOBAL (523.693 µs) : 501, 546
.   : milestone, 524,
iast_HARDCODED_SECRET_DISABLED (494.68 µs) : 472, 517
.   : milestone, 495,
iast_INACTIVE (443.752 µs) : 423, 464
.   : milestone, 444,
iast_TELEMETRY_OFF (479.932 µs) : 458, 502
.   : milestone, 480,
tracing (451.648 µs) : 431, 473
.   : milestone, 452,
section candidate
no_agent (375.783 µs) : 356, 395
.   : milestone, 376,
iast (491.327 µs) : 470, 513
.   : milestone, 491,
iast_FULL (647.848 µs) : 626, 669
.   : milestone, 648,
iast_GLOBAL (521.789 µs) : 499, 544
.   : milestone, 522,
iast_HARDCODED_SECRET_DISABLED (496.376 µs) : 475, 518
.   : milestone, 496,
iast_INACTIVE (452.617 µs) : 432, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (476.296 µs) : 455, 498
.   : milestone, 476,
tracing (451.638 µs) : 430, 474
.   : milestone, 452,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.078 µs [352.526 µs, 391.63 µs] -
iast 489.689 µs [468.2 µs, 511.178 µs] 117.611 µs (31.6%)
iast_FULL 652.876 µs [631.354 µs, 674.398 µs] 280.798 µs (75.5%)
iast_GLOBAL 523.693 µs [501.239 µs, 546.146 µs] 151.615 µs (40.7%)
iast_HARDCODED_SECRET_DISABLED 494.68 µs [472.142 µs, 517.219 µs] 122.602 µs (33.0%)
iast_INACTIVE 443.752 µs [423.146 µs, 464.358 µs] 71.674 µs (19.3%)
iast_TELEMETRY_OFF 479.932 µs [458.319 µs, 501.546 µs] 107.855 µs (29.0%)
tracing 451.648 µs [430.746 µs, 472.551 µs] 79.571 µs (21.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.783 µs [356.146 µs, 395.419 µs] -
iast 491.327 µs [469.663 µs, 512.99 µs] 115.544 µs (30.7%)
iast_FULL 647.848 µs [626.36 µs, 669.335 µs] 272.065 µs (72.4%)
iast_GLOBAL 521.789 µs [499.405 µs, 544.173 µs] 146.006 µs (38.9%)
iast_HARDCODED_SECRET_DISABLED 496.376 µs [474.671 µs, 518.08 µs] 120.593 µs (32.1%)
iast_INACTIVE 452.617 µs [431.719 µs, 473.514 µs] 76.834 µs (20.4%)
iast_TELEMETRY_OFF 476.296 µs [455.056 µs, 497.536 µs] 100.513 µs (26.7%)
tracing 451.638 µs [429.557 µs, 473.719 µs] 75.855 µs (20.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/fix-npe-in-flaky-tests-list
git_commit_date 1733749565 1733757733
git_commit_sha 88c9405 e90472b
release_version 1.44.0-SNAPSHOT~88c9405c96 1.44.0-SNAPSHOT~e90472b827
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1733759740 1733759740
ci_job_id 731094711 731094711
ci_pipeline_id 50582628 50582628
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.356 s) : 15356000, 15356000
.   : milestone, 15356000,
appsec (15.045 s) : 15045000, 15045000
.   : milestone, 15045000,
iast (18.712 s) : 18712000, 18712000
.   : milestone, 18712000,
iast_GLOBAL (18.052 s) : 18052000, 18052000
.   : milestone, 18052000,
profiling (15.109 s) : 15109000, 15109000
.   : milestone, 15109000,
tracing (15.048 s) : 15048000, 15048000
.   : milestone, 15048000,
section candidate
no_agent (15.484 s) : 15484000, 15484000
.   : milestone, 15484000,
appsec (14.837 s) : 14837000, 14837000
.   : milestone, 14837000,
iast (19.178 s) : 19178000, 19178000
.   : milestone, 19178000,
iast_GLOBAL (17.964 s) : 17964000, 17964000
.   : milestone, 17964000,
profiling (14.97 s) : 14970000, 14970000
.   : milestone, 14970000,
tracing (15.137 s) : 15137000, 15137000
.   : milestone, 15137000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.356 s [15.356 s, 15.356 s] -
appsec 15.045 s [15.045 s, 15.045 s] -311.0 ms (-2.0%)
iast 18.712 s [18.712 s, 18.712 s] 3.356 s (21.9%)
iast_GLOBAL 18.052 s [18.052 s, 18.052 s] 2.696 s (17.6%)
profiling 15.109 s [15.109 s, 15.109 s] -247.0 ms (-1.6%)
tracing 15.048 s [15.048 s, 15.048 s] -308.0 ms (-2.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.484 s [15.484 s, 15.484 s] -
appsec 14.837 s [14.837 s, 14.837 s] -647.0 ms (-4.2%)
iast 19.178 s [19.178 s, 19.178 s] 3.694 s (23.9%)
iast_GLOBAL 17.964 s [17.964 s, 17.964 s] 2.48 s (16.0%)
profiling 14.97 s [14.97 s, 14.97 s] -514.0 ms (-3.3%)
tracing 15.137 s [15.137 s, 15.137 s] -347.0 ms (-2.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~e90472b827, baseline=1.44.0-SNAPSHOT~88c9405c96
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (2.368 ms) : 2326, 2410
.   : milestone, 2368,
iast (2.097 ms) : 2044, 2151
.   : milestone, 2097,
iast_GLOBAL (2.134 ms) : 2081, 2186
.   : milestone, 2134,
profiling (1.956 ms) : 1914, 1998
.   : milestone, 1956,
tracing (1.93 ms) : 1889, 1970
.   : milestone, 1930,
section candidate
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (2.358 ms) : 2316, 2399
.   : milestone, 2358,
iast (2.101 ms) : 2049, 2154
.   : milestone, 2101,
iast_GLOBAL (2.139 ms) : 2087, 2192
.   : milestone, 2139,
profiling (2.421 ms) : 2248, 2594
.   : milestone, 2421,
tracing (1.94 ms) : 1900, 1981
.   : milestone, 1940,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.46 ms, 1.483 ms] -
appsec 2.368 ms [2.326 ms, 2.41 ms] 896.737 µs (60.9%)
iast 2.097 ms [2.044 ms, 2.151 ms] 626.057 µs (42.5%)
iast_GLOBAL 2.134 ms [2.081 ms, 2.186 ms] 662.267 µs (45.0%)
profiling 1.956 ms [1.914 ms, 1.998 ms] 484.688 µs (32.9%)
tracing 1.93 ms [1.889 ms, 1.97 ms] 458.483 µs (31.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.479 ms [1.468 ms, 1.491 ms] -
appsec 2.358 ms [2.316 ms, 2.399 ms] 878.676 µs (59.4%)
iast 2.101 ms [2.049 ms, 2.154 ms] 621.936 µs (42.0%)
iast_GLOBAL 2.139 ms [2.087 ms, 2.192 ms] 660.195 µs (44.6%)
profiling 2.421 ms [2.248 ms, 2.594 ms] 941.489 µs (63.6%)
tracing 1.94 ms [1.9 ms, 1.981 ms] 461.192 µs (31.2%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 4524f7b into master Dec 9, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/fix-npe-in-flaky-tests-list branch December 9, 2024 16:42
@github-actions github-actions Bot added this to the 1.44.0 milestone Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@daniel-mohedano daniel-mohedano daniel-mohedano approved these changes

+1 more reviewer

@romainkomorn-exdatadog romainkomorn-exdatadog romainkomorn-exdatadog approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp: ci visibility Continuous Integration Visibility tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

1.44.0

Development

Successfully merging this pull request may close these issues.

3 participants

@nikita-tkachenko-datadog @romainkomorn-exdatadog @daniel-mohedano