Skip to content

Do not report code coverage for skipped tests#7244

Merged
nikita-tkachenko-datadog merged 2 commits into
masterfrom
nikita-tkachenko/do-not-report-cov-for-skipped-tests
Jun 26, 2024
Merged

Do not report code coverage for skipped tests#7244
nikita-tkachenko-datadog merged 2 commits into
masterfrom
nikita-tkachenko/do-not-report-cov-for-skipped-tests

Conversation

@nikita-tkachenko-datadog

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Jun 26, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

What Does This Do

Updates code coverage processing logic so that code coverage is not processed/reported for skipped tests.

Motivation

If a test is skipped, code coverage is likely to be empty or incomplete. The backend has no use for this coverage, at least as of now, so we can save some resources by not processing it and not reporting it.

Jira ticket: SDTEST-493

@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Jun 26, 2024
@pr-commenter

pr-commenter Bot commented Jun 26, 2024 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/do-not-report-cov-for-skipped-tests
git_commit_date 1719416347 1719417351
git_commit_sha b6b3c7f d1925c8
release_version 1.36.0-SNAPSHOT~b6b3c7f32f 1.36.0-SNAPSHOT~d1925c8358
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1719420300 1719420300
ci_job_id 553806753 553806753
ci_pipeline_id 37595820 37595820
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081211
Total [baseline] (10.451 s) : 0, 10451109
Agent [candidate] (1.072 s) : 0, 1071863
Total [candidate] (10.322 s) : 0, 10321606
section appsec
Agent [baseline] (1.181 s) : 0, 1180840
Total [baseline] (10.516 s) : 0, 10516399
Agent [candidate] (1.181 s) : 0, 1181371
Total [candidate] (10.492 s) : 0, 10492378
section iast
Agent [baseline] (1.173 s) : 0, 1172693
Total [baseline] (10.813 s) : 0, 10813056
Agent [candidate] (1.175 s) : 0, 1174510
Total [candidate] (10.674 s) : 0, 10673731
section profiling
Agent [baseline] (1.266 s) : 0, 1266033
Total [baseline] (10.635 s) : 0, 10634842
Agent [candidate] (1.282 s) : 0, 1282404
Total [candidate] (10.626 s) : 0, 10626458
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent appsec 1.181 s 99.629 ms (9.2%)
Agent iast 1.173 s 91.483 ms (8.5%)
Agent profiling 1.266 s 184.823 ms (17.1%)
Total tracing 10.451 s -
Total appsec 10.516 s 65.29 ms (0.6%)
Total iast 10.813 s 361.947 ms (3.5%)
Total profiling 10.635 s 183.733 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.072 s -
Agent appsec 1.181 s 109.508 ms (10.2%)
Agent iast 1.175 s 102.647 ms (9.6%)
Agent profiling 1.282 s 210.542 ms (19.6%)
Total tracing 10.322 s -
Total appsec 10.492 s 170.772 ms (1.7%)
Total iast 10.674 s 352.125 ms (3.4%)
Total profiling 10.626 s 304.851 ms (3.0%) 
gantt
    title petclinic - break down per module: candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (675.978 ms) : 0, 675978
BytebuddyAgent [candidate] (670.989 ms) : 0, 670989
GlobalTracer [baseline] (310.72 ms) : 0, 310720
GlobalTracer [candidate] (307.446 ms) : 0, 307446
AppSec [baseline] (51.133 ms) : 0, 51133
AppSec [candidate] (50.301 ms) : 0, 50301
Remote Config [baseline] (738.139 µs) : 0, 738
Remote Config [candidate] (782.826 µs) : 0, 783
Telemetry [baseline] (7.757 ms) : 0, 7757
Telemetry [candidate] (7.644 ms) : 0, 7644
section appsec
BytebuddyAgent [baseline] (675.877 ms) : 0, 675877
BytebuddyAgent [candidate] (675.098 ms) : 0, 675098
GlobalTracer [baseline] (298.799 ms) : 0, 298799
GlobalTracer [candidate] (298.588 ms) : 0, 298588
AppSec [baseline] (153.748 ms) : 0, 153748
AppSec [candidate] (153.659 ms) : 0, 153659
Remote Config [baseline] (643.398 µs) : 0, 643
Remote Config [candidate] (641.002 µs) : 0, 641
Telemetry [baseline] (7.458 ms) : 0, 7458
Telemetry [candidate] (8.444 ms) : 0, 8444
IAST [baseline] (19.765 ms) : 0, 19765
IAST [candidate] (21.072 ms) : 0, 21072
section iast
BytebuddyAgent [baseline] (781.368 ms) : 0, 781368
BytebuddyAgent [candidate] (782.455 ms) : 0, 782455
GlobalTracer [baseline] (294.884 ms) : 0, 294884
GlobalTracer [candidate] (295.53 ms) : 0, 295530
AppSec [baseline] (47.266 ms) : 0, 47266
AppSec [candidate] (47.263 ms) : 0, 47263
Remote Config [baseline] (625.947 µs) : 0, 626
Remote Config [candidate] (569.377 µs) : 0, 569
Telemetry [baseline] (7.0 ms) : 0, 7000
Telemetry [candidate] (6.915 ms) : 0, 6915
IAST [baseline] (28.237 ms) : 0, 28237
IAST [candidate] (28.428 ms) : 0, 28428
section profiling
ProfilingAgent [baseline] (96.769 ms) : 0, 96769
ProfilingAgent [candidate] (98.487 ms) : 0, 98487
BytebuddyAgent [baseline] (664.217 ms) : 0, 664217
BytebuddyAgent [candidate] (672.745 ms) : 0, 672745
GlobalTracer [baseline] (388.256 ms) : 0, 388256
GlobalTracer [candidate] (392.509 ms) : 0, 392509
AppSec [baseline] (51.567 ms) : 0, 51567
AppSec [candidate] (52.61 ms) : 0, 52610
Remote Config [baseline] (664.853 µs) : 0, 665
Remote Config [candidate] (689.051 µs) : 0, 689
Telemetry [baseline] (7.441 ms) : 0, 7441
Telemetry [candidate] (7.554 ms) : 0, 7554
Profiling [baseline] (96.794 ms) : 0, 96794
Profiling [candidate] (98.512 ms) : 0, 98512
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1070812
Total [baseline] (8.576 s) : 0, 8576070
Agent [candidate] (1.063 s) : 0, 1063090
Total [candidate] (8.55 s) : 0, 8550273
section iast
Agent [baseline] (1.169 s) : 0, 1169230
Total [baseline] (9.004 s) : 0, 9003529
Agent [candidate] (1.172 s) : 0, 1171546
Total [candidate] (9.018 s) : 0, 9018295
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.171 s) : 0, 1170675
Total [baseline] (8.988 s) : 0, 8987504
Agent [candidate] (1.172 s) : 0, 1172173
Total [candidate] (8.974 s) : 0, 8974213
section iast_TELEMETRY_OFF
Agent [baseline] (1.166 s) : 0, 1166148
Total [baseline] (8.988 s) : 0, 8988013
Agent [candidate] (1.167 s) : 0, 1166645
Total [candidate] (8.969 s) : 0, 8968850
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.071 s -
Agent iast 1.169 s 98.418 ms (9.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 99.863 ms (9.3%)
Agent iast_TELEMETRY_OFF 1.166 s 95.336 ms (8.9%)
Total tracing 8.576 s -
Total iast 9.004 s 427.458 ms (5.0%)
Total iast_HARDCODED_SECRET_DISABLED 8.988 s 411.434 ms (4.8%)
Total iast_TELEMETRY_OFF 8.988 s 411.942 ms (4.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.172 s 108.456 ms (10.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 109.083 ms (10.3%)
Agent iast_TELEMETRY_OFF 1.167 s 103.555 ms (9.7%)
Total tracing 8.55 s -
Total iast 9.018 s 468.022 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 8.974 s 423.939 ms (5.0%)
Total iast_TELEMETRY_OFF 8.969 s 418.577 ms (4.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.329 ms) : 0, 670329
BytebuddyAgent [candidate] (665.227 ms) : 0, 665227
GlobalTracer [baseline] (306.812 ms) : 0, 306812
GlobalTracer [candidate] (305.304 ms) : 0, 305304
AppSec [baseline] (50.64 ms) : 0, 50640
AppSec [candidate] (49.9 ms) : 0, 49900
Remote Config [baseline] (751.537 µs) : 0, 752
Remote Config [candidate] (802.54 µs) : 0, 803
Telemetry [baseline] (7.698 ms) : 0, 7698
Telemetry [candidate] (7.473 ms) : 0, 7473
section iast
BytebuddyAgent [baseline] (779.417 ms) : 0, 779417
BytebuddyAgent [candidate] (780.944 ms) : 0, 780944
GlobalTracer [baseline] (294.12 ms) : 0, 294120
GlobalTracer [candidate] (294.258 ms) : 0, 294258
AppSec [baseline] (47.219 ms) : 0, 47219
AppSec [candidate] (47.144 ms) : 0, 47144
IAST [baseline] (26.82 ms) : 0, 26820
IAST [candidate] (27.469 ms) : 0, 27469
Remote Config [baseline] (593.268 µs) : 0, 593
Remote Config [candidate] (617.316 µs) : 0, 617
Telemetry [baseline] (7.78 ms) : 0, 7780
Telemetry [candidate] (7.812 ms) : 0, 7812
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (779.123 ms) : 0, 779123
BytebuddyAgent [candidate] (779.527 ms) : 0, 779527
GlobalTracer [baseline] (294.841 ms) : 0, 294841
GlobalTracer [candidate] (294.773 ms) : 0, 294773
AppSec [baseline] (47.195 ms) : 0, 47195
AppSec [candidate] (47.364 ms) : 0, 47364
IAST [baseline] (28.754 ms) : 0, 28754
IAST [candidate] (28.849 ms) : 0, 28849
Remote Config [baseline] (567.813 µs) : 0, 568
Remote Config [candidate] (618.444 µs) : 0, 618
Telemetry [baseline] (6.871 ms) : 0, 6871
Telemetry [candidate] (7.659 ms) : 0, 7659
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (776.852 ms) : 0, 776852
BytebuddyAgent [candidate] (777.524 ms) : 0, 777524
GlobalTracer [baseline] (294.442 ms) : 0, 294442
GlobalTracer [candidate] (294.74 ms) : 0, 294740
AppSec [baseline] (47.102 ms) : 0, 47102
AppSec [candidate] (47.01 ms) : 0, 47010
IAST [baseline] (26.235 ms) : 0, 26235
IAST [candidate] (25.876 ms) : 0, 25876
Remote Config [baseline] (591.277 µs) : 0, 591
Remote Config [candidate] (588.736 µs) : 0, 589
Telemetry [baseline] (7.593 ms) : 0, 7593
Telemetry [candidate] (7.536 ms) : 0, 7536
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-06-26T16:15:54 2024-06-26T16:22:45
git_branch master nikita-tkachenko/do-not-report-cov-for-skipped-tests
git_commit_date 1719416347 1719417351
git_commit_sha b6b3c7f d1925c8
release_version 1.36.0-SNAPSHOT~b6b3c7f32f 1.36.0-SNAPSHOT~d1925c8358
start_time 2024-06-26T16:15:41 2024-06-26T16:22:31
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1719419309 1719419309
ci_job_id 553806754 553806754
ci_pipeline_id 37595820 37595820
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.338 ms) : 1319, 1357
.   : milestone, 1338,
appsec (1.734 ms) : 1710, 1758
.   : milestone, 1734,
appsec_no_iast (1.707 ms) : 1682, 1733
.   : milestone, 1707,
iast (1.476 ms) : 1453, 1499
.   : milestone, 1476,
profiling (1.492 ms) : 1467, 1516
.   : milestone, 1492,
tracing (1.456 ms) : 1431, 1481
.   : milestone, 1456,
section candidate
no_agent (1.332 ms) : 1313, 1351
.   : milestone, 1332,
appsec (1.719 ms) : 1696, 1742
.   : milestone, 1719,
appsec_no_iast (1.715 ms) : 1689, 1740
.   : milestone, 1715,
iast (1.5 ms) : 1477, 1522
.   : milestone, 1500,
profiling (1.474 ms) : 1449, 1498
.   : milestone, 1474,
tracing (1.447 ms) : 1422, 1471
.   : milestone, 1447,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.338 ms [1.319 ms, 1.357 ms] -
appsec 1.734 ms [1.71 ms, 1.758 ms] 395.891 µs (29.6%)
appsec_no_iast 1.707 ms [1.682 ms, 1.733 ms] 369.363 µs (27.6%)
iast 1.476 ms [1.453 ms, 1.499 ms] 138.035 µs (10.3%)
profiling 1.492 ms [1.467 ms, 1.516 ms] 153.894 µs (11.5%)
tracing 1.456 ms [1.431 ms, 1.481 ms] 118.016 µs (8.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.332 ms [1.313 ms, 1.351 ms] -
appsec 1.719 ms [1.696 ms, 1.742 ms] 386.665 µs (29.0%)
appsec_no_iast 1.715 ms [1.689 ms, 1.74 ms] 382.623 µs (28.7%)
iast 1.5 ms [1.477 ms, 1.522 ms] 167.568 µs (12.6%)
profiling 1.474 ms [1.449 ms, 1.498 ms] 141.828 µs (10.6%)
tracing 1.447 ms [1.422 ms, 1.471 ms] 114.66 µs (8.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f
    dateFormat X
    axisFormat %s
section baseline
no_agent (371.334 µs) : 352, 391
.   : milestone, 371,
iast (493.27 µs) : 471, 515
.   : milestone, 493,
iast_FULL (550.271 µs) : 529, 571
.   : milestone, 550,
iast_GLOBAL (512.895 µs) : 490, 535
.   : milestone, 513,
iast_HARDCODED_SECRET_DISABLED (481.038 µs) : 460, 502
.   : milestone, 481,
iast_INACTIVE (455.156 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (477.522 µs) : 456, 499
.   : milestone, 478,
tracing (439.562 µs) : 419, 460
.   : milestone, 440,
section candidate
no_agent (367.297 µs) : 348, 387
.   : milestone, 367,
iast (480.524 µs) : 459, 502
.   : milestone, 481,
iast_FULL (548.818 µs) : 528, 570
.   : milestone, 549,
iast_GLOBAL (509.09 µs) : 487, 531
.   : milestone, 509,
iast_HARDCODED_SECRET_DISABLED (493.436 µs) : 472, 515
.   : milestone, 493,
iast_INACTIVE (458.649 µs) : 437, 481
.   : milestone, 459,
iast_TELEMETRY_OFF (475.187 µs) : 454, 497
.   : milestone, 475,
tracing (442.852 µs) : 422, 463
.   : milestone, 443,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.334 µs [351.717 µs, 390.951 µs] -
iast 493.27 µs [471.272 µs, 515.269 µs] 121.936 µs (32.8%)
iast_FULL 550.271 µs [529.206 µs, 571.335 µs] 178.937 µs (48.2%)
iast_GLOBAL 512.895 µs [490.428 µs, 535.362 µs] 141.561 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED 481.038 µs [459.822 µs, 502.254 µs] 109.704 µs (29.5%)
iast_INACTIVE 455.156 µs [434.016 µs, 476.297 µs] 83.822 µs (22.6%)
iast_TELEMETRY_OFF 477.522 µs [455.733 µs, 499.311 µs] 106.188 µs (28.6%)
tracing 439.562 µs [419.008 µs, 460.116 µs] 68.228 µs (18.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.297 µs [347.86 µs, 386.733 µs] -
iast 480.524 µs [459.311 µs, 501.737 µs] 113.227 µs (30.8%)
iast_FULL 548.818 µs [527.591 µs, 570.045 µs] 181.521 µs (49.4%)
iast_GLOBAL 509.09 µs [487.361 µs, 530.82 µs] 141.794 µs (38.6%)
iast_HARDCODED_SECRET_DISABLED 493.436 µs [472.228 µs, 514.644 µs] 126.139 µs (34.3%)
iast_INACTIVE 458.649 µs [436.512 µs, 480.786 µs] 91.352 µs (24.9%)
iast_TELEMETRY_OFF 475.187 µs [453.635 µs, 496.739 µs] 107.89 µs (29.4%)
tracing 442.852 µs [422.332 µs, 463.372 µs] 75.555 µs (20.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/do-not-report-cov-for-skipped-tests
git_commit_date 1719416347 1719417351
git_commit_sha b6b3c7f d1925c8
release_version 1.36.0-SNAPSHOT~b6b3c7f32f 1.36.0-SNAPSHOT~d1925c8358
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1719419810 1719419810
ci_job_id 553806755 553806755
ci_pipeline_id 37595820 37595820
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.358 s) : 15358000, 15358000
.   : milestone, 15358000,
appsec (15.392 s) : 15392000, 15392000
.   : milestone, 15392000,
iast (18.581 s) : 18581000, 18581000
.   : milestone, 18581000,
iast_GLOBAL (17.768 s) : 17768000, 17768000
.   : milestone, 17768000,
profiling (15.777 s) : 15777000, 15777000
.   : milestone, 15777000,
tracing (14.93 s) : 14930000, 14930000
.   : milestone, 14930000,
section candidate
no_agent (15.522 s) : 15522000, 15522000
.   : milestone, 15522000,
appsec (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
iast (19.069 s) : 19069000, 19069000
.   : milestone, 19069000,
iast_GLOBAL (17.688 s) : 17688000, 17688000
.   : milestone, 17688000,
profiling (15.306 s) : 15306000, 15306000
.   : milestone, 15306000,
tracing (14.99 s) : 14990000, 14990000
.   : milestone, 14990000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.358 s [15.358 s, 15.358 s] -
appsec 15.392 s [15.392 s, 15.392 s] 34.0 ms (0.2%)
iast 18.581 s [18.581 s, 18.581 s] 3.223 s (21.0%)
iast_GLOBAL 17.768 s [17.768 s, 17.768 s] 2.41 s (15.7%)
profiling 15.777 s [15.777 s, 15.777 s] 419.0 ms (2.7%)
tracing 14.93 s [14.93 s, 14.93 s] -428.0 ms (-2.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.522 s [15.522 s, 15.522 s] -
appsec 14.893 s [14.893 s, 14.893 s] -629.0 ms (-4.1%)
iast 19.069 s [19.069 s, 19.069 s] 3.547 s (22.9%)
iast_GLOBAL 17.688 s [17.688 s, 17.688 s] 2.166 s (14.0%)
profiling 15.306 s [15.306 s, 15.306 s] -216.0 ms (-1.4%)
tracing 14.99 s [14.99 s, 14.99 s] -532.0 ms (-3.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.36.0-SNAPSHOT~d1925c8358, baseline=1.36.0-SNAPSHOT~b6b3c7f32f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.215 ms) : 2181, 2249
.   : milestone, 2215,
iast (1.973 ms) : 1932, 2015
.   : milestone, 1973,
iast_GLOBAL (2.013 ms) : 1972, 2054
.   : milestone, 2013,
profiling (1.874 ms) : 1840, 1908
.   : milestone, 1874,
tracing (1.843 ms) : 1811, 1875
.   : milestone, 1843,
section candidate
no_agent (1.469 ms) : 1457, 1480
.   : milestone, 1469,
appsec (2.225 ms) : 2191, 2260
.   : milestone, 2225,
iast (1.98 ms) : 1939, 2021
.   : milestone, 1980,
iast_GLOBAL (2.01 ms) : 1969, 2050
.   : milestone, 2010,
profiling (1.868 ms) : 1834, 1901
.   : milestone, 1868,
tracing (1.84 ms) : 1807, 1873
.   : milestone, 1840,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.215 ms [2.181 ms, 2.249 ms] 745.449 µs (50.7%)
iast 1.973 ms [1.932 ms, 2.015 ms] 503.993 µs (34.3%)
iast_GLOBAL 2.013 ms [1.972 ms, 2.054 ms] 543.215 µs (37.0%)
profiling 1.874 ms [1.84 ms, 1.908 ms] 404.243 µs (27.5%)
tracing 1.843 ms [1.811 ms, 1.875 ms] 373.279 µs (25.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.457 ms, 1.48 ms] -
appsec 2.225 ms [2.191 ms, 2.26 ms] 756.404 µs (51.5%)
iast 1.98 ms [1.939 ms, 2.021 ms] 511.36 µs (34.8%)
iast_GLOBAL 2.01 ms [1.969 ms, 2.05 ms] 540.816 µs (36.8%)
profiling 1.868 ms [1.834 ms, 1.901 ms] 398.97 µs (27.2%)
tracing 1.84 ms [1.807 ms, 1.873 ms] 371.202 µs (25.3%)

@nikita-tkachenko-datadog
nikita-tkachenko-datadog marked this pull request as ready for review June 26, 2024 14:53
@nikita-tkachenko-datadog
nikita-tkachenko-datadog requested a review from a team as a code owner June 26, 2024 14:53
anmarchenko
anmarchenko reviewed Jun 26, 2024
View reviewed changes
Comment thread dd-java-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/domain/TestImpl.java
metricCollector.add(CiVisibilityCountMetric.CODE_COVERAGE_IS_EMPTY, 1);

// do not process coverage reports for skipped tests
if (span.getTag(Tags.TEST_STATUS) != TestStatus.skip) {

@anmarchenko anmarchenko Jun 26, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a test for that?

@nikita-tkachenko-datadog nikita-tkachenko-datadog Jun 26, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, that makes sense, added

@nikita-tkachenko-datadog
nikita-tkachenko-datadog merged commit 2f63f03 into master Jun 26, 2024
@nikita-tkachenko-datadog
nikita-tkachenko-datadog deleted the nikita-tkachenko/do-not-report-cov-for-skipped-tests branch June 26, 2024 20:43
@github-actions github-actions Bot added this to the 1.36.0 milestone Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anmarchenko anmarchenko anmarchenko approved these changes

Assignees

No one assigned

Labels

comp: ci visibility Continuous Integration Visibility

Projects

None yet

Milestone

1.36.0

Development

Successfully merging this pull request may close these issues.

2 participants

@nikita-tkachenko-datadog @anmarchenko