Skip to content

Exclude org.mockito package from CI Visibility code coverage by default#5712

Merged
nikita-tkachenko-datadog merged 1 commit into
masterfrom
nikita-tkachenko/exclude-mockito-from-coverage
Aug 14, 2023
Merged

Exclude org.mockito package from CI Visibility code coverage by default#5712
nikita-tkachenko-datadog merged 1 commit into
masterfrom
nikita-tkachenko/exclude-mockito-from-coverage

Conversation

@nikita-tkachenko-datadog

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Aug 11, 2023

Copy link
Copy Markdown
Contributor

What Does This Do

Excludes org.mockito package from CI Visibility code coverage.
When customer uses Jacoco injected by tracer, there is a list of packages that are by default excluded from Jacoco instrumentation.
This change extends this list with org.mockito.

Motivation

Instrumenting classes in org.mockito may cause StackOverflowError - some of the tracer code that handles coverage probe activations uses certain classes (ConcurrentHashMap in particular) that are transformed by Mockito.
As the result, activating a coverage probe in a Mockito class leads to invoking a Mockito class which leads to activating a coverage probe.
This continues until method execution stack is overflown.

@nikita-tkachenko-datadog
nikita-tkachenko-datadog requested a review from a team August 11, 2023 14:54
@nikita-tkachenko-datadog nikita-tkachenko-datadog added comp: ci visibility Continuous Integration Visibility type: bug fix Bug fix labels Aug 11, 2023
@pr-commenter

pr-commenter Bot commented Aug 11, 2023

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~9e16e31acb 1.20.0-SNAPSHOT~a252ac1fd9
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~a252ac1fd9, baseline=1.20.0-SNAPSHOT~9e16e31acb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (992.859 ms) : 0, 992859
Total [baseline] (8.696 s) : 0, 8695865
Agent [candidate] (994.056 ms) : 0, 994056
Total [candidate] (8.66 s) : 0, 8660371
section appsec
Agent [baseline] (1.078 s) : 0, 1078048
Total [baseline] (8.756 s) : 0, 8756259
Agent [candidate] (1.072 s) : 0, 1072252
Total [candidate] (8.742 s) : 0, 8741721
section iast
Agent [baseline] (1.087 s) : 0, 1086753
Total [baseline] (9.194 s) : 0, 9194245
Agent [candidate] (1.101 s) : 0, 1101350
Total [candidate] (9.233 s) : 0, 9233323
section profiling
Agent [baseline] (1.155 s) : 0, 1155468
Total [baseline] (8.893 s) : 0, 8892983
Agent [candidate] (1.158 s) : 0, 1157776
Total [candidate] (8.904 s) : 0, 8903753
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 992.859 ms -
Agent appsec 1.078 s 85.189 ms (8.6%)
Agent iast 1.087 s 93.894 ms (9.5%)
Agent profiling 1.155 s 162.609 ms (16.4%)
Total tracing 8.696 s -
Total appsec 8.756 s 60.394 ms (0.7%)
Total iast 9.194 s 498.379 ms (5.7%)
Total profiling 8.893 s 197.118 ms (2.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 994.056 ms -
Agent appsec 1.072 s 78.196 ms (7.9%)
Agent iast 1.101 s 107.294 ms (10.8%)
Agent profiling 1.158 s 163.72 ms (16.5%)
Total tracing 8.66 s -
Total appsec 8.742 s 81.349 ms (0.9%)
Total iast 9.233 s 572.952 ms (6.6%)
Total profiling 8.904 s 243.382 ms (2.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~a252ac1fd9, baseline=1.20.0-SNAPSHOT~9e16e31acb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (615.107 ms) : 0, 615107
BytebuddyAgent [candidate] (616.343 ms) : 0, 616343
GlobalTracer [baseline] (286.883 ms) : 0, 286883
GlobalTracer [candidate] (286.58 ms) : 0, 286580
AppSec [baseline] (50.197 ms) : 0, 50197
AppSec [candidate] (50.329 ms) : 0, 50329
Remote Config [baseline] (673.247 µs) : 0, 673
Remote Config [candidate] (670.067 µs) : 0, 670
Telemetry [baseline] (5.466 ms) : 0, 5466
Telemetry [candidate] (5.461 ms) : 0, 5461
section appsec
BytebuddyAgent [baseline] (615.522 ms) : 0, 615522
BytebuddyAgent [candidate] (611.263 ms) : 0, 611263
GlobalTracer [baseline] (286.604 ms) : 0, 286604
GlobalTracer [candidate] (285.56 ms) : 0, 285560
AppSec [baseline] (135.503 ms) : 0, 135503
AppSec [candidate] (135.296 ms) : 0, 135296
Remote Config [baseline] (736.363 µs) : 0, 736
Remote Config [candidate] (726.051 µs) : 0, 726
Telemetry [baseline] (5.188 ms) : 0, 5188
Telemetry [candidate] (5.158 ms) : 0, 5158
section iast
BytebuddyAgent [baseline] (714.05 ms) : 0, 714050
BytebuddyAgent [candidate] (724.373 ms) : 0, 724373
GlobalTracer [baseline] (271.271 ms) : 0, 271271
GlobalTracer [candidate] (274.032 ms) : 0, 274032
AppSec [baseline] (47.051 ms) : 0, 47051
AppSec [candidate] (47.518 ms) : 0, 47518
Remote Config [baseline] (643.236 µs) : 0, 643
Remote Config [candidate] (659.122 µs) : 0, 659
Telemetry [baseline] (5.513 ms) : 0, 5513
Telemetry [candidate] (5.693 ms) : 0, 5693
IAST [baseline] (13.88 ms) : 0, 13880
IAST [candidate] (14.242 ms) : 0, 14242
section profiling
BytebuddyAgent [baseline] (622.796 ms) : 0, 622796
BytebuddyAgent [candidate] (623.954 ms) : 0, 623954
GlobalTracer [baseline] (349.399 ms) : 0, 349399
GlobalTracer [candidate] (349.739 ms) : 0, 349739
AppSec [baseline] (49.099 ms) : 0, 49099
AppSec [candidate] (49.471 ms) : 0, 49471
Remote Config [baseline] (657.79 µs) : 0, 658
Remote Config [candidate] (657.009 µs) : 0, 657
Telemetry [baseline] (5.345 ms) : 0, 5345
Telemetry [candidate] (5.409 ms) : 0, 5409
ProfilingAgent [baseline] (75.418 ms) : 0, 75418
ProfilingAgent [candidate] (75.61 ms) : 0, 75610
Profiling [baseline] (75.442 ms) : 0, 75442
Profiling [candidate] (75.635 ms) : 0, 75635
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~a252ac1fd9, baseline=1.20.0-SNAPSHOT~9e16e31acb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (984.039 ms) : 0, 984039
Total [baseline] (9.183 s) : 0, 9183460
Agent [candidate] (994.651 ms) : 0, 994651
Total [candidate] (9.211 s) : 0, 9210662
section appsec
Agent [baseline] (1.076 s) : 0, 1076419
Total [baseline] (9.313 s) : 0, 9313026
Agent [candidate] (1.079 s) : 0, 1078822
Total [candidate] (9.285 s) : 0, 9285331
section iast
Agent [baseline] (1.092 s) : 0, 1092194
Total [baseline] (9.449 s) : 0, 9449400
Agent [candidate] (1.094 s) : 0, 1094483
Total [candidate] (9.537 s) : 0, 9536634
section profiling
Agent [baseline] (1.153 s) : 0, 1153466
Total [baseline] (9.479 s) : 0, 9478968
Agent [candidate] (1.158 s) : 0, 1157615
Total [candidate] (9.481 s) : 0, 9480879
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 984.039 ms -
Agent appsec 1.076 s 92.38 ms (9.4%)
Agent iast 1.092 s 108.155 ms (11.0%)
Agent profiling 1.153 s 169.427 ms (17.2%)
Total tracing 9.183 s -
Total appsec 9.313 s 129.566 ms (1.4%)
Total iast 9.449 s 265.94 ms (2.9%)
Total profiling 9.479 s 295.508 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 994.651 ms -
Agent appsec 1.079 s 84.171 ms (8.5%)
Agent iast 1.094 s 99.832 ms (10.0%)
Agent profiling 1.158 s 162.964 ms (16.4%)
Total tracing 9.211 s -
Total appsec 9.285 s 74.669 ms (0.8%)
Total iast 9.537 s 325.973 ms (3.5%)
Total profiling 9.481 s 270.218 ms (2.9%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~a252ac1fd9, baseline=1.20.0-SNAPSHOT~9e16e31acb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (609.72 ms) : 0, 609720
BytebuddyAgent [candidate] (616.176 ms) : 0, 616176
GlobalTracer [baseline] (283.626 ms) : 0, 283626
GlobalTracer [candidate] (287.444 ms) : 0, 287444
AppSec [baseline] (50.478 ms) : 0, 50478
AppSec [candidate] (50.386 ms) : 0, 50386
Remote Config [baseline] (668.177 µs) : 0, 668
Remote Config [candidate] (673.476 µs) : 0, 673
Telemetry [baseline] (5.383 ms) : 0, 5383
Telemetry [candidate] (5.449 ms) : 0, 5449
section appsec
BytebuddyAgent [baseline] (614.139 ms) : 0, 614139
BytebuddyAgent [candidate] (615.51 ms) : 0, 615510
GlobalTracer [baseline] (285.943 ms) : 0, 285943
GlobalTracer [candidate] (287.017 ms) : 0, 287017
AppSec [baseline] (135.966 ms) : 0, 135966
AppSec [candidate] (135.943 ms) : 0, 135943
Remote Config [baseline] (731.806 µs) : 0, 732
Remote Config [candidate] (719.913 µs) : 0, 720
Telemetry [baseline] (5.17 ms) : 0, 5170
Telemetry [candidate] (5.164 ms) : 0, 5164
section iast
BytebuddyAgent [baseline] (717.927 ms) : 0, 717927
BytebuddyAgent [candidate] (718.699 ms) : 0, 718699
GlobalTracer [baseline] (272.528 ms) : 0, 272528
GlobalTracer [candidate] (273.466 ms) : 0, 273466
AppSec [baseline] (47.146 ms) : 0, 47146
AppSec [candidate] (47.601 ms) : 0, 47601
Remote Config [baseline] (651.871 µs) : 0, 652
Remote Config [candidate] (650.074 µs) : 0, 650
Telemetry [baseline] (5.42 ms) : 0, 5420
Telemetry [candidate] (5.639 ms) : 0, 5639
IAST [baseline] (13.97 ms) : 0, 13970
IAST [candidate] (14.145 ms) : 0, 14145
section profiling
BytebuddyAgent [baseline] (621.635 ms) : 0, 621635
BytebuddyAgent [candidate] (623.181 ms) : 0, 623181
GlobalTracer [baseline] (348.13 ms) : 0, 348130
GlobalTracer [candidate] (350.151 ms) : 0, 350151
AppSec [baseline] (49.054 ms) : 0, 49054
AppSec [candidate] (49.297 ms) : 0, 49297
Remote Config [baseline] (659.857 µs) : 0, 660
Remote Config [candidate] (652.737 µs) : 0, 653
Telemetry [baseline] (5.382 ms) : 0, 5382
Telemetry [candidate] (5.371 ms) : 0, 5371
ProfilingAgent [baseline] (75.93 ms) : 0, 75930
ProfilingAgent [candidate] (76.165 ms) : 0, 76165
Profiling [baseline] (75.954 ms) : 0, 75954
Profiling [candidate] (76.189 ms) : 0, 76189
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~9e16e31acb 1.20.0-SNAPSHOT~a252ac1fd9
config baseline candidate
end_time 2023-08-11T15:12:01 2023-08-11T15:30:13
start_time 2023-08-11T15:11:44 2023-08-11T15:29:55
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~a252ac1fd9, baseline=1.20.0-SNAPSHOT~9e16e31acb
    dateFormat X
    axisFormat %s
section baseline
no_agent (361.925 µs) : 342, 382
.   : milestone, 362,
appsec (668.981 µs) : 648, 690
.   : milestone, 669,
iast (458.629 µs) : 438, 479
.   : milestone, 459,
iast_FULL (528.085 µs) : 507, 549
.   : milestone, 528,
iast_INACTIVE (441.296 µs) : 420, 462
.   : milestone, 441,
profiling (439.598 µs) : 419, 461
.   : milestone, 440,
tracing (440.133 µs) : 419, 461
.   : milestone, 440,
section candidate
no_agent (367.262 µs) : 346, 389
.   : milestone, 367,
appsec (680.545 µs) : 660, 701
.   : milestone, 681,
iast (457.707 µs) : 437, 479
.   : milestone, 458,
iast_FULL (533.573 µs) : 512, 555
.   : milestone, 534,
iast_INACTIVE (444.306 µs) : 423, 465
.   : milestone, 444,
profiling (450.15 µs) : 429, 471
.   : milestone, 450,
tracing (439.47 µs) : 419, 460
.   : milestone, 439,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.925 µs [341.734 µs, 382.116 µs] -
appsec 668.981 µs [647.747 µs, 690.215 µs] 307.056 µs (84.8%)
iast 458.629 µs [438.009 µs, 479.249 µs] 96.704 µs (26.7%)
iast_FULL 528.085 µs [507.041 µs, 549.129 µs] 166.16 µs (45.9%)
iast_INACTIVE 441.296 µs [420.21 µs, 462.383 µs] 79.372 µs (21.9%)
profiling 439.598 µs [418.598 µs, 460.599 µs] 77.673 µs (21.5%)
tracing 440.133 µs [419.068 µs, 461.199 µs] 78.208 µs (21.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.262 µs [346.015 µs, 388.509 µs] -
appsec 680.545 µs [659.779 µs, 701.311 µs] 313.283 µs (85.3%)
iast 457.707 µs [436.91 µs, 478.504 µs] 90.445 µs (24.6%)
iast_FULL 533.573 µs [512.404 µs, 554.742 µs] 166.311 µs (45.3%)
iast_INACTIVE 444.306 µs [423.269 µs, 465.344 µs] 77.044 µs (21.0%)
profiling 450.15 µs [429.132 µs, 471.168 µs] 82.888 µs (22.6%)
tracing 439.47 µs [418.736 µs, 460.205 µs] 72.208 µs (19.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~a252ac1fd9, baseline=1.20.0-SNAPSHOT~9e16e31acb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.344 ms) : 1324, 1363
.   : milestone, 1344,
appsec (1.699 ms) : 1674, 1724
.   : milestone, 1699,
iast (1.473 ms) : 1449, 1497
.   : milestone, 1473,
profiling (1.481 ms) : 1455, 1508
.   : milestone, 1481,
tracing (1.457 ms) : 1433, 1481
.   : milestone, 1457,
section candidate
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.688 ms) : 1664, 1712
.   : milestone, 1688,
iast (1.472 ms) : 1448, 1497
.   : milestone, 1472,
profiling (1.469 ms) : 1444, 1495
.   : milestone, 1469,
tracing (1.43 ms) : 1405, 1454
.   : milestone, 1430,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.324 ms, 1.363 ms] -
appsec 1.699 ms [1.674 ms, 1.724 ms] 355.743 µs (26.5%)
iast 1.473 ms [1.449 ms, 1.497 ms] 129.657 µs (9.7%)
profiling 1.481 ms [1.455 ms, 1.508 ms] 137.741 µs (10.3%)
tracing 1.457 ms [1.433 ms, 1.481 ms] 113.638 µs (8.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.322 ms, 1.36 ms] -
appsec 1.688 ms [1.664 ms, 1.712 ms] 346.687 µs (25.9%)
iast 1.472 ms [1.448 ms, 1.497 ms] 131.456 µs (9.8%)
profiling 1.469 ms [1.444 ms, 1.495 ms] 128.272 µs (9.6%)
tracing 1.43 ms [1.405 ms, 1.454 ms] 88.796 µs (6.6%)

@nikita-tkachenko-datadog
nikita-tkachenko-datadog marked this pull request as ready for review August 14, 2023 12:06
@nikita-tkachenko-datadog
nikita-tkachenko-datadog requested a review from a team as a code owner August 14, 2023 12:06
@nikita-tkachenko-datadog
nikita-tkachenko-datadog merged commit 7dfca99 into master Aug 14, 2023
@nikita-tkachenko-datadog
nikita-tkachenko-datadog deleted the nikita-tkachenko/exclude-mockito-from-coverage branch August 14, 2023 12:07
@github-actions github-actions Bot added this to the 1.20.0 milestone Aug 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@smola smola smola approved these changes

Assignees

No one assigned

Labels

comp: ci visibility Continuous Integration Visibility type: bug fix Bug fix

Projects

None yet

Milestone

1.20.0

Development

Successfully merging this pull request may close these issues.

2 participants

@nikita-tkachenko-datadog @smola