Skip to content

Use DD Javac Plugin metadata to resolve method lines#5746

Merged
nikita-tkachenko-datadog merged 1 commit into
masterfrom
nikita-tkachenko/compiler-aided-method-lines
Aug 24, 2023
Merged

Use DD Javac Plugin metadata to resolve method lines#5746
nikita-tkachenko-datadog merged 1 commit into
masterfrom
nikita-tkachenko/compiler-aided-method-lines

Conversation

@nikita-tkachenko-datadog

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Aug 21, 2023

Copy link
Copy Markdown
Contributor

What Does This Do

Updates CI Visibility logic that extracts lines metadata for test methods.
The update is to use @MethodLines annotation injected by dd-javac-plugin if available, and fall back to the previously used method (class bytecode traversing) if not.

Motivation

Data injected by the compiler plugin is more accurate (as it includes method declaration lines - method name, modifiers, annotations) and can be retrieved with less overhead (as getting a runtime annotation does not required reading class data from disk, which is needed for bytecode traversing).

@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Aug 21, 2023
@pr-commenter

pr-commenter Bot commented Aug 21, 2023

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~9bc95da6b5 1.20.0-SNAPSHOT~a1136e007e
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~a1136e007e, baseline=1.20.0-SNAPSHOT~9bc95da6b5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (994.767 ms) : 0, 994767
Total [baseline] (8.645 s) : 0, 8645312
Agent [candidate] (993.195 ms) : 0, 993195
Total [candidate] (8.657 s) : 0, 8656611
section appsec
Agent [baseline] (1.069 s) : 0, 1069163
Total [baseline] (8.705 s) : 0, 8704512
Agent [candidate] (1.071 s) : 0, 1071024
Total [candidate] (8.715 s) : 0, 8715047
section iast
Agent [baseline] (1.087 s) : 0, 1087113
Total [baseline] (9.152 s) : 0, 9151561
Agent [candidate] (1.097 s) : 0, 1097258
Total [candidate] (9.173 s) : 0, 9173077
section profiling
Agent [baseline] (1.156 s) : 0, 1155988
Total [baseline] (8.878 s) : 0, 8878350
Agent [candidate] (1.155 s) : 0, 1155486
Total [candidate] (8.85 s) : 0, 8850399
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 994.767 ms -
Agent appsec 1.069 s 74.396 ms (7.5%)
Agent iast 1.087 s 92.346 ms (9.3%)
Agent profiling 1.156 s 161.221 ms (16.2%)
Total tracing 8.645 s -
Total appsec 8.705 s 59.2 ms (0.7%)
Total iast 9.152 s 506.249 ms (5.9%)
Total profiling 8.878 s 233.039 ms (2.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 993.195 ms -
Agent appsec 1.071 s 77.829 ms (7.8%)
Agent iast 1.097 s 104.063 ms (10.5%)
Agent profiling 1.155 s 162.291 ms (16.3%)
Total tracing 8.657 s -
Total appsec 8.715 s 58.436 ms (0.7%)
Total iast 9.173 s 516.465 ms (6.0%)
Total profiling 8.85 s 193.788 ms (2.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~a1136e007e, baseline=1.20.0-SNAPSHOT~9bc95da6b5

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (616.656 ms) : 0, 616656
BytebuddyAgent [candidate] (615.966 ms) : 0, 615966
GlobalTracer [baseline] (286.603 ms) : 0, 286603
GlobalTracer [candidate] (286.072 ms) : 0, 286072
AppSec [baseline] (50.183 ms) : 0, 50183
AppSec [candidate] (49.898 ms) : 0, 49898
Remote Config [baseline] (674.183 µs) : 0, 674
Remote Config [candidate] (675.063 µs) : 0, 675
Telemetry [baseline] (6.069 ms) : 0, 6069
Telemetry [candidate] (6.012 ms) : 0, 6012
section appsec
BytebuddyAgent [baseline] (609.97 ms) : 0, 609970
BytebuddyAgent [candidate] (610.547 ms) : 0, 610547
GlobalTracer [baseline] (283.364 ms) : 0, 283364
GlobalTracer [candidate] (284.315 ms) : 0, 284315
AppSec [baseline] (135.138 ms) : 0, 135138
AppSec [candidate] (135.493 ms) : 0, 135493
Remote Config [baseline] (725.624 µs) : 0, 726
Remote Config [candidate] (708.951 µs) : 0, 709
Telemetry [baseline] (5.753 ms) : 0, 5753
Telemetry [candidate] (5.761 ms) : 0, 5761
section iast
BytebuddyAgent [baseline] (715.35 ms) : 0, 715350
BytebuddyAgent [candidate] (722.42 ms) : 0, 722420
GlobalTracer [baseline] (269.845 ms) : 0, 269845
GlobalTracer [candidate] (272.415 ms) : 0, 272415
AppSec [baseline] (47.346 ms) : 0, 47346
AppSec [candidate] (47.439 ms) : 0, 47439
Remote Config [baseline] (557.424 µs) : 0, 557
Remote Config [candidate] (567.672 µs) : 0, 568
Telemetry [baseline] (5.712 ms) : 0, 5712
Telemetry [candidate] (5.74 ms) : 0, 5740
IAST [baseline] (13.895 ms) : 0, 13895
IAST [candidate] (14.017 ms) : 0, 14017
section profiling
BytebuddyAgent [baseline] (622.428 ms) : 0, 622428
BytebuddyAgent [candidate] (621.394 ms) : 0, 621394
GlobalTracer [baseline] (349.183 ms) : 0, 349183
GlobalTracer [candidate] (348.829 ms) : 0, 348829
AppSec [baseline] (49.278 ms) : 0, 49278
AppSec [candidate] (49.362 ms) : 0, 49362
Remote Config [baseline] (646.062 µs) : 0, 646
Remote Config [candidate] (635.851 µs) : 0, 636
Telemetry [baseline] (5.955 ms) : 0, 5955
Telemetry [candidate] (5.931 ms) : 0, 5931
ProfilingAgent [baseline] (75.542 ms) : 0, 75542
ProfilingAgent [candidate] (76.702 ms) : 0, 76702
Profiling [baseline] (75.566 ms) : 0, 75566
Profiling [candidate] (76.726 ms) : 0, 76726
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~a1136e007e, baseline=1.20.0-SNAPSHOT~9bc95da6b5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (985.764 ms) : 0, 985764
Total [baseline] (9.125 s) : 0, 9124529
Agent [candidate] (992.334 ms) : 0, 992334
Total [candidate] (9.148 s) : 0, 9147664
section appsec
Agent [baseline] (1.078 s) : 0, 1077685
Total [baseline] (9.28 s) : 0, 9280223
Agent [candidate] (1.078 s) : 0, 1077512
Total [candidate] (9.293 s) : 0, 9292922
section iast
Agent [baseline] (1.087 s) : 0, 1087345
Total [baseline] (9.423 s) : 0, 9422667
Agent [candidate] (1.097 s) : 0, 1096936
Total [candidate] (9.491 s) : 0, 9491327
section profiling
Agent [baseline] (1.157 s) : 0, 1157302
Total [baseline] (9.441 s) : 0, 9441199
Agent [candidate] (1.158 s) : 0, 1157857
Total [candidate] (9.494 s) : 0, 9493542
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 985.764 ms -
Agent appsec 1.078 s 91.921 ms (9.3%)
Agent iast 1.087 s 101.58 ms (10.3%)
Agent profiling 1.157 s 171.537 ms (17.4%)
Total tracing 9.125 s -
Total appsec 9.28 s 155.694 ms (1.7%)
Total iast 9.423 s 298.138 ms (3.3%)
Total profiling 9.441 s 316.67 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 992.334 ms -
Agent appsec 1.078 s 85.178 ms (8.6%)
Agent iast 1.097 s 104.602 ms (10.5%)
Agent profiling 1.158 s 165.523 ms (16.7%)
Total tracing 9.148 s -
Total appsec 9.293 s 145.258 ms (1.6%)
Total iast 9.491 s 343.663 ms (3.8%)
Total profiling 9.494 s 345.878 ms (3.8%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~a1136e007e, baseline=1.20.0-SNAPSHOT~9bc95da6b5

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (611.465 ms) : 0, 611465
BytebuddyAgent [candidate] (614.79 ms) : 0, 614790
GlobalTracer [baseline] (283.502 ms) : 0, 283502
GlobalTracer [candidate] (286.145 ms) : 0, 286145
AppSec [baseline] (49.809 ms) : 0, 49809
AppSec [candidate] (50.217 ms) : 0, 50217
Remote Config [baseline] (661.118 µs) : 0, 661
Remote Config [candidate] (667.162 µs) : 0, 667
Telemetry [baseline] (5.985 ms) : 0, 5985
Telemetry [candidate] (6.003 ms) : 0, 6003
section appsec
BytebuddyAgent [baseline] (615.026 ms) : 0, 615026
BytebuddyAgent [candidate] (614.808 ms) : 0, 614808
GlobalTracer [baseline] (285.95 ms) : 0, 285950
GlobalTracer [candidate] (286.256 ms) : 0, 286256
AppSec [baseline] (135.649 ms) : 0, 135649
AppSec [candidate] (135.42 ms) : 0, 135420
Remote Config [baseline] (721.665 µs) : 0, 722
Remote Config [candidate] (721.566 µs) : 0, 722
Telemetry [baseline] (5.805 ms) : 0, 5805
Telemetry [candidate] (5.807 ms) : 0, 5807
section iast
BytebuddyAgent [baseline] (715.902 ms) : 0, 715902
BytebuddyAgent [candidate] (722.165 ms) : 0, 722165
GlobalTracer [baseline] (269.837 ms) : 0, 269837
GlobalTracer [candidate] (272.334 ms) : 0, 272334
AppSec [baseline] (47.123 ms) : 0, 47123
AppSec [candidate] (47.567 ms) : 0, 47567
Remote Config [baseline] (548.986 µs) : 0, 549
Remote Config [candidate] (557.048 µs) : 0, 557
Telemetry [baseline] (5.722 ms) : 0, 5722
Telemetry [candidate] (5.758 ms) : 0, 5758
IAST [baseline] (13.824 ms) : 0, 13824
IAST [candidate] (13.995 ms) : 0, 13995
section profiling
BytebuddyAgent [baseline] (622.715 ms) : 0, 622715
BytebuddyAgent [candidate] (624.131 ms) : 0, 624131
GlobalTracer [baseline] (348.69 ms) : 0, 348690
GlobalTracer [candidate] (349.211 ms) : 0, 349211
AppSec [baseline] (49.244 ms) : 0, 49244
AppSec [candidate] (48.943 ms) : 0, 48943
Remote Config [baseline] (648.668 µs) : 0, 649
Remote Config [candidate] (655.496 µs) : 0, 655
Telemetry [baseline] (6.038 ms) : 0, 6038
Telemetry [candidate] (5.956 ms) : 0, 5956
ProfilingAgent [baseline] (76.924 ms) : 0, 76924
ProfilingAgent [candidate] (76.152 ms) : 0, 76152
Profiling [baseline] (76.949 ms) : 0, 76949
Profiling [candidate] (76.176 ms) : 0, 76176
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~9bc95da6b5 1.20.0-SNAPSHOT~a1136e007e
config baseline candidate
end_time 2023-08-21T14:09:11 2023-08-21T14:27:14
start_time 2023-08-21T14:08:53 2023-08-21T14:26:57
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~a1136e007e, baseline=1.20.0-SNAPSHOT~9bc95da6b5
    dateFormat X
    axisFormat %s
section baseline
no_agent (361.282 µs) : 341, 382
.   : milestone, 361,
appsec (653.806 µs) : 633, 674
.   : milestone, 654,
iast (452.128 µs) : 431, 473
.   : milestone, 452,
iast_FULL (520.27 µs) : 500, 541
.   : milestone, 520,
iast_INACTIVE (432.289 µs) : 411, 453
.   : milestone, 432,
profiling (434.202 µs) : 414, 455
.   : milestone, 434,
tracing (429.98 µs) : 410, 450
.   : milestone, 430,
section candidate
no_agent (358.841 µs) : 338, 379
.   : milestone, 359,
appsec (655.564 µs) : 635, 676
.   : milestone, 656,
iast (455.134 µs) : 434, 476
.   : milestone, 455,
iast_FULL (515.697 µs) : 495, 536
.   : milestone, 516,
iast_INACTIVE (434.209 µs) : 413, 455
.   : milestone, 434,
profiling (434.767 µs) : 414, 456
.   : milestone, 435,
tracing (433.707 µs) : 413, 454
.   : milestone, 434,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.282 µs [340.561 µs, 382.003 µs] -
appsec 653.806 µs [633.186 µs, 674.427 µs] 292.524 µs (81.0%)
iast 452.128 µs [431.287 µs, 472.97 µs] 90.846 µs (25.1%)
iast_FULL 520.27 µs [499.723 µs, 540.817 µs] 158.988 µs (44.0%)
iast_INACTIVE 432.289 µs [411.25 µs, 453.327 µs] 71.007 µs (19.7%)
profiling 434.202 µs [413.692 µs, 454.713 µs] 72.921 µs (20.2%)
tracing 429.98 µs [409.724 µs, 450.237 µs] 68.698 µs (19.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 358.841 µs [338.449 µs, 379.232 µs] -
appsec 655.564 µs [634.834 µs, 676.293 µs] 296.723 µs (82.7%)
iast 455.134 µs [434.472 µs, 475.795 µs] 96.293 µs (26.8%)
iast_FULL 515.697 µs [495.175 µs, 536.218 µs] 156.856 µs (43.7%)
iast_INACTIVE 434.209 µs [413.484 µs, 454.933 µs] 75.368 µs (21.0%)
profiling 434.767 µs [413.981 µs, 455.554 µs] 75.927 µs (21.2%)
tracing 433.707 µs [413.1 µs, 454.314 µs] 74.866 µs (20.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~a1136e007e, baseline=1.20.0-SNAPSHOT~9bc95da6b5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.313 ms) : 1294, 1332
.   : milestone, 1313,
appsec (1.666 ms) : 1641, 1690
.   : milestone, 1666,
iast (1.455 ms) : 1431, 1479
.   : milestone, 1455,
profiling (1.479 ms) : 1454, 1505
.   : milestone, 1479,
tracing (1.421 ms) : 1397, 1445
.   : milestone, 1421,
section candidate
no_agent (1.323 ms) : 1304, 1342
.   : milestone, 1323,
appsec (1.649 ms) : 1625, 1673
.   : milestone, 1649,
iast (1.448 ms) : 1423, 1472
.   : milestone, 1448,
profiling (1.455 ms) : 1431, 1480
.   : milestone, 1455,
tracing (1.448 ms) : 1424, 1472
.   : milestone, 1448,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.313 ms [1.294 ms, 1.332 ms] -
appsec 1.666 ms [1.641 ms, 1.69 ms] 352.176 µs (26.8%)
iast 1.455 ms [1.431 ms, 1.479 ms] 141.423 µs (10.8%)
profiling 1.479 ms [1.454 ms, 1.505 ms] 165.968 µs (12.6%)
tracing 1.421 ms [1.397 ms, 1.445 ms] 107.588 µs (8.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.323 ms [1.304 ms, 1.342 ms] -
appsec 1.649 ms [1.625 ms, 1.673 ms] 325.988 µs (24.6%)
iast 1.448 ms [1.423 ms, 1.472 ms] 124.558 µs (9.4%)
profiling 1.455 ms [1.431 ms, 1.48 ms] 132.356 µs (10.0%)
tracing 1.448 ms [1.424 ms, 1.472 ms] 125.138 µs (9.5%)

@nikita-tkachenko-datadog
nikita-tkachenko-datadog marked this pull request as ready for review August 21, 2023 14:55
@nikita-tkachenko-datadog
nikita-tkachenko-datadog merged commit 199de1b into master Aug 24, 2023
@nikita-tkachenko-datadog
nikita-tkachenko-datadog deleted the nikita-tkachenko/compiler-aided-method-lines branch August 24, 2023 14:32
@github-actions github-actions Bot added this to the 1.20.0 milestone Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@juan-fernandez juan-fernandez juan-fernandez approved these changes

Assignees

No one assigned

Labels

comp: ci visibility Continuous Integration Visibility

Projects

None yet

Milestone

1.20.0

Development

Successfully merging this pull request may close these issues.

2 participants

@nikita-tkachenko-datadog @juan-fernandez