Skip to content

Ensure superclasses and interfaces are loaded before classes that extend/implement them#6677

Merged
mcculls merged 1 commit intomasterfrom
mcculls/predefine-type-hierarchies
Feb 17, 2024
Merged

Ensure superclasses and interfaces are loaded before classes that extend/implement them#6677
mcculls merged 1 commit intomasterfrom
mcculls/predefine-type-hierarchies

Conversation

@mcculls
Copy link
Copy Markdown
Contributor

@mcculls mcculls commented Feb 13, 2024

What Does This Do

Updates j.l.ClassLoader to notify the tracer before defining a class. The tracer can then attempt to preload the superclass and any interfaces, which will make sure those types are memoized/cached before the initial type is matched.

To disable this optimization add this JVM option:

-Ddd.integration.defineclass.enabled=false

or set this environment variable:

DD_INTEGRATION_DEFINECLASS_ENABLED=false

Motivation

For spring-petclinic (when memoization is enabled) this reduces the number of class-file lookups by over a third.

(from 1970 lookups to 1226 lookups)

Note on Benchmarking Data

Benchmarks indicate that there is a minor startup regression in this PR, but this is untrue because the Datadog benchmarking platform measures the time between phases of tracer installation, not application startup as a whole.

What actually happened here is we are now instrumenting an additional method from j.l.ClassLoader, which incurs a non-zero cost, but it has the benefit of optimizing how we use various type related caches. (Specifically around re-using memoized results from the type hierarchy.) This means that instrumented applications as a whole will startup faster, but they might spend a few milliseconds extra installing the agent. Those additional milliseconds are more than recouped over the life of the application by savings when matching application types.

Jira ticket: APMS-11443

@mcculls mcculls added comp: core Tracer core tag: performance Performance related changes labels Feb 13, 2024
@mcculls mcculls force-pushed the mcculls/predefine-type-hierarchies branch from 9d7dace to c17b1eb Compare February 13, 2024 15:20
@mcculls mcculls marked this pull request as ready for review February 13, 2024 15:32
@mcculls mcculls requested a review from a team as a code owner February 13, 2024 15:32
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 13, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/predefine-type-hierarchies
git_commit_date 1708089621 1708133631
git_commit_sha aee3ca5 db11295
release_version 1.31.0-SNAPSHOT~aee3ca59c6 1.31.0-SNAPSHOT~db1129529b
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1708180123 1708180123
ci_job_id 436753249 436753249
ci_pipeline_id 28469244 28469244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 42 metrics, 9 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:AppSec worse
[+5.208ms; +9.234ms] or [+10.379%; +18.402%]
57.400ms 50.179ms
scenario:startup:petclinic:iast:AppSec worse
[+2.155ms; +7.015ms] or [+4.310%; +14.029%]
54.592ms 50.007ms
scenario:startup:petclinic:profiling:BytebuddyAgent worse
[+17.111ms; +21.957ms] or [+2.595%; +3.330%]
678.917ms 659.383ms

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-02-17T14:05:07 2024-02-17T14:23:44
git_branch master mcculls/predefine-type-hierarchies
git_commit_date 1708089621 1708133631
git_commit_sha aee3ca5 db11295
release_version 1.31.0-SNAPSHOT~aee3ca59c6 1.31.0-SNAPSHOT~db1129529b
start_time 2024-02-17T14:04:53 2024-02-17T14:23:31
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1708180123 1708180123
ci_job_id 436753249 436753249
ci_pipeline_id 28469244 28469244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 15 unstable metrics.

Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~db1129529b, baseline=1.31.0-SNAPSHOT~aee3ca59c6
    dateFormat X
    axisFormat %s
section baseline
no_agent (367.04 µs) : 347, 387
.   : milestone, 367,
iast (477.188 µs) : 457, 498
.   : milestone, 477,
iast_FULL (532.938 µs) : 513, 553
.   : milestone, 533,
iast_GLOBAL (499.922 µs) : 480, 520
.   : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (465.464 µs) : 445, 486
.   : milestone, 465,
iast_INACTIVE (450.4 µs) : 429, 472
.   : milestone, 450,
iast_TELEMETRY_OFF (473.792 µs) : 453, 495
.   : milestone, 474,
tracing (452.805 µs) : 432, 474
.   : milestone, 453,
section candidate
no_agent (360.342 µs) : 341, 380
.   : milestone, 360,
iast (472.448 µs) : 452, 493
.   : milestone, 472,
iast_FULL (538.814 µs) : 518, 559
.   : milestone, 539,
iast_GLOBAL (497.757 µs) : 476, 520
.   : milestone, 498,
iast_HARDCODED_SECRET_DISABLED (468.158 µs) : 447, 489
.   : milestone, 468,
iast_INACTIVE (440.098 µs) : 420, 460
.   : milestone, 440,
iast_TELEMETRY_OFF (471.325 µs) : 450, 492
.   : milestone, 471,
tracing (447.451 µs) : 427, 468
.   : milestone, 447,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.04 µs [347.017 µs, 387.063 µs] -
iast 477.188 µs [456.667 µs, 497.708 µs] 110.148 µs (30.0%)
iast_FULL 532.938 µs [512.635 µs, 553.24 µs] 165.897 µs (45.2%)
iast_GLOBAL 499.922 µs [479.681 µs, 520.162 µs] 132.881 µs (36.2%)
iast_HARDCODED_SECRET_DISABLED 465.464 µs [445.382 µs, 485.546 µs] 98.424 µs (26.8%)
iast_INACTIVE 450.4 µs [428.819 µs, 471.981 µs] 83.36 µs (22.7%)
iast_TELEMETRY_OFF 473.792 µs [453.001 µs, 494.584 µs] 106.752 µs (29.1%)
tracing 452.805 µs [431.907 µs, 473.703 µs] 85.765 µs (23.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.342 µs [340.663 µs, 380.022 µs] -
iast 472.448 µs [451.914 µs, 492.982 µs] 112.106 µs (31.1%)
iast_FULL 538.814 µs [518.313 µs, 559.315 µs] 178.472 µs (49.5%)
iast_GLOBAL 497.757 µs [475.903 µs, 519.611 µs] 137.414 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED 468.158 µs [447.447 µs, 488.87 µs] 107.816 µs (29.9%)
iast_INACTIVE 440.098 µs [419.742 µs, 460.454 µs] 79.756 µs (22.1%)
iast_TELEMETRY_OFF 471.325 µs [450.306 µs, 492.343 µs] 110.982 µs (30.8%)
tracing 447.451 µs [426.946 µs, 467.955 µs] 87.108 µs (24.2%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~db1129529b, baseline=1.31.0-SNAPSHOT~aee3ca59c6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.774 ms) : 1750, 1798
.   : milestone, 1774,
iast (1.537 ms) : 1513, 1560
.   : milestone, 1537,
profiling (1.535 ms) : 1511, 1560
.   : milestone, 1535,
tracing (1.517 ms) : 1493, 1541
.   : milestone, 1517,
section candidate
no_agent (1.338 ms) : 1319, 1357
.   : milestone, 1338,
appsec (1.77 ms) : 1746, 1795
.   : milestone, 1770,
iast (1.539 ms) : 1515, 1562
.   : milestone, 1539,
profiling (1.512 ms) : 1487, 1536
.   : milestone, 1512,
tracing (1.509 ms) : 1486, 1533
.   : milestone, 1509,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.774 ms [1.75 ms, 1.798 ms] 429.305 µs (31.9%)
iast 1.537 ms [1.513 ms, 1.56 ms] 192.014 µs (14.3%)
profiling 1.535 ms [1.511 ms, 1.56 ms] 190.737 µs (14.2%)
tracing 1.517 ms [1.493 ms, 1.541 ms] 171.918 µs (12.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.338 ms [1.319 ms, 1.357 ms] -
appsec 1.77 ms [1.746 ms, 1.795 ms] 432.457 µs (32.3%)
iast 1.539 ms [1.515 ms, 1.562 ms] 200.674 µs (15.0%)
profiling 1.512 ms [1.487 ms, 1.536 ms] 173.623 µs (13.0%)
tracing 1.509 ms [1.486 ms, 1.533 ms] 171.24 µs (12.8%)

Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition! 👍

@mcculls mcculls force-pushed the mcculls/predefine-type-hierarchies branch 5 times, most recently from 40143a2 to 65a03e7 Compare February 17, 2024 01:18
…ded before classes that extend/implement them.
@mcculls mcculls force-pushed the mcculls/predefine-type-hierarchies branch from 65a03e7 to db11295 Compare February 17, 2024 01:34
@mcculls mcculls merged commit 1c33411 into master Feb 17, 2024
@mcculls mcculls deleted the mcculls/predefine-type-hierarchies branch February 17, 2024 19:22
@github-actions github-actions Bot added this to the 1.31.0 milestone Feb 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: performance Performance related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants