Fix call-site-plugin build script to avoid rebuilding plugin every time by PerfectSlayer · Pull Request #6921 · DataDog/dd-trace-java

PerfectSlayer · 2024-04-16T11:51:56Z

What Does This Do

This PR fixes the CSI (Call Site Plugin) build script which is part of the project build script.

There are two main fixes:

Avoid rebuilding CSI plugin jar for every Gradle command

CSI plugin shadowed jar was built for every gradle command runned as part of the project configuration:

With the changes, all plugin gradle tasks are properly cached (UP-TO-DATE state):

Avoid generating call site sources file for every build

CSI plugin drops generated call site source files (from build/generated/sources/csi/) for every build, breaking the task cache and making the plugin needed to re-run.

Running gradlew :dd-java-agent:shadowJar multiple times shows CSI plugin tasks are not cached and are the slowest taks to run:

After the fix, all generateCallSiteJava generated tasks are properly cached:

The generated source files will still be cleared as part of the clean gradle task and will still be updated if the sources files (build/classes/raw) are updated.

Other changes

Upgrade deprecated API usage,
Avoid cleaning every temporary files twice

Motivation

This should help to run gradle project configuration step faster.

Additional Notes

Fixes and upgrades are split in multiple commits to help reviewing the PR.

Jira ticket: [PROJ-IDENT]

pr-commenter · 2024-04-16T12:43:09Z

Benchmarks

Startup

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-04-17T09:15:55	2024-04-17T09:22:44
git_branch	master	bbujon/csi-gradle
git_commit_date	1713338255	1713344134
git_commit_sha	`34e7066`	`54a1da4`
release_version	1.33.0-SNAPSHOT~34e706697c	1.33.0-SNAPSHOT~54a1da4b19
start_time	2024-04-17T09:15:42	2024-04-17T09:22:31

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1713346108	1713346108
ci_job_id	488876752	488876752
ci_pipeline_id	32284286	32284286
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.33.0-SNAPSHOT~54a1da4b19, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.355 ms) : 1335, 1374
.   : milestone, 1355,
appsec (1.723 ms) : 1698, 1748
.   : milestone, 1723,
appsec_no_iast (1.719 ms) : 1695, 1744
.   : milestone, 1719,
iast (1.501 ms) : 1478, 1524
.   : milestone, 1501,
profiling (1.495 ms) : 1471, 1520
.   : milestone, 1495,
tracing (1.475 ms) : 1452, 1499
.   : milestone, 1475,
section candidate
no_agent (1.353 ms) : 1334, 1373
.   : milestone, 1353,
appsec (1.736 ms) : 1712, 1760
.   : milestone, 1736,
appsec_no_iast (1.705 ms) : 1681, 1730
.   : milestone, 1705,
iast (1.49 ms) : 1466, 1513
.   : milestone, 1490,
profiling (1.491 ms) : 1466, 1516
.   : milestone, 1491,
tracing (1.479 ms) : 1455, 1504
.   : milestone, 1479,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.355 ms [1.335 ms, 1.374 ms]	-
appsec	1.723 ms [1.698 ms, 1.748 ms]	368.432 µs (27.2%)
appsec_no_iast	1.719 ms [1.695 ms, 1.744 ms]	364.896 µs (26.9%)
iast	1.501 ms [1.478 ms, 1.524 ms]	146.247 µs (10.8%)
profiling	1.495 ms [1.471 ms, 1.52 ms]	140.659 µs (10.4%)
tracing	1.475 ms [1.452 ms, 1.499 ms]	120.794 µs (8.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.353 ms [1.334 ms, 1.373 ms]	-
appsec	1.736 ms [1.712 ms, 1.76 ms]	382.506 µs (28.3%)
appsec_no_iast	1.705 ms [1.681 ms, 1.73 ms]	351.862 µs (26.0%)
iast	1.49 ms [1.466 ms, 1.513 ms]	136.258 µs (10.1%)
profiling	1.491 ms [1.466 ms, 1.516 ms]	137.499 µs (10.2%)
tracing	1.479 ms [1.455 ms, 1.504 ms]	126.086 µs (9.3%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.33.0-SNAPSHOT~54a1da4b19, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (370.71 µs) : 349, 393
.   : milestone, 371,
iast (472.058 µs) : 451, 493
.   : milestone, 472,
iast_FULL (544.425 µs) : 523, 566
.   : milestone, 544,
iast_GLOBAL (501.782 µs) : 480, 523
.   : milestone, 502,
iast_HARDCODED_SECRET_DISABLED (477.115 µs) : 456, 498
.   : milestone, 477,
iast_INACTIVE (451.399 µs) : 430, 473
.   : milestone, 451,
iast_TELEMETRY_OFF (475.27 µs) : 454, 496
.   : milestone, 475,
tracing (446.887 µs) : 426, 468
.   : milestone, 447,
section candidate
no_agent (373.262 µs) : 354, 393
.   : milestone, 373,
iast (479.477 µs) : 459, 500
.   : milestone, 479,
iast_FULL (540.047 µs) : 519, 561
.   : milestone, 540,
iast_GLOBAL (519.141 µs) : 496, 543
.   : milestone, 519,
iast_HARDCODED_SECRET_DISABLED (473.191 µs) : 453, 494
.   : milestone, 473,
iast_INACTIVE (452.753 µs) : 432, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (476.023 µs) : 455, 497
.   : milestone, 476,
tracing (444.047 µs) : 424, 465
.   : milestone, 444,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	370.71 µs [348.768 µs, 392.651 µs]	-
iast	472.058 µs [451.197 µs, 492.919 µs]	101.348 µs (27.3%)
iast_FULL	544.425 µs [523.144 µs, 565.706 µs]	173.715 µs (46.9%)
iast_GLOBAL	501.782 µs [480.303 µs, 523.261 µs]	131.072 µs (35.4%)
iast_HARDCODED_SECRET_DISABLED	477.115 µs [456.198 µs, 498.033 µs]	106.406 µs (28.7%)
iast_INACTIVE	451.399 µs [430.279 µs, 472.519 µs]	80.689 µs (21.8%)
iast_TELEMETRY_OFF	475.27 µs [454.47 µs, 496.069 µs]	104.56 µs (28.2%)
tracing	446.887 µs [426.223 µs, 467.551 µs]	76.177 µs (20.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	373.262 µs [353.855 µs, 392.668 µs]	-
iast	479.477 µs [458.607 µs, 500.346 µs]	106.215 µs (28.5%)
iast_FULL	540.047 µs [519.443 µs, 560.651 µs]	166.785 µs (44.7%)
iast_GLOBAL	519.141 µs [495.522 µs, 542.76 µs]	145.879 µs (39.1%)
iast_HARDCODED_SECRET_DISABLED	473.191 µs [452.759 µs, 493.622 µs]	99.929 µs (26.8%)
iast_INACTIVE	452.753 µs [431.639 µs, 473.868 µs]	79.492 µs (21.3%)
iast_TELEMETRY_OFF	476.023 µs [454.557 µs, 497.488 µs]	102.761 µs (27.5%)
tracing	444.047 µs [423.505 µs, 464.588 µs]	70.785 µs (19.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bbujon/csi-gradle
git_commit_date	1713338255	1713344134
git_commit_sha	`34e7066`	`54a1da4`
release_version	1.33.0-SNAPSHOT~34e706697c	1.33.0-SNAPSHOT~54a1da4b19

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1713346643	1713346643
ci_job_id	488876753	488876753
ci_pipeline_id	32284286	32284286
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.33.0-SNAPSHOT~54a1da4b19, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.457 ms) : 1445, 1468
.   : milestone, 1457,
appsec (2.186 ms) : 2153, 2220
.   : milestone, 2186,
iast (1.875 ms) : 1840, 1910
.   : milestone, 1875,
iast_GLOBAL (1.912 ms) : 1876, 1947
.   : milestone, 1912,
profiling (1.829 ms) : 1796, 1862
.   : milestone, 1829,
tracing (1.816 ms) : 1785, 1848
.   : milestone, 1816,
section candidate
no_agent (1.454 ms) : 1443, 1465
.   : milestone, 1454,
appsec (2.182 ms) : 2149, 2214
.   : milestone, 2182,
iast (1.861 ms) : 1826, 1895
.   : milestone, 1861,
iast_GLOBAL (1.909 ms) : 1875, 1944
.   : milestone, 1909,
profiling (1.851 ms) : 1816, 1885
.   : milestone, 1851,
tracing (1.823 ms) : 1791, 1856
.   : milestone, 1823,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.457 ms [1.445 ms, 1.468 ms]	-
appsec	2.186 ms [2.153 ms, 2.22 ms]	729.706 µs (50.1%)
iast	1.875 ms [1.84 ms, 1.91 ms]	417.857 µs (28.7%)
iast_GLOBAL	1.912 ms [1.876 ms, 1.947 ms]	454.988 µs (31.2%)
profiling	1.829 ms [1.796 ms, 1.862 ms]	372.148 µs (25.5%)
tracing	1.816 ms [1.785 ms, 1.848 ms]	359.656 µs (24.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.454 ms [1.443 ms, 1.465 ms]	-
appsec	2.182 ms [2.149 ms, 2.214 ms]	727.558 µs (50.0%)
iast	1.861 ms [1.826 ms, 1.895 ms]	406.642 µs (28.0%)
iast_GLOBAL	1.909 ms [1.875 ms, 1.944 ms]	455.414 µs (31.3%)
profiling	1.851 ms [1.816 ms, 1.885 ms]	396.718 µs (27.3%)
tracing	1.823 ms [1.791 ms, 1.856 ms]	369.213 µs (25.4%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.33.0-SNAPSHOT~54a1da4b19, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.091 s) : 15091000, 15091000
.   : milestone, 15091000,
appsec (14.945 s) : 14945000, 14945000
.   : milestone, 14945000,
iast (18.677 s) : 18677000, 18677000
.   : milestone, 18677000,
iast_GLOBAL (18.093 s) : 18093000, 18093000
.   : milestone, 18093000,
profiling (15.19 s) : 15190000, 15190000
.   : milestone, 15190000,
tracing (15.268 s) : 15268000, 15268000
.   : milestone, 15268000,
section candidate
no_agent (15.4 s) : 15400000, 15400000
.   : milestone, 15400000,
appsec (14.874 s) : 14874000, 14874000
.   : milestone, 14874000,
iast (18.817 s) : 18817000, 18817000
.   : milestone, 18817000,
iast_GLOBAL (17.701 s) : 17701000, 17701000
.   : milestone, 17701000,
profiling (15.55 s) : 15550000, 15550000
.   : milestone, 15550000,
tracing (14.784 s) : 14784000, 14784000
.   : milestone, 14784000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.091 s [15.091 s, 15.091 s]	-
appsec	14.945 s [14.945 s, 14.945 s]	-146.0 ms (-1.0%)
iast	18.677 s [18.677 s, 18.677 s]	3.586 s (23.8%)
iast_GLOBAL	18.093 s [18.093 s, 18.093 s]	3.002 s (19.9%)
profiling	15.19 s [15.19 s, 15.19 s]	99.0 ms (0.7%)
tracing	15.268 s [15.268 s, 15.268 s]	177.0 ms (1.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.4 s [15.4 s, 15.4 s]	-
appsec	14.874 s [14.874 s, 14.874 s]	-526.0 ms (-3.4%)
iast	18.817 s [18.817 s, 18.817 s]	3.417 s (22.2%)
iast_GLOBAL	17.701 s [17.701 s, 17.701 s]	2.301 s (14.9%)
profiling	15.55 s [15.55 s, 15.55 s]	150.0 ms (1.0%)
tracing	14.784 s [14.784 s, 14.784 s]	-616.0 ms (-4.0%)

Build folder was cleared at each task creation, breaking task caching (output was removed) It does no more clear the build/generated/sources/csi. It will be cleared as part of the `clean` gradle task. Generated files will still be updated if the sources files (build/classes/raw) are updated.

newTempFile clears existing files as withWriter does. Avoid doing twice for build performance.

manuel-alvarez-alvarez · 2024-04-17T13:55:40Z

One of the reasons why the plugin was dropping the generated source files is that under very particular circumstances (e.g. after certain incompatible changes in the call site definition), the generated sources ended up in a bad state that forced us to delete them by hand.

It's very possible that this is no longer an issue (e.g. gradle has been updated a few times) and since we don't update call sites so often nowadays I think that shouldn't be a problem any more.

Many thanks for the change @PerfectSlayer

PerfectSlayer · 2024-04-22T08:54:48Z

One of the reasons why the plugin was dropping the generated source files is that under very particular circumstances (e.g. after certain incompatible changes in the call site definition), the generated sources ended up in a bad state that forced us to delete them by hand.

I checked the following things:

If you edit any file from the instrumentation source folder, it will generate CSI files,
If you manually edited the generated sources, they are well overridden with the new fresh generated file,

The only issue I found is if you delete a source file related to CSI, you might have build issue.
But it is also required to clean the build folder to remove the related class file so it should not be an issue.

PerfectSlayer added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Apr 16, 2024

PerfectSlayer requested a review from a team as a code owner April 16, 2024 11:51

PerfectSlayer requested review from nayeem-kamal and ygree April 16, 2024 11:51

PerfectSlayer added 4 commits April 17, 2024 10:55

fix(csi): Fix build script to avoid rebuilding plugin everytime

6f6251b

feat(csi): Upgrade any deprecated call

b05dc39

fix(csi): Avoid clearing temporary file twice

54a1da4

newTempFile clears existing files as withWriter does. Avoid doing twice for build performance.

PerfectSlayer force-pushed the bbujon/csi-gradle branch 2 times, most recently from 29e125b to 54a1da4 Compare April 17, 2024 09:02

smola requested a review from manuel-alvarez-alvarez April 17, 2024 13:10

smola approved these changes Apr 17, 2024

View reviewed changes

manuel-alvarez-alvarez approved these changes Apr 17, 2024

View reviewed changes

PerfectSlayer merged commit 8c37f09 into master Apr 22, 2024

PerfectSlayer deleted the bbujon/csi-gradle branch April 22, 2024 08:55

github-actions Bot added this to the 1.34.0 milestone Apr 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix call-site-plugin build script to avoid rebuilding plugin every time#6921

Fix call-site-plugin build script to avoid rebuilding plugin every time#6921
PerfectSlayer merged 4 commits intomasterfrom
bbujon/csi-gradle

PerfectSlayer commented Apr 16, 2024 •

edited

Loading

Uh oh!

pr-commenter Bot commented Apr 16, 2024 •

edited

Loading

Uh oh!

manuel-alvarez-alvarez commented Apr 17, 2024

Uh oh!

PerfectSlayer commented Apr 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

PerfectSlayer commented Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Avoid rebuilding CSI plugin jar for every Gradle command

Avoid generating call site sources file for every build

Other changes

Motivation

Additional Notes

Uh oh!

pr-commenter Bot commented Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

manuel-alvarez-alvarez commented Apr 17, 2024

Uh oh!

PerfectSlayer commented Apr 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

PerfectSlayer commented Apr 16, 2024 •

edited

Loading

pr-commenter Bot commented Apr 16, 2024 •

edited

Loading