Skip to content

Support ordering of InstrumenterModules#6840

Merged
mcculls merged 1 commit into
masterfrom
mcculls/support-instrumenter-module-ordering
Mar 27, 2024
Merged

Support ordering of InstrumenterModules#6840
mcculls merged 1 commit into
masterfrom
mcculls/support-instrumenter-module-ordering

Conversation

@mcculls

@mcculls mcculls commented Mar 25, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Motivation

Allows modules to be applied in a particular order, rather than just the order they appear in the buiild.

Note: this ordering is applied at build-time before generating the instrumenter.index used at runtime.

Jira ticket: AIT-9441

@mcculls mcculls added the comp: core Tracer core label Mar 25, 2024
@mcculls
mcculls force-pushed the mcculls/support-instrumenter-module-ordering branch from 5715b62 to 63dfdf5 Compare March 25, 2024 12:27
@mcculls
mcculls marked this pull request as ready for review March 25, 2024 12:30
@mcculls
mcculls requested a review from a team as a code owner March 25, 2024 12:30
@pr-commenter

pr-commenter Bot commented Mar 25, 2024 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/support-instrumenter-module-ordering
git_commit_date 1711569848 1711569901
git_commit_sha 1a0ea42 e367ec9
release_version 1.32.0-SNAPSHOT~1a0ea421c6 1.32.0-SNAPSHOT~e367ec93f4
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1711573267 1711573267
ci_job_id 471481802 471481802
ci_pipeline_id 30964942 30964942
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~e367ec93f4, baseline=1.32.0-SNAPSHOT~1a0ea421c6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.074 s) : 0, 1073755
Total [baseline] (10.344 s) : 0, 10344458
Agent [candidate] (1.077 s) : 0, 1076958
Total [candidate] (10.476 s) : 0, 10475693
section appsec
Agent [baseline] (1.197 s) : 0, 1196924
Total [baseline] (10.6 s) : 0, 10600310
Agent [candidate] (1.195 s) : 0, 1194892
Total [candidate] (10.565 s) : 0, 10564512
section iast
Agent [baseline] (1.216 s) : 0, 1215762
Total [baseline] (10.829 s) : 0, 10828770
Agent [candidate] (1.199 s) : 0, 1198804
Total [candidate] (10.821 s) : 0, 10821244
section profiling
Agent [baseline] (1.277 s) : 0, 1276888
Total [baseline] (10.72 s) : 0, 10719655
Agent [candidate] (1.278 s) : 0, 1278316
Total [candidate] (10.694 s) : 0, 10694396
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.074 s -
Agent appsec 1.197 s 123.169 ms (11.5%)
Agent iast 1.216 s 142.006 ms (13.2%)
Agent profiling 1.277 s 203.133 ms (18.9%)
Total tracing 10.344 s -
Total appsec 10.6 s 255.852 ms (2.5%)
Total iast 10.829 s 484.312 ms (4.7%)
Total profiling 10.72 s 375.197 ms (3.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent appsec 1.195 s 117.934 ms (11.0%)
Agent iast 1.199 s 121.846 ms (11.3%)
Agent profiling 1.278 s 201.358 ms (18.7%)
Total tracing 10.476 s -
Total appsec 10.565 s 88.819 ms (0.8%)
Total iast 10.821 s 345.55 ms (3.3%)
Total profiling 10.694 s 218.702 ms (2.1%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~e367ec93f4, baseline=1.32.0-SNAPSHOT~1a0ea421c6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.649 ms) : 0, 684649
BytebuddyAgent [candidate] (685.895 ms) : 0, 685895
GlobalTracer [baseline] (296.791 ms) : 0, 296791
GlobalTracer [candidate] (298.489 ms) : 0, 298489
AppSec [baseline] (49.751 ms) : 0, 49751
AppSec [candidate] (49.734 ms) : 0, 49734
Remote Config [baseline] (668.528 µs) : 0, 669
Remote Config [candidate] (671.263 µs) : 0, 671
Telemetry [baseline] (7.621 ms) : 0, 7621
Telemetry [candidate] (7.657 ms) : 0, 7657
section appsec
BytebuddyAgent [baseline] (694.732 ms) : 0, 694732
BytebuddyAgent [candidate] (693.436 ms) : 0, 693436
GlobalTracer [baseline] (291.224 ms) : 0, 291224
GlobalTracer [candidate] (291.636 ms) : 0, 291636
AppSec [baseline] (149.781 ms) : 0, 149781
AppSec [candidate] (149.389 ms) : 0, 149389
IAST [baseline] (18.768 ms) : 0, 18768
IAST [candidate] (18.772 ms) : 0, 18772
Remote Config [baseline] (605.789 µs) : 0, 606
Remote Config [candidate] (606.709 µs) : 0, 607
Telemetry [baseline] (7.381 ms) : 0, 7381
Telemetry [candidate] (6.764 ms) : 0, 6764
section iast
BytebuddyAgent [baseline] (805.961 ms) : 0, 805961
BytebuddyAgent [candidate] (794.227 ms) : 0, 794227
GlobalTracer [baseline] (292.16 ms) : 0, 292160
GlobalTracer [candidate] (288.988 ms) : 0, 288988
AppSec [baseline] (52.402 ms) : 0, 52402
AppSec [candidate] (49.709 ms) : 0, 49709
IAST [baseline] (23.216 ms) : 0, 23216
IAST [candidate] (23.614 ms) : 0, 23614
Remote Config [baseline] (583.462 µs) : 0, 583
Remote Config [candidate] (586.342 µs) : 0, 586
Telemetry [baseline] (6.685 ms) : 0, 6685
Telemetry [candidate] (7.366 ms) : 0, 7366
section profiling
BytebuddyAgent [baseline] (681.876 ms) : 0, 681876
BytebuddyAgent [candidate] (682.362 ms) : 0, 682362
GlobalTracer [baseline] (382.794 ms) : 0, 382794
GlobalTracer [candidate] (384.031 ms) : 0, 384031
AppSec [baseline] (50.758 ms) : 0, 50758
AppSec [candidate] (50.564 ms) : 0, 50564
Remote Config [baseline] (717.719 µs) : 0, 718
Remote Config [candidate] (711.717 µs) : 0, 712
Telemetry [baseline] (7.55 ms) : 0, 7550
Telemetry [candidate] (7.518 ms) : 0, 7518
ProfilingAgent [baseline] (96.524 ms) : 0, 96524
ProfilingAgent [candidate] (96.48 ms) : 0, 96480
Profiling [baseline] (96.548 ms) : 0, 96548
Profiling [candidate] (96.505 ms) : 0, 96505
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~e367ec93f4, baseline=1.32.0-SNAPSHOT~1a0ea421c6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.09 s) : 0, 1090073
Total [baseline] (8.613 s) : 0, 8612646
Agent [candidate] (1.075 s) : 0, 1075420
Total [candidate] (8.591 s) : 0, 8591358
section iast
Agent [baseline] (1.207 s) : 0, 1207038
Total [baseline] (9.108 s) : 0, 9107701
Agent [candidate] (1.208 s) : 0, 1208466
Total [candidate] (9.102 s) : 0, 9102010
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.197 s) : 0, 1197307
Total [baseline] (9.039 s) : 0, 9039412
Agent [candidate] (1.202 s) : 0, 1201872
Total [candidate] (9.084 s) : 0, 9083962
section iast_TELEMETRY_OFF
Agent [baseline] (1.203 s) : 0, 1203459
Total [baseline] (9.117 s) : 0, 9117479
Agent [candidate] (1.205 s) : 0, 1205078
Total [candidate] (9.12 s) : 0, 9119900
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.09 s -
Agent iast 1.207 s 116.965 ms (10.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.197 s 107.234 ms (9.8%)
Agent iast_TELEMETRY_OFF 1.203 s 113.387 ms (10.4%)
Total tracing 8.613 s -
Total iast 9.108 s 495.054 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.039 s 426.765 ms (5.0%)
Total iast_TELEMETRY_OFF 9.117 s 504.832 ms (5.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent iast 1.208 s 133.046 ms (12.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.202 s 126.452 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.205 s 129.658 ms (12.1%)
Total tracing 8.591 s -
Total iast 9.102 s 510.651 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.084 s 492.604 ms (5.7%)
Total iast_TELEMETRY_OFF 9.12 s 528.541 ms (6.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~e367ec93f4, baseline=1.32.0-SNAPSHOT~1a0ea421c6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (695.392 ms) : 0, 695392
BytebuddyAgent [candidate] (685.258 ms) : 0, 685258
GlobalTracer [baseline] (301.087 ms) : 0, 301087
GlobalTracer [candidate] (298.046 ms) : 0, 298046
AppSec [baseline] (50.295 ms) : 0, 50295
AppSec [candidate] (49.578 ms) : 0, 49578
Remote Config [baseline] (683.303 µs) : 0, 683
Remote Config [candidate] (653.183 µs) : 0, 653
Telemetry [baseline] (7.724 ms) : 0, 7724
Telemetry [candidate] (7.567 ms) : 0, 7567
section iast
BytebuddyAgent [baseline] (800.07 ms) : 0, 800070
BytebuddyAgent [candidate] (800.689 ms) : 0, 800689
GlobalTracer [baseline] (290.0 ms) : 0, 290000
GlobalTracer [candidate] (290.829 ms) : 0, 290829
AppSec [baseline] (51.778 ms) : 0, 51778
AppSec [candidate] (52.043 ms) : 0, 52043
IAST [baseline] (22.522 ms) : 0, 22522
IAST [candidate] (23.012 ms) : 0, 23012
Remote Config [baseline] (589.431 µs) : 0, 589
Remote Config [candidate] (571.972 µs) : 0, 572
Telemetry [baseline] (7.469 ms) : 0, 7469
Telemetry [candidate] (6.675 ms) : 0, 6675
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (794.577 ms) : 0, 794577
BytebuddyAgent [candidate] (795.154 ms) : 0, 795154
GlobalTracer [baseline] (288.16 ms) : 0, 288160
GlobalTracer [candidate] (289.814 ms) : 0, 289814
AppSec [baseline] (49.324 ms) : 0, 49324
AppSec [candidate] (50.435 ms) : 0, 50435
IAST [baseline] (23.668 ms) : 0, 23668
IAST [candidate] (23.943 ms) : 0, 23943
Remote Config [baseline] (588.655 µs) : 0, 589
Remote Config [candidate] (576.912 µs) : 0, 577
Telemetry [baseline] (6.621 ms) : 0, 6621
Telemetry [candidate] (7.361 ms) : 0, 7361
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (796.644 ms) : 0, 796644
BytebuddyAgent [candidate] (798.558 ms) : 0, 798558
GlobalTracer [baseline] (290.126 ms) : 0, 290126
GlobalTracer [candidate] (290.429 ms) : 0, 290429
AppSec [baseline] (48.386 ms) : 0, 48386
AppSec [candidate] (49.197 ms) : 0, 49197
IAST [baseline] (26.533 ms) : 0, 26533
IAST [candidate] (24.928 ms) : 0, 24928
Remote Config [baseline] (584.689 µs) : 0, 585
Remote Config [candidate] (583.862 µs) : 0, 584
Telemetry [baseline] (6.534 ms) : 0, 6534
Telemetry [candidate] (6.578 ms) : 0, 6578
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-03-27T20:33:16 2024-03-27T20:55:21
git_branch master mcculls/support-instrumenter-module-ordering
git_commit_date 1711569848 1711569901
git_commit_sha 1a0ea42 e367ec9
release_version 1.32.0-SNAPSHOT~1a0ea421c6 1.32.0-SNAPSHOT~e367ec93f4
start_time 2024-03-27T20:33:03 2024-03-27T20:55:08
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1711573267 1711573267
ci_job_id 471481802 471481802
ci_pipeline_id 30964942 30964942
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~e367ec93f4, baseline=1.32.0-SNAPSHOT~1a0ea421c6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.35 ms) : 1331, 1370
.   : milestone, 1350,
appsec (1.735 ms) : 1711, 1759
.   : milestone, 1735,
appsec_no_iast (1.714 ms) : 1690, 1739
.   : milestone, 1714,
iast (1.491 ms) : 1468, 1513
.   : milestone, 1491,
profiling (1.508 ms) : 1483, 1533
.   : milestone, 1508,
tracing (1.469 ms) : 1444, 1494
.   : milestone, 1469,
section candidate
no_agent (1.344 ms) : 1324, 1365
.   : milestone, 1344,
appsec (1.719 ms) : 1694, 1745
.   : milestone, 1719,
appsec_no_iast (1.71 ms) : 1685, 1735
.   : milestone, 1710,
iast (1.489 ms) : 1466, 1512
.   : milestone, 1489,
profiling (1.543 ms) : 1518, 1569
.   : milestone, 1543,
tracing (1.466 ms) : 1441, 1490
.   : milestone, 1466,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.35 ms [1.331 ms, 1.37 ms] -
appsec 1.735 ms [1.711 ms, 1.759 ms] 384.409 µs (28.5%)
appsec_no_iast 1.714 ms [1.69 ms, 1.739 ms] 363.988 µs (27.0%)
iast 1.491 ms [1.468 ms, 1.513 ms] 140.424 µs (10.4%)
profiling 1.508 ms [1.483 ms, 1.533 ms] 157.417 µs (11.7%)
tracing 1.469 ms [1.444 ms, 1.494 ms] 118.763 µs (8.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.324 ms, 1.365 ms] -
appsec 1.719 ms [1.694 ms, 1.745 ms] 374.996 µs (27.9%)
appsec_no_iast 1.71 ms [1.685 ms, 1.735 ms] 365.502 µs (27.2%)
iast 1.489 ms [1.466 ms, 1.512 ms] 144.524 µs (10.8%)
profiling 1.543 ms [1.518 ms, 1.569 ms] 198.945 µs (14.8%)
tracing 1.466 ms [1.441 ms, 1.49 ms] 121.173 µs (9.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~e367ec93f4, baseline=1.32.0-SNAPSHOT~1a0ea421c6
    dateFormat X
    axisFormat %s
section baseline
no_agent (368.414 µs) : 348, 389
.   : milestone, 368,
iast (473.886 µs) : 453, 494
.   : milestone, 474,
iast_FULL (542.939 µs) : 521, 565
.   : milestone, 543,
iast_GLOBAL (491.829 µs) : 471, 513
.   : milestone, 492,
iast_HARDCODED_SECRET_DISABLED (476.671 µs) : 456, 498
.   : milestone, 477,
iast_INACTIVE (447.617 µs) : 427, 468
.   : milestone, 448,
iast_TELEMETRY_OFF (473.426 µs) : 452, 494
.   : milestone, 473,
tracing (437.206 µs) : 417, 458
.   : milestone, 437,
section candidate
no_agent (369.498 µs) : 349, 390
.   : milestone, 369,
iast (468.945 µs) : 448, 490
.   : milestone, 469,
iast_FULL (538.526 µs) : 518, 559
.   : milestone, 539,
iast_GLOBAL (496.228 µs) : 476, 517
.   : milestone, 496,
iast_HARDCODED_SECRET_DISABLED (473.575 µs) : 453, 494
.   : milestone, 474,
iast_INACTIVE (449.196 µs) : 428, 471
.   : milestone, 449,
iast_TELEMETRY_OFF (470.263 µs) : 449, 491
.   : milestone, 470,
tracing (445.681 µs) : 425, 467
.   : milestone, 446,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 368.414 µs [348.076 µs, 388.752 µs] -
iast 473.886 µs [453.349 µs, 494.423 µs] 105.472 µs (28.6%)
iast_FULL 542.939 µs [521.344 µs, 564.534 µs] 174.525 µs (47.4%)
iast_GLOBAL 491.829 µs [470.852 µs, 512.805 µs] 123.415 µs (33.5%)
iast_HARDCODED_SECRET_DISABLED 476.671 µs [455.745 µs, 497.597 µs] 108.257 µs (29.4%)
iast_INACTIVE 447.617 µs [426.813 µs, 468.421 µs] 79.203 µs (21.5%)
iast_TELEMETRY_OFF 473.426 µs [452.381 µs, 494.472 µs] 105.012 µs (28.5%)
tracing 437.206 µs [416.75 µs, 457.662 µs] 68.792 µs (18.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.498 µs [348.969 µs, 390.027 µs] -
iast 468.945 µs [448.389 µs, 489.5 µs] 99.447 µs (26.9%)
iast_FULL 538.526 µs [517.954 µs, 559.098 µs] 169.028 µs (45.7%)
iast_GLOBAL 496.228 µs [475.772 µs, 516.684 µs] 126.73 µs (34.3%)
iast_HARDCODED_SECRET_DISABLED 473.575 µs [453.076 µs, 494.074 µs] 104.077 µs (28.2%)
iast_INACTIVE 449.196 µs [427.771 µs, 470.62 µs] 79.698 µs (21.6%)
iast_TELEMETRY_OFF 470.263 µs [449.118 µs, 491.407 µs] 100.765 µs (27.3%)
tracing 445.681 µs [424.561 µs, 466.801 µs] 76.183 µs (20.6%)

dougqh
dougqh approved these changes Mar 27, 2024
View reviewed changes

@dougqh dougqh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I presume this is coming from OpenTelemetry.
I worried about the brittleness of such a solution. I would kind of prefer more explicit dependencies.

But if we need to support this, we to support it.

@mcculls

mcculls commented Mar 27, 2024

Copy link
Copy Markdown
Contributor Author

I presume this is coming from OpenTelemetry.

Yes this is required to align with OpenTelemetry

Base automatically changed from mcculls/generate-instrumenter-index to master March 27, 2024 20:04
@mcculls
mcculls force-pushed the mcculls/support-instrumenter-module-ordering branch from 63dfdf5 to e367ec9 Compare March 27, 2024 20:05
@mcculls
mcculls merged commit 722243d into master Mar 27, 2024
@mcculls
mcculls deleted the mcculls/support-instrumenter-module-ordering branch March 27, 2024 21:06
@github-actions github-actions Bot added this to the 1.32.0 milestone Mar 27, 2024
@PerfectSlayer PerfectSlayer added the type: feature Enhancements and improvements label Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dougqh dougqh dougqh approved these changes
@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java
@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer

Assignees

No one assigned

Labels

comp: core Tracer core type: feature Enhancements and improvements

Projects

None yet

Milestone

1.32.0

Development

Successfully merging this pull request may close these issues.

3 participants

@mcculls @dougqh @PerfectSlayer