Skip to content

Separate config for Elasticsearch body and params#5771

Merged
bm1549 merged 1 commit into
masterfrom
bm1549/elastic-body-default
Aug 24, 2023
Merged

Separate config for Elasticsearch body and params#5771
bm1549 merged 1 commit into
masterfrom
bm1549/elastic-body-default

Conversation

@bm1549

@bm1549 bm1549 commented Aug 23, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

What Does This Do

Expands upon #5749 to separate the configuration options for body and params so that they can be individually selected

After this, the usage of dd.trace.elasticsearch.body-and-params.enabled will be discouraged in favor of the individual options, dd.trace.elasticsearch.body.enabled and dd.trace.elasticsearch.params.enabled

Motivation

Gives users of the library increased flexibility when it comes to configuring their elasticsearch instrumentation

Additional Notes

@bm1549
bm1549 requested a review from a team as a code owner August 23, 2023 21:51
@bm1549
bm1549 requested a review from mcculls August 23, 2023 21:51
@bm1549 bm1549 mentioned this pull request Aug 23, 2023
Merged
4 tasks
@pr-commenter

pr-commenter Bot commented Aug 23, 2023 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~6d21c3ecf6 1.20.0-SNAPSHOT~35c46da67b
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~35c46da67b, baseline=1.20.0-SNAPSHOT~6d21c3ecf6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (987.279 ms) : 0, 987279
Total [baseline] (9.154 s) : 0, 9153911
Agent [candidate] (989.82 ms) : 0, 989820
Total [candidate] (9.242 s) : 0, 9241996
section appsec
Agent [baseline] (1.071 s) : 0, 1070640
Total [baseline] (9.225 s) : 0, 9224608
Agent [candidate] (1.081 s) : 0, 1081212
Total [candidate] (9.246 s) : 0, 9246298
section iast
Agent [baseline] (1.09 s) : 0, 1089576
Total [baseline] (9.425 s) : 0, 9425384
Agent [candidate] (1.09 s) : 0, 1089921
Total [candidate] (9.389 s) : 0, 9388912
section profiling
Agent [baseline] (1.158 s) : 0, 1157769
Total [baseline] (9.412 s) : 0, 9412069
Agent [candidate] (1.158 s) : 0, 1158291
Total [candidate] (9.508 s) : 0, 9508117
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 987.279 ms -
Agent appsec 1.071 s 83.361 ms (8.4%)
Agent iast 1.09 s 102.297 ms (10.4%)
Agent profiling 1.158 s 170.49 ms (17.3%)
Total tracing 9.154 s -
Total appsec 9.225 s 70.698 ms (0.8%)
Total iast 9.425 s 271.473 ms (3.0%)
Total profiling 9.412 s 258.159 ms (2.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 989.82 ms -
Agent appsec 1.081 s 91.393 ms (9.2%)
Agent iast 1.09 s 100.102 ms (10.1%)
Agent profiling 1.158 s 168.471 ms (17.0%)
Total tracing 9.242 s -
Total appsec 9.246 s 4.302 ms (0.0%)
Total iast 9.389 s 146.916 ms (1.6%)
Total profiling 9.508 s 266.121 ms (2.9%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~35c46da67b, baseline=1.20.0-SNAPSHOT~6d21c3ecf6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (611.981 ms) : 0, 611981
BytebuddyAgent [candidate] (613.75 ms) : 0, 613750
GlobalTracer [baseline] (283.946 ms) : 0, 283946
GlobalTracer [candidate] (285.102 ms) : 0, 285102
AppSec [baseline] (50.318 ms) : 0, 50318
AppSec [candidate] (49.932 ms) : 0, 49932
Remote Config [baseline] (675.727 µs) : 0, 676
Remote Config [candidate] (665.844 µs) : 0, 666
Telemetry [baseline] (6.095 ms) : 0, 6095
Telemetry [candidate] (5.977 ms) : 0, 5977
section appsec
BytebuddyAgent [baseline] (611.539 ms) : 0, 611539
BytebuddyAgent [candidate] (617.633 ms) : 0, 617633
GlobalTracer [baseline] (283.273 ms) : 0, 283273
GlobalTracer [candidate] (287.41 ms) : 0, 287410
AppSec [baseline] (135.05 ms) : 0, 135050
AppSec [candidate] (135.034 ms) : 0, 135034
Remote Config [baseline] (723.864 µs) : 0, 724
Remote Config [candidate] (734.898 µs) : 0, 735
Telemetry [baseline] (5.727 ms) : 0, 5727
Telemetry [candidate] (5.762 ms) : 0, 5762
section iast
BytebuddyAgent [baseline] (717.585 ms) : 0, 717585
BytebuddyAgent [candidate] (717.741 ms) : 0, 717741
GlobalTracer [baseline] (270.185 ms) : 0, 270185
GlobalTracer [candidate] (270.579 ms) : 0, 270579
AppSec [baseline] (47.334 ms) : 0, 47334
AppSec [candidate] (47.355 ms) : 0, 47355
IAST [baseline] (13.861 ms) : 0, 13861
IAST [candidate] (13.735 ms) : 0, 13735
Remote Config [baseline] (558.307 µs) : 0, 558
Remote Config [candidate] (540.661 µs) : 0, 541
Telemetry [baseline] (5.743 ms) : 0, 5743
Telemetry [candidate] (5.624 ms) : 0, 5624
section profiling
BytebuddyAgent [baseline] (624.202 ms) : 0, 624202
BytebuddyAgent [candidate] (624.668 ms) : 0, 624668
GlobalTracer [baseline] (347.072 ms) : 0, 347072
GlobalTracer [candidate] (346.772 ms) : 0, 346772
AppSec [baseline] (49.236 ms) : 0, 49236
AppSec [candidate] (49.104 ms) : 0, 49104
Remote Config [baseline] (653.028 µs) : 0, 653
Remote Config [candidate] (646.394 µs) : 0, 646
Telemetry [baseline] (6.002 ms) : 0, 6002
Telemetry [candidate] (5.978 ms) : 0, 5978
ProfilingAgent [baseline] (77.761 ms) : 0, 77761
ProfilingAgent [candidate] (78.334 ms) : 0, 78334
Profiling [baseline] (77.785 ms) : 0, 77785
Profiling [candidate] (78.358 ms) : 0, 78358
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~35c46da67b, baseline=1.20.0-SNAPSHOT~6d21c3ecf6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (984.668 ms) : 0, 984668
Total [baseline] (8.65 s) : 0, 8650119
Agent [candidate] (988.141 ms) : 0, 988141
Total [candidate] (8.671 s) : 0, 8671143
section appsec
Agent [baseline] (1.069 s) : 0, 1069201
Total [baseline] (8.692 s) : 0, 8692139
Agent [candidate] (1.075 s) : 0, 1074998
Total [candidate] (8.696 s) : 0, 8695701
section iast
Agent [baseline] (1.088 s) : 0, 1087975
Total [baseline] (9.138 s) : 0, 9137970
Agent [candidate] (1.1 s) : 0, 1099620
Total [candidate] (9.193 s) : 0, 9193031
section profiling
Agent [baseline] (1.156 s) : 0, 1156077
Total [baseline] (8.868 s) : 0, 8867707
Agent [candidate] (1.157 s) : 0, 1157137
Total [candidate] (8.877 s) : 0, 8876837
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 984.668 ms -
Agent appsec 1.069 s 84.533 ms (8.6%)
Agent iast 1.088 s 103.307 ms (10.5%)
Agent profiling 1.156 s 171.409 ms (17.4%)
Total tracing 8.65 s -
Total appsec 8.692 s 42.02 ms (0.5%)
Total iast 9.138 s 487.851 ms (5.6%)
Total profiling 8.868 s 217.587 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 988.141 ms -
Agent appsec 1.075 s 86.857 ms (8.8%)
Agent iast 1.1 s 111.479 ms (11.3%)
Agent profiling 1.157 s 168.997 ms (17.1%)
Total tracing 8.671 s -
Total appsec 8.696 s 24.558 ms (0.3%)
Total iast 9.193 s 521.888 ms (6.0%)
Total profiling 8.877 s 205.694 ms (2.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~35c46da67b, baseline=1.20.0-SNAPSHOT~6d21c3ecf6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (611.145 ms) : 0, 611145
BytebuddyAgent [candidate] (612.34 ms) : 0, 612340
GlobalTracer [baseline] (282.981 ms) : 0, 282981
GlobalTracer [candidate] (284.479 ms) : 0, 284479
AppSec [baseline] (49.559 ms) : 0, 49559
AppSec [candidate] (50.296 ms) : 0, 50296
Remote Config [baseline] (664.461 µs) : 0, 664
Remote Config [candidate] (661.678 µs) : 0, 662
Telemetry [baseline] (5.976 ms) : 0, 5976
Telemetry [candidate] (5.995 ms) : 0, 5995
section appsec
BytebuddyAgent [baseline] (610.856 ms) : 0, 610856
BytebuddyAgent [candidate] (613.316 ms) : 0, 613316
GlobalTracer [baseline] (283.501 ms) : 0, 283501
GlobalTracer [candidate] (285.139 ms) : 0, 285139
AppSec [baseline] (134.117 ms) : 0, 134117
AppSec [candidate] (135.639 ms) : 0, 135639
Remote Config [baseline] (754.191 µs) : 0, 754
Remote Config [candidate] (718.591 µs) : 0, 719
Telemetry [baseline] (5.741 ms) : 0, 5741
Telemetry [candidate] (5.762 ms) : 0, 5762
section iast
BytebuddyAgent [baseline] (716.956 ms) : 0, 716956
BytebuddyAgent [candidate] (724.518 ms) : 0, 724518
GlobalTracer [baseline] (269.641 ms) : 0, 269641
GlobalTracer [candidate] (272.823 ms) : 0, 272823
AppSec [baseline] (47.188 ms) : 0, 47188
AppSec [candidate] (47.48 ms) : 0, 47480
IAST [baseline] (13.686 ms) : 0, 13686
IAST [candidate] (13.898 ms) : 0, 13898
Remote Config [baseline] (539.907 µs) : 0, 540
Remote Config [candidate] (549.783 µs) : 0, 550
Telemetry [baseline] (5.605 ms) : 0, 5605
Telemetry [candidate] (5.686 ms) : 0, 5686
section profiling
BytebuddyAgent [baseline] (624.389 ms) : 0, 624389
BytebuddyAgent [candidate] (625.898 ms) : 0, 625898
GlobalTracer [baseline] (345.659 ms) : 0, 345659
GlobalTracer [candidate] (346.899 ms) : 0, 346899
AppSec [baseline] (49.152 ms) : 0, 49152
AppSec [candidate] (49.164 ms) : 0, 49164
Remote Config [baseline] (650.075 µs) : 0, 650
Remote Config [candidate] (657.057 µs) : 0, 657
Telemetry [baseline] (6.004 ms) : 0, 6004
Telemetry [candidate] (5.976 ms) : 0, 5976
ProfilingAgent [baseline] (77.294 ms) : 0, 77294
ProfilingAgent [candidate] (75.626 ms) : 0, 75626
Profiling [baseline] (77.318 ms) : 0, 77318
Profiling [candidate] (75.65 ms) : 0, 75650
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~6d21c3ecf6 1.20.0-SNAPSHOT~35c46da67b
config baseline candidate
end_time 2023-08-23T22:15:29 2023-08-23T22:33:35
start_time 2023-08-23T22:15:11 2023-08-23T22:33:17
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~35c46da67b, baseline=1.20.0-SNAPSHOT~6d21c3ecf6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.329 ms) : 1310, 1349
.   : milestone, 1329,
appsec (1.68 ms) : 1656, 1704
.   : milestone, 1680,
iast (1.472 ms) : 1448, 1496
.   : milestone, 1472,
profiling (1.469 ms) : 1444, 1493
.   : milestone, 1469,
tracing (1.44 ms) : 1416, 1464
.   : milestone, 1440,
section candidate
no_agent (1.333 ms) : 1314, 1351
.   : milestone, 1333,
appsec (1.683 ms) : 1658, 1708
.   : milestone, 1683,
iast (1.476 ms) : 1452, 1500
.   : milestone, 1476,
profiling (1.47 ms) : 1444, 1495
.   : milestone, 1470,
tracing (1.46 ms) : 1436, 1484
.   : milestone, 1460,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.329 ms [1.31 ms, 1.349 ms] -
appsec 1.68 ms [1.656 ms, 1.704 ms] 351.01 µs (26.4%)
iast 1.472 ms [1.448 ms, 1.496 ms] 142.812 µs (10.7%)
profiling 1.469 ms [1.444 ms, 1.493 ms] 139.517 µs (10.5%)
tracing 1.44 ms [1.416 ms, 1.464 ms] 110.583 µs (8.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.333 ms [1.314 ms, 1.351 ms] -
appsec 1.683 ms [1.658 ms, 1.708 ms] 350.786 µs (26.3%)
iast 1.476 ms [1.452 ms, 1.5 ms] 143.264 µs (10.8%)
profiling 1.47 ms [1.444 ms, 1.495 ms] 136.933 µs (10.3%)
tracing 1.46 ms [1.436 ms, 1.484 ms] 127.147 µs (9.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~35c46da67b, baseline=1.20.0-SNAPSHOT~6d21c3ecf6
    dateFormat X
    axisFormat %s
section baseline
no_agent (359.371 µs) : 338, 381
.   : milestone, 359,
appsec (672.681 µs) : 651, 694
.   : milestone, 673,
iast (453.503 µs) : 432, 475
.   : milestone, 454,
iast_FULL (516.181 µs) : 495, 537
.   : milestone, 516,
iast_INACTIVE (425.699 µs) : 405, 447
.   : milestone, 426,
profiling (437.804 µs) : 417, 459
.   : milestone, 438,
tracing (426.844 µs) : 407, 447
.   : milestone, 427,
section candidate
no_agent (364.551 µs) : 343, 386
.   : milestone, 365,
appsec (669.476 µs) : 648, 691
.   : milestone, 669,
iast (454.476 µs) : 433, 475
.   : milestone, 454,
iast_FULL (515.738 µs) : 495, 536
.   : milestone, 516,
iast_INACTIVE (436.968 µs) : 416, 458
.   : milestone, 437,
profiling (457.306 µs) : 436, 479
.   : milestone, 457,
tracing (432.486 µs) : 412, 453
.   : milestone, 432,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.371 µs [337.55 µs, 381.192 µs] -
appsec 672.681 µs [651.275 µs, 694.086 µs] 313.31 µs (87.2%)
iast 453.503 µs [432.377 µs, 474.629 µs] 94.132 µs (26.2%)
iast_FULL 516.181 µs [495.161 µs, 537.201 µs] 156.81 µs (43.6%)
iast_INACTIVE 425.699 µs [404.818 µs, 446.579 µs] 66.328 µs (18.5%)
profiling 437.804 µs [416.61 µs, 458.997 µs] 78.433 µs (21.8%)
tracing 426.844 µs [406.734 µs, 446.954 µs] 67.473 µs (18.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.551 µs [342.642 µs, 386.46 µs] -
appsec 669.476 µs [647.888 µs, 691.064 µs] 304.925 µs (83.6%)
iast 454.476 µs [433.497 µs, 475.455 µs] 89.925 µs (24.7%)
iast_FULL 515.738 µs [495.148 µs, 536.329 µs] 151.187 µs (41.5%)
iast_INACTIVE 436.968 µs [415.91 µs, 458.027 µs] 72.417 µs (19.9%)
profiling 457.306 µs [435.693 µs, 478.919 µs] 92.755 µs (25.4%)
tracing 432.486 µs [412.417 µs, 452.556 µs] 67.935 µs (18.6%)

@PerfectSlayer

PerfectSlayer commented Aug 24, 2023

Copy link
Copy Markdown
Contributor

dd.trace.elasticsearch.body.enabled and dd.trace.elasticsearch.trace.enabled

Do you mean dd.trace.elasticseach.params.enabled instead?

But overall, why use two boolean flags to control one behavior rather than an unique settings with meaningful value: body, params or none?

@PerfectSlayer PerfectSlayer added the inst: elasticsearch Elasticsearch instrumentation label Aug 24, 2023
@bantonsson

bantonsson commented Aug 24, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

@PerfectSlayer I think that two separate *.enabled settings makes it clearer than the quad-value setting for body, params, body-params, none. But maybe that's just me.

@bm1549
bm1549 merged commit ccdadbf into master Aug 24, 2023
@bm1549
bm1549 deleted the bm1549/elastic-body-default branch August 24, 2023 19:29
@github-actions github-actions Bot added this to the 1.20.0 milestone Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bantonsson bantonsson bantonsson approved these changes
@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

inst: elasticsearch Elasticsearch instrumentation

Projects

None yet

Milestone

1.20.0

Development

Successfully merging this pull request may close these issues.

3 participants

@bm1549 @PerfectSlayer @bantonsson