Tag-based sampling rules by ygree · Pull Request #6298 · DataDog/dd-trace-java

ygree · 2023-11-30T05:42:29Z

What Does This Do

This is a naive implementation of trace tag-based sampling rules.

This PR does not mean to include:

Single Span Tag-Based Sampling rules
Target Span property (root/any span)
Enable tag-based tests once this PR merged

Motivation

RFC

Additional Notes

pr-commenter · 2023-11-30T06:26:00Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	ygree/tag-based-sampling-rules
git_commit_date	1702479976	1702505558
git_commit_sha	`4908b2c`	`f46448d`
release_version	1.26.0-SNAPSHOT~4908b2cf3e	1.26.0-SNAPSHOT~f46448dfab

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702508588	1702508588
ci_job_id	389416484	389416484
ci_pipeline_id	25057982	25057982
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 7 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.26.0-SNAPSHOT~f46448dfab, baseline=1.26.0-SNAPSHOT~4908b2cf3e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1046490
Total [baseline] (9.348 s) : 0, 9348227
Agent [candidate] (1.048 s) : 0, 1047864
Total [candidate] (9.284 s) : 0, 9284107
section appsec
Agent [baseline] (1.16 s) : 0, 1159970
Total [baseline] (9.461 s) : 0, 9461167
Agent [candidate] (1.145 s) : 0, 1144785
Total [candidate] (9.434 s) : 0, 9433596
section iast
Agent [baseline] (1.175 s) : 0, 1175102
Total [baseline] (9.591 s) : 0, 9590662
Agent [candidate] (1.17 s) : 0, 1170201
Total [candidate] (9.571 s) : 0, 9570640
section profiling
Agent [baseline] (1.261 s) : 0, 1260535
Total [baseline] (9.691 s) : 0, 9690569
Agent [candidate] (1.241 s) : 0, 1241462
Total [candidate] (9.659 s) : 0, 9658755

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.16 s	113.48 ms (10.8%)
Agent	iast	1.175 s	128.612 ms (12.3%)
Agent	profiling	1.261 s	214.046 ms (20.5%)
Total	tracing	9.348 s	-
Total	appsec	9.461 s	112.939 ms (1.2%)
Total	iast	9.591 s	242.435 ms (2.6%)
Total	profiling	9.691 s	342.342 ms (3.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.145 s	96.921 ms (9.2%)
Agent	iast	1.17 s	122.337 ms (11.7%)
Agent	profiling	1.241 s	193.598 ms (18.5%)
Total	tracing	9.284 s	-
Total	appsec	9.434 s	149.488 ms (1.6%)
Total	iast	9.571 s	286.533 ms (3.1%)
Total	profiling	9.659 s	374.648 ms (4.0%)

gantt
    title petclinic - break down per module: candidate=1.26.0-SNAPSHOT~f46448dfab, baseline=1.26.0-SNAPSHOT~4908b2cf3e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.566 ms) : 0, 646566
BytebuddyAgent [candidate] (647.963 ms) : 0, 647963
GlobalTracer [baseline] (306.488 ms) : 0, 306488
GlobalTracer [candidate] (306.755 ms) : 0, 306755
AppSec [baseline] (51.338 ms) : 0, 51338
AppSec [candidate] (51.052 ms) : 0, 51052
Remote Config [baseline] (668.937 µs) : 0, 669
Remote Config [candidate] (666.053 µs) : 0, 666
Telemetry [baseline] (7.12 ms) : 0, 7120
Telemetry [candidate] (7.105 ms) : 0, 7105
section appsec
BytebuddyAgent [baseline] (656.429 ms) : 0, 656429
BytebuddyAgent [candidate] (646.581 ms) : 0, 646581
GlobalTracer [baseline] (310.631 ms) : 0, 310631
GlobalTracer [candidate] (307.027 ms) : 0, 307027
AppSec [baseline] (150.585 ms) : 0, 150585
AppSec [candidate] (149.434 ms) : 0, 149434
Remote Config [baseline] (650.349 µs) : 0, 650
Remote Config [candidate] (643.674 µs) : 0, 644
Telemetry [baseline] (6.959 ms) : 0, 6959
Telemetry [candidate] (6.843 ms) : 0, 6843
section iast
BytebuddyAgent [baseline] (774.284 ms) : 0, 774284
BytebuddyAgent [candidate] (772.381 ms) : 0, 772381
GlobalTracer [baseline] (287.359 ms) : 0, 287359
GlobalTracer [candidate] (284.808 ms) : 0, 284808
AppSec [baseline] (49.528 ms) : 0, 49528
AppSec [candidate] (49.486 ms) : 0, 49486
IAST [baseline] (20.681 ms) : 0, 20681
IAST [candidate] (21.153 ms) : 0, 21153
Remote Config [baseline] (629.344 µs) : 0, 629
Remote Config [candidate] (610.405 µs) : 0, 610
Telemetry [baseline] (7.993 ms) : 0, 7993
Telemetry [candidate] (7.234 ms) : 0, 7234
section profiling
BytebuddyAgent [baseline] (667.787 ms) : 0, 667787
BytebuddyAgent [candidate] (657.532 ms) : 0, 657532
GlobalTracer [baseline] (381.946 ms) : 0, 381946
GlobalTracer [candidate] (375.307 ms) : 0, 375307
AppSec [baseline] (51.819 ms) : 0, 51819
AppSec [candidate] (51.271 ms) : 0, 51271
Remote Config [baseline] (685.188 µs) : 0, 685
Remote Config [candidate] (689.706 µs) : 0, 690
Telemetry [baseline] (7.601 ms) : 0, 7601
Telemetry [candidate] (7.49 ms) : 0, 7490
ProfilingAgent [baseline] (95.6 ms) : 0, 95600
ProfilingAgent [candidate] (95.028 ms) : 0, 95028
Profiling [baseline] (95.625 ms) : 0, 95625
Profiling [candidate] (95.052 ms) : 0, 95052

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.26.0-SNAPSHOT~f46448dfab, baseline=1.26.0-SNAPSHOT~4908b2cf3e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046709
Total [baseline] (8.709 s) : 0, 8709144
Agent [candidate] (1.047 s) : 0, 1047051
Total [candidate] (8.709 s) : 0, 8709340
section iast
Agent [baseline] (1.172 s) : 0, 1171767
Total [baseline] (9.264 s) : 0, 9264115
Agent [candidate] (1.174 s) : 0, 1174423
Total [candidate] (9.34 s) : 0, 9339811
section iast_TELEMETRY_OFF
Agent [baseline] (1.165 s) : 0, 1164688
Total [baseline] (9.267 s) : 0, 9267337
Agent [candidate] (1.163 s) : 0, 1163145
Total [candidate] (9.247 s) : 0, 9246884

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	iast	1.172 s	125.059 ms (11.9%)
Agent	iast_TELEMETRY_OFF	1.165 s	117.979 ms (11.3%)
Total	tracing	8.709 s	-
Total	iast	9.264 s	554.972 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.267 s	558.193 ms (6.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	iast	1.174 s	127.372 ms (12.2%)
Agent	iast_TELEMETRY_OFF	1.163 s	116.094 ms (11.1%)
Total	tracing	8.709 s	-
Total	iast	9.34 s	630.47 ms (7.2%)
Total	iast_TELEMETRY_OFF	9.247 s	537.544 ms (6.2%)

gantt
    title insecure-bank - break down per module: candidate=1.26.0-SNAPSHOT~f46448dfab, baseline=1.26.0-SNAPSHOT~4908b2cf3e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.887 ms) : 0, 646887
BytebuddyAgent [candidate] (647.046 ms) : 0, 647046
GlobalTracer [baseline] (306.324 ms) : 0, 306324
GlobalTracer [candidate] (306.665 ms) : 0, 306665
AppSec [baseline] (51.381 ms) : 0, 51381
AppSec [candidate] (51.232 ms) : 0, 51232
Remote Config [baseline] (676.489 µs) : 0, 676
Remote Config [candidate] (672.189 µs) : 0, 672
Telemetry [baseline] (7.176 ms) : 0, 7176
Telemetry [candidate] (7.184 ms) : 0, 7184
section iast
BytebuddyAgent [baseline] (771.928 ms) : 0, 771928
BytebuddyAgent [candidate] (774.907 ms) : 0, 774907
GlobalTracer [baseline] (286.304 ms) : 0, 286304
GlobalTracer [candidate] (286.577 ms) : 0, 286577
AppSec [baseline] (49.804 ms) : 0, 49804
AppSec [candidate] (49.698 ms) : 0, 49698
IAST [baseline] (21.311 ms) : 0, 21311
IAST [candidate] (20.685 ms) : 0, 20685
Remote Config [baseline] (622.822 µs) : 0, 623
Remote Config [candidate] (631.453 µs) : 0, 631
Telemetry [baseline] (7.227 ms) : 0, 7227
Telemetry [candidate] (7.331 ms) : 0, 7331
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (766.496 ms) : 0, 766496
BytebuddyAgent [candidate] (765.863 ms) : 0, 765863
GlobalTracer [baseline] (287.598 ms) : 0, 287598
GlobalTracer [candidate] (286.969 ms) : 0, 286969
AppSec [baseline] (49.542 ms) : 0, 49542
AppSec [candidate] (49.62 ms) : 0, 49620
IAST [baseline] (18.675 ms) : 0, 18675
IAST [candidate] (18.758 ms) : 0, 18758
Remote Config [baseline] (607.948 µs) : 0, 608
Remote Config [candidate] (620.096 µs) : 0, 620
Telemetry [baseline] (7.311 ms) : 0, 7311
Telemetry [candidate] (6.767 ms) : 0, 6767

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2023-12-13T22:42:23	2023-12-13T22:58:56
git_branch	master	ygree/tag-based-sampling-rules
git_commit_date	1702479976	1702505558
git_commit_sha	`4908b2c`	`f46448d`
release_version	1.26.0-SNAPSHOT~4908b2cf3e	1.26.0-SNAPSHOT~f46448dfab
start_time	2023-12-13T22:42:10	2023-12-13T22:58:43

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702508588	1702508588
ci_job_id	389416484	389416484
ci_pipeline_id	25057982	25057982
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 7 metrics, 14 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	better [-127.745µs; -72.852µs] or [-8.001%; -4.563%]	unstable [-444.900op/s; +672.820op/s] or [-15.015%; +22.708%]	1.496ms	3076.923op/s	1.597ms	2962.963op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~f46448dfab, baseline=1.26.0-SNAPSHOT~4908b2cf3e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.354 ms) : 1335, 1374
.   : milestone, 1354,
appsec (1.74 ms) : 1715, 1764
.   : milestone, 1740,
iast (1.5 ms) : 1475, 1525
.   : milestone, 1500,
profiling (1.597 ms) : 1570, 1623
.   : milestone, 1597,
tracing (1.485 ms) : 1461, 1510
.   : milestone, 1485,
section candidate
no_agent (1.352 ms) : 1333, 1372
.   : milestone, 1352,
appsec (1.76 ms) : 1734, 1786
.   : milestone, 1760,
iast (1.533 ms) : 1509, 1557
.   : milestone, 1533,
profiling (1.496 ms) : 1472, 1521
.   : milestone, 1496,
tracing (1.494 ms) : 1470, 1518
.   : milestone, 1494,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.354 ms [1.335 ms, 1.374 ms]	-
appsec	1.74 ms [1.715 ms, 1.764 ms]	385.117 µs (28.4%)
iast	1.5 ms [1.475 ms, 1.525 ms]	145.116 µs (10.7%)
profiling	1.597 ms [1.57 ms, 1.623 ms]	242.216 µs (17.9%)
tracing	1.485 ms [1.461 ms, 1.51 ms]	130.948 µs (9.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.352 ms [1.333 ms, 1.372 ms]	-
appsec	1.76 ms [1.734 ms, 1.786 ms]	407.311 µs (30.1%)
iast	1.533 ms [1.509 ms, 1.557 ms]	180.825 µs (13.4%)
profiling	1.496 ms [1.472 ms, 1.521 ms]	144.147 µs (10.7%)
tracing	1.494 ms [1.47 ms, 1.518 ms]	141.627 µs (10.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~f46448dfab, baseline=1.26.0-SNAPSHOT~4908b2cf3e
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.7 µs) : 344, 385
.   : milestone, 365,
iast (473.077 µs) : 452, 494
.   : milestone, 473,
iast_FULL (528.363 µs) : 508, 549
.   : milestone, 528,
iast_INACTIVE (442.441 µs) : 421, 464
.   : milestone, 442,
iast_TELEMETRY_OFF (464.906 µs) : 443, 487
.   : milestone, 465,
tracing (439.463 µs) : 419, 460
.   : milestone, 439,
section candidate
no_agent (368.071 µs) : 347, 389
.   : milestone, 368,
iast (468.718 µs) : 448, 489
.   : milestone, 469,
iast_FULL (526.679 µs) : 506, 547
.   : milestone, 527,
iast_INACTIVE (434.325 µs) : 414, 455
.   : milestone, 434,
iast_TELEMETRY_OFF (465.749 µs) : 445, 486
.   : milestone, 466,
tracing (439.789 µs) : 419, 460
.   : milestone, 440,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	364.7 µs [344.245 µs, 385.154 µs]	-
iast	473.077 µs [452.189 µs, 493.965 µs]	108.377 µs (29.7%)
iast_FULL	528.363 µs [507.824 µs, 548.903 µs]	163.664 µs (44.9%)
iast_INACTIVE	442.441 µs [421.306 µs, 463.577 µs]	77.742 µs (21.3%)
iast_TELEMETRY_OFF	464.906 µs [442.882 µs, 486.93 µs]	100.206 µs (27.5%)
tracing	439.463 µs [418.568 µs, 460.357 µs]	74.763 µs (20.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	368.071 µs [347.125 µs, 389.017 µs]	-
iast	468.718 µs [448.161 µs, 489.275 µs]	100.647 µs (27.3%)
iast_FULL	526.679 µs [506.305 µs, 547.052 µs]	158.607 µs (43.1%)
iast_INACTIVE	434.325 µs [414.127 µs, 454.522 µs]	66.253 µs (18.0%)
iast_TELEMETRY_OFF	465.749 µs [445.038 µs, 486.461 µs]	97.678 µs (26.5%)
tracing	439.789 µs [419.158 µs, 460.42 µs]	71.718 µs (19.5%)

am312 · 2023-12-14T15:11:39Z

+
+  private final Map<String, Matcher> matchers;
+
+  public TagsMatcher(Map<String, Matcher> matchers) {


could this constructor be private?

am312 · 2023-12-14T15:27:03Z

-      ANY
-    }
-  }
+  interface TraceSamplingRule extends SamplingRule {}


Is it clearer if usages of TraceSamplingRule elsewhere are just replaced with SamplingRule or better to maintain the additional interface?

dougqh · 2023-12-15T18:09:28Z

+    "[{\"tags\": {\"env\": \"x??\"}, \"sample_rate\": 1}, {\"tags\": {\"env\": \"b?r\", \"tag\": \"f??\"}, \"sample_rate\": 0}]"                         | null        | 0                | null              | null              | USER_DROP
+
+    // Select matching service + resource + operation + tag rules
+    "[{\"service\": \"service\", \"resource\": \"xxx\", \"tags\": {\"env\": \"x??\"}, \"sample_rate\": 0}, {\"service\": \"service\", \"resource\": \"resource\", \"name\": \"operation\", \"tags\": {\"env\": \"b?r\", \"tag\": \"f??\"}, \"sample_rate\": 1}]"                          | null        | 1.0              | 50                | null              | USER_KEEP


Do we test numeric tags at all?

ygree added the comp: core Tracer core label Nov 30, 2023

ygree self-assigned this Nov 30, 2023

ygree requested a review from a team as a code owner November 30, 2023 05:42

ygree requested review from am312 and dougqh November 30, 2023 05:42

ygree marked this pull request as draft November 30, 2023 05:42

Base automatically changed from bbujon/sampling to master December 1, 2023 22:12

Add tag-based TraceSamplingRules naive implementation

3934061

ygree force-pushed the ygree/tag-based-sampling-rules branch from 739048a to 3934061 Compare December 5, 2023 19:43

ygree added 2 commits December 13, 2023 14:12

Remove Target Span (root/any span)

f275fa2

Merge branch 'master' into ygree/tag-based-sampling-rules

f46448d

ygree marked this pull request as ready for review December 13, 2023 22:25

am312 reviewed Dec 14, 2023

View reviewed changes

dougqh reviewed Dec 15, 2023

View reviewed changes

dougqh approved these changes Dec 15, 2023

View reviewed changes

bm1549 merged commit aa3228d into master Dec 18, 2023

bm1549 deleted the ygree/tag-based-sampling-rules branch December 18, 2023 16:15

github-actions Bot added this to the 1.26.0 milestone Dec 18, 2023

ygree mentioned this pull request Dec 28, 2023

Enable tag-based sampling tests for Java DataDog/system-tests#1972

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Tag-based sampling rules#6298

Tag-based sampling rules#6298
bm1549 merged 3 commits into
masterfrom
ygree/tag-based-sampling-rules

ygree commented Nov 30, 2023 •

edited

Loading

Uh oh!

pr-commenter Bot commented Nov 30, 2023 •

edited

Loading

Uh oh!

am312 Dec 14, 2023

Uh oh!

am312 Dec 14, 2023

Uh oh!

dougqh Dec 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		private final Map<String, Matcher> matchers;

		public TagsMatcher(Map<String, Matcher> matchers) {

Uh oh!

Conversation

ygree commented Nov 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Uh oh!

pr-commenter Bot commented Nov 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

am312 Dec 14, 2023

Choose a reason for hiding this comment

Uh oh!

am312 Dec 14, 2023

Choose a reason for hiding this comment

Uh oh!

dougqh Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ygree commented Nov 30, 2023 •

edited

Loading

pr-commenter Bot commented Nov 30, 2023 •

edited

Loading