Skip to content

Add more tests about OpenTelemetry attributes conventions#7163

Merged
PerfectSlayer merged 1 commit into
masterfrom
bbujon/otel-attributes-test
Jun 17, 2024
Merged

Add more tests about OpenTelemetry attributes conventions#7163
PerfectSlayer merged 1 commit into
masterfrom
bbujon/otel-attributes-test

Conversation

@PerfectSlayer

@PerfectSlayer PerfectSlayer commented Jun 11, 2024

Copy link
Copy Markdown
Contributor

What Does This Do

This PR adds more tests about the OpenTelemetry attributes conventions.

Motivation

This is a follow up from the discussion on #7138

Additional Notes

Jira ticket: [PROJ-IDENT]

@PerfectSlayer PerfectSlayer added the inst: opentelemetry OpenTelemetry instrumentation label Jun 11, 2024
@PerfectSlayer
PerfectSlayer requested a review from a team as a code owner June 11, 2024 15:25
@PerfectSlayer
PerfectSlayer requested review from mcculls and removed request for a team June 11, 2024 15:25
@PerfectSlayer
PerfectSlayer enabled auto-merge June 11, 2024 16:08
@pr-commenter

pr-commenter Bot commented Jun 11, 2024

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/otel-attributes-test
git_commit_date 1718118542 1718119380
git_commit_sha 54461ee 129b372
release_version 1.36.0-SNAPSHOT~54461eefea 1.36.0-SNAPSHOT~129b37289d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1718122325 1718122325
ci_job_id 539460621 539460621
ci_pipeline_id 36412871 36412871
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 18 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063097
Total [baseline] (10.457 s) : 0, 10457282
Agent [candidate] (1.062 s) : 0, 1062035
Total [candidate] (10.429 s) : 0, 10428992
section appsec
Agent [baseline] (1.179 s) : 0, 1179403
Total [baseline] (10.507 s) : 0, 10507087
Agent [candidate] (1.184 s) : 0, 1183805
Total [candidate] (10.474 s) : 0, 10474379
section iast
Agent [baseline] (1.167 s) : 0, 1167353
Total [baseline] (10.832 s) : 0, 10831907
Agent [candidate] (1.186 s) : 0, 1185673
Total [candidate] (10.805 s) : 0, 10804577
section profiling
Agent [baseline] (1.266 s) : 0, 1266261
Total [baseline] (10.6 s) : 0, 10599796
Agent [candidate] (1.269 s) : 0, 1269105
Total [candidate] (10.665 s) : 0, 10664808
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.179 s 116.306 ms (10.9%)
Agent iast 1.167 s 104.256 ms (9.8%)
Agent profiling 1.266 s 203.164 ms (19.1%)
Total tracing 10.457 s -
Total appsec 10.507 s 49.805 ms (0.5%)
Total iast 10.832 s 374.625 ms (3.6%)
Total profiling 10.6 s 142.514 ms (1.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.184 s 121.77 ms (11.5%)
Agent iast 1.186 s 123.638 ms (11.6%)
Agent profiling 1.269 s 207.07 ms (19.5%)
Total tracing 10.429 s -
Total appsec 10.474 s 45.387 ms (0.4%)
Total iast 10.805 s 375.585 ms (3.6%)
Total profiling 10.665 s 235.816 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.745 ms) : 0, 665745
BytebuddyAgent [candidate] (665.006 ms) : 0, 665006
GlobalTracer [baseline] (303.977 ms) : 0, 303977
GlobalTracer [candidate] (304.051 ms) : 0, 304051
AppSec [baseline] (50.206 ms) : 0, 50206
AppSec [candidate] (49.935 ms) : 0, 49935
Logs Intake [baseline] (386.973 µs) : 0, 387
Logs Intake [candidate] (399.4 µs) : 0, 399
Remote Config [baseline] (694.446 µs) : 0, 694
Remote Config [candidate] (661.277 µs) : 0, 661
Telemetry [baseline] (7.645 ms) : 0, 7645
Telemetry [candidate] (7.556 ms) : 0, 7556
section appsec
BytebuddyAgent [baseline] (675.438 ms) : 0, 675438
BytebuddyAgent [candidate] (677.868 ms) : 0, 677868
GlobalTracer [baseline] (297.377 ms) : 0, 297377
GlobalTracer [candidate] (298.328 ms) : 0, 298328
AppSec [baseline] (152.834 ms) : 0, 152834
AppSec [candidate] (153.232 ms) : 0, 153232
IAST [baseline] (19.76 ms) : 0, 19760
IAST [candidate] (21.363 ms) : 0, 21363
Logs Intake [baseline] (325.892 µs) : 0, 326
Logs Intake [candidate] (322.857 µs) : 0, 323
Remote Config [baseline] (631.304 µs) : 0, 631
Remote Config [candidate] (645.238 µs) : 0, 645
Telemetry [baseline] (8.556 ms) : 0, 8556
Telemetry [candidate] (8.182 ms) : 0, 8182
section iast
BytebuddyAgent [baseline] (778.728 ms) : 0, 778728
BytebuddyAgent [candidate] (791.783 ms) : 0, 791783
GlobalTracer [baseline] (293.724 ms) : 0, 293724
GlobalTracer [candidate] (297.542 ms) : 0, 297542
AppSec [baseline] (46.648 ms) : 0, 46648
AppSec [candidate] (47.269 ms) : 0, 47269
IAST [baseline] (26.344 ms) : 0, 26344
IAST [candidate] (25.187 ms) : 0, 25187
Logs Intake [baseline] (303.712 µs) : 0, 304
Logs Intake [candidate] (311.747 µs) : 0, 312
Remote Config [baseline] (579.488 µs) : 0, 579
Remote Config [candidate] (595.239 µs) : 0, 595
Telemetry [baseline] (7.76 ms) : 0, 7760
Telemetry [candidate] (9.481 ms) : 0, 9481
section profiling
BytebuddyAgent [baseline] (667.618 ms) : 0, 667618
BytebuddyAgent [candidate] (667.707 ms) : 0, 667707
GlobalTracer [baseline] (386.643 ms) : 0, 386643
GlobalTracer [candidate] (389.16 ms) : 0, 389160
AppSec [baseline] (50.41 ms) : 0, 50410
AppSec [candidate] (51.185 ms) : 0, 51185
Logs Intake [baseline] (330.183 µs) : 0, 330
Logs Intake [candidate] (338.93 µs) : 0, 339
Remote Config [baseline] (754.708 µs) : 0, 755
Remote Config [candidate] (745.467 µs) : 0, 745
Telemetry [baseline] (7.381 ms) : 0, 7381
Telemetry [candidate] (7.406 ms) : 0, 7406
ProfilingAgent [baseline] (95.985 ms) : 0, 95985
ProfilingAgent [candidate] (95.458 ms) : 0, 95458
Profiling [baseline] (96.01 ms) : 0, 96010
Profiling [candidate] (95.483 ms) : 0, 95483
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063386
Total [baseline] (8.547 s) : 0, 8546897
Agent [candidate] (1.064 s) : 0, 1063931
Total [candidate] (8.62 s) : 0, 8619984
section iast
Agent [baseline] (1.17 s) : 0, 1169988
Total [baseline] (9.001 s) : 0, 9000521
Agent [candidate] (1.168 s) : 0, 1167549
Total [candidate] (8.983 s) : 0, 8982612
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.169 s) : 0, 1168666
Total [baseline] (8.968 s) : 0, 8967523
Agent [candidate] (1.171 s) : 0, 1170929
Total [candidate] (8.979 s) : 0, 8979187
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1169027
Total [baseline] (8.975 s) : 0, 8974788
Agent [candidate] (1.172 s) : 0, 1172442
Total [candidate] (9.056 s) : 0, 9055928
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.17 s 106.602 ms (10.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.169 s 105.28 ms (9.9%)
Agent iast_TELEMETRY_OFF 1.169 s 105.641 ms (9.9%)
Total tracing 8.547 s -
Total iast 9.001 s 453.623 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.968 s 420.625 ms (4.9%)
Total iast_TELEMETRY_OFF 8.975 s 427.89 ms (5.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.168 s 103.618 ms (9.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 106.999 ms (10.1%)
Agent iast_TELEMETRY_OFF 1.172 s 108.511 ms (10.2%)
Total tracing 8.62 s -
Total iast 8.983 s 362.628 ms (4.2%)
Total iast_HARDCODED_SECRET_DISABLED 8.979 s 359.203 ms (4.2%)
Total iast_TELEMETRY_OFF 9.056 s 435.944 ms (5.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.738 ms) : 0, 665738
BytebuddyAgent [candidate] (666.253 ms) : 0, 666253
GlobalTracer [baseline] (304.384 ms) : 0, 304384
GlobalTracer [candidate] (304.361 ms) : 0, 304361
AppSec [baseline] (50.119 ms) : 0, 50119
AppSec [candidate] (50.034 ms) : 0, 50034
Logs Intake [baseline] (411.634 µs) : 0, 412
Logs Intake [candidate] (425.831 µs) : 0, 426
Remote Config [baseline] (667.046 µs) : 0, 667
Remote Config [candidate] (678.428 µs) : 0, 678
Telemetry [baseline] (7.514 ms) : 0, 7514
Telemetry [candidate] (7.575 ms) : 0, 7575
section iast
BytebuddyAgent [baseline] (780.72 ms) : 0, 780720
BytebuddyAgent [candidate] (778.123 ms) : 0, 778123
GlobalTracer [baseline] (293.941 ms) : 0, 293941
GlobalTracer [candidate] (293.604 ms) : 0, 293604
AppSec [baseline] (46.516 ms) : 0, 46516
AppSec [candidate] (46.745 ms) : 0, 46745
IAST [baseline] (25.426 ms) : 0, 25426
IAST [candidate] (26.398 ms) : 0, 26398
Logs Intake [baseline] (302.502 µs) : 0, 303
Logs Intake [candidate] (300.147 µs) : 0, 300
Remote Config [baseline] (573.902 µs) : 0, 574
Remote Config [candidate] (569.108 µs) : 0, 569
Telemetry [baseline] (9.152 ms) : 0, 9152
Telemetry [candidate] (8.452 ms) : 0, 8452
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (778.502 ms) : 0, 778502
BytebuddyAgent [candidate] (780.322 ms) : 0, 780322
GlobalTracer [baseline] (293.863 ms) : 0, 293863
GlobalTracer [candidate] (294.511 ms) : 0, 294511
AppSec [baseline] (46.625 ms) : 0, 46625
AppSec [candidate] (46.733 ms) : 0, 46733
IAST [baseline] (27.113 ms) : 0, 27113
IAST [candidate] (26.565 ms) : 0, 26565
Logs Intake [baseline] (297.399 µs) : 0, 297
Logs Intake [candidate] (303.914 µs) : 0, 304
Remote Config [baseline] (568.916 µs) : 0, 569
Remote Config [candidate] (588.604 µs) : 0, 589
Telemetry [baseline] (8.328 ms) : 0, 8328
Telemetry [candidate] (8.509 ms) : 0, 8509
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.136 ms) : 0, 779136
BytebuddyAgent [candidate] (781.113 ms) : 0, 781113
GlobalTracer [baseline] (294.785 ms) : 0, 294785
GlobalTracer [candidate] (295.276 ms) : 0, 295276
AppSec [baseline] (50.082 ms) : 0, 50082
AppSec [candidate] (48.541 ms) : 0, 48541
IAST [baseline] (24.07 ms) : 0, 24070
IAST [candidate] (25.587 ms) : 0, 25587
Logs Intake [baseline] (295.289 µs) : 0, 295
Logs Intake [candidate] (316.795 µs) : 0, 317
Remote Config [baseline] (558.102 µs) : 0, 558
Remote Config [candidate] (576.431 µs) : 0, 576
Telemetry [baseline] (6.716 ms) : 0, 6716
Telemetry [candidate] (7.543 ms) : 0, 7543
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-06-11T15:44:34 2024-06-11T15:51:22
git_branch master bbujon/otel-attributes-test
git_commit_date 1718118542 1718119380
git_commit_sha 54461ee 129b372
release_version 1.36.0-SNAPSHOT~54461eefea 1.36.0-SNAPSHOT~129b37289d
start_time 2024-06-11T15:44:20 2024-06-11T15:51:09
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1718121428 1718121428
ci_job_id 539460622 539460622
ci_pipeline_id 36412871 36412871
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 11 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling worse
[+32.411µs; +87.561µs] or [+2.170%; +5.861%]		 unstable
[-577.854op/s; +577.854op/s] or [-18.780%; +18.780%]		 1.554ms 3076.923op/s 1.494ms 3076.923op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1333, 1371
.   : milestone, 1352,
appsec (1.693 ms) : 1668, 1718
.   : milestone, 1693,
appsec_no_iast (1.717 ms) : 1692, 1742
.   : milestone, 1717,
iast (1.477 ms) : 1455, 1500
.   : milestone, 1477,
profiling (1.494 ms) : 1468, 1519
.   : milestone, 1494,
tracing (1.469 ms) : 1445, 1493
.   : milestone, 1469,
section candidate
no_agent (1.33 ms) : 1310, 1350
.   : milestone, 1330,
appsec (1.712 ms) : 1688, 1736
.   : milestone, 1712,
appsec_no_iast (1.721 ms) : 1697, 1746
.   : milestone, 1721,
iast (1.494 ms) : 1471, 1517
.   : milestone, 1494,
profiling (1.554 ms) : 1528, 1580
.   : milestone, 1554,
tracing (1.444 ms) : 1419, 1468
.   : milestone, 1444,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.333 ms, 1.371 ms] -
appsec 1.693 ms [1.668 ms, 1.718 ms] 341.234 µs (25.2%)
appsec_no_iast 1.717 ms [1.692 ms, 1.742 ms] 365.431 µs (27.0%)
iast 1.477 ms [1.455 ms, 1.5 ms] 125.83 µs (9.3%)
profiling 1.494 ms [1.468 ms, 1.519 ms] 142.284 µs (10.5%)
tracing 1.469 ms [1.445 ms, 1.493 ms] 117.401 µs (8.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.33 ms [1.31 ms, 1.35 ms] -
appsec 1.712 ms [1.688 ms, 1.736 ms] 381.581 µs (28.7%)
appsec_no_iast 1.721 ms [1.697 ms, 1.746 ms] 391.212 µs (29.4%)
iast 1.494 ms [1.471 ms, 1.517 ms] 163.779 µs (12.3%)
profiling 1.554 ms [1.528 ms, 1.58 ms] 223.556 µs (16.8%)
tracing 1.444 ms [1.419 ms, 1.468 ms] 113.628 µs (8.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.083 µs) : 349, 389
.   : milestone, 369,
iast (485.726 µs) : 464, 507
.   : milestone, 486,
iast_FULL (553.45 µs) : 532, 575
.   : milestone, 553,
iast_GLOBAL (505.342 µs) : 484, 527
.   : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (483.944 µs) : 463, 505
.   : milestone, 484,
iast_INACTIVE (453.888 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (472.602 µs) : 451, 494
.   : milestone, 473,
tracing (446.151 µs) : 426, 467
.   : milestone, 446,
section candidate
no_agent (364.782 µs) : 345, 385
.   : milestone, 365,
iast (486.331 µs) : 465, 508
.   : milestone, 486,
iast_FULL (552.548 µs) : 531, 574
.   : milestone, 553,
iast_GLOBAL (510.126 µs) : 488, 532
.   : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (484.814 µs) : 463, 506
.   : milestone, 485,
iast_INACTIVE (454.993 µs) : 433, 477
.   : milestone, 455,
iast_TELEMETRY_OFF (478.788 µs) : 457, 500
.   : milestone, 479,
tracing (439.183 µs) : 419, 460
.   : milestone, 439,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.083 µs [349.361 µs, 388.804 µs] -
iast 485.726 µs [464.445 µs, 507.008 µs] 116.644 µs (31.6%)
iast_FULL 553.45 µs [532.161 µs, 574.738 µs] 184.367 µs (50.0%)
iast_GLOBAL 505.342 µs [483.933 µs, 526.752 µs] 136.26 µs (36.9%)
iast_HARDCODED_SECRET_DISABLED 483.944 µs [462.676 µs, 505.211 µs] 114.861 µs (31.1%)
iast_INACTIVE 453.888 µs [432.774 µs, 475.003 µs] 84.806 µs (23.0%)
iast_TELEMETRY_OFF 472.602 µs [451.296 µs, 493.908 µs] 103.519 µs (28.0%)
tracing 446.151 µs [425.801 µs, 466.502 µs] 77.069 µs (20.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.782 µs [344.811 µs, 384.753 µs] -
iast 486.331 µs [464.904 µs, 507.758 µs] 121.549 µs (33.3%)
iast_FULL 552.548 µs [531.48 µs, 573.616 µs] 187.766 µs (51.5%)
iast_GLOBAL 510.126 µs [488.018 µs, 532.234 µs] 145.344 µs (39.8%)
iast_HARDCODED_SECRET_DISABLED 484.814 µs [463.159 µs, 506.47 µs] 120.032 µs (32.9%)
iast_INACTIVE 454.993 µs [433.473 µs, 476.513 µs] 90.211 µs (24.7%)
iast_TELEMETRY_OFF 478.788 µs [457.268 µs, 500.308 µs] 114.006 µs (31.3%)
tracing 439.183 µs [418.582 µs, 459.785 µs] 74.401 µs (20.4%)

Dacapo

@PerfectSlayer
PerfectSlayer merged commit a19c6d5 into master Jun 17, 2024
@PerfectSlayer
PerfectSlayer deleted the bbujon/otel-attributes-test branch June 17, 2024 11:58
@github-actions github-actions Bot added this to the 1.35.1 milestone Jun 17, 2024
@mcculls mcculls modified the milestones: 1.35.1, 1.36.0 Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls approved these changes

Assignees

No one assigned

Labels

inst: opentelemetry OpenTelemetry instrumentation

Projects

None yet

Milestone

1.36.0

Development

Successfully merging this pull request may close these issues.

2 participants

@PerfectSlayer @mcculls