Ensure module name is consistent between test.module, test.suite and test spans when build system is instrumented by nikita-tkachenko-datadog · Pull Request #6354 · DataDog/dd-trace-java

nikita-tkachenko-datadog · 2023-12-12T19:57:42Z

What Does This Do

Ensures that test.module tag value is consistent in test module, test suite and test spans, when using CI Visibility to instrument Maven or Gradle runs.

Motivation

If a test suite or a test case belong to a module, it is expected that their test.module tag has the same value as the one the their parent module.

Additional Notes

When instrumenting Maven or Gradle, test.module value computation is based on build system logic: for Gradle it is the path of a corresponding task (e.g. :module-name:test), for Maven - project name + plugin name + goal name.
When running the tests in a child JVM, it is not possible to compute the same value, so previous version of the code computed module name by calculating current path relative to repository root.
The new version of the code propagates computed value from parent process to child process by setting a system property.

Jira ticket: CIVIS-8291

…test spans when build system is instrumented

pr-commenter · 2023-12-12T20:37:23Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/test-module-name-fix
git_commit_date	1702409587	1702410238
git_commit_sha	`1878db6`	`9c0a077`
release_version	1.26.0-SNAPSHOT~1878db6329	1.26.0-SNAPSHOT~9c0a0772ea

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702413401	1702413401
ci_job_id	388262073	388262073
ci_pipeline_id	24970933	24970933
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 8 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.26.0-SNAPSHOT~9c0a0772ea, baseline=1.26.0-SNAPSHOT~1878db6329

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1041201
Total [baseline] (8.689 s) : 0, 8688948
Agent [candidate] (1.044 s) : 0, 1044116
Total [candidate] (8.691 s) : 0, 8691415
section iast
Agent [baseline] (1.158 s) : 0, 1157914
Total [baseline] (9.217 s) : 0, 9216585
Agent [candidate] (1.161 s) : 0, 1160968
Total [candidate] (9.258 s) : 0, 9257628
section iast_TELEMETRY_OFF
Agent [baseline] (1.151 s) : 0, 1150686
Total [baseline] (9.217 s) : 0, 9216980
Agent [candidate] (1.152 s) : 0, 1152484
Total [candidate] (9.225 s) : 0, 9225305

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.041 s	-
Agent	iast	1.158 s	116.713 ms (11.2%)
Agent	iast_TELEMETRY_OFF	1.151 s	109.486 ms (10.5%)
Total	tracing	8.689 s	-
Total	iast	9.217 s	527.636 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.217 s	528.032 ms (6.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.161 s	116.852 ms (11.2%)
Agent	iast_TELEMETRY_OFF	1.152 s	108.368 ms (10.4%)
Total	tracing	8.691 s	-
Total	iast	9.258 s	566.213 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.225 s	533.89 ms (6.1%)

gantt
    title insecure-bank - break down per module: candidate=1.26.0-SNAPSHOT~9c0a0772ea, baseline=1.26.0-SNAPSHOT~1878db6329

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (644.711 ms) : 0, 644711
BytebuddyAgent [candidate] (645.559 ms) : 0, 645559
GlobalTracer [baseline] (306.162 ms) : 0, 306162
GlobalTracer [candidate] (307.81 ms) : 0, 307810
AppSec [baseline] (48.339 ms) : 0, 48339
AppSec [candidate] (48.451 ms) : 0, 48451
Remote Config [baseline] (671.905 µs) : 0, 672
Remote Config [candidate] (673.576 µs) : 0, 674
Telemetry [baseline] (7.159 ms) : 0, 7159
Telemetry [candidate] (7.222 ms) : 0, 7222
section iast
BytebuddyAgent [baseline] (764.805 ms) : 0, 764805
BytebuddyAgent [candidate] (766.846 ms) : 0, 766846
GlobalTracer [baseline] (284.422 ms) : 0, 284422
GlobalTracer [candidate] (285.283 ms) : 0, 285283
AppSec [baseline] (46.323 ms) : 0, 46323
AppSec [candidate] (46.226 ms) : 0, 46226
IAST [baseline] (20.813 ms) : 0, 20813
IAST [candidate] (20.323 ms) : 0, 20323
Remote Config [baseline] (599.247 µs) : 0, 599
Remote Config [candidate] (616.837 µs) : 0, 617
Telemetry [baseline] (6.582 ms) : 0, 6582
Telemetry [candidate] (7.324 ms) : 0, 7324
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (757.999 ms) : 0, 757999
BytebuddyAgent [candidate] (758.277 ms) : 0, 758277
GlobalTracer [baseline] (284.624 ms) : 0, 284624
GlobalTracer [candidate] (285.275 ms) : 0, 285275
AppSec [baseline] (46.924 ms) : 0, 46924
AppSec [candidate] (46.985 ms) : 0, 46985
IAST [baseline] (19.157 ms) : 0, 19157
IAST [candidate] (19.21 ms) : 0, 19210
Remote Config [baseline] (587.282 µs) : 0, 587
Remote Config [candidate] (594.001 µs) : 0, 594
Telemetry [baseline] (7.192 ms) : 0, 7192
Telemetry [candidate] (7.928 ms) : 0, 7928

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.26.0-SNAPSHOT~9c0a0772ea, baseline=1.26.0-SNAPSHOT~1878db6329

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.042 s) : 0, 1042092
Total [baseline] (9.329 s) : 0, 9329135
Agent [candidate] (1.044 s) : 0, 1044052
Total [candidate] (9.301 s) : 0, 9300559
section appsec
Agent [baseline] (1.133 s) : 0, 1133312
Total [baseline] (9.451 s) : 0, 9451372
Agent [candidate] (1.131 s) : 0, 1130720
Total [candidate] (9.386 s) : 0, 9386257
section iast
Agent [baseline] (1.158 s) : 0, 1157885
Total [baseline] (9.53 s) : 0, 9529623
Agent [candidate] (1.16 s) : 0, 1160444
Total [candidate] (9.53 s) : 0, 9529904
section profiling
Agent [baseline] (1.234 s) : 0, 1234201
Total [baseline] (9.622 s) : 0, 9621844
Agent [candidate] (1.232 s) : 0, 1232136
Total [candidate] (9.604 s) : 0, 9603809

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.042 s	-
Agent	appsec	1.133 s	91.22 ms (8.8%)
Agent	iast	1.158 s	115.793 ms (11.1%)
Agent	profiling	1.234 s	192.109 ms (18.4%)
Total	tracing	9.329 s	-
Total	appsec	9.451 s	122.237 ms (1.3%)
Total	iast	9.53 s	200.488 ms (2.1%)
Total	profiling	9.622 s	292.709 ms (3.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	appsec	1.131 s	86.668 ms (8.3%)
Agent	iast	1.16 s	116.392 ms (11.1%)
Agent	profiling	1.232 s	188.083 ms (18.0%)
Total	tracing	9.301 s	-
Total	appsec	9.386 s	85.698 ms (0.9%)
Total	iast	9.53 s	229.345 ms (2.5%)
Total	profiling	9.604 s	303.25 ms (3.3%)

gantt
    title petclinic - break down per module: candidate=1.26.0-SNAPSHOT~9c0a0772ea, baseline=1.26.0-SNAPSHOT~1878db6329

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.675 ms) : 0, 645675
BytebuddyAgent [candidate] (645.436 ms) : 0, 645436
GlobalTracer [baseline] (305.889 ms) : 0, 305889
GlobalTracer [candidate] (307.955 ms) : 0, 307955
AppSec [baseline] (48.404 ms) : 0, 48404
AppSec [candidate] (48.508 ms) : 0, 48508
Remote Config [baseline] (673.751 µs) : 0, 674
Remote Config [candidate] (670.426 µs) : 0, 670
Telemetry [baseline] (7.146 ms) : 0, 7146
Telemetry [candidate] (7.221 ms) : 0, 7221
section appsec
BytebuddyAgent [baseline] (647.685 ms) : 0, 647685
BytebuddyAgent [candidate] (645.122 ms) : 0, 645122
GlobalTracer [baseline] (307.263 ms) : 0, 307263
GlobalTracer [candidate] (306.883 ms) : 0, 306883
AppSec [baseline] (136.508 ms) : 0, 136508
AppSec [candidate] (136.515 ms) : 0, 136515
Remote Config [baseline] (644.585 µs) : 0, 645
Remote Config [candidate] (643.716 µs) : 0, 644
Telemetry [baseline] (6.816 ms) : 0, 6816
Telemetry [candidate] (7.421 ms) : 0, 7421
section iast
BytebuddyAgent [baseline] (765.17 ms) : 0, 765170
BytebuddyAgent [candidate] (766.361 ms) : 0, 766361
GlobalTracer [baseline] (284.297 ms) : 0, 284297
GlobalTracer [candidate] (285.361 ms) : 0, 285361
AppSec [baseline] (46.299 ms) : 0, 46299
AppSec [candidate] (46.362 ms) : 0, 46362
IAST [baseline] (19.93 ms) : 0, 19930
IAST [candidate] (19.418 ms) : 0, 19418
Remote Config [baseline] (595.403 µs) : 0, 595
Remote Config [candidate] (601.506 µs) : 0, 602
Telemetry [baseline] (7.234 ms) : 0, 7234
Telemetry [candidate] (8.008 ms) : 0, 8008
section profiling
BytebuddyAgent [baseline] (658.331 ms) : 0, 658331
BytebuddyAgent [candidate] (656.051 ms) : 0, 656051
GlobalTracer [baseline] (376.552 ms) : 0, 376552
GlobalTracer [candidate] (377.475 ms) : 0, 377475
AppSec [baseline] (48.739 ms) : 0, 48739
AppSec [candidate] (48.44 ms) : 0, 48440
Remote Config [baseline] (655.317 µs) : 0, 655
Remote Config [candidate] (660.192 µs) : 0, 660
Telemetry [baseline] (7.355 ms) : 0, 7355
Telemetry [candidate] (7.362 ms) : 0, 7362
ProfilingAgent [baseline] (88.084 ms) : 0, 88084
ProfilingAgent [candidate] (88.07 ms) : 0, 88070
Profiling [baseline] (88.109 ms) : 0, 88109
Profiling [candidate] (88.095 ms) : 0, 88095

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2023-12-12T20:16:00	2023-12-12T20:32:30
git_branch	master	nikita-tkachenko/test-module-name-fix
git_commit_date	1702409587	1702410238
git_commit_sha	`1878db6`	`9c0a077`
release_version	1.26.0-SNAPSHOT~1878db6329	1.26.0-SNAPSHOT~9c0a0772ea
start_time	2023-12-12T20:15:47	2023-12-12T20:32:17

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1702413401	1702413401
ci_job_id	388262073	388262073
ci_pipeline_id	24970933	24970933
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~9c0a0772ea, baseline=1.26.0-SNAPSHOT~1878db6329
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.65 µs) : 345, 387
.   : milestone, 366,
iast (469.392 µs) : 448, 490
.   : milestone, 469,
iast_FULL (527.091 µs) : 507, 547
.   : milestone, 527,
iast_INACTIVE (445.36 µs) : 424, 466
.   : milestone, 445,
iast_TELEMETRY_OFF (463.238 µs) : 442, 484
.   : milestone, 463,
tracing (439.054 µs) : 418, 460
.   : milestone, 439,
section candidate
no_agent (356.776 µs) : 337, 376
.   : milestone, 357,
iast (475.752 µs) : 455, 496
.   : milestone, 476,
iast_FULL (533.143 µs) : 513, 554
.   : milestone, 533,
iast_INACTIVE (446.975 µs) : 426, 468
.   : milestone, 447,
iast_TELEMETRY_OFF (464.591 µs) : 444, 485
.   : milestone, 465,
tracing (445.512 µs) : 425, 466
.   : milestone, 446,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	365.65 µs [344.78 µs, 386.52 µs]	-
iast	469.392 µs [448.361 µs, 490.423 µs]	103.742 µs (28.4%)
iast_FULL	527.091 µs [506.912 µs, 547.27 µs]	161.441 µs (44.2%)
iast_INACTIVE	445.36 µs [424.45 µs, 466.269 µs]	79.71 µs (21.8%)
iast_TELEMETRY_OFF	463.238 µs [442.293 µs, 484.182 µs]	97.588 µs (26.7%)
tracing	439.054 µs [418.2 µs, 459.908 µs]	73.404 µs (20.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	356.776 µs [337.371 µs, 376.181 µs]	-
iast	475.752 µs [455.139 µs, 496.365 µs]	118.977 µs (33.3%)
iast_FULL	533.143 µs [512.669 µs, 553.617 µs]	176.368 µs (49.4%)
iast_INACTIVE	446.975 µs [425.782 µs, 468.167 µs]	90.199 µs (25.3%)
iast_TELEMETRY_OFF	464.591 µs [443.809 µs, 485.373 µs]	107.815 µs (30.2%)
tracing	445.512 µs [425.111 µs, 465.914 µs]	88.737 µs (24.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~9c0a0772ea, baseline=1.26.0-SNAPSHOT~1878db6329
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.346 ms) : 1327, 1365
.   : milestone, 1346,
appsec (1.743 ms) : 1717, 1768
.   : milestone, 1743,
iast (1.531 ms) : 1507, 1555
.   : milestone, 1531,
profiling (1.531 ms) : 1505, 1558
.   : milestone, 1531,
tracing (1.487 ms) : 1462, 1511
.   : milestone, 1487,
section candidate
no_agent (1.348 ms) : 1329, 1367
.   : milestone, 1348,
appsec (1.766 ms) : 1742, 1791
.   : milestone, 1766,
iast (1.479 ms) : 1454, 1503
.   : milestone, 1479,
profiling (1.542 ms) : 1516, 1569
.   : milestone, 1542,
tracing (1.458 ms) : 1433, 1483
.   : milestone, 1458,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.346 ms [1.327 ms, 1.365 ms]	-
appsec	1.743 ms [1.717 ms, 1.768 ms]	396.659 µs (29.5%)
iast	1.531 ms [1.507 ms, 1.555 ms]	185.115 µs (13.8%)
profiling	1.531 ms [1.505 ms, 1.558 ms]	185.148 µs (13.8%)
tracing	1.487 ms [1.462 ms, 1.511 ms]	140.683 µs (10.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.348 ms [1.329 ms, 1.367 ms]	-
appsec	1.766 ms [1.742 ms, 1.791 ms]	418.108 µs (31.0%)
iast	1.479 ms [1.454 ms, 1.503 ms]	130.126 µs (9.7%)
profiling	1.542 ms [1.516 ms, 1.569 ms]	193.923 µs (14.4%)
tracing	1.458 ms [1.433 ms, 1.483 ms]	109.807 µs (8.1%)

Ensure module name is consistent between test.module, test.suite and …

9c0a077

…test spans when build system is instrumented

nikita-tkachenko-datadog added type: bug fix Bug fix comp: ci visibility Continuous Integration Visibility labels Dec 12, 2023

nikita-tkachenko-datadog marked this pull request as ready for review December 12, 2023 20:35

nikita-tkachenko-datadog requested a review from a team as a code owner December 12, 2023 20:35

nikita-tkachenko-datadog requested a review from tonyredondo December 12, 2023 20:35

romainkomorn-exdatadog approved these changes Dec 13, 2023

View reviewed changes

nikita-tkachenko-datadog merged commit e26f1a4 into master Dec 13, 2023

nikita-tkachenko-datadog deleted the nikita-tkachenko/test-module-name-fix branch December 13, 2023 10:01

github-actions Bot added this to the 1.26.0 milestone Dec 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Ensure module name is consistent between test.module, test.suite and test spans when build system is instrumented#6354

Ensure module name is consistent between test.module, test.suite and test spans when build system is instrumented#6354
nikita-tkachenko-datadog merged 1 commit into
masterfrom
nikita-tkachenko/test-module-name-fix

nikita-tkachenko-datadog commented Dec 12, 2023 •

edited by atlassian Bot

Loading

Uh oh!

pr-commenter Bot commented Dec 12, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

nikita-tkachenko-datadog commented Dec 12, 2023 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Uh oh!

pr-commenter Bot commented Dec 12, 2023

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nikita-tkachenko-datadog commented Dec 12, 2023 •

edited by atlassian Bot

Loading