Cherrypick: Fix root coverage packages calculation in large repositories (#6317) by bm1549 · Pull Request #6321 · DataDog/dd-trace-java

bm1549 · 2023-12-05T20:49:37Z

What Does This Do

Motivation

Additional Notes

Jira ticket: [PROJ-IDENT]

pr-commenter · 2023-12-05T21:33:33Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bm1549/cherry-pick
git_commit_date	1701801899	1701809347
git_commit_sha	`f4980d2`	`8908066`
release_version	1.26.0-SNAPSHOT~f4980d293f	1.26.0-SNAPSHOT~8908066de9

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1701811970	1701811970
ci_job_id	382664099	382664099
ci_pipeline_id	24461019	24461019
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.26.0-SNAPSHOT~8908066de9, baseline=1.26.0-SNAPSHOT~f4980d293f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1047793
Total [baseline] (9.314 s) : 0, 9313534
Agent [candidate] (1.05 s) : 0, 1050245
Total [candidate] (9.328 s) : 0, 9328327
section appsec
Agent [baseline] (1.147 s) : 0, 1146557
Total [baseline] (9.606 s) : 0, 9605913
Agent [candidate] (1.132 s) : 0, 1131707
Total [candidate] (9.469 s) : 0, 9468865
section iast
Agent [baseline] (1.159 s) : 0, 1159206
Total [baseline] (9.602 s) : 0, 9602422
Agent [candidate] (1.161 s) : 0, 1161493
Total [candidate] (9.525 s) : 0, 9525472
section profiling
Agent [baseline] (1.234 s) : 0, 1233582
Total [baseline] (9.585 s) : 0, 9585365
Agent [candidate] (1.24 s) : 0, 1240104
Total [candidate] (9.684 s) : 0, 9683597

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.147 s	98.764 ms (9.4%)
Agent	iast	1.159 s	111.413 ms (10.6%)
Agent	profiling	1.234 s	185.789 ms (17.7%)
Total	tracing	9.314 s	-
Total	appsec	9.606 s	292.379 ms (3.1%)
Total	iast	9.602 s	288.888 ms (3.1%)
Total	profiling	9.585 s	271.831 ms (2.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.05 s	-
Agent	appsec	1.132 s	81.462 ms (7.8%)
Agent	iast	1.161 s	111.248 ms (10.6%)
Agent	profiling	1.24 s	189.859 ms (18.1%)
Total	tracing	9.328 s	-
Total	appsec	9.469 s	140.537 ms (1.5%)
Total	iast	9.525 s	197.145 ms (2.1%)
Total	profiling	9.684 s	355.27 ms (3.8%)

gantt
    title petclinic - break down per module: candidate=1.26.0-SNAPSHOT~8908066de9, baseline=1.26.0-SNAPSHOT~f4980d293f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (648.771 ms) : 0, 648771
BytebuddyAgent [candidate] (649.729 ms) : 0, 649729
GlobalTracer [baseline] (308.04 ms) : 0, 308040
GlobalTracer [candidate] (309.165 ms) : 0, 309165
AppSec [baseline] (48.697 ms) : 0, 48697
AppSec [candidate] (49.031 ms) : 0, 49031
Remote Config [baseline] (668.726 µs) : 0, 669
Remote Config [candidate] (676.461 µs) : 0, 676
Telemetry [baseline] (7.196 ms) : 0, 7196
Telemetry [candidate] (7.172 ms) : 0, 7172
section appsec
BytebuddyAgent [baseline] (655.7 ms) : 0, 655700
BytebuddyAgent [candidate] (645.433 ms) : 0, 645433
GlobalTracer [baseline] (310.58 ms) : 0, 310580
GlobalTracer [candidate] (306.508 ms) : 0, 306508
AppSec [baseline] (137.949 ms) : 0, 137949
AppSec [candidate] (136.682 ms) : 0, 136682
Remote Config [baseline] (655.764 µs) : 0, 656
Remote Config [candidate] (677.178 µs) : 0, 677
Telemetry [baseline] (6.928 ms) : 0, 6928
Telemetry [candidate] (8.092 ms) : 0, 8092
section iast
BytebuddyAgent [baseline] (765.399 ms) : 0, 765399
BytebuddyAgent [candidate] (766.576 ms) : 0, 766576
GlobalTracer [baseline] (284.768 ms) : 0, 284768
GlobalTracer [candidate] (286.013 ms) : 0, 286013
AppSec [baseline] (46.056 ms) : 0, 46056
AppSec [candidate] (46.335 ms) : 0, 46335
Remote Config [baseline] (604.921 µs) : 0, 605
Remote Config [candidate] (597.447 µs) : 0, 597
Telemetry [baseline] (7.951 ms) : 0, 7951
Telemetry [candidate] (8.136 ms) : 0, 8136
IAST [baseline] (20.082 ms) : 0, 20082
IAST [candidate] (19.562 ms) : 0, 19562
section profiling
BytebuddyAgent [baseline] (656.513 ms) : 0, 656513
BytebuddyAgent [candidate] (660.875 ms) : 0, 660875
GlobalTracer [baseline] (377.758 ms) : 0, 377758
GlobalTracer [candidate] (378.743 ms) : 0, 378743
AppSec [baseline] (48.407 ms) : 0, 48407
AppSec [candidate] (48.774 ms) : 0, 48774
Remote Config [baseline] (681.655 µs) : 0, 682
Remote Config [candidate] (703.585 µs) : 0, 704
Telemetry [baseline] (7.394 ms) : 0, 7394
Telemetry [candidate] (8.1 ms) : 0, 8100
ProfilingAgent [baseline] (88.747 ms) : 0, 88747
ProfilingAgent [candidate] (88.395 ms) : 0, 88395
Profiling [baseline] (88.771 ms) : 0, 88771
Profiling [candidate] (88.42 ms) : 0, 88420

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.26.0-SNAPSHOT~8908066de9, baseline=1.26.0-SNAPSHOT~f4980d293f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.051 s) : 0, 1051229
Total [baseline] (8.713 s) : 0, 8713413
Agent [candidate] (1.044 s) : 0, 1044372
Total [candidate] (8.712 s) : 0, 8712212
section iast
Agent [baseline] (1.166 s) : 0, 1166028
Total [baseline] (9.263 s) : 0, 9262849
Agent [candidate] (1.165 s) : 0, 1164712
Total [candidate] (9.279 s) : 0, 9279306
section iast_TELEMETRY_OFF
Agent [baseline] (1.166 s) : 0, 1166044
Total [baseline] (9.281 s) : 0, 9280979
Agent [candidate] (1.156 s) : 0, 1155613
Total [candidate] (9.251 s) : 0, 9250918

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	iast	1.166 s	114.799 ms (10.9%)
Agent	iast_TELEMETRY_OFF	1.166 s	114.815 ms (10.9%)
Total	tracing	8.713 s	-
Total	iast	9.263 s	549.436 ms (6.3%)
Total	iast_TELEMETRY_OFF	9.281 s	567.566 ms (6.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.165 s	120.34 ms (11.5%)
Agent	iast_TELEMETRY_OFF	1.156 s	111.241 ms (10.7%)
Total	tracing	8.712 s	-
Total	iast	9.279 s	567.094 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.251 s	538.706 ms (6.2%)

gantt
    title insecure-bank - break down per module: candidate=1.26.0-SNAPSHOT~8908066de9, baseline=1.26.0-SNAPSHOT~f4980d293f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (651.149 ms) : 0, 651149
BytebuddyAgent [candidate] (646.318 ms) : 0, 646318
GlobalTracer [baseline] (309.187 ms) : 0, 309187
GlobalTracer [candidate] (306.854 ms) : 0, 306854
AppSec [baseline] (48.516 ms) : 0, 48516
AppSec [candidate] (48.979 ms) : 0, 48979
Remote Config [baseline] (674.282 µs) : 0, 674
Remote Config [candidate] (676.092 µs) : 0, 676
Telemetry [baseline] (7.142 ms) : 0, 7142
Telemetry [candidate] (7.135 ms) : 0, 7135
section iast
BytebuddyAgent [baseline] (770.833 ms) : 0, 770833
BytebuddyAgent [candidate] (767.571 ms) : 0, 767571
GlobalTracer [baseline] (286.792 ms) : 0, 286792
GlobalTracer [candidate] (287.843 ms) : 0, 287843
AppSec [baseline] (46.509 ms) : 0, 46509
AppSec [candidate] (46.64 ms) : 0, 46640
Remote Config [baseline] (600.926 µs) : 0, 601
Remote Config [candidate] (604.816 µs) : 0, 605
Telemetry [baseline] (8.098 ms) : 0, 8098
Telemetry [candidate] (8.848 ms) : 0, 8848
IAST [baseline] (18.716 ms) : 0, 18716
IAST [candidate] (18.759 ms) : 0, 18759
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (769.775 ms) : 0, 769775
BytebuddyAgent [candidate] (760.179 ms) : 0, 760179
GlobalTracer [baseline] (287.899 ms) : 0, 287899
GlobalTracer [candidate] (286.501 ms) : 0, 286501
AppSec [baseline] (46.631 ms) : 0, 46631
AppSec [candidate] (46.3 ms) : 0, 46300
Remote Config [baseline] (603.748 µs) : 0, 604
Remote Config [candidate] (651.586 µs) : 0, 652
Telemetry [baseline] (7.918 ms) : 0, 7918
Telemetry [candidate] (8.687 ms) : 0, 8687
IAST [baseline] (18.367 ms) : 0, 18367
IAST [candidate] (18.938 ms) : 0, 18938

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2023-12-05T21:12:03	2023-12-05T21:28:36
git_branch	master	bm1549/cherry-pick
git_commit_date	1701801899	1701809347
git_commit_sha	`f4980d2`	`8908066`
release_version	1.26.0-SNAPSHOT~f4980d293f	1.26.0-SNAPSHOT~8908066de9
start_time	2023-12-05T21:11:50	2023-12-05T21:28:23

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1701811970	1701811970
ci_job_id	382664099	382664099
ci_pipeline_id	24461019	24461019
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~8908066de9, baseline=1.26.0-SNAPSHOT~f4980d293f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.364 ms) : 1345, 1383
.   : milestone, 1364,
appsec (1.758 ms) : 1734, 1783
.   : milestone, 1758,
iast (1.547 ms) : 1523, 1571
.   : milestone, 1547,
profiling (1.531 ms) : 1506, 1556
.   : milestone, 1531,
tracing (1.492 ms) : 1468, 1517
.   : milestone, 1492,
section candidate
no_agent (1.344 ms) : 1325, 1363
.   : milestone, 1344,
appsec (1.759 ms) : 1734, 1784
.   : milestone, 1759,
iast (1.508 ms) : 1483, 1533
.   : milestone, 1508,
profiling (1.566 ms) : 1540, 1592
.   : milestone, 1566,
tracing (1.502 ms) : 1477, 1526
.   : milestone, 1502,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.364 ms [1.345 ms, 1.383 ms]	-
appsec	1.758 ms [1.734 ms, 1.783 ms]	394.457 µs (28.9%)
iast	1.547 ms [1.523 ms, 1.571 ms]	183.133 µs (13.4%)
profiling	1.531 ms [1.506 ms, 1.556 ms]	166.683 µs (12.2%)
tracing	1.492 ms [1.468 ms, 1.517 ms]	128.475 µs (9.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.344 ms [1.325 ms, 1.363 ms]	-
appsec	1.759 ms [1.734 ms, 1.784 ms]	414.979 µs (30.9%)
iast	1.508 ms [1.483 ms, 1.533 ms]	164.043 µs (12.2%)
profiling	1.566 ms [1.54 ms, 1.592 ms]	222.047 µs (16.5%)
tracing	1.502 ms [1.477 ms, 1.526 ms]	157.645 µs (11.7%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~8908066de9, baseline=1.26.0-SNAPSHOT~f4980d293f
    dateFormat X
    axisFormat %s
section baseline
no_agent (366.453 µs) : 346, 387
.   : milestone, 366,
iast (472.33 µs) : 452, 493
.   : milestone, 472,
iast_FULL (535.668 µs) : 515, 556
.   : milestone, 536,
iast_INACTIVE (443.252 µs) : 422, 464
.   : milestone, 443,
iast_TELEMETRY_OFF (470.204 µs) : 449, 491
.   : milestone, 470,
tracing (438.981 µs) : 419, 459
.   : milestone, 439,
section candidate
no_agent (363.833 µs) : 344, 384
.   : milestone, 364,
iast (488.028 µs) : 468, 509
.   : milestone, 488,
iast_FULL (537.963 µs) : 518, 558
.   : milestone, 538,
iast_INACTIVE (457.472 µs) : 437, 478
.   : milestone, 457,
iast_TELEMETRY_OFF (470.101 µs) : 449, 491
.   : milestone, 470,
tracing (442.474 µs) : 422, 463
.   : milestone, 442,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	366.453 µs [346.103 µs, 386.803 µs]	-
iast	472.33 µs [451.844 µs, 492.817 µs]	105.878 µs (28.9%)
iast_FULL	535.668 µs [515.057 µs, 556.28 µs]	169.216 µs (46.2%)
iast_INACTIVE	443.252 µs [422.345 µs, 464.159 µs]	76.8 µs (21.0%)
iast_TELEMETRY_OFF	470.204 µs [448.93 µs, 491.478 µs]	103.751 µs (28.3%)
tracing	438.981 µs [418.53 µs, 459.432 µs]	72.528 µs (19.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	363.833 µs [343.916 µs, 383.751 µs]	-
iast	488.028 µs [467.526 µs, 508.531 µs]	124.195 µs (34.1%)
iast_FULL	537.963 µs [517.814 µs, 558.113 µs]	174.13 µs (47.9%)
iast_INACTIVE	457.472 µs [436.561 µs, 478.383 µs]	93.639 µs (25.7%)
iast_TELEMETRY_OFF	470.101 µs [449.312 µs, 490.891 µs]	106.268 µs (29.2%)
tracing	442.474 µs [421.579 µs, 463.369 µs]	78.64 µs (21.6%)

Fix root coverage packages calculation in large repositories (#6317)

8908066

bm1549 requested review from a team as code owners December 5, 2023 20:49

bm1549 requested review from am312, dougqh and nikita-tkachenko-datadog December 5, 2023 20:49

nikita-tkachenko-datadog approved these changes Dec 5, 2023

View reviewed changes

bm1549 enabled auto-merge December 5, 2023 20:50

bm1549 added type: bug fix Bug fix comp: ci visibility Continuous Integration Visibility labels Dec 5, 2023

bm1549 merged commit 476c346 into release/v1.25.x Dec 6, 2023

bm1549 deleted the bm1549/cherry-pick branch December 6, 2023 15:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Cherrypick: Fix root coverage packages calculation in large repositories (#6317)#6321

Cherrypick: Fix root coverage packages calculation in large repositories (#6317)#6321
bm1549 merged 1 commit into
release/v1.25.xfrom
bm1549/cherry-pick

bm1549 commented Dec 5, 2023

Uh oh!

pr-commenter Bot commented Dec 5, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

bm1549 commented Dec 5, 2023

What Does This Do

Motivation

Additional Notes

Uh oh!

pr-commenter Bot commented Dec 5, 2023

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants