Add git command line client builder to GitInfoProvider by nikita-tkachenko-datadog · Pull Request #5711 · DataDog/dd-trace-java

nikita-tkachenko-datadog · 2023-08-11T14:48:26Z

What Does This Do

Adds a new GitInfoBuilder instance to GitInfoProvider.
The new instance fetches git data using shell commands.

Also repository URL in Git data upload now uses GitInfoProvider instead of directly invoking the shell command.

Motivation

Previously Git data upload relied solely on executing a shell command to obtain Git remote URL, while the other logic that used Git remote URL extracted it from a variety of sources: CI provider or user-supplied environment variables, locally available .git folder, etc.
This change makes Git data upload consistent with the other places in the code that use Git info.

In addition to that, the other places can now benefit from info fetched with shell commands, in case it cannot be obtained from other sources.

Additional notes

The new shell-command-based GitInfoBuilder has lower priority than the other existing sources.

+    } catch (Exception e) {
+      LOGGER.debug("Error while getting Git folder in " + repositoryPath, e);
+    }
+    return Paths.get(repositoryPath, gitFolderName);


pr-commenter · 2023-08-14T13:37:28Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.20.0-SNAPSHOT~78d3f137c8	1.20.0-SNAPSHOT~6236cde918
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~6236cde918, baseline=1.20.0-SNAPSHOT~78d3f137c8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (994.234 ms) : 0, 994234
Total [baseline] (8.69 s) : 0, 8690030
Agent [candidate] (985.98 ms) : 0, 985980
Total [candidate] (8.661 s) : 0, 8660853
section appsec
Agent [baseline] (1.072 s) : 0, 1071679
Total [baseline] (8.729 s) : 0, 8729325
Agent [candidate] (1.087 s) : 0, 1087368
Total [candidate] (8.761 s) : 0, 8760575
section iast
Agent [baseline] (1.082 s) : 0, 1082190
Total [baseline] (9.202 s) : 0, 9201585
Agent [candidate] (1.085 s) : 0, 1084902
Total [candidate] (9.218 s) : 0, 9217695
section profiling
Agent [baseline] (1.164 s) : 0, 1164025
Total [baseline] (8.927 s) : 0, 8927435
Agent [candidate] (1.161 s) : 0, 1160874
Total [candidate] (8.933 s) : 0, 8932707

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	994.234 ms	-
Agent	appsec	1.072 s	77.445 ms (7.8%)
Agent	iast	1.082 s	87.956 ms (8.8%)
Agent	profiling	1.164 s	169.792 ms (17.1%)
Total	tracing	8.69 s	-
Total	appsec	8.729 s	39.295 ms (0.5%)
Total	iast	9.202 s	511.555 ms (5.9%)
Total	profiling	8.927 s	237.405 ms (2.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	985.98 ms	-
Agent	appsec	1.087 s	101.388 ms (10.3%)
Agent	iast	1.085 s	98.922 ms (10.0%)
Agent	profiling	1.161 s	174.894 ms (17.7%)
Total	tracing	8.661 s	-
Total	appsec	8.761 s	99.722 ms (1.2%)
Total	iast	9.218 s	556.842 ms (6.4%)
Total	profiling	8.933 s	271.854 ms (3.1%)

gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~6236cde918, baseline=1.20.0-SNAPSHOT~78d3f137c8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (616.125 ms) : 0, 616125
BytebuddyAgent [candidate] (610.579 ms) : 0, 610579
GlobalTracer [baseline] (287.231 ms) : 0, 287231
GlobalTracer [candidate] (284.543 ms) : 0, 284543
AppSec [baseline] (50.14 ms) : 0, 50140
AppSec [candidate] (50.374 ms) : 0, 50374
Remote Config [baseline] (669.322 µs) : 0, 669
Remote Config [candidate] (669.181 µs) : 0, 669
Telemetry [baseline] (5.423 ms) : 0, 5423
Telemetry [candidate] (5.471 ms) : 0, 5471
section appsec
BytebuddyAgent [baseline] (611.347 ms) : 0, 611347
BytebuddyAgent [candidate] (621.469 ms) : 0, 621469
GlobalTracer [baseline] (284.939 ms) : 0, 284939
GlobalTracer [candidate] (289.63 ms) : 0, 289630
AppSec [baseline] (135.147 ms) : 0, 135147
AppSec [candidate] (135.509 ms) : 0, 135509
Remote Config [baseline] (710.9 µs) : 0, 711
Remote Config [candidate] (741.001 µs) : 0, 741
Telemetry [baseline] (5.186 ms) : 0, 5186
Telemetry [candidate] (5.217 ms) : 0, 5217
section iast
BytebuddyAgent [baseline] (710.787 ms) : 0, 710787
BytebuddyAgent [candidate] (712.251 ms) : 0, 712251
GlobalTracer [baseline] (270.286 ms) : 0, 270286
GlobalTracer [candidate] (271.071 ms) : 0, 271071
AppSec [baseline] (46.835 ms) : 0, 46835
AppSec [candidate] (47.181 ms) : 0, 47181
Remote Config [baseline] (630.756 µs) : 0, 631
Remote Config [candidate] (641.719 µs) : 0, 642
Telemetry [baseline] (5.55 ms) : 0, 5550
Telemetry [candidate] (5.617 ms) : 0, 5617
IAST [baseline] (13.955 ms) : 0, 13955
IAST [candidate] (13.957 ms) : 0, 13957
section profiling
BytebuddyAgent [baseline] (626.797 ms) : 0, 626797
BytebuddyAgent [candidate] (624.261 ms) : 0, 624261
GlobalTracer [baseline] (351.909 ms) : 0, 351909
GlobalTracer [candidate] (350.378 ms) : 0, 350378
AppSec [baseline] (49.779 ms) : 0, 49779
AppSec [candidate] (49.448 ms) : 0, 49448
Remote Config [baseline] (664.006 µs) : 0, 664
Remote Config [candidate] (651.925 µs) : 0, 652
Telemetry [baseline] (5.455 ms) : 0, 5455
Telemetry [candidate] (5.381 ms) : 0, 5381
ProfilingAgent [baseline] (76.035 ms) : 0, 76035
ProfilingAgent [candidate] (77.895 ms) : 0, 77895
Profiling [baseline] (76.059 ms) : 0, 76059
Profiling [candidate] (77.919 ms) : 0, 77919

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~6236cde918, baseline=1.20.0-SNAPSHOT~78d3f137c8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.008 s) : 0, 1007755
Total [baseline] (9.303 s) : 0, 9302856
Agent [candidate] (989.765 ms) : 0, 989765
Total [candidate] (9.176 s) : 0, 9175765
section appsec
Agent [baseline] (1.076 s) : 0, 1076301
Total [baseline] (9.274 s) : 0, 9273695
Agent [candidate] (1.072 s) : 0, 1071575
Total [candidate] (9.276 s) : 0, 9276308
section iast
Agent [baseline] (1.086 s) : 0, 1086108
Total [baseline] (9.403 s) : 0, 9403468
Agent [candidate] (1.092 s) : 0, 1091899
Total [candidate] (9.439 s) : 0, 9439050
section profiling
Agent [baseline] (1.165 s) : 0, 1164584
Total [baseline] (9.532 s) : 0, 9531507
Agent [candidate] (1.158 s) : 0, 1158026
Total [candidate] (9.509 s) : 0, 9509398

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.008 s	-
Agent	appsec	1.076 s	68.547 ms (6.8%)
Agent	iast	1.086 s	78.353 ms (7.8%)
Agent	profiling	1.165 s	156.829 ms (15.6%)
Total	tracing	9.303 s	-
Total	appsec	9.274 s	-29.16 ms (-0.3%)
Total	iast	9.403 s	100.613 ms (1.1%)
Total	profiling	9.532 s	228.651 ms (2.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	989.765 ms	-
Agent	appsec	1.072 s	81.81 ms (8.3%)
Agent	iast	1.092 s	102.135 ms (10.3%)
Agent	profiling	1.158 s	168.261 ms (17.0%)
Total	tracing	9.176 s	-
Total	appsec	9.276 s	100.543 ms (1.1%)
Total	iast	9.439 s	263.285 ms (2.9%)
Total	profiling	9.509 s	333.632 ms (3.6%)

gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~6236cde918, baseline=1.20.0-SNAPSHOT~78d3f137c8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (624.698 ms) : 0, 624698
BytebuddyAgent [candidate] (613.059 ms) : 0, 613059
GlobalTracer [baseline] (291.27 ms) : 0, 291270
GlobalTracer [candidate] (286.272 ms) : 0, 286272
AppSec [baseline] (50.477 ms) : 0, 50477
AppSec [candidate] (50.009 ms) : 0, 50009
Remote Config [baseline] (681.861 µs) : 0, 682
Remote Config [candidate] (664.442 µs) : 0, 664
Telemetry [baseline] (5.481 ms) : 0, 5481
Telemetry [candidate] (5.396 ms) : 0, 5396
section appsec
BytebuddyAgent [baseline] (614.556 ms) : 0, 614556
BytebuddyAgent [candidate] (611.523 ms) : 0, 611523
GlobalTracer [baseline] (285.881 ms) : 0, 285881
GlobalTracer [candidate] (284.934 ms) : 0, 284934
AppSec [baseline] (135.528 ms) : 0, 135528
AppSec [candidate] (134.939 ms) : 0, 134939
Remote Config [baseline] (723.527 µs) : 0, 724
Remote Config [candidate] (723.687 µs) : 0, 724
Telemetry [baseline] (5.158 ms) : 0, 5158
Telemetry [candidate] (5.145 ms) : 0, 5145
section iast
BytebuddyAgent [baseline] (713.61 ms) : 0, 713610
BytebuddyAgent [candidate] (716.988 ms) : 0, 716988
GlobalTracer [baseline] (270.908 ms) : 0, 270908
GlobalTracer [candidate] (272.807 ms) : 0, 272807
AppSec [baseline] (47.232 ms) : 0, 47232
AppSec [candidate] (47.351 ms) : 0, 47351
Remote Config [baseline] (630.748 µs) : 0, 631
Remote Config [candidate] (653.834 µs) : 0, 654
Telemetry [baseline] (5.487 ms) : 0, 5487
Telemetry [candidate] (5.568 ms) : 0, 5568
IAST [baseline] (13.87 ms) : 0, 13870
IAST [candidate] (14.073 ms) : 0, 14073
section profiling
BytebuddyAgent [baseline] (627.573 ms) : 0, 627573
BytebuddyAgent [candidate] (622.634 ms) : 0, 622634
GlobalTracer [baseline] (351.526 ms) : 0, 351526
GlobalTracer [candidate] (348.701 ms) : 0, 348701
AppSec [baseline] (49.341 ms) : 0, 49341
AppSec [candidate] (49.236 ms) : 0, 49236
Remote Config [baseline] (653.768 µs) : 0, 654
Remote Config [candidate] (652.895 µs) : 0, 653
Telemetry [baseline] (5.434 ms) : 0, 5434
Telemetry [candidate] (5.372 ms) : 0, 5372
ProfilingAgent [baseline] (76.826 ms) : 0, 76826
ProfilingAgent [candidate] (78.744 ms) : 0, 78744
Profiling [baseline] (76.85 ms) : 0, 76850
Profiling [candidate] (78.769 ms) : 0, 78769

Load

Parameters

	Baseline	Candidate
commit	1.20.0-SNAPSHOT~78d3f137c8	1.20.0-SNAPSHOT~6236cde918
config	baseline	candidate
end_time	2023-08-16T09:01:30	2023-08-16T09:19:39
start_time	2023-08-16T09:01:12	2023-08-16T09:19:21

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~6236cde918, baseline=1.20.0-SNAPSHOT~78d3f137c8
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.734 µs) : 341, 381
.   : milestone, 361,
appsec (666.549 µs) : 645, 688
.   : milestone, 667,
iast (462.046 µs) : 441, 483
.   : milestone, 462,
iast_FULL (538.757 µs) : 518, 560
.   : milestone, 539,
iast_INACTIVE (444.269 µs) : 422, 466
.   : milestone, 444,
profiling (439.202 µs) : 418, 460
.   : milestone, 439,
tracing (438.85 µs) : 418, 460
.   : milestone, 439,
section candidate
no_agent (362.811 µs) : 342, 384
.   : milestone, 363,
appsec (666.126 µs) : 645, 687
.   : milestone, 666,
iast (463.869 µs) : 443, 485
.   : milestone, 464,
iast_FULL (533.036 µs) : 512, 554
.   : milestone, 533,
iast_INACTIVE (439.116 µs) : 418, 460
.   : milestone, 439,
profiling (439.264 µs) : 419, 460
.   : milestone, 439,
tracing (440.795 µs) : 420, 462
.   : milestone, 441,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	360.734 µs [340.94 µs, 380.527 µs]	-
appsec	666.549 µs [645.422 µs, 687.676 µs]	305.815 µs (84.8%)
iast	462.046 µs [441.268 µs, 482.823 µs]	101.312 µs (28.1%)
iast_FULL	538.757 µs [517.817 µs, 559.697 µs]	178.024 µs (49.4%)
iast_INACTIVE	444.269 µs [422.207 µs, 466.331 µs]	83.535 µs (23.2%)
profiling	439.202 µs [418.407 µs, 459.996 µs]	78.468 µs (21.8%)
tracing	438.85 µs [417.685 µs, 460.016 µs]	78.117 µs (21.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	362.811 µs [341.623 µs, 383.999 µs]	-
appsec	666.126 µs [645.429 µs, 686.823 µs]	303.316 µs (83.6%)
iast	463.869 µs [442.618 µs, 485.12 µs]	101.059 µs (27.9%)
iast_FULL	533.036 µs [511.693 µs, 554.379 µs]	170.225 µs (46.9%)
iast_INACTIVE	439.116 µs [417.818 µs, 460.413 µs]	76.305 µs (21.0%)
profiling	439.264 µs [418.757 µs, 459.771 µs]	76.453 µs (21.1%)
tracing	440.795 µs [419.963 µs, 461.626 µs]	77.984 µs (21.5%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~6236cde918, baseline=1.20.0-SNAPSHOT~78d3f137c8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.342 ms) : 1323, 1361
.   : milestone, 1342,
appsec (1.699 ms) : 1675, 1723
.   : milestone, 1699,
iast (1.467 ms) : 1444, 1491
.   : milestone, 1467,
profiling (1.471 ms) : 1446, 1496
.   : milestone, 1471,
tracing (1.452 ms) : 1427, 1476
.   : milestone, 1452,
section candidate
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.682 ms) : 1658, 1706
.   : milestone, 1682,
iast (1.477 ms) : 1453, 1501
.   : milestone, 1477,
profiling (1.483 ms) : 1457, 1510
.   : milestone, 1483,
tracing (1.453 ms) : 1429, 1477
.   : milestone, 1453,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.342 ms [1.323 ms, 1.361 ms]	-
appsec	1.699 ms [1.675 ms, 1.723 ms]	357.25 µs (26.6%)
iast	1.467 ms [1.444 ms, 1.491 ms]	125.334 µs (9.3%)
profiling	1.471 ms [1.446 ms, 1.496 ms]	129.15 µs (9.6%)
tracing	1.452 ms [1.427 ms, 1.476 ms]	109.849 µs (8.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.341 ms [1.322 ms, 1.36 ms]	-
appsec	1.682 ms [1.658 ms, 1.706 ms]	341.125 µs (25.4%)
iast	1.477 ms [1.453 ms, 1.501 ms]	135.694 µs (10.1%)
profiling	1.483 ms [1.457 ms, 1.51 ms]	142.384 µs (10.6%)
tracing	1.453 ms [1.429 ms, 1.477 ms]	112.111 µs (8.4%)

romainkomorn-exdatadog

Looks sound enough to me but I'm still not enough of a Java dev to approve. 😅

PerfectSlayer

LGTM.
I notice you checked the GitHub vuln report too.

Update repository URL extraction logic for Git upload

cfce5f6

nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Aug 11, 2023

Add git command line client builder to centralized git info provider

109e50e

nikita-tkachenko-datadog changed the title ~~Update repository URL extraction logic for Git upload~~ Add git command line client builder to GitInfoProvider Aug 14, 2023

github-advanced-security AI found potential problems Aug 14, 2023

View reviewed changes

Ignore Git remote URLs that start with file: protocol

1ab6f4d

Add code coverage exclusion

6236cde

nikita-tkachenko-datadog marked this pull request as ready for review August 17, 2023 14:00

nikita-tkachenko-datadog requested review from a team as code owners August 17, 2023 14:00

romainkomorn-exdatadog reviewed Aug 21, 2023

View reviewed changes

PerfectSlayer reviewed Aug 21, 2023

View reviewed changes

PerfectSlayer approved these changes Aug 21, 2023

View reviewed changes

nikita-tkachenko-datadog merged commit 238c3c1 into master Aug 21, 2023

nikita-tkachenko-datadog deleted the nikita-tkachenko/itr-git-remote-url branch August 21, 2023 08:19

github-actions Bot added this to the 1.20.0 milestone Aug 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add git command line client builder to GitInfoProvider#5711

Add git command line client builder to GitInfoProvider#5711
nikita-tkachenko-datadog merged 4 commits into
masterfrom
nikita-tkachenko/itr-git-remote-url

nikita-tkachenko-datadog commented Aug 11, 2023 •

edited

Loading

Uh oh!

Check failure

pr-commenter Bot commented Aug 14, 2023 •

edited

Loading

Uh oh!

romainkomorn-exdatadog left a comment

Uh oh!

PerfectSlayer left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

nikita-tkachenko-datadog commented Aug 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional notes

Uh oh!

Check failure

pr-commenter Bot commented Aug 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

romainkomorn-exdatadog left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nikita-tkachenko-datadog commented Aug 11, 2023 •

edited

Loading

pr-commenter Bot commented Aug 14, 2023 •

edited

Loading