Implement a fallback method for getting effective JVM for Maven Surefire executions#7493
Merged
nikita-tkachenko-datadog merged 1 commit intomasterfrom Aug 23, 2024
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 13 unstable metrics.
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.546 s) : 0, 1545742
Total [baseline] (11.802 s) : 0, 11801975
Agent [candidate] (1.553 s) : 0, 1552932
Total [candidate] (11.846 s) : 0, 11846012
section iast
Agent [baseline] (1.714 s) : 0, 1713715
Total [baseline] (12.489 s) : 0, 12489250
Agent [candidate] (1.717 s) : 0, 1716979
Total [candidate] (12.483 s) : 0, 12482720
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.718 s) : 0, 1717915
Total [baseline] (12.471 s) : 0, 12470679
Agent [candidate] (1.714 s) : 0, 1714005
Total [candidate] (12.474 s) : 0, 12474103
section iast_TELEMETRY_OFF
Agent [baseline] (1.71 s) : 0, 1709713
Total [baseline] (12.453 s) : 0, 12452676
Agent [candidate] (1.709 s) : 0, 1708888
Total [candidate] (12.476 s) : 0, 12475985
gantt
title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (988.835 ms) : 0, 988835
BytebuddyAgent [candidate] (994.918 ms) : 0, 994918
GlobalTracer [baseline] (453.445 ms) : 0, 453445
GlobalTracer [candidate] (454.772 ms) : 0, 454772
AppSec [baseline] (72.899 ms) : 0, 72899
AppSec [candidate] (72.51 ms) : 0, 72510
Remote Config [baseline] (835.353 µs) : 0, 835
Remote Config [candidate] (827.292 µs) : 0, 827
Telemetry [baseline] (9.798 ms) : 0, 9798
Telemetry [candidate] (9.815 ms) : 0, 9815
section iast
BytebuddyAgent [baseline] (1.146 s) : 0, 1146173
BytebuddyAgent [candidate] (1.149 s) : 0, 1148859
GlobalTracer [baseline] (433.763 ms) : 0, 433763
GlobalTracer [candidate] (435.065 ms) : 0, 435065
AppSec [baseline] (70.939 ms) : 0, 70939
AppSec [candidate] (73.839 ms) : 0, 73839
IAST [baseline] (30.205 ms) : 0, 30205
IAST [candidate] (27.424 ms) : 0, 27424
Remote Config [baseline] (753.362 µs) : 0, 753
Remote Config [candidate] (753.687 µs) : 0, 754
Telemetry [baseline] (11.845 ms) : 0, 11845
Telemetry [candidate] (10.957 ms) : 0, 10957
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (1.149 s) : 0, 1149025
BytebuddyAgent [candidate] (1.146 s) : 0, 1146044
GlobalTracer [baseline] (435.664 ms) : 0, 435664
GlobalTracer [candidate] (435.542 ms) : 0, 435542
AppSec [baseline] (72.164 ms) : 0, 72164
AppSec [candidate] (71.271 ms) : 0, 71271
IAST [baseline] (29.334 ms) : 0, 29334
IAST [candidate] (29.367 ms) : 0, 29367
Remote Config [baseline] (742.359 µs) : 0, 742
Remote Config [candidate] (740.888 µs) : 0, 741
Telemetry [baseline] (10.903 ms) : 0, 10903
Telemetry [candidate] (10.903 ms) : 0, 10903
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (1.142 s) : 0, 1141929
BytebuddyAgent [candidate] (1.142 s) : 0, 1141897
GlobalTracer [baseline] (434.601 ms) : 0, 434601
GlobalTracer [candidate] (435.143 ms) : 0, 435143
AppSec [baseline] (68.693 ms) : 0, 68693
AppSec [candidate] (70.818 ms) : 0, 70818
IAST [baseline] (33.824 ms) : 0, 33824
IAST [candidate] (30.383 ms) : 0, 30383
Remote Config [baseline] (743.214 µs) : 0, 743
Remote Config [candidate] (741.613 µs) : 0, 742
Telemetry [baseline] (9.894 ms) : 0, 9894
Telemetry [candidate] (9.838 ms) : 0, 9838
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.548 s) : 0, 1548411
Total [baseline] (14.341 s) : 0, 14340999
Agent [candidate] (1.549 s) : 0, 1548520
Total [candidate] (14.323 s) : 0, 14322596
section appsec
Agent [baseline] (1.736 s) : 0, 1735575
Total [baseline] (14.448 s) : 0, 14448213
Agent [candidate] (1.734 s) : 0, 1734481
Total [candidate] (14.385 s) : 0, 14385306
section iast
Agent [baseline] (1.736 s) : 0, 1735650
Total [baseline] (14.862 s) : 0, 14862105
Agent [candidate] (1.714 s) : 0, 1713902
Total [candidate] (14.944 s) : 0, 14944257
section profiling
Agent [baseline] (1.858 s) : 0, 1857967
Total [baseline] (14.735 s) : 0, 14735082
Agent [candidate] (1.864 s) : 0, 1863506
Total [candidate] (14.624 s) : 0, 14624114
gantt
title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (991.757 ms) : 0, 991757
BytebuddyAgent [candidate] (991.877 ms) : 0, 991877
GlobalTracer [baseline] (453.358 ms) : 0, 453358
GlobalTracer [candidate] (453.713 ms) : 0, 453713
AppSec [baseline] (72.55 ms) : 0, 72550
AppSec [candidate] (72.233 ms) : 0, 72233
Remote Config [baseline] (827.677 µs) : 0, 828
Remote Config [candidate] (823.517 µs) : 0, 824
Telemetry [baseline] (9.911 ms) : 0, 9911
Telemetry [candidate] (9.816 ms) : 0, 9816
section appsec
BytebuddyAgent [baseline] (1.008 s) : 0, 1008129
BytebuddyAgent [candidate] (1.008 s) : 0, 1007629
GlobalTracer [baseline] (447.403 ms) : 0, 447403
GlobalTracer [candidate] (446.561 ms) : 0, 446561
AppSec [baseline] (234.87 ms) : 0, 234870
AppSec [candidate] (235.144 ms) : 0, 235144
IAST [baseline] (26.214 ms) : 0, 26214
IAST [candidate] (26.845 ms) : 0, 26845
Remote Config [baseline] (756.823 µs) : 0, 757
Remote Config [candidate] (756.533 µs) : 0, 757
Telemetry [baseline] (11.311 ms) : 0, 11311
Telemetry [candidate] (10.919 ms) : 0, 10919
section iast
BytebuddyAgent [baseline] (1.162 s) : 0, 1161544
BytebuddyAgent [candidate] (1.146 s) : 0, 1145946
GlobalTracer [baseline] (438.738 ms) : 0, 438738
GlobalTracer [candidate] (434.502 ms) : 0, 434502
AppSec [baseline] (76.332 ms) : 0, 76332
AppSec [candidate] (71.313 ms) : 0, 71313
IAST [baseline] (27.841 ms) : 0, 27841
IAST [candidate] (30.316 ms) : 0, 30316
Remote Config [baseline] (774.516 µs) : 0, 775
Remote Config [candidate] (793.986 µs) : 0, 794
Telemetry [baseline] (10.216 ms) : 0, 10216
Telemetry [candidate] (10.971 ms) : 0, 10971
section profiling
BytebuddyAgent [baseline] (986.151 ms) : 0, 986151
BytebuddyAgent [candidate] (987.551 ms) : 0, 987551
GlobalTracer [baseline] (584.031 ms) : 0, 584031
GlobalTracer [candidate] (584.337 ms) : 0, 584337
AppSec [baseline] (73.8 ms) : 0, 73800
AppSec [candidate] (73.875 ms) : 0, 73875
Remote Config [baseline] (877.279 µs) : 0, 877
Remote Config [candidate] (894.194 µs) : 0, 894
Telemetry [baseline] (9.532 ms) : 0, 9532
Telemetry [candidate] (9.541 ms) : 0, 9541
ProfilingAgent [baseline] (147.711 ms) : 0, 147711
ProfilingAgent [candidate] (151.238 ms) : 0, 151238
Profiling [baseline] (147.773 ms) : 0, 147773
Profiling [candidate] (151.31 ms) : 0, 151310
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section baseline
no_agent (1.335 ms) : 1316, 1353
. : milestone, 1335,
appsec (1.714 ms) : 1691, 1738
. : milestone, 1714,
appsec_no_iast (1.701 ms) : 1677, 1726
. : milestone, 1701,
iast (1.458 ms) : 1436, 1481
. : milestone, 1458,
profiling (1.54 ms) : 1515, 1566
. : milestone, 1540,
tracing (1.461 ms) : 1437, 1485
. : milestone, 1461,
section candidate
no_agent (1.336 ms) : 1318, 1355
. : milestone, 1336,
appsec (1.725 ms) : 1701, 1748
. : milestone, 1725,
appsec_no_iast (1.692 ms) : 1668, 1715
. : milestone, 1692,
iast (1.445 ms) : 1422, 1467
. : milestone, 1445,
profiling (1.523 ms) : 1496, 1549
. : milestone, 1523,
tracing (1.458 ms) : 1433, 1483
. : milestone, 1458,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section baseline
no_agent (363.603 µs) : 343, 384
. : milestone, 364,
iast (484.842 µs) : 462, 508
. : milestone, 485,
iast_FULL (547.69 µs) : 526, 569
. : milestone, 548,
iast_GLOBAL (499.932 µs) : 478, 522
. : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (482.98 µs) : 460, 506
. : milestone, 483,
iast_INACTIVE (442.674 µs) : 422, 464
. : milestone, 443,
iast_TELEMETRY_OFF (474.48 µs) : 452, 497
. : milestone, 474,
tracing (431.893 µs) : 410, 454
. : milestone, 432,
section candidate
no_agent (361.948 µs) : 342, 382
. : milestone, 362,
iast (477.771 µs) : 455, 500
. : milestone, 478,
iast_FULL (545.595 µs) : 524, 567
. : milestone, 546,
iast_GLOBAL (496.859 µs) : 475, 519
. : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (474.803 µs) : 453, 497
. : milestone, 475,
iast_INACTIVE (435.864 µs) : 415, 456
. : milestone, 436,
iast_TELEMETRY_OFF (470.949 µs) : 449, 493
. : milestone, 471,
tracing (439.066 µs) : 419, 460
. : milestone, 439,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section baseline
no_agent (14.98 s) : 14980000, 14980000
. : milestone, 14980000,
appsec (15.098 s) : 15098000, 15098000
. : milestone, 15098000,
iast (18.614 s) : 18614000, 18614000
. : milestone, 18614000,
iast_GLOBAL (18.189 s) : 18189000, 18189000
. : milestone, 18189000,
profiling (15.414 s) : 15414000, 15414000
. : milestone, 15414000,
tracing (14.835 s) : 14835000, 14835000
. : milestone, 14835000,
section candidate
no_agent (15.358 s) : 15358000, 15358000
. : milestone, 15358000,
appsec (15.117 s) : 15117000, 15117000
. : milestone, 15117000,
iast (18.996 s) : 18996000, 18996000
. : milestone, 18996000,
iast_GLOBAL (17.9 s) : 17900000, 17900000
. : milestone, 17900000,
profiling (15.336 s) : 15336000, 15336000
. : milestone, 15336000,
tracing (14.943 s) : 14943000, 14943000
. : milestone, 14943000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~3d6914a14f, baseline=1.39.0-SNAPSHOT~f6c87de39a
dateFormat X
axisFormat %s
section baseline
no_agent (1.458 ms) : 1447, 1470
. : milestone, 1458,
appsec (2.222 ms) : 2187, 2257
. : milestone, 2222,
iast (1.959 ms) : 1917, 2002
. : milestone, 1959,
iast_GLOBAL (2.03 ms) : 1986, 2074
. : milestone, 2030,
profiling (1.845 ms) : 1811, 1878
. : milestone, 1845,
tracing (1.834 ms) : 1802, 1867
. : milestone, 1834,
section candidate
no_agent (1.456 ms) : 1444, 1467
. : milestone, 1456,
appsec (2.216 ms) : 2181, 2251
. : milestone, 2216,
iast (1.957 ms) : 1916, 1999
. : milestone, 1957,
iast_GLOBAL (2.01 ms) : 1967, 2053
. : milestone, 2010,
profiling (1.868 ms) : 1832, 1903
. : milestone, 1868,
tracing (1.835 ms) : 1802, 1868
. : milestone, 1835,
|
juan-fernandez
approved these changes
Aug 23, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Implements a fallback method for getting effective JVM for a traced Maven Surefire plugin invocation.
Additional Notes
The original logic looks up and invokes a private
getEffectiveJvm()method from the plugin's configured MOJO - this is the method that Surefire plugin itself uses to determine the JVM to use for forking.It is not always possible to look up the method and invoke it - one example is when the MOJO is proxied (although I could not reproduce this locally, this happens for some customers).
The fallback logic tries to follow the steps of the
getEffectiveJvm()as closely as possible.Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: SDTEST-723