Add config source reading from Test Optimization remote environment #9701
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nikita-tkachenko-datadog
added
type: enhancement
Contributor
|
🎯 Code Coverage 🔗 Commit SHA: 76dead1 | Docs | Was this helpful? Give us feedback! |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 5 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.016 s) : 0, 1016350
Total [baseline] (10.732 s) : 0, 10731857
Agent [candidate] (1.016 s) : 0, 1015644
Total [candidate] (10.706 s) : 0, 10706268
section appsec
Agent [baseline] (1.198 s) : 0, 1197892
Total [baseline] (10.772 s) : 0, 10771980
Agent [candidate] (1.193 s) : 0, 1192656
Total [candidate] (11.104 s) : 0, 11103928
section iast
Agent [baseline] (1.153 s) : 0, 1153224
Total [baseline] (11.082 s) : 0, 11081515
Agent [candidate] (1.149 s) : 0, 1148763
Total [candidate] (11.081 s) : 0, 11081032
section profiling
Agent [baseline] (1.171 s) : 0, 1171411
Total [baseline] (10.88 s) : 0, 10879744
Agent [candidate] (1.164 s) : 0, 1163945
Total [candidate] (11.008 s) : 0, 11007620
gantt
title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (691.878 ms) : 0, 691878
BytebuddyAgent [candidate] (693.261 ms) : 0, 693261
GlobalTracer [baseline] (242.43 ms) : 0, 242430
GlobalTracer [candidate] (241.711 ms) : 0, 241711
AppSec [baseline] (32.662 ms) : 0, 32662
AppSec [candidate] (32.305 ms) : 0, 32305
Debugger [baseline] (6.404 ms) : 0, 6404
Debugger [candidate] (6.304 ms) : 0, 6304
Remote Config [baseline] (676.454 µs) : 0, 676
Remote Config [candidate] (679.439 µs) : 0, 679
Telemetry [baseline] (9.416 ms) : 0, 9416
Telemetry [candidate] (9.301 ms) : 0, 9301
Flare Poller [baseline] (10.307 ms) : 0, 10307
Flare Poller [candidate] (9.641 ms) : 0, 9641
section appsec
crashtracking [baseline] (1.461 ms) : 0, 1461
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (719.348 ms) : 0, 719348
BytebuddyAgent [candidate] (717.337 ms) : 0, 717337
GlobalTracer [baseline] (236.674 ms) : 0, 236674
GlobalTracer [candidate] (233.973 ms) : 0, 233973
AppSec [baseline] (175.0 ms) : 0, 175000
AppSec [candidate] (174.874 ms) : 0, 174874
Debugger [baseline] (6.072 ms) : 0, 6072
Debugger [candidate] (6.08 ms) : 0, 6080
Remote Config [baseline] (623.59 µs) : 0, 624
Remote Config [candidate] (639.915 µs) : 0, 640
Telemetry [baseline] (8.462 ms) : 0, 8462
Telemetry [candidate] (8.458 ms) : 0, 8458
Flare Poller [baseline] (3.867 ms) : 0, 3867
Flare Poller [candidate] (3.88 ms) : 0, 3880
IAST [baseline] (25.226 ms) : 0, 25226
IAST [candidate] (24.834 ms) : 0, 24834
section iast
crashtracking [baseline] (1.468 ms) : 0, 1468
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (816.329 ms) : 0, 816329
BytebuddyAgent [candidate] (813.492 ms) : 0, 813492
GlobalTracer [baseline] (232.543 ms) : 0, 232543
GlobalTracer [candidate] (230.465 ms) : 0, 230465
AppSec [baseline] (35.032 ms) : 0, 35032
AppSec [candidate] (35.399 ms) : 0, 35399
Debugger [baseline] (6.186 ms) : 0, 6186
Debugger [candidate] (6.196 ms) : 0, 6196
Remote Config [baseline] (599.175 µs) : 0, 599
Remote Config [candidate] (598.847 µs) : 0, 599
Telemetry [baseline] (8.677 ms) : 0, 8677
Telemetry [candidate] (8.779 ms) : 0, 8779
Flare Poller [baseline] (4.224 ms) : 0, 4224
Flare Poller [candidate] (4.322 ms) : 0, 4322
IAST [baseline] (26.637 ms) : 0, 26637
IAST [candidate] (26.664 ms) : 0, 26664
section profiling
ProfilingAgent [baseline] (109.313 ms) : 0, 109313
ProfilingAgent [candidate] (109.053 ms) : 0, 109053
crashtracking [baseline] (1.447 ms) : 0, 1447
crashtracking [candidate] (1.463 ms) : 0, 1463
BytebuddyAgent [baseline] (726.117 ms) : 0, 726117
BytebuddyAgent [candidate] (721.727 ms) : 0, 721727
GlobalTracer [baseline] (220.537 ms) : 0, 220537
GlobalTracer [candidate] (217.963 ms) : 0, 217963
AppSec [baseline] (32.663 ms) : 0, 32663
AppSec [candidate] (32.455 ms) : 0, 32455
Debugger [baseline] (7.546 ms) : 0, 7546
Debugger [candidate] (6.597 ms) : 0, 6597
Remote Config [baseline] (743.809 µs) : 0, 744
Remote Config [candidate] (828.349 µs) : 0, 828
Telemetry [baseline] (15.254 ms) : 0, 15254
Telemetry [candidate] (15.856 ms) : 0, 15856
Flare Poller [baseline] (4.156 ms) : 0, 4156
Flare Poller [candidate] (4.138 ms) : 0, 4138
Profiling [baseline] (110.479 ms) : 0, 110479
Profiling [candidate] (109.641 ms) : 0, 109641
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.016 s) : 0, 1015518
Total [baseline] (8.66 s) : 0, 8660101
Agent [candidate] (1.026 s) : 0, 1025802
Total [candidate] (8.711 s) : 0, 8711352
section iast
Agent [baseline] (1.153 s) : 0, 1153026
Total [baseline] (9.28 s) : 0, 9279725
Agent [candidate] (1.149 s) : 0, 1149340
Total [candidate] (9.267 s) : 0, 9267301
gantt
title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.487 ms) : 0, 1487
crashtracking [candidate] (1.48 ms) : 0, 1480
BytebuddyAgent [baseline] (692.317 ms) : 0, 692317
BytebuddyAgent [candidate] (698.647 ms) : 0, 698647
GlobalTracer [baseline] (242.233 ms) : 0, 242233
GlobalTracer [candidate] (243.857 ms) : 0, 243857
AppSec [baseline] (32.562 ms) : 0, 32562
AppSec [candidate] (32.914 ms) : 0, 32914
Debugger [baseline] (6.385 ms) : 0, 6385
Debugger [candidate] (6.459 ms) : 0, 6459
Remote Config [baseline] (676.431 µs) : 0, 676
Remote Config [candidate] (688.806 µs) : 0, 689
Telemetry [baseline] (9.45 ms) : 0, 9450
Telemetry [candidate] (9.548 ms) : 0, 9548
Flare Poller [baseline] (9.287 ms) : 0, 9287
Flare Poller [candidate] (10.983 ms) : 0, 10983
section iast
crashtracking [baseline] (1.487 ms) : 0, 1487
crashtracking [candidate] (1.484 ms) : 0, 1484
BytebuddyAgent [baseline] (816.118 ms) : 0, 816118
BytebuddyAgent [candidate] (814.4 ms) : 0, 814400
GlobalTracer [baseline] (232.412 ms) : 0, 232412
GlobalTracer [candidate] (230.759 ms) : 0, 230759
IAST [baseline] (26.65 ms) : 0, 26650
IAST [candidate] (26.633 ms) : 0, 26633
AppSec [baseline] (35.163 ms) : 0, 35163
AppSec [candidate] (35.009 ms) : 0, 35009
Debugger [baseline] (6.158 ms) : 0, 6158
Debugger [candidate] (6.148 ms) : 0, 6148
Remote Config [baseline] (601.492 µs) : 0, 601
Remote Config [candidate] (600.625 µs) : 0, 601
Telemetry [baseline] (8.704 ms) : 0, 8704
Telemetry [candidate] (8.708 ms) : 0, 8708
Flare Poller [baseline] (4.269 ms) : 0, 4269
Flare Poller [candidate] (4.218 ms) : 0, 4218
LoadParameters
See matching parameters
SummaryFound 3 performance improvements and 4 performance regressions! Performance is the same for 5 metrics, 12 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section baseline
no_agent (37.296 ms) : 36994, 37598
. : milestone, 37296,
appsec (49.156 ms) : 48717, 49595
. : milestone, 49156,
code_origins (43.148 ms) : 42774, 43522
. : milestone, 43148,
iast (44.263 ms) : 43861, 44664
. : milestone, 44263,
profiling (49.78 ms) : 49306, 50254
. : milestone, 49780,
tracing (43.771 ms) : 43400, 44143
. : milestone, 43771,
section candidate
no_agent (35.899 ms) : 35621, 36177
. : milestone, 35899,
appsec (48.414 ms) : 47998, 48830
. : milestone, 48414,
code_origins (45.87 ms) : 45486, 46253
. : milestone, 45870,
iast (44.204 ms) : 43821, 44586
. : milestone, 44204,
profiling (49.068 ms) : 48610, 49525
. : milestone, 49068,
tracing (43.755 ms) : 43389, 44121
. : milestone, 43755,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section baseline
no_agent (4.347 ms) : 4293, 4401
. : milestone, 4347,
iast (9.138 ms) : 8989, 9287
. : milestone, 9138,
iast_FULL (14.214 ms) : 13934, 14495
. : milestone, 14214,
iast_GLOBAL (10.838 ms) : 10630, 11046
. : milestone, 10838,
profiling (9.011 ms) : 8871, 9150
. : milestone, 9011,
tracing (8.285 ms) : 8159, 8411
. : milestone, 8285,
section candidate
no_agent (4.525 ms) : 4475, 4575
. : milestone, 4525,
iast (9.974 ms) : 9803, 10145
. : milestone, 9974,
iast_FULL (14.36 ms) : 14073, 14648
. : milestone, 14360,
iast_GLOBAL (11.55 ms) : 11340, 11760
. : milestone, 11550,
profiling (8.357 ms) : 8220, 8493
. : milestone, 8357,
tracing (7.812 ms) : 7685, 7938
. : milestone, 7812,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section baseline
no_agent (15.318 s) : 15318000, 15318000
. : milestone, 15318000,
appsec (14.95 s) : 14950000, 14950000
. : milestone, 14950000,
iast (18.639 s) : 18639000, 18639000
. : milestone, 18639000,
iast_GLOBAL (17.718 s) : 17718000, 17718000
. : milestone, 17718000,
profiling (15.528 s) : 15528000, 15528000
. : milestone, 15528000,
tracing (15.16 s) : 15160000, 15160000
. : milestone, 15160000,
section candidate
no_agent (15.57 s) : 15570000, 15570000
. : milestone, 15570000,
appsec (14.805 s) : 14805000, 14805000
. : milestone, 14805000,
iast (18.393 s) : 18393000, 18393000
. : milestone, 18393000,
iast_GLOBAL (18.071 s) : 18071000, 18071000
. : milestone, 18071000,
profiling (15.563 s) : 15563000, 15563000
. : milestone, 15563000,
tracing (15.278 s) : 15278000, 15278000
. : milestone, 15278000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~76dead19bf, baseline=1.55.0-SNAPSHOT~664b9a412c
dateFormat X
axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
. : milestone, 1480,
appsec (3.7 ms) : 3484, 3915
. : milestone, 3700,
iast (2.202 ms) : 2139, 2265
. : milestone, 2202,
iast_GLOBAL (2.252 ms) : 2189, 2316
. : milestone, 2252,
profiling (2.06 ms) : 2008, 2113
. : milestone, 2060,
tracing (2.042 ms) : 1992, 2092
. : milestone, 2042,
section candidate
no_agent (1.477 ms) : 1465, 1488
. : milestone, 1477,
appsec (3.727 ms) : 3507, 3947
. : milestone, 3727,
iast (2.209 ms) : 2145, 2272
. : milestone, 2209,
iast_GLOBAL (2.253 ms) : 2189, 2317
. : milestone, 2253,
profiling (2.08 ms) : 2027, 2133
. : milestone, 2080,
tracing (2.043 ms) : 1992, 2093
. : milestone, 2043,
|
nikita-tkachenko-datadog
requested review from
Mariovido and
PerfectSlayer
and removed request for
a team
daniel-mohedano
approved these changes
Oct 10, 2025
Contributor
There was a problem hiding this comment.
💭 thought: I have few concerns about this change.
First, it introduce both product -- CI Visibility -- and feature -- Config -- into a platform component. It also introduces a dependency to a logger.
I would very like to keep the platform component as light and "single-concerned" as possible.
🎯 suggestion: As an alternative, what about implementing an EnvironmentVariablesProvider with your logic, and apply is as source for the EnvironmentVariable?
If it works, that would leave both :environment and :config-utils untouched.
because Bazel runs tests in a sandbox where environment variables from the "outside" are not available by default.
❔ question: Just to be sure I get it right, it concerns cases where the dd-java-agent runs with the bazel tools when users runs their tests, right?
nikita-tkachenko-datadog
force-pushed
the
nikita-tkachenko/ci-environment-service
branch
from
6c45053 to
434fc92
Compare
nikita-tkachenko-datadog
force-pushed
the
nikita-tkachenko/ci-environment-service
branch
from
e902d3a to
75e4253
Compare
PerfectSlayer
approved these changes
Oct 17, 2025
Contributor
PerfectSlayer
left a comment
PerfectSlayer
left a comment
There was a problem hiding this comment.
Left minor comments.
Thanks for the follow changes! 🙏
Comment on lines
57
to
59
| private static String toEnvVar(String string) { | ||
| return string.replace('.', '_').replace('-', '_').toUpperCase(); | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 suggestion: You can use ConfigStrings.toEnvVar() instead
Contributor
Author
There was a problem hiding this comment.
Good catch, thanks! I had to duplicate the method initially since the class lived in a different module which didn't have the access to ConfigStrings. Fixed now
|
|
||
| @Nullable | ||
| public static Map<String, String> getAll() { | ||
| return REMOTE_ENVIRONMENT != null ? Collections.unmodifiableMap(REMOTE_ENVIRONMENT) : null; |
Contributor
There was a problem hiding this comment.
🎯 suggestion: What about returning an empty collection rather than null? Is there a semantic difference?
I'm asking because the propertiesSource is checking for empty collection too before creating a PropertiesConfigSource.
Contributor
Author
There was a problem hiding this comment.
From my perspective there is a semantic difference, as an empty collection could mean we made the call to the remote service and received a successful empty response, while a null means we didn't/couldn't make the request. While in this particular case there isn't much difference, I'd prefer to leave it as it is to be able to know the difference
nikita-tkachenko-datadog
deleted the
nikita-tkachenko/ci-environment-service
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Adds a new config source that reads from Test Optimization remote environment.
The tracer will only use the new config source when explicitly configured to do so.
Motivation
For Test Optimization runs with Bazel we cannot read environment variables as usual, because Bazel runs tests in a sandbox where environment variables from the "outside" are not available by default.
Any environment variable exposed to the tests messes with Bazel's cache, as all env vars are considered as test tasks input, so if any of them changes, the previously cached outcome of the task cannot be used anymore.
Some environment vars read by the tracer (such as git commit SHA or CI job ID) change constantly making Bazel cache virtually unusable.
The temporary solution for now is to use a service that the tracer can query over HTTP to get the environment variables, which allows to circumvent Bazel's isolation without breaking caching.
Additional Notes
The solution described above is already being used for the env vars that the tracer reads from the CI provider.
This PR extends it to also be used for the env vars that control config values.
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: LANGTOOLS-3400