ci(test-optimization): make STS API key fetch non-fatal#9320
Conversation
If the OIDC token exchange fails due to a transient infrastructure issue, jobs were failing before tests even ran. With continue-on-error, ci/init.js gracefully skips reporting when DD_API_KEY is absent and the tests still execute normally. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Overall package sizeSelf size: 6.71 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.1 | 122.62 kB | 438.86 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
![datadog-prod-us1-3[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
More details
PR adds continue-on-error: true to all 7 dd-sts-api-key steps in test-optimization.yml. Verification confirms ci/init.js gracefully handles missing API keys when agentless mode is enabled—it logs an error and skips initialization, allowing tests to run. The change is safe: it prevents transient STS failures from breaking CI runs when the test initialization already tolerates missing keys.
📊 Validated against 4 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit 1350e45 · What is Autotest? · Any feedback? Reach out in #autotest
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 1350e45 | Docs | Datadog PR Page | Give us feedback! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9320 +/- ##
========================================
Coverage 96.60% 96.60%
========================================
Files 919 919
Lines 121692 121692
Branches 21027 20895 -132
========================================
+ Hits 117562 117564 +2
+ Misses 4130 4128 -2 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
BenchmarksBenchmark execution time: 2026-07-14 03:02:17 Comparing candidate commit 1350e45 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2313 metrics, 45 unstable metrics.
|
juan-fernandez
left a comment
juan-fernandez
left a comment
There was a problem hiding this comment.
I'm OK with the change as long as we're aware that this makes test optimization unusable
If the OIDC token exchange fails due to a transient infrastructure issue, jobs were failing before tests even ran. With continue-on-error, ci/init.js gracefully skips reporting when DD_API_KEY is absent and the tests still execute normally. Co-authored-by: Claude Sonnet 4.6 <[email protected]>
If the OIDC token exchange fails due to a transient infrastructure issue, jobs were failing before tests even ran. With continue-on-error, ci/init.js gracefully skips reporting when DD_API_KEY is absent and the tests still execute normally. Co-authored-by: Claude Sonnet 4.6 <[email protected]>
3 participants
What does this PR do?
Adds
continue-on-error: trueto thedd-sts-api-keystep in all 7 jobs of the Test Optimization workflow (playwright, mocha, jest, cucumber, selenium, cypress, vitest).Motivation
Each job independently exchanges a GitHub OIDC token for a Datadog API key via STS. If this exchange fails due to a transient infrastructure issue, the entire job fails before any tests run — producing false failures unrelated to the code under test.
ci/init.jsalready handles a missingDD_API_KEYgracefully: whenDD_CIVISIBILITY_AGENTLESS_ENABLEDis set but no key is present, it logs an error and skips initialization rather than crashing. So making the STS step non-fatal is safe — tests still run and results are still reported if the key is available.Alternative considered
A more robust long-term approach would be to intercept the agentless intake requests in each job and forward them to a single final job for batched submission. This is possible because
DD_CIVISIBILITY_AGENTLESS_URLcan redirect all intake traffic (citestcycle,citestcov, etc.) to an arbitrary URL. Each job would run a local proxy that saves payloads to disk, upload them as GitHub Actions artifacts, and a single final job would download all artifacts, fetch one STS key, and replay the payloads to the real Datadog intake. This would eliminate per-job STS exchanges entirely but requires building a proxy server and replayer. Left as a future improvement.Checklist
continue-on-error: trueadded to all 7dd-sts-api-keysteps