fix(test-optimization): support concurrent Vitest tests#8412
Conversation
Overall package sizeSelf size: 6.51 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8412 +/- ##
=======================================
Coverage 93.63% 93.63%
=======================================
Files 896 896
Lines 52220 52241 +21
Branches 12265 12271 +6
=======================================
+ Hits 48895 48916 +21
Misses 3325 3325 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 9de766e | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-07-02 14:54:42 Comparing candidate commit 9de766e in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2250 metrics, 36 unstable metrics.
|
b032447 to
810201b
Compare
juan-fernandez
changed the title
![datadog-prod-us1-6[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
The hook-deduplication and per-task WeakSet refactors are sound: hooks wrap once and dynamically resolve the active task from args[0]?.task, and retry-reason state is correctly isolated per task. One omission: onAfterTryTask lacks the !isConcurrentTask guard that onBeforeTryTask correctly has for the DI probe wait, causing a spurious 400 ms waitForHitProbe() per concurrent ATR retry when DI is enabled (probe is never set for concurrent tasks, so the wait always times out needlessly). Fix applied.
📊 Validated against 16 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit 810201b · What is Autotest? · Any feedback? Reach out in #autotest
810201b to
9de766e
Compare
|
@codex review |
|
/datadog autotest review |
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
More details
The hook-wrapping overhaul is architecturally sound. The key correctness property — that wrapSuiteHookFn wraps each suite hook only once and then resolves the active test via args[0]?.task at call time — was verified by running adversarial simulations covering sequential, concurrent, and describe.concurrent scenarios. All 22 unit-level checks passed, confirming the dynamic task resolution works correctly when Vitest passes the test context (which it does for beforeEach/afterEach hooks across supported versions ≥1.6.0). The hasConcurrentTask recursion correctly detects both test.concurrent and describe.concurrent trees. No additional tests recommended: the integration test suite already exercises every new concurrent path end-to-end.
📊 Validated against 22 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit 9de766e · What is Autotest? · Any feedback? Reach out in #autotest
E2E Test Report: SUCCESSTested by: Shepherd Agent (autonomous QA for Datadog Test Optimization) Test Environment
Results
Test Methodology
Issues Found
Verification
This E2E test was performed by Shepherd - autonomous QA agent for Datadog Test Optimization. |
2 participants
What does this PR do?
Adds support for Vitest
test.concurrentin Test Optimization when retries, early flake detection, impacted tests, test management, and hook context propagation are enabled.Motivation
Vitest can interleave concurrent tasks in the same worker. The worker instrumentation had task state that was effectively process-global for retry reasons, and shared suite hooks were repeatedly wrapped with a task-specific closure. That could attribute retry metadata or hook-created spans to the wrong test when concurrent tests ran together.
This keeps the existing Test Optimization session definition: one test session and module for the Vitest invocation, with each concurrent test reported as its own test span under that session.
Additional Notes
vitest-main.js,vitest-worker.js,vitest-util.js).