test(test-optimization): stabilize Playwright retry tagging flake#9216
Conversation
Overall package sizeSelf size: 6.53 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.0 | 117.14 kB | 432.47 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
|
2998691 to
1227b0e
Compare
1227b0e to
9d7f67e
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9216 +/- ##
==========================================
+ Coverage 93.64% 93.65% +0.01%
==========================================
Files 898 899 +1
Lines 52361 52473 +112
Branches 12321 12352 +31
==========================================
+ Hits 49033 49144 +111
- Misses 3328 3329 +1 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
More details
The root-cause flake is confirmed synthetic: gatherPayloadsMaxTimeout has an early-exit path that calls the assertion callback on each arriving payload and resolves immediately if it passes — or, if still failing at the 15-second mark, rejects with a partial buffer. Under CI timing pressure, only 2 of the expected 4 retry events had arrived when the timeout fired. gatherPayloadsUntilChildExit eliminates the race entirely by deferring the single assertion until child exit + 1-second grace period, collecting the full buffer first.
📊 Validated against 6 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit 9d7f67e · What is Autotest? · Any feedback? Reach out in #autotest
BenchmarksBenchmark execution time: 2026-07-06 12:15:21 Comparing candidate commit 9d7f67e in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2247 metrics, 39 unstable metrics.
|
What does this PR do?
Stabilizes the Playwright retry-tagging integration test by waiting for the Playwright child process to exit before asserting the final intake payload buffer.
Motivation
The linked GitHub Actions job failed on its first attempt and passed on retry. The failing attempt asserted after seeing only 2 of the expected 4 retry test events, so the test could fail before all retry payloads had drained into the fake intake.
Flaky job evidence:
Additional Notes
Validated with:
./node_modules/.bin/eslint integration-tests/playwright/playwright-active-test-span.spec.js --max-warnings 0 PLAYWRIGHT_VERSION=latest SPEC=playwright-active-test-span npm run test:integration:playwright:coverage -- --grep "does not send attempt to fix tags" git diff --check