feat(cypress): tag failure screenshot upload result#9232
Conversation
Overall package sizeSelf size: 6.6 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.0 | 117.14 kB | 432.47 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: ac923b6 | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-07-07 13:30:43 Comparing candidate commit ac923b6 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2246 metrics, 40 unstable metrics.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9232 +/- ##
==========================================
- Coverage 93.55% 93.49% -0.07%
==========================================
Files 900 901 +1
Lines 52941 53380 +439
Branches 12491 12602 +111
==========================================
+ Hits 49531 49907 +376
- Misses 3410 3473 +63
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@codex review |
|
/datadog autotest review |
There was a problem hiding this comment.
More details
Implementation correctly tracks screenshot upload outcomes and tags test events. Promises are properly chained to ensure test spans finish with tags before suite spans complete. No behavioral regressions detected through code analysis and synthetic testing.
📊 Validated against 8 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit 7f2147d · What is Autotest? · Any feedback? Reach out in #autotest
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7f2147d083
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Found a screenshot filtering logic inconsistency in the cypress-plugin.js: isFailureScreenshotForUpload was overly restrictive compared to isFailureScreenshot. When a screenshot object lacks explicit testFailure metadata, isFailureScreenshotForUpload would not fall back to filename-based detection, whereas isFailureScreenshot would. This PR inadvertently introduced this bug while refactoring. Fix applied: added && screenshot.testFailure !== undefined check to align both functions' behavior, ensuring consistent screenshot identification across upload paths.
📊 Validated against 6 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit 346bf85 · What is Autotest? · Any feedback? Reach out in #autotest
There was a problem hiding this comment.
More details
42 adversarial scenarios across getScreenshotUploadResult, isFailureScreenshotByMetadata/isFailureScreenshotForUpload divergence, the addScreenshotUploadPromise/getScreenshotUploadResultPromise tracker, and the async span-finishing order all pass. The finishSuite() promotion to before testSpanFinishPromises is safe — timestamps are pre-stamped and spans never reject, so the ordering change cannot cause dropped or misattributed spans.
📊 Validated against 42 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit ac923b6 · What is Autotest? · Any feedback? Reach out in #autotest
What does this PR do?
Adds Cypress test-event tags for failure screenshot upload outcomes:
test.failure_screenshot.uploaded:truewhen a failure screenshot upload succeeds.test.failure_screenshot.upload_error:truewhen upload is enabled but an attempted upload fails.DD_TEST_FAILURE_SCREENSHOTS_ENABLEDis not enabled or no upload is possible.The Cypress plugin now waits for the upload outcome before finishing the failed test span, so the upload result lands on the test event. It also keeps suite duration independent from media upload latency, preserves custom
after:screenshothandling in the auto-instrumented path, and reports dropped media requests as upload errors.Motivation
Follow-up to #8981 so users can distinguish disabled screenshot uploads from successful uploads and upload errors in Test Optimization events.
Additional Notes
Validation covered syntax, lint, the screenshot upload request unit spec, and focused Cypress failure screenshot upload scenarios. The focused Cypress run passed the agentless cases; the EVP proxy variants remain pending as expected.
Semver:
semver-minor.