[test optimization] Improve cypress testing time#7814
Conversation
Overall package sizeSelf size: 4.97 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.0 | 81.15 kB | 815.98 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7814 +/- ##
==========================================
- Coverage 80.45% 80.36% -0.09%
==========================================
Files 748 747 -1
Lines 32411 32327 -84
==========================================
- Hits 26075 25981 -94
- Misses 6336 6346 +10 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This comment has been minimized.
This comment has been minimized.
BenchmarksBenchmark execution time: 2026-03-18 14:47:36 Comparing candidate commit c30c1ba in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 230 metrics, 30 unstable metrics. |
BridgeAR
left a comment
BridgeAR
left a comment
There was a problem hiding this comment.
Nice, reducing the runtime is always great!
BridgeAR
left a comment
BridgeAR
left a comment
There was a problem hiding this comment.
Code wise LGTM. I did not check that the tests still match all cases.
3 participants
What does this PR do?
Reduces Cypress integration test wall-clock time by eliminating redundant Cypress process spawns and unnecessary test execution. Each Cypress spawn costs ~10-20s (browser launch, module loading), so removing them has a significant impact.
Changes:
Merge duplicate ATR tests: Combined "retries flaky tests in the correct order" into "retries flaky tests" — both spawned identical Cypress processes with the same settings and spec file. The order assertions now run in the same callback.
Merge duplicate EFD tests: Combined "sets TEST_HAS_FAILED_ALL_RETRIES when all EFD attempts fail" into "retries new tests in the correct order" — identical settings, known tests, and spec file.
Merge "catches errors in hooks" into "can run and report tests": Both ran all
.cy.jsspecs with noSPEC_PATTERN. Now a single Cypress run verifies both reporting correctness and hook error handling. AddedSPEC_PATTERNto excludespec-source-line-invocation.cy.jswhich neither test needed.Merge "after:spec" into "after:run" test: Both had identical assertions (4 suites, 9 tests) and only differed by which custom hook env var was set. The hooks are independent, so testing both simultaneously is equivalent.
Add
SPEC_PATTERNto "user_provided_service" test: Was running all 10.cy.jstests but only needed one spec to verify theDD_TEST_IS_USER_PROVIDED_SERVICEtag.Estimated savings: ~75-85s per version × module-type matrix combination.
Motivation
The Cypress integration tests run for every version × module-type combination ([email protected]/10.2.0/14.5.4/latest × commonJS/esm). Several tests were spawning separate Cypress processes with identical configurations, and others were running more Cypress specs than their assertions required. Since Cypress startup is the dominant cost per test (~10-20s), eliminating redundant spawns is the most effective optimization available.