Skip to content

test: make dev server tests deterministic by replacing fixed sleeps with event-driven polling#8561

Merged
Boshen merged 3 commits intomainfrom
fix/deterministic-dev-server-tests
Mar 24, 2026
Merged

test: make dev server tests deterministic by replacing fixed sleeps with event-driven polling#8561
Boshen merged 3 commits intomainfrom
fix/deterministic-dev-server-tests

Conversation

@Boshen
Copy link
Copy Markdown
Member

@Boshen Boshen commented Mar 6, 2026

Summary

  • Add a /_dev/status endpoint to the dev server exposing buildSeq, moduleRegistrationSeq, and bundle state counters
  • Replace all fixed setTimeout sleeps in test synchronization with event-driven polling utilities (waitForNextBuild, waitForBuildStable, waitForModuleRegistration)
  • Remove sensibleTimeoutInMs helper and unused CONFIG.watch references
  • On retry, fully restart the dev server with fresh files instead of relying on sleep + page reload
  • Fix buildSeq double-increment on full-reload HMR builds (skip increment in onHmrUpdates when FullReload is present, since onOutput will handle it)

Eliminates all hardcoded sleeps (710ms-6000ms, x3 on CI) that caused flakiness and wasted CI minutes.

Local test results (macOS, 3 runs each)

main (sleeps) this PR (polling) Improvement
Fixtures 17.24s avg 7.54s avg 56% faster
Browser 10.76s avg 9.43s avg 12% faster
Total 28.00s avg 16.97s avg 39% faster
Reliability 3/3 passed 3/3 passed OK

CI job time comparison (avg of 3 runs each)

Platform main this PR saved
Ubuntu 5m28s 3m36s 1m52s (34% faster)
macOS 5m05s 2m55s 2m10s (43% faster)
Windows 7m30s 5m36s 1m54s (25% faster)

Closes #8556

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 6, 2026

Deploy Preview for rolldown-rs canceled.

Name Link
🔨 Latest commit 08fd4fa
🔍 Latest deploy log https://app.netlify.com/projects/rolldown-rs/deploys/69c2932204d60e000805d604

@shulaoda shulaoda requested review from Copilot and removed request for Copilot March 6, 2026 13:22
@Boshen Boshen marked this pull request as ready for review March 6, 2026 13:26
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make @rolldown/test-dev-server’s dev-server test suite deterministic and faster by replacing fixed sleeps with polling utilities driven by an internal dev-server status endpoint.

Changes:

  • Added a dev-server status endpoint (GET /_dev/status) exposing build/activity counters for tests.
  • Replaced fixed setTimeout sleeps in dev-server tests with polling helpers (waitForNextBuild, waitForBuildStable, waitForModuleRegistration).
  • Removed the CI timeout multiplier helper (sensibleTimeoutInMs) and eliminated unused watch config plumbing in tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
packages/test-dev-server/src/dev-server.ts Adds build/module-registration counters and /_dev/status endpoint used by tests for synchronization.
packages/test-dev-server/tests/test-utils.ts Introduces polling helpers that query /_dev/status and updates editFile to use them.
packages/test-dev-server/tests/vitest-setup-playwright.ts Replaces retry sleep with a “wait for rebuild” helper before reloading the page.
packages/test-dev-server/tests/fixtures.test.ts Switches fixture synchronization to build/module-registration polling instead of fixed sleeps.
packages/test-dev-server/tests/src/utils.ts Removes sensibleTimeoutInMs sleep multiplier helper.
packages/test-dev-server/tests/src/config.ts Removes unused CI watch option wiring from test config.

Copilot AI review requested due to automatic review settings March 12, 2026 08:58
@Boshen Boshen force-pushed the fix/deterministic-dev-server-tests branch from 2d07fe8 to bf814f4 Compare March 12, 2026 08:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.


You can also share your feedback on Copilot code review. Take the survey.

@sapphi-red sapphi-red changed the title fix: make dev server tests deterministic by replacing fixed sleeps with event-driven polling test: make dev server tests deterministic by replacing fixed sleeps with event-driven polling Mar 12, 2026
Copilot AI review requested due to automatic review settings March 12, 2026 10:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.


You can also share your feedback on Copilot code review. Take the survey.

Copilot AI review requested due to automatic review settings March 12, 2026 13:36
@Boshen Boshen force-pushed the fix/deterministic-dev-server-tests branch from 16026a4 to 03f335c Compare March 12, 2026 13:36
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.


You can also share your feedback on Copilot code review. Take the survey.

@Boshen Boshen force-pushed the fix/deterministic-dev-server-tests branch from 03f335c to 2cfc568 Compare March 13, 2026 08:12
Boshen and others added 2 commits March 24, 2026 19:14
…th event-driven polling

Add a `/_dev/status` endpoint exposing `buildSeq` and `registeredClients` counters.
Tests now poll this endpoint instead of using hardcoded sleeps, eliminating flakiness
from timing-dependent synchronization.

Closes #8556

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Without this wait, consecutive HMR steps could be coalesced by the
watcher debounce, causing the invalidation fixture to intermittently
fail when ok-1 was never written.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@Boshen Boshen force-pushed the fix/deterministic-dev-server-tests branch from 2cfc568 to 1ed73fb Compare March 24, 2026 11:23
When HMR produces a FullReload update, the engine also triggers a
rebuild which fires onOutput. Previously both callbacks incremented
buildSeq, double-counting a single build cycle. Now onHmrUpdates
skips the increment when FullReload is present, letting onOutput
handle it.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@Boshen Boshen merged commit 557d377 into main Mar 24, 2026
30 checks passed
@Boshen Boshen deleted the fix/deterministic-dev-server-tests branch March 24, 2026 14:02
@github-actions github-actions bot mentioned this pull request Mar 25, 2026
@github-actions github-actions bot mentioned this pull request Mar 25, 2026
shulaoda added a commit that referenced this pull request Mar 25, 2026
## [1.0.0-rc.12] - 2026-03-25

### 🚀 Features

- chunk-optimizer: skip circular dependency check when strict execution order is enabled (#8886) by @hyf0

### 🐛 Bug Fixes

- emit build warnings during watch mode rebuilds (#8897) by @IWANABETHATGUY
- lazy-barrel: load import-then-export specifiers when barrel has local exports (#8895) by @shulaoda
- correct execution order of transferred CJS init calls (#8877) by @IWANABETHATGUY
- mcs: `entriesAware` should calculate sizes without duplication (#8887) by @hyf0
- non-deterministic chunk generation (#8882) by @sapphi-red
- `is_top_level` incorrectly treats strict-mode scopes as top-level (#8878) by @Dunqing

### 🚜 Refactor

- treeshake: migrate SideEffectDetector to Oxc's MayHaveSideEffects trait (#8624) by @Dunqing

### 🧪 Testing

- make dev server tests deterministic by replacing fixed sleeps with event-driven polling (#8561) by @Boshen

### ⚙️ Miscellaneous Tasks

- deps: update dependency vite-plus to v0.1.14 (#8902) by @camc314
- deps: update dependency oxfmt to ^0.42.0 (#8891) by @renovate[bot]
- deps: update rust crate oxc_sourcemap to v6.1.1 (#8890) by @renovate[bot]
- remove Rolldown MF plan (#8883) by @shulaoda
- deps: update rollup submodule for tests to v4.60.0 (#8881) by @sapphi-red
- deps: update test262 submodule for tests (#8880) by @sapphi-red
- deps: upgrade oxc crates to 0.122.0 (#8879) by @shulaoda

Co-authored-by: shulaoda <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dev-server tests: replace time-based synchronization with event-driven approach

3 participants