-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Closed
Copy link
Labels
feat: browserIssues and PRs related to the browser runnerIssues and PRs related to the browser runnerp3-minor-bugAn edge case that only affects very specific usage (priority)An edge case that only affects very specific usage (priority)
Description
Describe the bug
When vitest is run with --update, toMatchScreenshot unconditionally emits new screenshots (even when the test would otherwise pass). This means that if you have a non-deterministic UI element, every run with --update produces a new screenshot.
This causes problems when used in CI (where it's recommended to run with --update). The presence of updated screenshots does not indicate the need to update the screenshots. This means we have to run CI twice:
- First, to see if we need to update the screenshot files
- Second, to get those updates. Even then, we have to be clever not to include unecessary changes.
It seems that --update shouldn't unconditionally update. If you want unconditional updates, it's easy enough to wipe the __screenshots__ directory and run the tests.
Reproduction
- Clone the example repo:
git clone https://github.com/djrenren/vitestScreenshotTestFailure - Run tests (successfully)
cd vitestScreenshotTestFailure && npm test:browser - Observe the screenshots have changed:
git status
The example repo renders a single random pixel on a canvas. Enough to differ but not enough to fail the test.
System Info
System:
OS: macOS 26.1
CPU: (14) arm64 Apple M3 Max
Memory: 2.05 GB / 36.00 GB
Shell: 5.9 - /bin/zsh
Binaries:
Node: 22.16.0 - /Users/john/.nvm/versions/node/v22.16.0/bin/node
npm: 10.9.2 - /Users/john/.nvm/versions/node/v22.16.0/bin/npm
pnpm: 10.18.3 - /Users/john/.nvm/versions/node/v22.16.0/bin/pnpm
bun: 1.1.29 - /Users/john/.bun/bin/bun
Deno: 1.41.3 - /Users/john/.deno/bin/deno
npmPackages:
@vitejs/plugin-react: ^5.1.1 => 5.1.1
@vitest/browser-playwright: ^4.0.15 => 4.0.15
vitest: ^4.0.15 => 4.0.15
vitest-browser-react: ^2.0.2 => 2.0.2Used Package Manager
npm
Validations
- Follow our Code of Conduct
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
- Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
- The provided reproduction is a minimal reproducible example of the bug.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
feat: browserIssues and PRs related to the browser runnerIssues and PRs related to the browser runnerp3-minor-bugAn edge case that only affects very specific usage (priority)An edge case that only affects very specific usage (priority)