Skip to content

Snapshot differs and fail when running all tests vs one test at time #3129

@ghiscoding

Description

@ghiscoding

Describe the bug

Migrating from Jest to Vitest (to support ESM), I see that all snapshots created by Vitest have different escaping and fail when running the entire tests but if I run them as a single filename pattern then they always pass (either by filename pattern in the CLI or via the VSCode debugger). When run one file at a time, it always pass and are equivalent to Jest snaphots (escaped with double quotes) but they are very different when running all tests at once (sometime escaped with single quotes & sometime without and sometime with double quotes).

The only thing that I can think of to bypass the problem for now would be to update all snapshots so that it passes in the CI but that is basically inversing the problem, it will then fail when run one at a time. I also think that the one at a time snapshot are the correct ones since they are equivalent to Jest before I started the migration to Vitest.

Reproduction

For a reproduction you can do a git checkout of this Lerna-Lite PR 537 and run all unit tests (follow installation instructions). I will merge the PR with the snapshot failures so that it can be inspected and I can continue with my migration to ESM. The only thing that is consistent in all of this is that it always fail when all tests are ran but always pass when run one at a time.

Also worth to know that I need to run Vitest with --no-threads since Lerna/Lerna-Lite unit tests are using process.chdir() extensively, what I wonder is why it is different when running the test one at a time, technically speaking it is supposed to be isolated because of the --no-threads so why does it fail when running all tests? Is there another flag to really isolate each tests? I tried isolate, useAtomic and singleThread but none have any effects and are irrelevant, if I read correctly, when executed with --no-threads, so... ?

Fails when running all unit tests

image

Always pass when running one file at a time

image

System Info

System:
    OS: Windows 10 10.0.19045
    CPU: (12) x64 Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz  
    Memory: 10.87 GB / 31.93 GB
  Binaries:
    Node: 16.20.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - C:\Program Files (x86)\Yarn\bin\yarn.CMD 
    npm: 7.0.11 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Chrome: 111.0.5563.147
    Edge: Spartan (44.19041.1266.0), Chromium (111.0.1661.54)
  npmPackages:
    @vitest/coverage-c8: ^0.29.8 => 0.29.8 
    vite: ^4.2.1 => 4.2.1 
    vitest: ^0.29.8 => 0.29.8

Used Package Manager

pnpm

Validations

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions