🐛 Fix flaky worker test by improving Jest child process handling by dubzzz · Pull Request #6863 · dubzzz/fast-check

dubzzz · 2026-04-14T22:06:49Z

Description

Fix the flaky "should fail on property blocking the main thread" test in @fast-check/jest by improving how the test captures Jest child process output.

The Jest child process could produce output on either stdout or stderr depending on timing. The test was only capturing stderr, which sometimes missed the expected markers. Additionally, tightened timeout thresholds and reduced setTimeout values to make assertions more reliable.

Changes:

Capture both stdout and stderr from the Jest child process (output can end up in either stream)
Collect both streams on failure too (they may be undefined when the process errors)
Reduce jest.setTimeout / testTimeoutCLI from 3000ms to 2000ms for consistency
Tighten expectTimeout upper bound from timeout * 2 to timeout * 1.5

Related: #6864 (extracted AI env var stripping), #6865 (vitest equivalent)

Checklist

— Don't delete this checklist and make sure you do the following before opening the PR

I have a full understanding of every line in this PR — whether the code was hand-written, AI-generated, copied from external sources or produced by any other tool
I flagged the impact of my change (minor / patch / major) either by running pnpm run bump or by following the instructions from the changeset bot
I kept this PR focused on a single concern and did not bundle unrelated changes
I followed the gitmoji specification for the name of the PR, including the package scope (e.g. 🐛(vitest) Something...) when the change targets a package other than fast-check
I added relevant tests and they would have failed without my PR (when applicable)

changeset-bot · 2026-04-14T22:06:55Z

⚠️ No Changeset found

Latest commit: e06f0a8

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-04-14T22:07:00Z

👋 It looks like the PR template was not used for this pull request.

Please edit the PR description to use the template, or close this PR and open a new one that uses it.
Not using the template may result in your change not being reviewed.

The template includes:

A Description section
A Checklist section with items to verify before submitting

Thank you for your contribution! 🙏

pkg-pr-new · 2026-04-14T22:08:57Z

@fast-check/examples

@fast-check/ava

npm i https://pkg.pr.new/@fast-check/ava@6863

fast-check

npm i https://pkg.pr.new/fast-check@6863

@fast-check/jest

npm i https://pkg.pr.new/@fast-check/jest@6863

@fast-check/packaged

npm i https://pkg.pr.new/@fast-check/packaged@6863

@fast-check/poisoning

npm i https://pkg.pr.new/@fast-check/poisoning@6863

@fast-check/vitest

npm i https://pkg.pr.new/@fast-check/vitest@6863

@fast-check/worker

npm i https://pkg.pr.new/@fast-check/worker@6863

commit: e06f0a8

dubzzz · 2026-04-14T22:10:01Z

@claude extract a dedicated pr just for the ai related env thing

claude · 2026-04-14T22:10:54Z

Claude finished @dubzzz's task in 2m 25s —— View job

Extracted AI env var stripping into a dedicated PR

Understand the current PR changes and identify the AI env var part
Create a new branch from main with just the env var change
Open a dedicated PR for the env var fix
Update this PR to remove the extracted change

Done! Created #6864 with just the AI agent env var stripping (jestEnv constant + env: jestEnv in execFile).

This PR (#6863) has been updated to remove the env var change — it now only contains the stdout/stderr capture, --forceExit, and timeout fixes.

dubzzz · 2026-04-14T22:11:39Z

@claude if applicable to vitest tests too (in fast-check/vitest) do apply something similar for env trick for vitest running in ai workers. Before doing it confirm it's needed for vitest suites

claude · 2026-04-14T22:11:53Z

Claude finished @dubzzz's task in 3m 42s —— View job

Applied AI env var stripping to vitest tests

Review the Jest env var stripping in this PR
Examine vitest test files for similar child process spawning
Determine if the fix is needed for vitest — Yes, confirmed needed
Apply the env var stripping to vitest test

Why it's needed: Vitest 4.x uses std-env's isAgent to detect AI agent environments via the same env vars (CLAUDE_CODE, AI_AGENT, CURSOR_AGENT, etc.). When detected, vitest switches to AgentReporter which only prints output for failed tests — suppressing the ✓ pass markers that expectPass() asserts on.

What was done: Added the same env var filtering to packages/vitest/test/vitest-fast-check.spec.ts — a vitestEnv constant that strips AI agent env vars, passed to the execFile call in runSpec().
·

dubzzz · 2026-04-14T22:12:35Z

@claude add comment on top of exit flag, timeout... To explain in one line why they have to be wdded

claude · 2026-04-14T22:12:52Z

Claude finished @dubzzz's task in 2m 13s —— View job

Adding explanatory comments to test options

Read the changed file to understand current code
Add comments explaining --forceExit, timeout, and output handling
Commit and push

Added inline comments in runSpec() explaining why each option was added:

stdout + stderr: "Capture both stdout and stderr as Jest may output to either stream."
--forceExit: "Force Jest to exit after tests complete, even if worker threads are still hanging (e.g. while(true))."
timeout: 60_000: "Prevent the child process from hanging indefinitely if Jest gets stuck."
catch block: "On failure, stdout/stderr may be undefined; collect whatever is available."

Note: there was a rebase conflict with the concurrent env var extraction — resolved by keeping comments but without env: jestEnv (which was already extracted to the other PR).
·

dubzzz · 2026-04-14T22:17:30Z

@claude move strip ai env for vitest into a dedicated pr

claude · 2026-04-14T22:17:46Z

Claude finished @dubzzz's task in 3m 28s —— View job

Extracted vitest AI env var stripping into a dedicated PR

Understand the vitest env var changes in this PR
Create a new branch from main with just the vitest env var change
Open a dedicated PR for the vitest env var fix
Update this PR to remove the vitest env var change
Push updated PR branch

Done! Created #6865 with just the vitest AI agent env var stripping (vitestEnv constant + env: vitestEnv in execFile).

This PR (#6863) has been updated to remove the vitest env var change — it now only contains the Jest stdout/stderr capture, --forceExit, timeout, and comment fixes.

View job

## Description Strip AI agent environment variables (CLAUDE_CODE, CURSOR_AGENT, etc.) from the environment passed to Jest child processes in tests. When tests run inside an AI agent environment, Jest may pick up these env vars and use AgentReporter instead of its default reporter, which strips the verbose test result markers (`PASS`/`FAIL`) that the test assertions rely on. This causes spurious test failures. Extracted from #6863. ## Checklist — _Don't delete this checklist and make sure you do the following before opening the PR_ - [x] I have a full understanding of every line in this PR — whether the code was hand-written, AI-generated, copied from external sources or produced by any other tool - [ ] I flagged the impact of my change (minor / patch / major) either by running `pnpm run bump` or by following the instructions from the changeset bot - [x] I kept this PR focused on a single concern and did not bundle unrelated changes - [x] I followed the [gitmoji](https://gitmoji.dev/) specification for the name of the PR, including the package scope (e.g. `🐛(vitest) Something...`) when the change targets a package other than `fast-check` - [x] I added relevant tests and they would have failed without my PR (when applicable) Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Nicolas DUBIEN <[email protected]>

The "should fail on property blocking the main thread" test was flaky because when the Jest child process exited with an error (e.g. after killing a hung worker thread), the output could end up in stdout instead of stderr. The catch block now returns both streams combined. https://claude.ai/code/session_014MbtNcfKa6gyj8iWAdG5wp

codecov · 2026-04-15T18:30:09Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.86%. Comparing base (6cb7e94) to head (e06f0a8).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6863   +/-   ##
=======================================
  Coverage   94.86%   94.86%           
=======================================
  Files         209      209           
  Lines        5665     5665           
  Branches     1488     1488           
=======================================
  Hits         5374     5374           
  Misses        283      283           
  Partials        8        8

Flag	Coverage Δ
tests	`94.86% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

## Description Strip AI agent environment variables (CLAUDE_CODE, CURSOR_AGENT, etc.) from the environment passed to vitest child processes in tests. When tests run inside an AI agent environment, vitest may pick up these env vars and use AgentReporter instead of its default reporter, which strips the verbose test result markers (`✓`) that the test assertions rely on. This causes spurious test failures. Extracted from #6863. Mirrors the Jest equivalent in #6864. ## Checklist — _Don't delete this checklist and make sure you do the following before opening the PR_ - [x] I have a full understanding of every line in this PR — whether the code was hand-written, AI-generated, copied from external sources or produced by any other tool - [ ] I flagged the impact of my change (minor / patch / major) either by running `pnpm run bump` or by following the instructions from the changeset bot - [x] I kept this PR focused on a single concern and did not bundle unrelated changes - [x] I followed the [gitmoji](https://gitmoji.dev/) specification for the name of the PR, including the package scope (e.g. `🐛(vitest) Something...`) when the change targets a package other than `fast-check` - [x] I added relevant tests and they would have failed without my PR (when applicable) Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Nicolas DUBIEN <[email protected]>

https://claude.ai/code/session_014MbtNcfKa6gyj8iWAdG5wp

dubzzz · 2026-04-15T20:29:26Z

@claude use pr template

claude · 2026-04-15T20:29:41Z

Claude finished @dubzzz's task in 2m 11s —— View job

Updating PR to use template

Read PR template
Read current PR changes
Update PR description with template

Updated the PR description to use the repository's PR template from .github/PULL_REQUEST_TEMPLATE.md, including the Description and Checklist sections with details about the stdout/stderr capture fix, timeout adjustments, and references to the extracted PRs (#6864, #6865).

claude Bot mentioned this pull request Apr 14, 2026

✅(jest) Strip AI env vars in child processes #6864

Merged

5 tasks

claude Bot mentioned this pull request Apr 14, 2026

✅(vitest) Strip AI env vars in child processes #6865

Merged

5 tasks

dubzzz force-pushed the claude/fix-flaky-worker-test-zL3oz branch from 5fab53a to 4691b3a Compare April 15, 2026 18:21

🐛 Also capture stdout in success path of runSpec

e06f0a8

https://claude.ai/code/session_014MbtNcfKa6gyj8iWAdG5wp

dubzzz closed this Apr 15, 2026

Uh oh!

Conversation

dubzzz commented Apr 14, 2026 • edited by claude Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

changeset-bot Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions Bot commented Apr 14, 2026

Uh oh!

pkg-pr-new Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dubzzz commented Apr 14, 2026

Uh oh!

claude Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Extracted AI env var stripping into a dedicated PR

Uh oh!

dubzzz commented Apr 14, 2026

Uh oh!

claude Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Applied AI env var stripping to vitest tests

Uh oh!

dubzzz commented Apr 14, 2026

Uh oh!

claude Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Adding explanatory comments to test options

Uh oh!

dubzzz commented Apr 14, 2026

Uh oh!

claude Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Extracted vitest AI env var stripping into a dedicated PR

Uh oh!

codecov Bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dubzzz commented Apr 15, 2026

Uh oh!

claude Bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Updating PR to use template

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dubzzz commented Apr 14, 2026 •

edited by claude Bot

Loading

changeset-bot Bot commented Apr 14, 2026 •

edited

Loading

pkg-pr-new Bot commented Apr 14, 2026 •

edited

Loading

claude Bot commented Apr 14, 2026 •

edited

Loading

claude Bot commented Apr 14, 2026 •

edited

Loading

claude Bot commented Apr 14, 2026 •

edited

Loading

claude Bot commented Apr 14, 2026 •

edited

Loading

codecov Bot commented Apr 15, 2026 •

edited

Loading

claude Bot commented Apr 15, 2026 •

edited

Loading