Skip to content

fix(ci): economyEngine.test.js worker-IPC-flake — deterministisk isolation (#3172) - #3222

Merged
NicolaiDolmer merged 3 commits into
mainfrom
fix/3172-economy-test-flake
Aug 3, 2026
Merged

NicolaiDolmer merged 3 commits into
mainfrom
fix/3172-economy-test-flake

Conversation

@NicolaiDolmer

@NicolaiDolmer NicolaiDolmer commented Aug 3, 2026 •

Copy link
Copy Markdown
Owner

Hvad

Stopper CI-flaket i backend-tests: lib/economyEngine.test.js (den klart største backend-testfil, 103 tests / 6513 linjer / ~250KB) kørte tidligere sammen med de øvrige ~343 testfiler i Node's testrunners parallelle worker-pool. Nyt backend/scripts/run-tests.js kører den store fil alene først, derefter resten som før — så den store fils IPC-resultat-payload aldrig konkurrerer med andre workers samtidig.

Refs #3172

Hvorfor

2x samme dag (31/7, #3169 + #3171) fejlede backend-tests med præcis 1 fil og en runner-intern fejl (Unable to deserialize cloned data due to invalid or unsupported version, node:internal/test_runner). Filen var altid 102/102 grøn isoleret — kun fuld-suite-kørsler flakede. Root cause: kendt Node-testrunner-bug (nodejs/node#64061, rettet upstream i nodejs/node#64706, merged 2026-07-26, men endnu ikke i vores pinnede Node 24.x-linje) i processRawBuffer's håndtering af IPC-buffere fra worker-processer — trigges af samtidig IPC-trafik fra mange workers, og den unormalt store fil er den mest sandsynlige trigger. Fremfor at vente på en Node-bump eller maskere fejlen med en retry-wrapper, fjerner denne PR selve race-betingelsen: filen kører aldrig samtidig med andre testfiler.

Detaljeret root-cause-analyse + alternativer overvejet: .claude/learnings/2026-08-03-economy-test-worker-ipc-flake.md.

Test plan

  • Frontend build: ikke rørt af denne PR (backend-only, dev-tooling)
  • Backend tests: npm test --prefix backend passerer — kørt 5 gange i træk, alle 5 grønne (103/103 + 4773/4773, exit 0, ingen deserialize-fejl)
  • Lint: npm run lint --prefix backend passerer (frontend ikke rørt)
  • Manuel: N/A — ingen UI-ændring
  • Regression: bekræftet at begge passes tilsammen dækker ALLE 344 testfiler (ingen fil droppet af den nye fil-enumerering) og at testtallet matcher baseline (103 + 4773 = 4876, svarer til issuets "4729/4730" + tilvækst siden 31/7)

Brugerverifikation (KRÆVET)

  • Jeg har åbnet feature i prod eller preview-deploy og bekræftet at brugere ser forventet UI/data
  • Jeg har testet som authenticated user (ikke kun service_role / admin)
  • Hvis ikke testet end-to-end: forklar hvorfor — ren CI/test-infrastruktur-ændring (test-orchestration), ingen bruger-vendt kode, ingen UI, ingen DB-ændring. Verificeret ved 5x lokal npm test-kørsel (se ovenfor) + npm run lint grøn.

Hvad jeg verificerede konkret: Ingen browser-flow relevant. Verificerede at npm test (backend) er deterministisk grøn 5/5 gange lokalt efter ændringen, og at node scripts/run-tests.js finder og kører alle 344 .test.js-filer (103 isoleret + 4773 i den samlede resterende kørsel = ingen fil tabt).

Risk / auto-merge

  • auto-merge er kun sat hvis PR'en er lav-risiko og ikke kræver menneskelig beslutning — ikke sat: PR er draft, afventer ejer-review af test-orchestration-ændringen (rører package.json's test-linje + verify-local.ps1)
  • Dependency/security-impact er vurderet: ingen — package-lock.json urørt, ingen nye dependencies, kun package.json's "test"-linje ændret til at pege på et lokalt script

Filer rørt

  • backend/package.json — "test"-scriptet peger nu på node scripts/run-tests.js (eneste tilladte package.json-ændring; ingen dependency-ændring)
  • backend/scripts/run-tests.js (ny) — to-pass test-orchestration: isolerer lib/economyEngine.test.js, kører resten bagefter
  • scripts/verify-local.ps1 — backend-test-steppet kalder nu samme run-tests.js i stedet for rå node --test (så lokale pre-push-kørsler får samme beskyttelse — flaket blev også set her 31/7)
  • .claude/learnings/2026-08-03-economy-test-worker-ipc-flake.md (ny) — postmortem

Skærmbillede / video

N/A — ingen UI-ændring.


Close-out (efter merge til main)

  • PatchNotesPage.jsx opdateret — N/A, ren dev-tooling/CI-ændring, ingen brugerrettet ændring
  • docs/NOW.md opdateret
  • docs/FEATURE_STATUS.md opdateret — N/A, ingen kontrakt-/feature-ændring
  • .claude/learnings/2026-08-03-economy-test-worker-ipc-flake.md oprettet
  • Issue-comment med shipped-status + verifikation, label skiftet til claude:done
  • Branch slettet efter merge

Summary by CodeRabbit

  • Bug Fixes

    • Improved backend test reliability by isolating a flaky economy engine test.
    • Preserved command-line options, test output, and exit-status reporting across test runs.
  • Chores

    • Updated standard and local verification commands to use the improved test runner.
    • Added documentation covering the test flake, resolution, and validation results.

NicolaiDolmer and others added 2 commits August 3, 2026 15:34
backend/lib/economyEngine.test.js is by far the largest backend test
file (100+ tests, 6500+ lines, ~250KB) and its per-worker result
payload appears large enough to trip a known Node test-runner bug
(nodejs/node#64061, root-caused and fixed upstream in
nodejs/node#64706, merged 2026-07-26 but not yet in our pinned Node
24.x line) when many other test-file workers stream results back to
the parent process concurrently. Isolated runs were always
102/102 (now 103/103) green; the flake only ever showed up in the
full suite (#3169-CI, #3171-CI).

Add backend/scripts/run-tests.js: runs economyEngine.test.js alone
first, then every other backend test file exactly as before (same
flags, same default concurrency) via explicit relative file paths so
the two passes never overlap. Wire both npm test (backend/package.json
- the one permitted package.json edit) and scripts/verify-local.ps1
(which called `node --test` directly, bypassing npm entirely) through
this script so CI and local pre-push runs get the same protection.

No dependency or lockfile changes. Verified with 5 consecutive
`npm test` runs locally (103/103 + 4773/4773 every time, exit 0).

Refs #3172
Co-Authored-By: Claude <[email protected]>
@vercel

vercel Bot commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cycling-zone Ready Ready Preview Aug 3, 2026 2:28pm

Request Review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026 •

Copy link
Copy Markdown

Review Change Stack

Walkthrough

The change adds a two-pass backend test runner. It runs economyEngine.test.js first, then runs remaining tests. npm and local verification use the runner. A postmortem documents the worker-IPC flake and validation.

Changes

Backend test execution

Layer / File(s) Summary
Two-pass test execution
backend/scripts/run-tests.js
The runner discovers test files, runs lib/economyEngine.test.js first, forwards CLI arguments, and returns failure statuses.
Verification command wiring
backend/package.json, scripts/verify-local.ps1, .claude/learnings/...
npm and local verification invoke the new runner. The postmortem records the failure cause, workaround, and validation.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant run-tests.js
  participant economyEngine.test.js
  participant remaining test files
  run-tests.js->>economyEngine.test.js: run isolated test file
  economyEngine.test.js-->>run-tests.js: return exit status
  run-tests.js->>remaining test files: run remaining tests with forwarded arguments
  remaining test files-->>run-tests.js: return suite exit status
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the CI fix for the economyEngine.test.js worker IPC flake and the deterministic isolation approach.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/3172-economy-test-flake

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@supabase

supabase Bot commented Aug 3, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ghwvkxzhsbbltzfnuhhz because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@NicolaiDolmer
NicolaiDolmer marked this pull request as ready for review August 3, 2026 13:39
@NicolaiDolmer
NicolaiDolmer enabled auto-merge (squash) August 3, 2026 13:39
@NicolaiDolmer
NicolaiDolmer merged commit 70ef2e1 into main Aug 3, 2026
31 of 32 checks passed
@NicolaiDolmer
NicolaiDolmer deleted the fix/3172-economy-test-flake branch August 3, 2026 14:29
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

✅ Deploy verificeret LIVE — 70ef2e1

  • Vercel deploy: skipped (no frontend changes)
  • Railway deploy: success
  • Backend /health = 200
  • Backend /api/auctions (no auth) = 401 (auth gates virker)
  • Frontend root = 200

Du kan trygt lukke telefonen — det er live.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@backend/scripts/run-tests.js`:
- Around line 47-55: Update runNodeTest and its callers to classify forwarded
CLI arguments before launching either test pass: apply supported lifecycle/watch
flags consistently or reject them before any tests start, and route positional
test selectors only to their intended pass or reject them. Preserve the
documented isolated-then-rest order and prevent --watch from making the first
pass persistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7e0de8b7-0194-4f91-a7f0-1bf03fbc9f95

📥 Commits

Reviewing files that changed from the base of the PR and between 81ae338 and 058d326.

📒 Files selected for processing (4)
  • .claude/learnings/2026-08-03-economy-test-worker-ipc-flake.md
  • backend/package.json
  • backend/scripts/run-tests.js
  • scripts/verify-local.ps1

Comment on lines +47 to +55
function runNodeTest(extraArgs, files) {
const result = spawnSync(
process.execPath,
["--test", "--import", "./test-setup.js", ...extraArgs, ...files],
{ stdio: "inherit", cwd: backendRoot },
);
if (result.error) throw result.error;
return result.status ?? 1;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

# Audit existing callers that may rely on forwarded selectors or watch mode.
rg -n --hidden \
  --glob '!**/node_modules/**' \
  --glob '!**/.git/**' \
  'npm(\s+run)?\s+test.*--|run-tests\.js.*--|--watch' .

Repository: NicolaiDolmer/CyclingZone

Length of output: 7178


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf 'repository root: '
git rev-parse --show-toplevel

printf '\nNode runtime:\n'
node --version || true

printf '\nbackend/package.json dependencies/scripts excerpt:\n'
python3 - <<'PY'
import json
from pathlib import Path
p=Path("backend/package.json")
data=json.loads(p.read_text())
for k in ("packageManager","type","scripts","engines","dependencies","devDependencies"):
    print(f"{k}: {data.get(k)}")
PY

printf '\nbackend/scripts/run-tests.js with line numbers:\n'
cat -n backend/scripts/run-tests.js

printf '\nRelevant package.json entries:\n'
cat -n backend/package.json | sed -n '1,80p'

Repository: NicolaiDolmer/CyclingZone

Length of output: 10689


🌐 Web query:

Node v24 node:test CLI positional arguments test file patterns --watch does not exit

💡 Result:

In Node.js, the --watch flag is designed to keep the test runner process alive to monitor files for changes [1][2]. By design, when running with --watch, the process does not exit after the initial test execution is complete [1][2]. Instead, it enters a persistent "watch mode" where it waits for file changes in order to rerun the relevant tests [1][2]. If you are experiencing issues where the runner does not exit, it is likely because this behavior is intended [1][2]. To terminate the process, you must manually stop it, typically by sending an interrupt signal (such as pressing Ctrl+C in your terminal) [1]. If you encounter issues where positional arguments (test file patterns) seem to be ignored or cause unexpected behavior, consider the following: 1. Intentional Persistence: Ensure you are not expecting the process to exit automatically; watch mode inherently requires manual termination [1][2]. 2. Positional Argument Parsing: Node.js has known issues where certain CLI arguments or options placed after positional arguments can be silently dropped or cause parsing confusion [3]. It is recommended to place your --watch flag before your positional test file patterns to ensure proper CLI parsing [1]. 3. File Discovery Issues: If you find that new files are not being picked up or that watch mode is not behaving as expected, ensure you are using a recent version of Node.js. Issues related to the watcher not correctly identifying new test files matching glob patterns have been addressed in recent releases (e.g., via improvements to the FileWatcher's ability to re-scan for new files) [4]. If the process is failing to exit even after you have terminated the parent process, or if it is stuck in a loop despite no file changes, this may indicate a bug in the specific Node.js version you are using. Check the official Node.js GitHub issue tracker for reports matching your environment [3][4].

Citations:


🌐 Web query:

Node.js v24 node:test CLI positional test file globs pattern argument "node --test"

💡 Result:

In Node.js v24, the --test flag invokes the built-in test runner [1]. When no specific files are provided, the runner automatically executes files matching several default glob patterns [1]. These defaults include various test-related naming conventions (e.g., /.test.js, /test-.js, /test//*.{js,cjs,mjs}) and, depending on configuration, TypeScript-related extensions (e.g., *.ts, *.cts, .mts) [1]. You can override or supplement these defaults by providing one or more glob patterns as positional arguments to the node --test command [1]. Key behavioral details for positional glob patterns include: - Positioning: Glob patterns must be provided as the final argument(s) to the node --test command [1]. - Shell Expansion: To prevent the shell from expanding these patterns prematurely, they should always be enclosed in double quotes (e.g., node --test "/.test.js") [1]. - Syntax: The patterns follow standard glob(7) behavior [1]. - Multiple Patterns: You can pass multiple glob patterns as separate arguments to execute tests from multiple sets of files simultaneously [1]. Because these patterns must be at the end of the command, you cannot easily place other flags (such as --test-only or --test-reporter) after the glob arguments [2]. If you need to use additional CLI flags alongside specific file patterns, ensure the flags precede the glob arguments [2].

Citations:


Classify forwarded CLI arguments before starting both passes.

runNodeTest() appends every caller argument and then adds test-file arguments. npm test -- "lib/foo.test.js" therefore runs as node --test --import ./test-setup.js lib/foo.test.js lib/economyEngine.test.js for pass 1 and again with every remaining file for pass 2. Node 24 treats final positional args as test-file globs, so forwarded positional selectors change both passes.

--watch also makes pass 1 persistent. Passing --watch means pass 2 does not start, unlike the documented isolated-then-rest run order. Split forwarded arguments by type: apply lifecycle/watch flags consistently or reject them before starting, and pass positional test selectors only to the appropriate pass or reject them before any test starts.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@backend/scripts/run-tests.js` around lines 47 - 55, Update runNodeTest and
its callers to classify forwarded CLI arguments before launching either test
pass: apply supported lifecycle/watch flags consistently or reject them before
any tests start, and route positional test selectors only to their intended pass
or reject them. Preserve the documented isolated-then-rest order and prevent
--watch from making the first pass persistent.

NicolaiDolmer added a commit that referenced this pull request Aug 3, 2026
NicolaiDolmer added a commit that referenced this pull request Aug 3, 2026
…tion alone wasn't the fix) (#3276)

PR #3222's fix (run the file alone in its own pass) theorized the
"Unable to deserialize cloned data" test-runner bug needed concurrent
IPC traffic from other workers to trigger. That theory was disproven
the same evening: the isolated pass itself failed with the identical
error twice more (CI runs 30836291715 attempt 1 and 30840936886), both
with zero other workers running. One of the two was masked in
`gh run list` by a subsequent green rerun and only surfaced by
checking run_attempt > 1 directly via the GitHub REST API.

The bug (nodejs/node#64061, fixed upstream but not yet in our pinned
Node 24.x line) is triggered by this file's IPC payload size/shape
alone, not by concurrency. Running it "alone" only changed the odds of
hitting the exact buffer boundary; it never removed the vulnerable
code path (the file still ran as a child worker streaming results back
via socket + structured-clone).

Fix: run this one file with --test-isolation=none, which executes it
in the same process as the node --test invocation instead of spawning
a child worker. No child worker means no IPC round-trip for this file,
so processRawBuffer/FileTest.parseMessage (present in every crash
stack trace) are never invoked for it - the bug class cannot trigger
regardless of payload size or concurrent CI load. Scope is limited to
this one file/pass; the other ~340 backend test files keep default
process-per-file isolation unchanged.

Verified locally: 25/25 green runs with --test-isolation=none, 25/25
green with default isolation (no local repro - expected, the bug is
rare and CI-Linux-weighted), failure-propagation confirmed correct
(a deliberately failing test still exits 1), and the per-file
FileTest wrapper line from the crash traces is structurally absent
from --test-isolation=none output. 4x full npm test (4988/4988) green
after the change.

No dependency, lockfile, or package.json changes.

Refs #3172

Co-authored-by: Nicolai Dolmer <[email protected]>
Co-authored-by: Claude <[email protected]>

This branch was successfully deployed

1 active deployment
Preview — 058d326b Deployed Aug 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant