|
| 1 | +# Bug 83 — a V8 GC race kills a jest worker and fails an arbitrary suite |
| 2 | + |
| 3 | +| | | |
| 4 | +|---|---| |
| 5 | +| **Status** | **Fixed in v4** (worked around; the defect itself is upstream V8's) | |
| 6 | +| **Found** | 2026-08-20 (endemic long before that — the crash signature appears in session logs at least back to 2026-08-11, misattributed the whole time) | |
| 7 | +| **Fixed** | 2026-08-20 | |
| 8 | +| **Severity** | Medium — no user data at risk, but roughly **1 full `npm run test:unit` run in 5** on the macOS arm64 dev machine lost a worker and failed one suite. Worse than the wasted rerun is the trained reflex: "red suite? run it again" is exactly the habit that lets a *real* intermittent failure slide by unexamined | |
| 9 | +| **Who it bites** | anyone running the jest suites on Node 24+ (the current engines floor). Reproduced upstream on macOS arm64; the defect is in shared V8 code, so other platforms are presumed exposed | |
| 10 | +| **Provenance** | Not this codebase's at all. V8 13.6 (Node 24) has a GC race — [nodejs/node#62393](https://github.com/nodejs/node/issues/62393), Chromium issue 483731079. Still reproduces upstream on Node 26.7.0, so waiting out a Node upgrade was not an option | |
| 11 | +| **Fix site** | `package.json` (the five jest scripts now launch `node --no-sparkplug node_modules/jest/bin/jest.js`), `jest.global-setup.js` (`armSparkplugGuard()` appends `--no-sparkplug` to `process.execArgv` before any worker forks), `jest.integration.config.ts` (now wired to the same globalSetup) | |
| 12 | +| **Defect site** | upstream: `v8::internal::ClearStaleLeftTrimmedPointerVisitor::VisitRootPointers` dereferencing a junk slot of an `InternalFrame` mid-`Builtins_BaselineOutOfLinePrologue` | |
| 13 | +| **v5 status** | Nothing owed — the Rust core doesn't run V8. The workaround should ride along only as long as v4's jest suites do | |
| 14 | +| **Index** | [bugs.md](../../bugs.md) | |
| 15 | + |
| 16 | +--- |
| 17 | + |
| 18 | +**FIXED in v4 (2026-08-20)** — by disabling V8's Sparkplug baseline compiler for |
| 19 | +every jest process, at two chokepoints so no invocation style escapes: |
| 20 | + |
| 21 | +1. The five jest scripts in `package.json` (`test:all`, `test:coverage`, |
| 22 | + `test:integration`, `test:unit`, `test:watch`) launch jest as |
| 23 | + `node --no-sparkplug node_modules/jest/bin/jest.js`. (`NODE_OPTIONS` can't |
| 24 | + carry the flag — it's not on Node's allowlist — hence the explicit |
| 25 | + interpreter invocation.) jest-worker forks each worker with the parent's |
| 26 | + `process.execArgv`, so the flag propagates to the whole fleet, and the main |
| 27 | + process — which runs the tests itself under `--runInBand` or a single-file |
| 28 | + run — is covered directly. |
| 29 | +2. `jest.global-setup.js` (`armSparkplugGuard()`) appends `--no-sparkplug` to |
| 30 | + `process.execArgv` when absent. `process.execArgv` is a plain mutable array |
| 31 | + that jest-worker reads at fork time, and globalSetup runs in the main |
| 32 | + process before any worker forks — so ad-hoc invocations that skip the npm |
| 33 | + scripts (`npx jest`, `npx jest -u some.test.ts`) still get protected |
| 34 | + workers. The integration config now points at the same globalSetup, which |
| 35 | + also brings it the native-ABI self-heal it never had. |
| 36 | + |
| 37 | +## Symptom |
| 38 | + |
| 39 | +Intermittently — about 1 full unit run in 5 locally — a run ends: |
| 40 | + |
| 41 | +``` |
| 42 | +FAIL __tests__/unit/<some suite>.test.ts |
| 43 | + ● Test suite failed to run |
| 44 | + A jest worker process (pid=NNNNN) was terminated by another process: |
| 45 | + signal=SIGSEGV, exitCode=null. |
| 46 | +``` |
| 47 | + |
| 48 | +with **every individual test in the run passing**. A *different* suite fails |
| 49 | +each time — plugin tests, component tests, API tests — because the dying worker |
| 50 | +takes down whatever file it happens to hold. A rerun essentially always passes, |
| 51 | +which is what kept this filed under "flake" instead of "bug" for months. |
| 52 | + |
| 53 | +## Root cause |
| 54 | + |
| 55 | +A V8 13.6 garbage-collection race, [nodejs/node#62393](https://github.com/nodejs/node/issues/62393). |
| 56 | +When a stack-guard interrupt fires inside Sparkplug's |
| 57 | +`Builtins_BaselineOutOfLinePrologue` and starts a mark-compact collection, the |
| 58 | +conservative root scan walks the half-built baseline `InternalFrame`, and |
| 59 | +`ClearStaleLeftTrimmedPointerVisitor` dereferences a junk slot — |
| 60 | +`EXC_BAD_ACCESS / KERN_INVALID_ADDRESS at 0xe`, every time, on every machine in |
| 61 | +the upstream thread and in this repo's own crash report |
| 62 | +(`~/Library/Logs/DiagnosticReports/node-2026-08-20-155211.ips`, pid 77470 — |
| 63 | +the exact pid jest reported that afternoon). The trigger population is "lots of |
| 64 | +JS compiled through the baseline tier under `vm`" — i.e. jest's module |
| 65 | +sandboxing — which is why test runs hit it and the dev server doesn't seem to. |
| 66 | + |
| 67 | +Timing-dependent, so anything that shifts GC pressure shifts the odds: a warm |
| 68 | +SWC transform cache makes runs lighter and faster, which is the entire truth |
| 69 | +behind "it always works the second time." |
| 70 | + |
| 71 | +## Why it survived — a two-year misattribution |
| 72 | + |
| 73 | +This repo *had* a real native-binding segfault in June 2026: real-SQLCipher |
| 74 | +suites running under jsdom, fixed then with `@jest-environment node` docblocks |
| 75 | +plus `workerIdleMemoryLimit`. Every SIGSEGV since inherited that diagnosis — |
| 76 | +"the known native-SQLCipher worker-teardown flake, run it again." The crash |
| 77 | +report that broke the story open showed the dying worker had **no** |
| 78 | +`better_sqlite3.node` loaded at all: whatever was crashing, it wasn't the |
| 79 | +binding. From there the faulting stack matched nodejs/node#62393 |
| 80 | +frame-for-frame, down to the `0xe`. |
| 81 | + |
| 82 | +Two lessons worth the ink: a *fixed* bug's ghost can misfile every later bug |
| 83 | +that shares its symptom, and macOS keeps the receipts — one `.ips` file in |
| 84 | +`DiagnosticReports` ended a months-long guessing game in about a minute. |
| 85 | + |
| 86 | +## How to verify |
| 87 | + |
| 88 | +- Workers carry the flag: while a suite runs, |
| 89 | + `ps -axo command | grep -c 'no-sparkplug.*processChild'` counts the fleet. |
| 90 | +- Soak: repeated full `npm run test:unit` runs (alternating `npx jest |
| 91 | + --clearCache` to recreate the heavier cold-cache condition) with |
| 92 | + `~/Library/Logs/DiagnosticReports/node-*.ips` watched for new arrivals. |
| 93 | + Upstream's matrices say the same thing at larger n: 0 crashes in 40, 6, and |
| 94 | + 4 flagged runs against 1-in-5-ish unflagged, at no measurable wall-time cost. |
| 95 | +- Retiring it someday: the guard should outlive the upstream bug, not the |
| 96 | + repo. When nodejs/node#62393 is actually fixed in the supported Node lines |
| 97 | + (verify against the issue, not the changelog), delete `armSparkplugGuard()` |
| 98 | + and return the scripts to plain `jest`. |
0 commit comments