You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f6e6a94
Browse filesBrowse the repository at this point in the historyBrowse files
fix(dev-stack): identify the dev-server deaths as a Node v24.15.0 bug (#224)
SOLVED. The Vite dev server has been dying mid-e2e-run since 2026-08-17 with
"rc=127", no error output, and a healthy backend beside it. It was never Vite.
Node v24.15.0 corrupts its own stack under concurrent load on Windows. Windows'
/GS stack-cookie check catches the corruption and fail-fasts the process
instantly and silently -- which is why there was never a message, a dialog, or a
Windows Error Reporting event.
Evidence
- Minidump (ProcDump, no admin), parsed for its exception record and modules:
code : 0xC0000409
fast-fail code : 2 (STACK_COOKIE_CHECK_FAILURE)
faulting module : C:\Program Files\nodejs\node.exe
Subcode 2, not 7. Code 7 (FATAL_APP_EXIT) is abort() -- a deliberate panic.
Code 2 is the stack cookie found overwritten: real memory corruption.
- A/B on this repo's own e2e suite at E2E_WORKERS=11:
v24.15.0 2 crashes in 3 runs (+3 earlier hunts, each crashed run 1)
v24.14.1 0 crashes in 8 runs
v24.20.0 0 crashes in 8 runs
- Reproduces identically on Vite 7.3.6 AND Vite 8.2.1, so neither the new
toolchain nor its Rust native addons are implicated.
- Rust panic ruled out with RUST_BACKTRACE=full (no panic message); V8 fatal
error ruled out with --report-on-fatalerror (no report, and the flag was
verified to fire on a real V8 OOM).
- Upstream: nodejs/node#62991 -- v24.15.0 native
crashes on Windows, v24.14.1 called out as known-good. Windows-only, so CI on
ubuntu was never at risk and is deliberately left alone.
What changes
- scripts/check-node-version.sh: warns at up.sh time when the running Node is on
the known-bad list. A warning, not a hard failure -- someone with only the bad
build installed still needs to run the stack, and the ledger now types these
crashes correctly. Add to the list rather than rediscovering this.
- supervise-server.js: use a shell only for a .cmd. Spawning a real .exe through
cmd.exe breaks on any space in its path and tripped DEP0190 on every start.
- dev-stack.md: replaced the speculation with the answer, and retired the two
inferences that cost the most -- "no WER event, therefore not a crash" (
__fastfail produces none by design) and "commit was comfortable, therefore not
memory" (it measured host commit, a different quantity).
- docs/incidents/2026-09-01-...: the full narrative, including why four previous
investigations each reached a different wrong conclusion.
Verified: full e2e suite 215 passed after the change.
Co-authored-by: Claude Opus 5 <[email protected]>
A/B on this repo's own e2e suite at `E2E_WORKERS=11`:
50
+
51
+
| Node | Crashes |
52
+
|---|---|
53
+
|**v24.15.0**|**2 in 3** (plus three earlier hunts that each crashed on the first run) |
54
+
| v24.14.1 | 0 in 8 |
55
+
| v24.20.0 | 0 in 8 |
56
+
57
+
Reproduces identically on Vite **7.3.6** and Vite **8.2.1**, so it is neither the new toolchain
58
+
nor its Rust native addons. Upstream: [nodejs/node#62991](https://github.com/nodejs/node/issues/62991)
59
+
(Windows-only; Linux/macOS unaffected, so CI on ubuntu was never at risk).
60
+
61
+
`scripts/check-node-version.sh` warns at `up.sh` time when the running Node is on the known-bad
62
+
list. **Add to that list rather than rediscovering this**, and don't pin CI for it — the bug is
63
+
Windows-only and CI does not run there.
64
+
65
+
### Read the native exit code, never `rc=127`
66
+
67
+
`scripts/supervise-server.js` records the real code, because bash collapses every abnormal end to
68
+
127 — an external `Stop-Process -Force` and a fail-fast crash are indistinguishable through it.
69
+
70
+
| Native | Meaning |
71
+
|---|---|
72
+
|`0xFFFFFFFF`|`TerminateProcess(-1)` — something killed it. Usually `down.sh`, so expect `intent=planned`|
Copy file name to clipboardExpand all lines: docs/incidents/README.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,6 +34,7 @@ need the "why" or the debugging story.
34
34
| 2026-08-19 |[Creating an exercise stopped opening it, online only](2026-08-19-exercise-create-navigation-lost-online.md) — an awaited invalidation evicted the optimistic row before it was selected; invisible in all three degraded modes, and the flow had no online coverage | Log screen / offline |
35
35
| 2026-08-31 |[The boot white screen came back, and the 2026-08-25 fix's recovery path didn't help](2026-08-31-boot-white-screen-recurrence.md) — "Try again" re-threw against the same restored state; root cause still not identified, but a real `/login` link plus an outside-React watchdog now guarantee a way out regardless | Frontend / resilience |
36
36
| 2026-08-31 |[X-Forwarded-For spoofing bypassed every per-IP rate limit](2026-08-31-xff-spoofing-bypassed-per-ip-rate-limits.md) — the 2026-07-17 fix traded a shared-bucket bug for a header-spoofing one; confirmed live against lower before the fix | Auth / infra / security |
37
+
| 2026-09-01 |[The Vite dev server died mid-run for months — it was a Node.js bug](2026-09-01-vite-dev-server-node-stack-corruption.md) — Node v24.15.0 corrupts its own stack on Windows and fail-fasts silently; four investigations failed because `rc=127`, the exit marker and the planned/unexpected tag were all broken | Local dev tooling |
0 commit comments