You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 0f4b7c4
Browse filesBrowse the repository at this point in the historyBrowse files
ci(unit): run npm single-threaded so node 24's exit cannot deadlock it
The merge queue's unit (node 24) job hung in npm ci after the install had
finished and ejected #3176 (run 43399170758) and #3182 (43420654281); it
had done so four times in September. The cause is a deadlock in node 24's
process.exit(): npm calls it, node joins V8's worker threads, and a worker
parked mid-Maglev-compile waits for a garbage collection only the joining
main thread could run (nodejs/node#64274, fix proposed in #66171,
unreleased). gdb stacks of hung npm ci runs in the sandbox show exactly
that; on the runners a hung run had the same three threads in futex waits,
and npm's debug log ended in its exit handler.
npm's own process now runs with --single-threaded, which turns off V8's
background tasks, so no compile can park. The flag is refused in
NODE_OPTIONS, so npm-cli.js is run directly and npm's children run as
before.
On the runners: plain npm ci hung 4 of 60; the flagged form hung 0 of 145
(60 in the same run, 43407363473, and 85 in 43421579091), at no measurable
cost (median 57 s plain, 56 s flagged). In the sandbox, with stdio on
pipes: 4 of 34 before, 0 of 20 after.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DPMMjGGdttTJWpWrHaaNL2
-**CI:** the merge queue's `unit (node 24)` job no longer hangs in `npm ci` after the install has finished. The cause was a deadlock in node 24's `process.exit()` (nodejs/node#64274): a background compile waited for a garbage collection that the exiting main thread never ran. The job now runs npm itself with V8's background tasks off (`--single-threaded`).
Upstream: nodejs/node#64274, fix proposed in nodejs/node#66171, in no release on 2026-11-13.
939
+
-**How to reproduce:** loop `npm ci` on node 24 and treat a run past 200 s as hung. On the runners (ubuntu-24.04,
940
+
node 24.22.2), the probe on scratch PR #3179 hung 4 of 60 plain runs with output redirected to a FILE (run
941
+
43407363473). In the cloud sandbox (node 24.22.1) a file never hung in 9 runs, but stdout and stderr on PIPES
942
+
(`npm ci 2> >(cat >e) | cat >o`) hung 4 of 34 (3 of 14, then 1 of 20 in a loop that alternated with the fix), so
943
+
locally, use pipes. A hung process is in state `Sl` with wchan `futex_do_wait`, not `ep_poll`, so a JS-level handle
944
+
dump never runs; attach `gdb -p <pid> -batch -ex "thread apply all bt"` for the stacks above (taken locally; on the
945
+
runners the probe captured the threads' kernel futex stacks, and npm's debug log of a hung run ends `verbose exit 0`
946
+
/ `info ok`, which npm writes from its `exit` handler, so process.exit() had been called).
947
+
-**Fix:** the `unit` job runs npm's own process with V8's background tasks off:
948
+
`node --single-threaded "$(dirname "$(readlink -f "$(command -v npm)")")/npm-cli.js" ci`. With no background compile
949
+
there is nothing to park. Measured after: 0 of 20 locally (the alternating loop), 0 of 145 on the runners: 60 in the same run as the
950
+
4 of 60 (the other arm), 85 more in run 43421579091. At the plain rate, 0 of 145 by chance is about 1 in 20,000. It costs nothing measurable: the 56 finished plain runs took 57 s at the median and 66 s at p90, the 60 flagged
951
+
runs 56 s and 69 s. The flag cannot go in `NODE_OPTIONS` (node refuses `--single-threaded`, `--no-maglev` and
952
+
`--no-concurrent-sparkplug` there), so it is passed on the command line of npm itself, and npm's children run as
953
+
before. `--no-maglev --no-concurrent-sparkplug` is the narrower mitigation others report; one reproduction upstream
954
+
still hung with it, so it was not chosen.
955
+
-**Not a fix:** a longer timeout or a retry. The 10-minute step cap only bounds the cost.
956
+
-**Any node 24 process that calls `process.exit()` is exposed,** not only npm. Nothing else in CI runs on node 24 except
957
+
this job's later steps, which have shown no such hang; if one appears, the same stacks will say so.
958
+
-**Removable when:** the fix in nodejs/node#66171 (or another for #64274) ships in the node 24 line CI uses; then the
0 commit comments