Skip to content

fix(npm): widen node_modules lock staleness margin to avoid false preemption#35806

Merged
bartlomieju merged 1 commit into
denoland:mainfrom
bartlomieju:fix/node-modules-concurrent-lock-margin
Jul 7, 2026
Merged

fix(npm): widen node_modules lock staleness margin to avoid false preemption#35806
bartlomieju merged 1 commit into
denoland:mainfrom
bartlomieju:fix/node-modules-concurrent-lock-margin

Conversation

@bartlomieju

@bartlomieju bartlomieju commented Jul 6, 2026

Copy link
Copy Markdown
Member

Summary

Fixes concurrent node_modules corruption when multiple Deno processes run in parallel against the same project (nodeModulesDir: "auto"), reported in #35804.

node_modules setup is guarded by LaxSingleProcessFsFlag (libs/npm_installer/flag.rs), which combines an OS file lock with a poll-file heartbeat so waiters can detect a lock holder that was killed without releasing the lock. A waiter concluded the holder was dead once the heartbeat went stale for just 200ms (2× the 100ms update interval) and then proceeded without the lock.

Under parallel installs of native --allow-scripts packages (notably on Windows), a busy but perfectly alive holder — spawning compilers/node-gyp, creating junctions, renaming files — can easily miss that 200ms window. Every waiter then falsely concludes it died and mutates node_modules concurrently, corrupting the tree. This matches the report's fingerprint: users see "Blocking waiting for file lock on node_modules directory" (the lock is working) and corruption at the same time (os error 3 junction failures, EBUSY renames, half-installed native addons, flaky ERR_MODULE_NOT_FOUND).

Heartbeat starvation isn't the only way to trip a 200ms margin:

  • The heartbeat write itself can silently fail — PollFile::touch() ignores write errors — so antivirus interference or a transient EBUSY on Windows stalls the poll file's mtime without the holder ever knowing. Two consecutive failed writes were previously enough to cause false preemption.
  • Coarse filesystem mtime granularity (2 seconds on FAT/exFAT) could exceed the old threshold all by itself.

The 5s margin comfortably covers all of these.

Change

  • Widen the staleness threshold from 200ms to 5s.
  • Pull the two timing values into named constants (POLL_FILE_UPDATE_INTERVAL_MS, POLL_FILE_STALE_THRESHOLD_MS) with a comment explaining why the margin must be generous.

OS file locks are released automatically when a process exits, so this poll check only needs to backstop the rare case of delayed release — it can afford a generous margin. The same lock is used by both the local (.deno layout) and hoisted installers, so both are covered.

Notes

  • No automated test: the bug only reproduces under real heartbeat-thread starvation, which isn't deterministic without refactoring the staleness decision into an injectable function — a wall-clock test would be flaky. Existing flag:: lock tests still pass.
  • A follow-up is worth considering: a read-only fast path so already-initialized node_modules skips mutation/lock contention entirely in child processes (also helps the startup-cost report in skip/background package check when booting with autoinstall & lock: false #35761). Left out of this PR to keep it focused on the correctness fix.
  • Known residual behavior: when preemption fires for a genuinely dead holder, all waiters cross the threshold within the same ~20ms polling window and proceed concurrently — the wider margin makes false preemption rare but doesn't serialize the true-positive case. Acceptable for now (the lock is "lax" by design); could be improved alongside the fast-path follow-up.

Refs #35804

…emption

The lax file lock guarding node_modules setup treated the lock holder as
dead once its poll-file heartbeat went stale for just 200ms (2x the 100ms
update interval), letting waiters proceed without the lock. Under parallel
installs with native --allow-scripts packages (notably on Windows), a busy
but alive holder can easily miss that window, so multiple processes mutate
node_modules concurrently and corrupt the tree.

Widen the staleness threshold to 5s and pull the timing values into named
constants. OS file locks are released automatically on process exit, so the
poll check only needs to backstop delayed release and can afford a generous
margin.

Refs denoland#35804
@bartlomieju
bartlomieju merged commit 186f17a into denoland:main Jul 7, 2026
138 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant