Skip to content

fix(core): prevent hang when inotify watches are exhausted - #1

Open
sailod wants to merge 4 commits into
devfrom
fix/watcher-inotify-hang
Open

sailod wants to merge 4 commits into
devfrom
fix/watcher-inotify-hang

Conversation

@sailod

@sailod sailod commented Jun 19, 2026

Copy link
Copy Markdown
Owner

Issue

Fixes anomalyco#16610

Type of change

  • Bug fix

What does this PR do?

Prevents opencode from hanging or crashing at startup when inotify resources are exhausted on Linux. This happens when max_user_instances or max_user_watches is near the system limit (e.g., other processes like IDEs, file managers, or containers have consumed most of the available inotify resources).

Root cause

Two bugs in @parcel/watcher v2.5.1 cause hangs/crashes when inotify resources are exhausted:

Bug 1: Instance exhaustion → hang

When inotify_init1() fails (EMFILE — no more inotify instances), @parcel/watchers Backend::run() blocks forever on mStartedSignal.wait() because InotifyBackend::start() throws before calling notifyStarted(). This blocks the main thread — no JS-level timeout can recover.

Bug 2: Watch exhaustion → crash

When inotify_add_watch() fails (ENOSPC — no more watches), subscribe() throws a WatcherError. But WatcherError is not a subclass of std::exception, so it escapes all catch (std::exception&) blocks and triggers std::terminate(), crashing the process.

The .git watcher needs ~228 inotify watches (one per subdirectory in .git/). On systems near the max_user_watches limit, this exhausts remaining watches and triggers the crash.

Fix

1. Replace .git watcher with fs.watch() on .git/HEAD

The .git watcher previously used @parcel/watcher.subscribe() on the entire .git/ directory tree, needing ~228 inotify watches. The VCS service only listens for HEAD changes (branch switching — see vcs.ts:325-336 where it filters if (!data.file.endsWith("HEAD")) return Effect.void), so watching the entire tree is unnecessary.

The fix replaces it with fs.watch() on .git/HEAD only — 1 watch instead of 228. This completely bypasses both @parcel/watcher bugs for the always-active .git watcher.

2. canWatch() pre-check for the root watcher

Before subscribing the root watcher (behind OPENCODE_EXPERIMENTAL_FILEWATCHER), the fix runs canWatch() — tries fs.watch() on a temp directory. If it fails (either instance or watch exhaustion), the entire watcher layer is skipped with a warning log, preventing the hang/crash.

The root watcher still uses @parcel/watcher when inotify is available, but the pre-check prevents calling it when resources are exhausted.

How did you verify your code works?

  • All 6 existing watcher tests pass (bun test test/filesystem/watcher.test.ts)
  • All 36 filesystem tests pass (bun test test/filesystem/)
  • Typecheck passes clean (bun typecheck)
  • Reproduced the hang using LD_PRELOAD to intercept inotify_init1() — confirmed subscribe() blocks the main thread forever
  • C diagnostic on the problematic machine confirmed inotify_add_watch() fails with ENOSPC (watch exhaustion, not instance exhaustion)
  • Built standalone binary and tested on the problematic machine — opencode starts normally instead of hanging

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Roi added 4 commits June 19, 2026 00:31
@parcel/watcher has two bugs that cause hangs/crashes when inotify
resources are exhausted:

1. Instance exhaustion: Backend::run() blocks forever on
   mStartedSignal.wait() when inotify_init1() fails, because
   InotifyBackend::start() throws before calling notifyStarted().

2. Watch exhaustion: inotify_add_watch() fails with ENOSPC when
   max_user_watches is full. WatcherError thrown by subscribe() is
   not a subclass of std::exception, so it escapes all catch blocks
   and triggers std::terminate().

Fix by replacing the .git watcher's @parcel/watcher subscription
(which needs ~228 inotify watches for a typical .git directory tree)
with fs.watch() on .git/HEAD (needs 1 watch). The VCS service only
listens for HEAD changes for branch tracking, so this is sufficient.

Also add canWatch() pre-check using fs.watch() on a temp directory
before subscribing the root watcher (experimental flag), covering
both instance and watch exhaustion.

Fixes anomalyco#16610
Git updates HEAD atomically (write to HEAD.lock, rename to HEAD).
fs.watch on the file watches the inode, which goes stale after
atomic replace. Watch the parent directory instead and filter for
filename === 'HEAD', which detects the rename event.
fs.watch on overlayfs silently fails — the watch is created but no
events fire, and no error event is emitted, so fallback logic never
activates. Remove the fs.watch attempt entirely and use fs.watchFile
(stat polling) which uses zero inotify watches and works on any
filesystem. Also remove canWatch() pre-check that tested /tmp and
could fail due to overlayfs inotify quirks unrelated to the .git
watcher.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Opencode hangs at startup if a .git repo is present and inotify user instances run out

1 participant