Skip to content

fix(ext/node): implement StatWatcher.ref() / unref()#33408

Merged
bartlomieju merged 4 commits into
denoland:mainfrom
nathanwhitbot:fix/node-compat-iter9
Apr 25, 2026
Merged

fix(ext/node): implement StatWatcher.ref() / unref()#33408
bartlomieju merged 4 commits into
denoland:mainfrom
nathanwhitbot:fix/node-compat-iter9

Conversation

@nathanwhitbot

@nathanwhitbot nathanwhitbot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Node's StatWatcher.ref() / unref() toggle whether the watcher's internal handle keeps the event loop alive (see lib/internal/fs/watchers.js). Deno's polyfill was throwing notImplemented from both methods, so scripts calling watcher.unref() to let the process exit once other work completes would crash instead.

Deno's StatWatcher polls via an async loop that used std/async/delay. Inline the delay into the watcher and hang onto the in-flight Timeout so ref()/unref() can call the corresponding methods on it. Record the current ref state and reapply it on each new interval timer so the setting persists across poll iterations.

Drop the now-unused _util/async.ts file from the polyfill registry — this was its only in-tree caller.

Enables parallel/test-fs-watchfile-ref-unref.js in the node compat suite.

Closes #23042

Node's StatWatcher.ref()/unref() toggle whether the watcher's internal
handle keeps the event loop alive (see
lib/internal/fs/watchers.js). Deno's polyfill threw `notImplemented`
from both methods, which broke scripts that legitimately call
`watcher.unref()` to let the process exit once other work completes.

Deno's StatWatcher polls via an async loop that was using
std/async/delay. Inline the delay into the watcher and track the
in-flight Timeout so ref()/unref() can call the corresponding methods
on it. Apply the current ref state to each new interval timer so the
setting sticks across poll iterations.

Also drop the now-unused `_util/async.ts` file from the polyfill
registry since this was its only caller in-tree.

Enables parallel/test-fs-watchfile-ref-unref.js in the node compat
suite.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@nathanwhitbot

Copy link
Copy Markdown
Contributor Author

Reviewed — functional and matches the test's observable behavior, with a couple of notes on Node conformance and one minor correctness concern.

What's correct

  • The #sleep helper correctly propagates abort: pre-check signal.aborted, create a timer, add a one-shot listener, and clean up the listener on success. No listener leak on repeated polls.
  • #refed persists across poll iterations and is reapplied when each new setTimeout is created (line: if (!this.#refed) timer.unref()). That's the key correctness point for "call .unref() now, let process exit whenever the current interval elapses."
  • Dropping _util/async.ts from deno_node is fine — rg "_util/async" shows no other callers in-tree.

Divergences from Node

  1. No refcount semantics. Node's ref/unref at lib/internal/fs/watchers.js:231-246 use KFSStatWatcherRefCount / KFSStatWatcherMaxRefCount so that only the 0↔1 transition actually touches this._handle.ref()/.unref(). The PR calls timer.ref()/.unref() on every call. Node's Timeout.ref/unref are idempotent, so the end state agrees for the test's ref/unref pattern — just more work per call.

  2. kFSStatWatcherAddOrCleanRef("clean") gap (pre-existing, not this PR). Node decrements maxRefCount and calls this.unref() at line 222 so the handle unrefs when the last listener is removed. Deno's version just decrements #refCount without touching ref state. This would also explain why unwatchFile + no explicit .unref() doesn't let the process exit. Out of scope here, flagging for future.

Minor correctness concern

In #sleep, the `abort` callback calls clearTimeout(timer) and then reject(signal.reason), but signal.addEventListener("abort", abort, { once: true }) is called after the sync if (signal.aborted) reject(...); return; guard. That's fine. However, inside abort, we `clearTimeout(timer)` but the `done` handler registered via `setTimeout(done, ms)` might still race if the timer fires at the same instant as the abort — in which case done runs first, clears this.#timer, calls resolve(), and then abort runs (since it hasn't been removed yet), clears an already-cleared timer, nulls #timer again (fine — already null), and calls reject on an already-resolved promise (no-op). Benign, but the done handler should signal.removeEventListener("abort", abort) before resolve() to avoid this chain. Tiny nit.

Performance

Per-poll allocation of a new Promise + timer + abort listener is heavier than the old std/async/delay, but at default 5007ms poll interval it's negligible.

No blocking issues. The refcount divergence only matters if something in userland (or a future test) depends on transition-triggered behavior — worth a follow-up but fine for landing.

@nathanwhitbot nathanwhitbot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Verified against Node source and the enabled test.

Correctness — matches Node's intent

Node's StatWatcher.prototype.ref() / unref() at lib/internal/fs/watchers.js:231-246 toggle whether the internal _StatWatcher libuv handle keeps the loop alive. Node uses a KFSStatWatcherRefCount / KFSStatWatcherMaxRefCount pair for idempotency against repeated ref()/unref() calls. Deno's polyfill has no libuv handle — the "handle" is the setTimeout between polls — so toggling #timer.ref()/#timer.unref() is the correct analogue.

The boolean #refed in Deno is a simplification of Node's refcount/maxrefcount pair, but behaviorally equivalent for the externally-observable ref/unref pattern: repeat calls to unref() after the handle is already unref'd are no-ops in both. The test (unref unref ref unref ref ref unref) doesn't differentiate between the two since it only checks that ref/unref don't throw and the process exits.

Walk-through of #sleep

#sleep(ms: number): Promise<void> {
  return new Promise((resolve, reject) => {
    const signal = this.#abortController.signal;
    if (signal.aborted) { reject(signal.reason); return; }
    const abort = () => { clearTimeout(timer); this.#timer = null; reject(signal.reason); };
    const done = () => { signal.removeEventListener("abort", abort); this.#timer = null; resolve(); };
    const timer = setTimeout(done, ms);
    if (!this.#refed) { timer.unref(); }
    this.#timer = timer;
    signal.addEventListener("abort", abort, { once: true });
  });
}

Verified:

  • ✅ Fast-exits if signal already aborted (matches delay's pre-condition).
  • ✅ Promise constructor body is synchronous; setTimeout(done, ms) can't fire before the addEventListener("abort", …) registers, so no ordering race.
  • ✅ Both abort and done paths null #timer and release listeners. { once: true } covers the abort-then-GC case even if done never runs.
  • ✅ On re-entry, a new timer picks up the current #refed, so changes made while #timer was null (during the statAsync portion of the loop) take effect immediately on the next poll.

Test trace (parallel/test-fs-watchfile-ref-unref.js)

First watcher, assuming default #refed = true:

Call #timer state #refed after timer.ref/unref OK
unref() likely live (mid-sleep) false timer.unref()
unref() live false timer.unref() (idempotent on libuv side)
ref() live true timer.ref()
unref() live false timer.unref()
ref() live true timer.ref()
ref() live true timer.ref()
unref() live false timer.unref()

unwatchFile triggers stop()abortController.abort()#sleep rejects → outer catch swallows AbortError. Clean shutdown.

Second part (two listeners, 100ms setTimeout): within 100ms the default 5007ms watcher interval hasn't elapsed, so watcher2.#timer is still the live initial setTimeout. watcher2.unref() at the end flips #refed=false + timer.unref(). The test's explicit setTimeout(100ms) is what was keeping the loop alive; once its callback fires and watcher2 is unref'd, the process exits. ✓

Issues

1. Timeout type is not imported (likely TS error under strict checking).

#timer: Timeout | null = null;

fs.ts imports only { clearTimeout, setTimeout } from node:timers — neither brings in the Timeout type. The actual Timeout constructor lives at ext:deno_node/internal/timers.mjs:71, and the timers.ts polyfill imports it for its own : Timeout annotations. Two options:

  • Add a value + type import: import { clearTimeout, setTimeout, type Timeout } from "node:timers"; (if node:timers re-exports the type — check cli/tsc/dts/node/timers.d.cts).
  • Or import type { Timeout } from "ext:deno_node/internal/timers.mjs";.

If the current code passes tsc, it's because polyfill type-checking isn't strict enough to catch unresolved names in type position — but that's fragile. The test passing just means runtime behavior is correct; it doesn't validate the type annotation.

2. _util/async.ts file orphaned but not deleted.

The PR removes the lib.rs polyfill registry entry and the import site in fs.ts, but leaves:

  • ext/node/polyfills/_util/async.ts (the file itself — 30ish lines of dead code)
  • tools/core_import_map.json:552 ("ext:deno_node/_util/async.ts": "../ext/node/polyfills/_util/async.ts")

I searched the repo — no other callers. Deleting the file + removing the import-map entry would be a clean follow-up (or part of this PR). Leaving the file stranded invites a future contributor to wire it back in by accident.

3. Pre-existing (not this PR) — kFSStatWatcherAddOrCleanRef is a no-op.

Deno's kFSStatWatcherAddOrCleanRef symbol handler updates #refCount but nothing else reads it, so "clean" / "cleanAll" don't actually unref anything on the handle. Node's equivalent calls this.unref() / this._handle?.unref(). Not this PR's job — but the way this PR wires ref/unref to #timer means a straightforward follow-up is possible: if (#refCount === 0) this.#timer?.unref() in the "clean" / "cleanAll" branches.

Performance / safety

  • Inlining #sleep removes one await-layer (no more import delay from std) — net-zero perf.
  • No leaks: each timer/listener pair is torn down on either resolve or reject; { once: true } guards the GC path.
  • The private #refed + #timer fields aren't observable externally — no API surface leak.

Nits

  • The constructor of AbortController is allocated unconditionally; doesn't matter for correctness but the pre-PR code also did this. Leaving as-is.
  • watcher.ref() / watcher.unref() now return this — matches Node's return value, which enables watcher.unref().watchFile(...) chaining. Good.

LGTM modulo the Timeout type import (issue 1) — the async.ts orphan (issue 2) is worth cleaning up either here or in a follow-up.

@nathanwhitbot nathanwhitbot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean-up commit removed stray <<<<<<</>>>>>>> markers in fs.ts; merge resolution kept the upstream side (deprecate import) and discarded the unused delay import. Workflow sync trivial. LGTM.

@bartlomieju
bartlomieju merged commit 9d67e02 into denoland:main Apr 25, 2026
113 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StatWatcher.unref is not implemented

3 participants