fix(core): prevent shared-buffer timer expiry race from losing timers#35312
Conversation
Restore the timer expiry protocol where JS returns the next expiry from __processTimers(now), and schedule that value before resolving async ops. This avoids stale shared-buffer expiry snapshots overwriting timers armed during op resolution or promise hooks. Add regression coverage for stale zero, negative, and positive expiry states. Ref: denoland#32844
bartlomieju
left a comment
There was a problem hiding this comment.
The fix is correct and well-justified. I traced the scheduling path: createTimer -> insert() arms the native timer synchronously via op_timer_schedule(msecs), and the old shared-buffer snapshot (written before op resolution) was read back by Rust after promise hooks armed new timers, clobbering that op_timer_schedule. Committing the next-expiry inside dispatch_user_timers before dispatch_event_loop_tick makes the op-path the last writer, which closes the race. Phase ordering (timers before ops) and processTimers internal microtask draining are preserved, so no behavioral regression.
Approving. A few minor, non-blocking notes inline. One last thing worth confirming before merge: that the renamed schedule_timer_expiry doc and the dispatch_event_loop_tick doc no longer reference the old combined single-call tick model anywhere (looks handled in the diff).
| return Ok(false); | ||
| } | ||
|
|
||
| let now = context_state.user_timer.now(); |
There was a problem hiding this comment.
Minor: the old dispatch_event_loop_tick clamped this with if now <= 0.0 { f64::EPSILON } ("even if elapsed time is exactly 0.0 at startup"). Here now is passed raw. I traced processTimers(0) and it's safe (list.expiry > now holds for any real timer, so nothing mis-fires), so this is fine in practice, but the guard was dropped silently. Could you confirm the clamp is intentionally unnecessary under the return-value protocol?
| if let Some(exception) = tc_scope.exception() { | ||
| return exception_to_err_result(tc_scope, exception, false, true); | ||
| } | ||
| if tc_scope.has_terminated() || tc_scope.is_execution_terminating() { |
There was a problem hiding this comment.
Very minor: when the timer call terminates, this returns Ok(false), so the caller's did_work |= timer_ready won't record that timer work was attempted. Matches prior behavior closely enough, just flagging.
| ) | ||
| .await; | ||
|
|
||
| if let Ok(result) = run_result { |
There was a problem hiding this comment.
Minor: this treats "timed out but FIRED == true" as a pass. If a future regression let the timer fire but then left the loop hung (e.g. a timer armed without a wakeup), this test would still go green. Asserting run_result.is_ok() (i.e. the loop actually completed) would make the test strictly stronger. Same applies to the run_next_ticks test below.
| ) | ||
| .await; | ||
|
|
||
| if let Ok(result) = run_result { |
There was a problem hiding this comment.
Same note as above: consider asserting the event loop completes (run_result.is_ok()) rather than accepting a timeout as long as FIRED is set.
Address review feedback on denoland#35312. The two timer return-protocol regression tests previously accepted a run_event_loop timeout as a pass whenever FIRED was set. That masks the exact regression class the PR guards against: a timer that fires but leaves the loop hung without a wakeup. Require run_result.is_ok() so a hang fails the test instead of going green on FIRED alone. Also document why dispatch_user_timers passes `now` to __processTimers without the old `f64::EPSILON` startup clamp: the clamp only existed to avoid colliding with the negative "skip timers" sentinel of the former single-function __eventLoopTick protocol, which no longer exists
Summary
This restores the timer expiry handoff changed in #32844 from the shared
Float64Arraybuffer back to the older return-value protocol.The shared-buffer protocol wrote the next timer expiry during
__eventLoopTick(...), then continued resolving completed async ops. Promise hooks or op-resolution side effects could synchronously arm or mutate timers after that snapshot was written. Rust then read the stale snapshot after the JS call returned, allowing older timer state to clear, delay, ref, or unref newer timer state.With this change, Rust calls
Deno.core.__processTimers(now), immediately commits the returned timer expiry, and only then resolves completed async ops. This removes the stale shared-buffer overwrite class.What changed
Deno.core.__processTimers(now).Deno.core.__eventLoopTick(...)so it only resolves completed async op results.Float64Arraytimer expiry handoff.__processTimers(now)return value:0: no timers remainprocessTimers()drains microtasks viarunNextTicks().Why return to the return-value protocol?
The shared-buffer design made the JS -> Rust handoff depend on a snapshot that could become stale before Rust consumed it. Covering all timer transitions correctly (
setTimeout,clearTimeout,refTimer, andunrefTimer) would require the shared state to represent both the next deadline and ref/unref liveness without being overwritten by older snapshots. That is hard to make robust when JS can synchronously mutate timers while async op promises are being resolved in the same event-loop turn.The return-value protocol restores a simpler ordering guarantee: JS processes the currently expired timers, returns the current next expiry, and Rust commits that value immediately before resolving completed async ops. That keeps JS and Rust timer state synchronized at the handoff boundary.
The performance tradeoff also looks acceptable. The extra Rust -> JS call can occur only in turns where both a native user timer and completed async ops are ready, and the overlap-focused benchmarks below did not show a stable, perceivable overhead from using the return-value protocol.
What kind of code could trigger this?
Reduced repro
This uses
Deno.core.setPromiseHooks(...)only to make the race deterministic: it lets the repro run JS synchronously while an async op promise is being resolved.On Deno 2.8.3, this locally reproduced the stale-
0failure shape. In runs where the hook armed the short timer, the broken output was:In a 50-run local sample, this
SCHEDULED-without-FIREDoutput occurred 34 times. The expired timer publishestimerExpiry[0] = 0beforeDeno.stat(...)is resolved. The promise hook then schedulessetTimeout(..., 1), but Rust applies the older0snapshot after returning from JS and clears the native timer. The newly scheduled timer is left without a native wakeup.With this change, the expected output is:
Real-world scenario
A production-shaped scenario was a Deno Deploy build timeout while running concurrent Slidev/Vite/esbuild builds. Several child processes printed successful build output, but the parent sometimes stayed stuck waiting for child status until the build timed out. The core of the relevant application pattern is:
This is not using
setPromiseHooksdirectly. It is ordinary child-process orchestration with async op completions under concurrency. Hook-like synchronous side effects may come from runtime internals, Node compatibility, async context, observability, APM, or a test/build harness. If such code arms or mutates a timer while Deno is resolving an async op in the same turn as an expired timer, the old shared-buffer protocol could apply an older timer snapshot afterwards and lose or delay the newer timer state.Why this fixes the race
Old shared-buffer ordering:
sequenceDiagram participant Rust participant JS participant Shared as shared Float64Array Rust->>JS: __eventLoopTick(timerNow, completed op results...) JS->>JS: process expired timers JS->>Shared: write next-expiry snapshot Note over Shared: This snapshot is only current until JS mutates timers again. JS->>JS: resolve completed async ops JS->>JS: promise hooks / op-resolution side effects arm or mutate timers Note over JS,Shared: Any timer armed or changed here can make<br/> the shared snapshot stale. JS-->>Rust: return Rust->>Shared: read older next-expiry snapshot Rust->>Rust: schedule/ref/unref from stale stateRestored return-value ordering:
sequenceDiagram participant Rust participant JS Rust->>JS: __processTimers(now) JS->>JS: process expired timers JS-->>Rust: return current next expiry Rust->>Rust: immediately schedule/ref/unref from return value Rust->>JS: __eventLoopTick(completed op results...) JS->>JS: resolve completed async ops JS->>JS: promise hooks / op-resolution side effects arm or mutate timers JS-->>Rust: returnIf op resolution or promise hooks arm new timers, those updates happen after the previous expiry has already been committed, so they cannot be overwritten by a stale shared-buffer snapshot.
Performance
The return protocol can add an extra Rust -> JS call only when both are true in the same event-loop turn:
I benchmarked overlap-focused workloads that intentionally exercise that shape.
Benchmark environment:
Benchmark workload details (click to expand)
Each benchmark sample ran in a fresh Deno process. The full suite used 3 warmups and 11 measured runs.
mixed_barrier_batchDeno.stat(...)async ops.timeout_guarded_statlong_guard: 10,000Deno.stat(...)calls guarded by a 1000 ms timeout.short_guard: 10,000Deno.stat(...)calls guarded by a 1 ms timeout.ready_op_plus_due_timerDeno.stat(...)async ops.timer_op_chainDeno.stat(...).mixed_high_concurrency_tailDeno.stat(...).Every measured sample completed the expected timer and async-op counts.
mixed_barrier_batchtimeout_guarded_statready_op_plus_due_timertimer_op_chainmixed_high_concurrency_tailThese overlap-focused benchmarks do not show a stable, perceivable overhead from the return-value protocol in the mixed timer/async-op shape where the extra JS call can occur.