Repository navigation
fix(server): log memory high-water marks and snapshot the heap at 4 GiB - #1718
Merged
Merged
Conversation
Refs #1717 Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
🩺 Aftercare
|
| Head | Result | Time | Check |
|---|---|---|---|
436c1ce |
9 min | run |
This was referenced Oct 12, 2026
donjor
added a commit
that referenced
this pull request
Oct 12, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 12, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 12, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 12, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 12, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 13, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 13, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 15, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 15, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 15, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 15, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 16, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 16, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 16, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 16, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 18, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 18, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 18, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 18, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 18, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 18, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 19, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 20, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 20, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 21, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 21, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 21, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 21, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes Co-authored-by: donjor <[email protected]>
donjor
added a commit
that referenced
this pull request
Oct 21, 2026
donjor
pushed a commit
that referenced
this pull request
Oct 21, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 21, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 21, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 21, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 21, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 21, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 22, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 22, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 22, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 22, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 23, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 23, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 24, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 24, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 27, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 27, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 27, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
donjor
pushed a commit
that referenced
this pull request
Oct 27, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
rsi-software-agent Bot
pushed a commit
that referenced
this pull request
Oct 27, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
rsi-software-agent Bot
pushed a commit
that referenced
this pull request
Oct 27, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 0b7add0 fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - 03a61db fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
rsi-software-agent Bot
pushed a commit
that referenced
this pull request
Oct 28, 2026
…iB (#1718) ## Problem `t3 serve` grew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop. Nothing recorded which memory pool grew, so the cause is still unknown. ## Change New fork layer `MemoryWatch.fork.ts`, hooked beside `HeapSnapshot.layer`: - **Sample:** `process.memoryUsage()` every minute - **Log:** a warning at each new GiB past 2 GiB - **Fields:** rss, heap, external, arrayBuffers - **Snapshot:** once per process at 4 GiB - **Written by:** `writeHeapSnapshot` into `logsDir` `external` and `arrayBuffers` show whether the growth is buffers or native heap. The snapshot names what retains them. ## Scope and approval Refs #1717. The maintainer approved telemetry as the next step there. Diagnostics only; no behavior change below 2 GiB RSS. ## Verification - **Unit test:** `MemoryWatch.fork.test.ts` passes - **Covers:** thresholds, repeat steps, one snapshot - **Typecheck:** `tsc` on `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB run; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 96a3e9f fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) - 9db81d1 fixup! fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) (#1718) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
rsi-software-agent Bot
pushed a commit
that referenced
this pull request
Oct 28, 2026
…1719) ## Problem In the 2026-10-12 burst, the event loop was blocked from the start. Every timer and log stopped, so the memory watch from #1718 cannot fire in that case. Signal-based reports wait for the loop too. ## Change `startStackWatch` in `MemoryWatch.fork.ts`, started by the existing layer: ### Capture - **Runs on:** a worker thread, off the main loop - **Trigger:** process RSS past 4 GiB - **Action:** inspector `Debugger.pause`, 3 times - **Interval:** 1 s between pauses ### Output - **Output:** `server-<pid>-<time>.stacks.json` - **Location:** `logsDir`, mode 0600 - **Released:** with the layer scope The inspector interrupts running JavaScript, so each pause names the stuck frame. ## Scope and approval Refs #1717. The maintainer asked for the root cause; this captures it on the next burst. No main-thread cost below 4 GiB; the worker only reads RSS once a second. ## Verification - **Unit test:** 2 of 2 pass - **Covers:** a 3 s sync loop on the main thread - **Asserts:** first frame is the stuck function - **Typecheck:** `apps/server`, exit 0 - **Fork CI:** `vp run fork:ci`, exit 0 - **Not checked:** a live 4 GiB burst; no repro yet Claude Code, claude-opus-5-5. ## Fork trailers Squashes: - 0b7add0 fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - 03a61db fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - 75a24fc fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) - d85d60d fixup! fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) (#1719) Fork-Domain: upstream-fixes Fork-Tier: bugfix Fork-Upstreamable: yes
rsi-software-agent Bot
pushed a commit
that referenced
this pull request
Oct 28, 2026
pick 85cd0f8 fix(upstream-fixes): mobile checkout moves keep upstream's task-permission gate pick 4f18a87 fix(ci): provision browser and fold replay prerequisites (#1666) pick adf496d fix(server): conversation forks keep their title, links, and children isolated (#1667) pick aa22745 fix(server): settle checkout-owned zmux sessions (#1665) pick 5d52d3b feat(web): add all-except project filtering (#1663) fixup 292c7e6 fixup! feat(web): add all-except project filtering (#1663) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/5d52d3b37617cacd6abe68efef3b114b308cc0ce.msg)" pick 35d7d7e feat(web): simplify the window project chooser (#1669) pick 49cad64 fix(server): deliver terminal exits to the owning viewer (#1676) pick 7c996b4 feat(terminal): open a plain shell alongside managed zmux (#1677) pick e6eb930 fix(web): preserve terminal selection across output updates (#1679) pick 92d6058 feat(server): scaffold external voice dictation (#1672) pick 24529cb feat(server): transcribe recordings with Meta Muse (#1673) pick 6257c11 feat(desktop): record dictation into editable chat drafts (#1675) fixup 89bc6e7 fixup! feat(desktop): record dictation into editable chat drafts (#1675) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/6257c1192810ad42c6e149f9cbd9a20ef60b7206.msg)" pick 7c8db8b docs(agents): tighten fork development priorities (#1683) pick 6caa4d8 fix(desktop): keep dictation transcribing across thread switches (#1684) fixup ead68e7 fixup! fix(desktop): keep dictation transcribing across thread switches (#1684) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/6caa4d8811115e55390a8c336bcc76234e9faa23.msg)" pick 800ed30 feat(desktop): stream live Meta dictation into the composer (#1686) fixup 740bd49 fixup! feat(desktop): stream live Meta dictation into the composer (#1686) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/800ed307c0d26f464320892b0941babe07baf08b.msg)" pick 3983853 docs(fork-meta): allow isolated browser verification (#1689) pick a47c10c docs(fork): showcase recent features with existing PR media (#1691) pick 67e6894 feat(desktop): duck selected speakers during dictation (#1687) fixup 36850d5 fixup! feat(desktop): duck selected speakers during dictation (#1687) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/67e689405e382b3c59fa1b52f98e237df38ed698.msg)" pick c40227a fix(fork-meta): inherit assigned Hypr workspace in desktop runner (#1690) pick b4451af fix(server): tear down zmux sessions on thread settle and delete (#1688) fixup 2af598f fixup! fix(server): tear down zmux sessions on thread settle and delete (#1688) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/b4451afea83a8018f4914e64e0a7a9c44b6203af.msg)" pick 1e056b4 feat(dictation): add configurable cleanup and formatting (#1685) fixup 06f209e fixup! feat(dictation): add configurable cleanup and formatting (#1685) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/1e056b479845c3e771897d9d340a94e5697aeda6.msg)" pick fb52655 fix(upstream-fixes): use extracted provider and replay services pick 80c87b6 fix(upstream-fixes): adapt fork services to extracted source-control and host process APIs pick 287c2d7 fix(upstream-fixes): align fork focus rings with inset indicators pick de9176b fix(fork-sync): the trunk moves only to a sha hyprws CI passed on sync/candidate (#1701) pick 2f74ffb feat(terminal): choose zmux or plain per terminal and restore sessions on unsettle (#1699) pick f43fd48 fix(upstream-fixes): provide fork Git workflow services in managed folder tests pick e6ad2ff fix(voice-input): keep Meta realtime dictation on HTTP/1.1 WebSockets (#1710) pick 08a2f80 fix(ci): run hyprws CI on stacked pull requests (#1715) pick a1cfdb3 fix(server): skip the update backup when no migrations are pending (#1711) pick ac34343 fix(fork): keep the client usable while the attached server restarts (#1712) pick 8df49c9 fix(web): show server update progress through reconnect (#1713) fixup 1c56ae7 fixup! fix(web): show server update progress through reconnect (#1713) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/8df49c97f6925ae381ffb7ecd53d3e457e28cc24.msg)" pick afe2734 fix(server): log memory high-water marks and snapshot the heap at 4 GiB (#1718) pick e905bde fix(server): record main-thread stacks when memory bursts past 4 GiB (#1719) pick ba8335a fix(desktop): keep the attached server through a stall (#1721) pick cb27f62 fix(server): stop syncing the SQLite WAL on every commit (#1722) pick 3ee6cba fix(server): purge imported v1 rows the v2 runtime never reads (#1723) pick 021b837 fix(ci): force upstream tag fetches so a moved tag cannot block releases (#1724) pick a473b09 fix(server): return thread waits when input is needed (#1736) fixup 7f92cc8 fixup! fix(server): return thread waits when input is needed (#1736) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/a473b09e495cad1421bb0e4cb2a9049f2c24889a.msg)" pick a958a4e fix(checkout): preserve turns and shared sources during moves (#1733) pick 7ef7b86 fix(server): retire live work when provider sessions detach (#1734) pick 0313964 fix(terminal): keep terminals through checkout moves (#1738) fixup 7b70026 fixup! fix(terminal): keep terminals through checkout moves (#1738) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/0313964c3c2d131d675637a9da738891d0eb0b32.msg)" pick f622f4a fix(web): pair on same-tab token hash navigation (#1740) pick 33743d6 fix(zmux): handle workspace basename collisions (#1741) pick 44b7dbc fix(mcp): hide tools refused by read-only grants (#1742) fixup 2585ace fixup! fix(mcp): hide tools refused by read-only grants (#1742) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/44b7dbc751a12a7b9fbf585ca10e6daffcf233c1.msg)" pick cdd21bc fix(web): preserve formatter-clean rich Markdown markers (#1743) pick 7c7faf7 fix(issues): persist controls, show parents, and search all issues (#1744) pick 8cd1cd8 fix(upstream-fixes): fork code calls the Effect 4.0.4 randomUUIDv4 function pick 49e5314 refactor(upstream-fixes): sampled planner statistics defer to upstream ; empty pick 5b8b460 fix(fork): hook guard matches replayed lines across re-indentation pick 7d1a2aa fix(web): update everything updates the attached service in client-only mode (#1751) pick f053d33 refactor(upstream-fixes): fork GitHub issue and attachment paths use upstream's request policy after `pingdotgg#19224` pick 40e475e refactor(upstream-fixes): fork WS RPCs get their own handler layer under TypeScript's instantiation limit pick bb196b3 test(upstream-fixes): fork ingestor tests provide upstream's GitVcsDriver (`pingdotgg#20079`) and CommandOutputHub (`pingdotgg#15094`) pick 953db7a fix(git): name the thread in checkout move refusals (#1762) pick 284943c fix(web): retarget hidden terminal splits when shown (#1763) pick 20b4aa4 fix(voice): show redacted provider detail when cleanup fails (#1765) pick 22e1a1e fix(release): build only AppImages for fork Linux releases (#1768) pick 6421c4d fix(web): align group title generation budgets (#1769) pick 4b03981 fix(web): autosave rich Markdown and save files with Ctrl+S (#1770) fixup c6f0754 fixup! fix(web): autosave rich Markdown and save files with Ctrl+S (#1770) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/4b03981264d1eb75c70dc2f8d0eec2abd9ba58cc.msg)" pick 545c871 fix(pr): stop turn refreshes from rereading review activity (#1771) pick 3bb83d3 fix(server): strip desktop launcher environment before providers start (#1766) pick 10ce76a fix(desktop): ignore installed dependencies in pack watchers (#1767) pick 7bec9b3 feat(voice-input): live dictation preview for any OpenAI Realtime endpoint (#1775) fixup 59d45ec fixup! feat(voice-input): live dictation preview for any OpenAI Realtime endpoint (#1775) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/7bec9b3103dee6f3e8b96cc3d6db7ad4fd7c6897.msg)" pick 10f7a3d fix(upstream-fixes): fork scripts drop section dividers upstream's no-low-value-comments lint refuses pick eb97ac5 refactor(upstream-fixes): retire the fork's activities purge now upstream drops the table pick eeef974 fix(ci): install Chromium for release web browser tests (#1778) pick 74f1379 fix(fork-sync): a file's second hooked commit re-applies; hooks must stay pure insertions (#1780) pick 179b030 refactor(desktop): retire the fork second-instance buffer for upstream's pick 53c1d1a fix(fork-sync): hook guard, hook re-apply, and fold check each drop a manual repair (#1782) pick 260d02a chore(fork-meta): run Macroscope check-run agents as rsi-pr-agent workflows (#1786) pick 850335e chore(fork-meta): run rsi-pr-agent workflows on the merged pr-change trigger (#1788) pick 02b0311 fix(web): terminal OSC 52 copies reach the client clipboard (#1790) pick 448da8c fix(web): use shared custom agent menu heading (#1796) pick 012cb09 fix(web): compose thread group section headers (#1797) pick 5238881 fix(web): compose bookmark rows from shared primitives (#1798) pick 874ffea fix(web): use shared primitives for GitHub issue controls (#1799) pick 7f24727 fix(voice-input): use InlineButton for processing error dismissal (#1800) pick b7af052 fix(ci): release web tests pin the upstream release repository (#1802) pick e1fad55 fix(web): dictation fills AskUserQuestion custom answers (#1803) pick f27d098 fix(fork-sync): an exact upstream backport may drop its PR link (#1805) pick fe75d4c fix(fork-meta): browser test siblings keep the .browser infix (#1806) pick 323a48d test(upstream-fixes): retire fork resume-routing tests superseded by upstream `pingdotgg#21615` pick ee8c9fe refactor(multi-window): route-navigation seams become marked insertions pick d2003ea refactor(multi-window): createWindow seams become marked insertions (#1811) pick 6f5494e fix(fork-meta): fork-fold folds a sync's trailerless fixup into its owner (#1814) pick 9a87a2c perf(fork-meta): fork scan reads blobs through one git cat-file, not one git show per path (#1815) pick bcc5966 refactor(server): retire duplicate fork SQLite WAL sync policy (#1823) pick a3d23d5 refactor(upstream-fixes): retire legacy V1 event purge (#1824) pick 9b254b3 refactor(desktop): extract remaining window helper seams (#1820) pick bd63a47 refactor(fork): share one sync attention issue (#1822) pick a9230b5 refactor(fork-meta): isolate scan git reading (#1819) pick 5d5f3eb refactor(fork-meta): fork scan refuses every finding; the replay baseline is gone (#1826) pick 96ccee7 fix(upstream-fixes): agents no longer inherit the live server's port settings (#1828) pick de66979 refactor(fork-meta): drop the dev-app launcher; agents run upstream dev and place windows with donjor hypr (#1827) pick ffe001b feat(web): thread groups span projects and can mark a parent thread (#1829) fixup 0a387aa fixup! feat(web): thread groups span projects and can mark a parent thread (#1829) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/ffe001b8efcae27f417731f0eb795bc49ab8f665.msg)" pick b7b6592 fix(server): in-thread MCP thread reads show background work (#1831) fixup 35433a7 fixup! fix(server): in-thread MCP thread reads show background work (#1831) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/b7b6592383dc1952310f272f62c42df013b23313.msg)" pick 80b6e0f refactor(upstream-fixes): retire the fork's separate WS handler layer for upstream's split table pick 2cb66a3 fix(fork-sync): reruns continue the rebase, fork:ci gates the candidate, tip commits survive a recreate (#1833) pick 3f3567c fix(ci): release CLI executable builds on the engines.node pin (#1834) pick 978268a fix(fork-sync): wait for setup, follow the trunk, root at the main checkout (#1835) pick 1498f21 fix(desktop): wait for the service update handoff before startup discovery (#1838) pick 94bb436 fix(server): t3 update waits for a pending remote update (#1837) pick 41187c6 chore(fork-meta): teach rsi-pr-agent workflows the fork hook seam (#1839) pick 01547bb feat(server): thread groups live on the server with agent tools (#1840) fixup a4b48f8 fixup! feat(server): thread groups live on the server with agent tools (#1840) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/01547bbc3c864174352e9cb4d32ec0fa8d6ce372.msg)" pick 6f33ca0 feat(fork-sync): resolutions persist as git data between runs (#1849) pick ec5756e feat(fork-sync): a pushed resolution reruns the sync from the trunk (#1850) pick 21a1043 feat(fork-sync): run the fork-sync skill through rsi-pr-agent (#1848) pick 434880c feat(fork-sync): a blocked sync labels its attention issue rsi:fork-sync (#1847) pick 8eadb9c fix(fork-sync): job mode adds the upstream remote before the dry run (#1852) pick 7c7dc13 fix(fork-meta): give the fork-sync job 45 minutes (#1853) pick dd1b537 fix(fork-sync): the job always decides and runs from an rsi-pr-agent checkout (#1854) pick 4c97819 fix(fork-sync): a red candidate starts the fork-sync job (#1855) pick ecc87d3 fix(fork-sync): carry a retired delete/modify through sync/resolution (#1856) pick dbbd60d docs(backend-attach): describe the attached primary and its seams (#1861) pick b069e64 fix(fork-meta): install pins gh's default repository to the fork (#1862) pick cdd794e test(desktop): the cross-arch probe test ignores a host ELECTRON_RUN_AS_NODE (#1859) pick fb406ff fix(github-issues): a V2 unlink of a V1 issue link survives compaction (#1858) pick 7cd6ea1 fix(fork-meta): fold apply keeps cancelling folds and never consults rerere (#1863) pick 9585bef fix(fork): refresh legacy setup scripts in project settings overrides (#1866) pick c20d6ef fix(web): client-only Connections hides managed-server controls (#1867) pick 49b3642 fix(server): fork notice follows a first /compact and native replacement (#1864) fixup 59a7b46 fixup! fix(server): fork notice follows a first /compact and native replacement (#1864) exec git commit --amend --no-verify -F "$(git rev-parse --git-path fork-sync-folds/49b3642171ae7e7e1418d76a98b1d60e89fece08.msg)" pick d4cc6a0 fix(web): group titles name the shared subject, anchored on the parent (#1868) pick 19b9067 fix(fork-sync): export partial progress before the job limit (#1872) pick 013ccfc test(server): pin core.excludesFile in the GitManager fixture (#1874) pick 8bd2820 fix(server): issue MCP tools target a thread like the pull request tools (#1860) pick cf2e183 fix(server): a stop during the migration backup no longer kills the update (#1873) pick 7f4c625 fix(terminal): close T3 viewers of a settled lane's removed zmux session (#1876) pick 6a2844f fix(fork-sync): time the partial export by the job's clock (#1879) pick e012572 feat(web): group title becomes a tab on its parent thread (#1878) pick e6cfd23 fix(fork-sync): keep a job to one checkout, no proof clones (#1880) pick 1231787 fix(web): file saves no longer overwrite newer disk changes (#1869) pick b6dd169 fix(voice-input): one base URL derives dictation upload and live paths (#1881) pick f1e284e feat(voice-input): send per-project speech terms to the dictation service (#1882) pick a8a3005 fix(fork-sync): a job pushes its rebased tip and the driver lands it without replay (#1887) pick 6424f8a refactor(upstream-fixes): retire the terminal focus-request gate; upstream focusTerminal owns focus fixup 205c723 fixup! refactor(upstream-fixes): retire the terminal focus-request gate; upstream focusTerminal owns focus pick d48be70 fix(fork-hooks): the marker scanner skips regex literals (#1888) pick c7d647c fix(fork-sync): a no-tag rerun keeps the kept worktree's tag (#1889) pick 998088f fix(fork-sync): a session runs fork:ci on its tip before the hand-off (#1890) pick 7f8dad0 fix(fork-sync): a sync refuses a tip that lost fork hook lines (#1891) pick a1c4ebc fix(fork-sync): a landed hook retirement exempts no later sync (#1892) pick 5498116 fix(fork-ci): fork:ci runs the merge-tree gate, so a session hands off no tip CI refuses (#1893) pick d30a4fb fix(fork-ci): fork:ci runs knip and typecheck, so a session hands off no tip the Check job refuses (#1894) pick 664abd2 refactor(upstream-fixes): retire the conversation-fork title guard; upstream titles forks from inherited history pick e3e4a4e fix(fork-sync): the land run waits out candidate CI, so the report names every red job (#1895) pick 9701f0c fix(upstream-fixes): storage cleanup treats a broken gitfile as gone on newer git pick 0d463ba fix(fork-sync): the driver adds the upstream remote, and the skill allows its kept worktree (#1897) pick 1a03480 fix(fork-sync): a job session leaves no background process before it replies (#1898) pick a1ea33c refactor(markdown-editing): retire the flush hook supplied by upstream ; empty pick 1320745 chore(fork-meta): account for existing context without changing the upstream budget pick 534ee07 docs(fork-meta): track the resolved sync seams in domain scans pick 66deb1a fix(fork-ci): fork:ci runs every app and package fork test, so a session hands off no tip they fail (#1899) pick 314965b fix(fork-sync): a job out of budget hands off a red finished marker, so its replay survives the pod (#1901) pick d3e402c fix(fork-sync): a sync session folds the stack before its hand-off, so the driver stays the only trunk writer (#1902) pick 8392dad feat(web): every thread group has a parent that carries it to Working (#1904) pick 9c32ad7 fix(fork-ci): the agent context budget compares against the PR base, not upstream main (#1905) Fork-Sync-Tag: v0.1.4-nightly.20261028.3809 Fork-Sync-Lease: 9c32ad7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
t3 servegrew to 60 GB RSS plus 16 GB swap in 74 minutes, then froze the desktop.Nothing recorded which memory pool grew, so the cause is still unknown.
Change
New fork layer
MemoryWatch.fork.ts, hooked besideHeapSnapshot.layer:process.memoryUsage()every minutewriteHeapSnapshotintologsDirexternalandarrayBuffersshow whether the growth is buffers or native heap.The snapshot names what retains them.
Scope and approval
Refs #1717.
The maintainer approved telemetry as the next step there.
Diagnostics only; no behavior change below 2 GiB RSS.
Verification
MemoryWatch.fork.test.tspassestsconapps/server, exit 0vp run fork:ci, exit 0Claude Code, claude-opus-5-5.
Fork trailers
Fork-Domain: upstream-fixes
Fork-Tier: bugfix
Fork-Upstreamable: yes