Skip to content

Commit 8faf385

Browse files
t3dotggclaude
andcommitted
fix(dev): long-running dev servers no longer leak memory when other worktrees install
Bundled dev and `--built` watch node_modules through Rolldown, which skips Vite's default ignore. Those files are pnpm-store hardlinks, so every `vp i` or worktree removal elsewhere rebuilds every running server and leaks native memory. Exclude node_modules from the Rolldown watcher, use stable chunk names so bundled dev stops keeping a copy of each rebuilt chunk (vitejs/vite#23781), and stop the dev runner when its process tree outgrows a memory limit. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 79daf43 commit 8faf385

6 files changed

Lines changed: 164 additions & 2 deletions

File tree

‎AGENTS.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,8 @@ The most common defect in this repo is a change that works on the path you teste
8383
- Ports derive from the worktree path and are stable across restarts, but read the real ones from the `[dev-runner]` line since occupied ports shift.
8484
- Sharing over the tailnet is three steps: run `vp run dev --share` in the background, wait for the `pairingUrl:` line in its output, then give that full URL to an unpaired browser. Do not wire up `tailscale serve` by hand, open the URL yourself, or consume the user's pairing link. A browser with the reusable dev cookie can use the bare origin. If a normal one-time token was consumed, mint a fresh one with `node apps/server/src/bin.ts pair`. It carries standard scopes, while the startup URL carries admin scopes needed for Connections settings.
8585
- To reuse web dev auth across worktrees, configure one fixed `T3CODE_DEV_AUTH_TOKEN` in the main checkout's gitignored `.env`. The `t3.json` setup links that file into worktrees. Never commit or publish the token or a startup URL. See [Reusable dev credential](docs/operations/development.md#reusable-dev-credential).
86-
- Stop what you started, by the PID you tracked. See rule 1.
86+
- Stop what you started, by the PID you tracked. See rule 1. `vp` does not pass SIGTERM on, so signalling the `vp run dev` PID alone leaves Vite and the server running. Stop the background task with your harness's own stop, which ends the whole tree.
87+
- Run dev servers, Metro, and builds from your own shell so they end with your session. Do not detach them with `systemd-run`, `nohup`, `setsid`, or `tmux` unless the user asks for one that outlives the session. Detached servers ran for weeks and filled RAM and swap. The dev runner stops itself above `--memory-limit-gb` (half of RAM, at most 16 GiB) on Linux.
8788
- Need a second checkout? `git worktree add` it next to this worktree, not in `/tmp`. Remove it when done.
8889

8990
## Test data

‎apps/web/vite.config.ts‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,11 @@ const builtDev = process.env.T3CODE_BUILT_DEV === "1";
7373
const bundledDevEnv = process.env.T3CODE_BUNDLED_DEV?.trim().toLowerCase();
7474
const bundledDev = bundledDevEnv === "1" || bundledDevEnv === "true";
7575

76+
// Bundled dev and `build --watch` pass Rolldown `exclude` but not Vite's default `ignored`, so
77+
// they watch node_modules. Those files are pnpm-store hardlinks shared by every worktree, so each
78+
// `vp i` or worktree removal elsewhere rebuilds every long-running server and leaks native memory.
79+
const rolldownWatchExclude = [/[\\/]node_modules[\\/]/];
80+
7681
const buildSourcemap: boolean | "hidden" =
7782
sourcemapEnv === "0" || sourcemapEnv === "false"
7883
? false
@@ -247,6 +252,20 @@ function chunkCycleGuardPlugin(): Plugin {
247252
};
248253
}
249254

255+
// Bundled dev never evicts emitted files from memory (vitejs/vite#23781), so hashed names keep
256+
// a copy of every changed chunk from every rebuild. Fixed names overwrite the previous copy.
257+
function bundledDevStableNamesPlugin(): Plugin {
258+
return {
259+
name: "t3code:bundled-dev-stable-names",
260+
apply: "serve",
261+
outputOptions: (options) => ({
262+
...options,
263+
chunkFileNames: "assets/[name].js",
264+
assetFileNames: "assets/[name][extname]",
265+
}),
266+
};
267+
}
268+
250269
// Stamps index.html's title with the name branding.ts resolves once the app
251270
// loads, so a tab opens on the right channel instead of flashing another's.
252271
// Desktop swaps in its injected branding from an inline script in index.html.
@@ -307,6 +326,7 @@ export default defineConfig(() => {
307326
react({ compiler: true }),
308327
tailwindPlugins(bundledDev),
309328
warmBundledDevPlugin(),
329+
bundledDev ? bundledDevStableNamesPlugin() : undefined,
310330
],
311331
optimizeDeps: {
312332
include: [
@@ -362,6 +382,7 @@ export default defineConfig(() => {
362382
warmup: {
363383
clientFiles: ["./src/main.tsx"],
364384
},
385+
watch: { exclude: rolldownWatchExclude },
365386
...(devProxyTarget
366387
? {
367388
// Share prefixes with the backend catch-all so the proxy and 404 lists cannot drift.
@@ -397,6 +418,7 @@ export default defineConfig(() => {
397418
emptyOutDir: true,
398419
manifest: true,
399420
sourcemap: buildSourcemap,
421+
...(builtDev ? { watch: { exclude: rolldownWatchExclude } } : {}),
400422
rolldownOptions: {
401423
output: {
402424
// PDF.js resolves decoder modules and ICC data relative to wasmUrl.

‎docs/operations/development.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ Leave `VITE_HTTP_URL` and `VITE_WS_URL` unset. Vite proxies the backend through
141141
origin so the same build works over localhost and remote connections.
142142

143143
Shared runs enable bundled dev to avoid a network round trip for each import level.
144-
`T3CODE_BUNDLED_DEV=0` opts out when debugging bundler differences. Two reload traps matter
144+
`T3CODE_BUNDLED_DEV=0` opts out when debugging bundler differences. These traps matter
145145
when changing this setup:
146146

147147
- The web entry must dynamically import the app so React refresh initializes before application
@@ -151,6 +151,9 @@ when changing this setup:
151151
- The app and its routes are lazy chunks, built on first request. A plugin pre-bundles the
152152
startup chain when the server starts. Renaming or moving a startup route or the app entry
153153
silently drops it from that list.
154+
- Bundled dev and `--built` hand Rolldown its own watcher, which skips Vite's default
155+
`node_modules` ignore. Those files are pnpm-store hardlinks, so another worktree's install or
156+
removal rebuilds every long-running server and leaks memory. Keep `node_modules` excluded.
154157

155158
The workarounds live in the [web entry](../../apps/web/src/bootstrap.ts),
156159
[Tailwind plugin](../../apps/web/vite/tailwind.ts), and

‎scripts/dev-runner.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ import * as Schema from "effect/Schema";
2020
import { Argument, Command, Flag } from "effect/cli";
2121
import { ChildProcess } from "effect/process";
2222

23+
import { defaultDevMemoryLimitBytes, enforceDevMemoryLimit } from "./lib/dev-memory-limit.ts";
2324
import { type DevShareError, shareDevServer, unshareDevServer } from "./lib/dev-share.ts";
2425
import { loadRepoEnv } from "./lib/public-config.ts";
2526

@@ -690,6 +691,7 @@ interface DevRunnerCliInput {
690691
readonly share: boolean;
691692
readonly seed?: boolean | undefined;
692693
readonly built: boolean;
694+
readonly memoryLimitGb?: number | undefined;
693695
readonly runArgs: ReadonlyArray<string>;
694696
}
695697

@@ -901,6 +903,10 @@ export const runDevRunnerWithInput = Effect.fn("dev-runner.runDevRunnerWithInput
901903
),
902904
);
903905

906+
const memoryLimitBytes =
907+
input.memoryLimitGb === undefined
908+
? defaultDevMemoryLimitBytes()
909+
: input.memoryLimitGb * 1024 ** 3;
904910
const exitCode = yield* child.exitCode.pipe(
905911
Effect.mapError(
906912
(cause) =>
@@ -910,6 +916,7 @@ export const runDevRunnerWithInput = Effect.fn("dev-runner.runDevRunnerWithInput
910916
cause,
911917
}),
912918
),
919+
Effect.raceFirst(enforceDevMemoryLimit(memoryLimitBytes)),
913920
);
914921
if (exitCode !== 0) {
915922
return yield* new DevRunnerProcessExitError({
@@ -984,6 +991,13 @@ const devRunnerCli = Command.make("dev-runner", {
984991
),
985992
Flag.withDefault(false),
986993
),
994+
memoryLimitGb: Flag.Int("memory-limit-gb").pipe(
995+
Flag.withSchema(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))),
996+
Flag.withDescription(
997+
"Stop the dev server when it and its child processes use more than this many GiB of RAM plus swap (equivalent to T3CODE_DEV_MEMORY_LIMIT_GB). Defaults to half of RAM, at most 16. 0 turns it off. Linux only.",
998+
),
999+
Flag.withFallbackConfig(optionalIntegerConfig("T3CODE_DEV_MEMORY_LIMIT_GB")),
1000+
),
9871001
runArgs: Argument.String("run-arg").pipe(
9881002
Argument.withDescription("Additional Vite+ run args (pass after `--`)."),
9891003
Argument.variadic(),
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
import { assert, it } from "@effect/vitest";
2+
3+
import {
4+
parseProcStatParentPid,
5+
parseProcStatusBytes,
6+
processTreePids,
7+
} from "./dev-memory-limit.ts";
8+
9+
it("counts RAM and swap for the dev runner's descendants only", () => {
10+
assert.strictEqual(parseProcStatParentPid("812 (node (vite) 2) S 77 812 812 0 -1"), 77);
11+
assert.strictEqual(
12+
parseProcStatusBytes("Name:\tnode\nVmRSS:\t 2048 kB\nVmSwap:\t 1024 kB\nThreads:\t9\n"),
13+
3 * 1024 * 1024,
14+
);
15+
// Listed before its parent, as /proc may order them.
16+
const processes = [
17+
{ pid: 30, ppid: 20 },
18+
{ pid: 20, ppid: 10 },
19+
{ pid: 11, ppid: 1 },
20+
{ pid: 10, ppid: 1 },
21+
];
22+
assert.deepStrictEqual([...processTreePids(10, processes)].sort(), [10, 20, 30]);
23+
});

‎scripts/lib/dev-memory-limit.ts‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
/** Stops a dev server whose process tree outgrows a memory limit, so a leak cannot fill RAM and swap. */
2+
3+
import * as NodeOS from "node:os";
4+
5+
import * as HostProcess from "@t3tools/shared/HostProcess";
6+
import * as Effect from "effect/Effect";
7+
import * as FileSystem from "effect/FileSystem";
8+
import * as Schema from "effect/Schema";
9+
10+
const GiB = 1024 ** 3;
11+
12+
/** Half of RAM, at most 16 GiB. A fresh bundled-dev server uses about 7 GiB. */
13+
export const defaultDevMemoryLimitBytes = (totalBytes = NodeOS.totalmem()) =>
14+
Math.min(16 * GiB, Math.floor(totalBytes / 2));
15+
16+
export class DevRunnerMemoryLimitError extends Schema.TaggedError<DevRunnerMemoryLimitError>()(
17+
"DevRunnerMemoryLimitError",
18+
{ usedBytes: Schema.Number, limitBytes: Schema.Number },
19+
) {
20+
override get message(): string {
21+
const gib = (bytes: number) => (bytes / GiB).toFixed(1);
22+
return `The dev server used ${gib(this.usedBytes)} GiB of RAM and swap, over its ${gib(this.limitBytes)} GiB limit, so it was stopped. Start it again, or set --memory-limit-gb (T3CODE_DEV_MEMORY_LIMIT_GB; 0 turns the limit off).`;
23+
}
24+
}
25+
26+
export interface ProcessSample {
27+
readonly pid: number;
28+
readonly ppid: number;
29+
}
30+
31+
/** `comm` can hold spaces and parentheses, so fields are read after its last `)`. */
32+
export function parseProcStatParentPid(stat: string): number | undefined {
33+
const ppid = Number(stat.slice(stat.lastIndexOf(")") + 2).split(" ")[1]);
34+
return Number.isInteger(ppid) ? ppid : undefined;
35+
}
36+
37+
export function parseProcStatusBytes(status: string): number {
38+
const kib = (field: string) =>
39+
Number(new RegExp(`^${field}:\\s+(\\d+) kB`, "m").exec(status)?.[1] ?? 0);
40+
return (kib("VmRSS") + kib("VmSwap")) * 1024;
41+
}
42+
43+
export function processTreePids(rootPid: number, processes: ReadonlyArray<ProcessSample>) {
44+
const tree = new Set([rootPid]);
45+
for (let grew = true; grew;) {
46+
grew = false;
47+
for (const { pid, ppid } of processes) {
48+
if (tree.has(ppid) && !tree.has(pid)) {
49+
tree.add(pid);
50+
grew = true;
51+
}
52+
}
53+
}
54+
return tree;
55+
}
56+
57+
const readProcessTreeBytes = Effect.fn("DevRunner.readProcessTreeBytes")(function* (
58+
rootPid: number,
59+
) {
60+
const fs = yield* FileSystem.FileSystem;
61+
// Processes exit between listing and reading, so each read may fail.
62+
const read = (path: string) => fs.readFileString(path).pipe(Effect.option);
63+
const entries = yield* fs.readDirectory("/proc").pipe(Effect.orElseSucceed(() => []));
64+
const pids = entries.filter((entry) => /^\d+$/.test(entry));
65+
const processes: ProcessSample[] = [];
66+
for (const pid of pids) {
67+
const stat = yield* read(`/proc/${pid}/stat`);
68+
const ppid = stat._tag === "Some" ? parseProcStatParentPid(stat.value) : undefined;
69+
if (ppid !== undefined) processes.push({ pid: Number(pid), ppid });
70+
}
71+
let bytes = 0;
72+
for (const pid of processTreePids(rootPid, processes)) {
73+
const status = yield* read(`/proc/${pid}/status`);
74+
if (status._tag === "Some") bytes += parseProcStatusBytes(status.value);
75+
}
76+
return bytes;
77+
});
78+
79+
/**
80+
* Samples this process and its descendants every 30 seconds. Once they hold
81+
* more than `limitBytes` of RAM plus swap, it logs why and sends SIGTERM to
82+
* this process group. Linux only; elsewhere it never completes.
83+
*/
84+
export const enforceDevMemoryLimit = Effect.fn("DevRunner.enforceDevMemoryLimit")(function* (
85+
limitBytes: number,
86+
) {
87+
if ((yield* HostProcess.Platform) !== "linux" || limitBytes <= 0) return yield* Effect.never;
88+
while (true) {
89+
yield* Effect.sleep("30 seconds");
90+
const usedBytes = yield* readProcessTreeBytes(process.pid);
91+
if (usedBytes > limitBytes) {
92+
const error = new DevRunnerMemoryLimitError({ usedBytes, limitBytes });
93+
yield* Effect.logError(error.message);
94+
// vp does not pass SIGTERM to its tasks, so stopping only the child would orphan Vite and the server.
95+
process.kill(0, "SIGTERM");
96+
return yield* error;
97+
}
98+
}
99+
});

0 commit comments

Comments
 (0)