[Bug]: On a 16 GB Linux host the server is swapped out under agent load and freezes for minutes (once 2 h); t3code.service has no MemoryLow #26004

Before submitting

• I searched existing issues and did not find a duplicate. (Closest: macOS desktop main process hangs under swap pressure from uncapped agent children on a 16 GB Mac (0.1.0) #24048 is the same symptom on macOS; Design: report Linux pressure stalls and make shared-provider recovery scope explicit #25918 is a closed design issue about reporting Linux pressure stalls; fix(server): agents run in their own systemd scopes so an OOM kill spares the server #17662 introduced the agents-slice limits. None covers protecting the Linux server's own memory.)
• I included enough detail to reproduce or investigate the problem.

Area

apps/server

Summary

On a 16 GB Linux host that also runs other services, the T3 server (t3code.service) freezes for minutes, once for two hours, while agent sessions run test suites. The server is not killed and nothing is logged except event-loop-stalls.ndjson. Every freeze shows the server's resident set shrinking while major page faults explode: the server is being swapped out, because it has no memory protection and the agents slice is allowed to push the host into reclaim.

Verified facts

Source (main, apps/server/src/process/agentScope.ts, apps/server/src/cloud/bootService.ts):

• agentSliceMemoryLimits(total) sets the agents slice to MemoryHigh = total - 6 GiB, MemoryMax = total - 4 GiB. The comment says this leaves about 6 GB free "so the server, sshd, and tailscaled keep working". The formula assumes nothing else on the host uses that 6 GB.
• t3code.service has CPUWeight=1000, IOWeight=1000, ManagedOOMPreference=avoid. It has no MemoryLow/MemoryMin, so nothing stops the kernel from reclaiming the server's pages first or at the same rate as the agents'.
• The agents slice has no CPU bound.

On the affected host (T3 0.1.2, 15.5 GiB RAM, 8 cores, 4 GiB swap file + 4 GiB zram added later):

• Agents slice before any local change: MemoryHigh=10241781760, MemoryMax=12513174528 (exactly the formula above). Other consumers on the same host: dockerd and buildkitd (about 1.4 GB RSS together), 14 small Postgres/Redis containers (about 1.7 GB), self-hosted GitHub runners. So "6 GB free" is used up before the agents hit MemoryHigh.
• ~/.t3/userdata/logs/event-loop-stalls.ndjson has 153 records between 2026-10-17 and 2026-10-27; 75 are 10 s or longer. The long ones, all with the server's rssMb at 230-430 MB (it is being paged out, not growing):

time                      delayMaxMs  majorPageFaults  rssMb
2026-10-20T22:04:23Z         238847        54478        413
2026-10-20T22:57:48Z        3110704       252683        397
2026-10-22T22:49:52Z        1352988        93415        370
2026-10-25T22:23:33Z         852625       702538        304
2026-10-26T05:38:50Z        7300444      6509327        275
2026-10-26T19:02:58Z         772631       456677        313
2026-10-27T14:06:17Z          50103        63107        304

• No kernel OOM kill in any of these windows (oom_kill 0, no Killed process in the journal). The user rebooted the machine by hand after the freezes; the journal shows clean systemd-reboot shutdowns, not crashes.
• At the time of the report, 8 agent sessions were running tsc, vitest, biome, go build and docker compose in separate worktrees. Load average was 29 on 8 cores; /proc/pressure/cpu some avg10 was 96%.
• A local cap I had put on the whole user-1000.slice (not T3's doing) made it worse, because the server shares that slice; that cap has been removed and is not the explanation for the earlier freezes (the rows before 2026-10-25 predate any local cap).

Hypotheses (not verified)

1. Under sustained reclaim in the agents slice, the kernel also evicts the server's pages because they have no memory.low protection; the server then page-faults on every event-loop turn. MemoryLow on t3code.service should keep its working set resident. The host mounts cgroup2 with memory_recursiveprot, so a leaf MemoryLow is effective without setting it on parent slices.
2. A fixed total - 6 GiB threshold is too loose on hosts that run other memory consumers. A limit based on what is actually available, or an override, would avoid it.
3. A default CPU bound on the agents slice would keep a build wave from starving sshd and the server's I/O threads; load 29 on 8 cores is the second symptom users notice.

Steps to reproduce

Not deterministic; the freezes follow bursts of parallel agent work.

1. Linux host with 16 GB RAM, swap enabled, other services using about 3-4 GB (docker + a few containers).
2. Run 6-8 agent sessions in separate git worktrees of a TypeScript monorepo, each running tsc, vitest, biome and a docker build at once.
3. Watch ~/.t3/userdata/logs/event-loop-stalls.ndjson: delayMaxMs climbs to tens of seconds or more with majorPageFaults in the tens of thousands, the UI stops responding, and /proc/pressure/memory shows sustained some.

Expected behavior

The server stays responsive while the agents slice is under pressure: agents are reclaimed and throttled first, and a stall that does happen is reported with its cause.

Actual behavior

The server is swapped out together with the agents, freezes for 1 minute to 2 hours, and recovers only after the load ends or the user restarts it.

Impact

Major degradation or frequent failure. It happens several times a day on this host, and the only recovery is a manual restart or reboot.

Suggested fix, in order of cost

1. Add MemoryLow=1G (or a size scaled to the server's RSS) to the t3code.service unit in bootService.ts. One line; testable with the reproduction above by comparing delayMaxMs and majorPageFaults with and without it.
2. Let users override the agents slice limits (env or settings) and document the total - 6 GiB assumption.
3. Optional default CPUQuota for the agents slice, or document the drop-in ~/.config/systemd/user/app-t3code-agents.slice.d/.
4. Extend the stall record with /proc/pressure/memory and the agents slice's memory.events high count so a pressure stall is distinguishable from a code stall (follows Design: report Linux pressure stalls and make shared-provider recovery scope explicit #25918).

Version or commit

t3 v0.1.2 (server), code read at main as of 2026-10-27.

Environment

Ubuntu 24.04.5, kernel 6.8.0-147-generic, systemd 255, cgroup2 with memory_recursiveprot, Intel i5-9300HF 8 threads, 15.5 GiB RAM, swap 4 GiB file + 4 GiB zram, vm.swappiness=60. Claude Code 2.1.311 agents (Opus and Sonnet), 6-8 concurrent sessions.

Workaround

Applied locally, so far verified only by a few minutes of lower load:

# ~/.config/systemd/user/app-t3code-agents.slice.d/60-memory.conf
[Slice]
MemoryHigh=7G
MemoryMax=8G
MemorySwapMax=2G
# ~/.config/systemd/user/t3code.service.d/50-memory-low.conf
[Service]
MemoryLow=1G

then systemctl --user daemon-reload. After this, t3code.service reports memory.low=1073741824. I will add a comment here with the stall log after a few days under the same workload.

Acceptance criteria

• With the reproduction above and fix 1, no delayMaxMs above 10 s in event-loop-stalls.ndjson over a comparable 24-hour run, or a before/after capture showing majorPageFaults per stall dropping by an order of magnitude.
• The stall record (fix 4) shows which cgroup was under reclaim.