Skip to content

fix(server): avoid uresourced loops in Linux service units - #25185

Open
maria-rcks wants to merge 2 commits into
mainfrom
maria/takeover-pr-20968
Open

maria-rcks wants to merge 2 commits into
mainfrom
maria/takeover-pr-20968

Conversation

@maria-rcks

@maria-rcks maria-rcks commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Linux service units emit ManagedOOMPreference=avoid, which triggers the reported uresourced 0.5.4 CPU loop on affected Fedora/GNOME systems (#20931).

The directive and its obsolete comment are removed, restoring systemd's default OOM preference. OOMPolicy=continue, the CPU and IO weights, and agent isolation stay. Existing installations pick up the new unit on the next t3 service install.

The existing install test failed on main because the written unit contained ManagedOOMPreference= and passes with the fix; the rendered unit also passed systemd-analyze verify. Unverified: live Fedora/GNOME/uresourced behavior and memory-pressure handling. Human review of the changed Linux service default is still needed.

Replaces https://git.cafe/pingdotgg/t3code/pulls/20968, with thanks to @nateEc for the original implementation.

Built with GPT-6.1-Sol in the Codex harness on T3 Code, reviewed by Claude Opus 5.5 in Claude Code.

Fixes #20931

@github-actions github-actions Bot added vouch:trusted Author is trusted by repo permissions or the VOUCHED list. size:XS 0-9 changed lines (additions + deletions). labels Oct 26, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — Removing ManagedOOMPreference=avoid restores systemd's default OOM preference to address reported uresourced CPU loops. Human review is needed for this Linux service-default change and its memory-pressure availability tradeoff.

You can add or adjust custom eligibility rules. Learn more.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 21.5 KiB 21.5 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 1 1 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.9 KiB 21.9 KiB 0 B (0.0%) 24.4 KiB ✅
Claude Live turn messages 1 1 0 (0.0%) 8 ✅

Other scenarios

All 64 scenario measurements match main.

All 64 scenario measurements
Provider Scenario Metric Main baseline This PR Impact PR ceiling
Codex short turn resume WebSocket decoded 10.5 KiB 10.5 KiB 0 B (0.0%) 12.7 KiB ✅
Codex short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn running, cold open HTTP decoded 281.7 KiB 281.7 KiB 0 B (0.0%) 302.7 KiB ✅
Codex long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Codex long turn running, history page HTTP decoded 92.4 KiB 92.4 KiB 0 B (0.0%) 107.4 KiB ✅
Codex long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Codex long turn finished, cold open HTTP decoded 381.5 KiB 381.5 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, socket cold open WebSocket decoded 381.6 KiB 381.6 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, work page HTTP decoded 186.2 KiB 186.2 KiB 0 B (0.0%) 200.2 KiB ✅
Codex long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Codex idle thread resume WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell snapshot HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Codex fleet shell subscribe WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Codex fleet shell resume check rows 0 0 0 0 ✅
Codex stale shell reload HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex stale shell reload rows 27 27 0 (0.0%) 27 ✅
Codex stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Codex stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex stale todo list cold open HTTP decoded 72.4 KiB 72.4 KiB 0 B (0.0%) 83.0 KiB ✅
Codex stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Codex stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅
Claude short turn resume WebSocket decoded 10.6 KiB 10.6 KiB 0 B (0.0%) 12.7 KiB ✅
Claude short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn running, cold open HTTP decoded 283.3 KiB 283.3 KiB 0 B (0.0%) 302.7 KiB ✅
Claude long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Claude long turn running, history page HTTP decoded 93.4 KiB 93.4 KiB 0 B (0.0%) 107.4 KiB ✅
Claude long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Claude long turn finished, cold open HTTP decoded 384.2 KiB 384.2 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, socket cold open WebSocket decoded 384.3 KiB 384.3 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, work page HTTP decoded 188.1 KiB 188.1 KiB 0 B (0.0%) 200.2 KiB ✅
Claude long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Claude idle thread resume WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell snapshot HTTP decoded 42.9 KiB 42.9 KiB 0 B (0.0%) 53.7 KiB ✅
Claude fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Claude fleet shell subscribe WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Claude fleet shell resume check rows 0 0 0 0 ✅
Claude stale shell reload HTTP decoded 42.9 KiB 42.9 KiB 0 B (0.0%) 53.7 KiB ✅
Claude stale shell reload rows 27 27 0 (0.0%) 27 ✅
Claude stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Claude stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude stale todo list cold open HTTP decoded 73.4 KiB 73.4 KiB 0 B (0.0%) 83.0 KiB ✅
Claude stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Claude stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅

Baseline: c00a373 · PR result: dc3c18e · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The generated Linux systemd unit no longer includes ManagedOOMPreference=avoid. The test checks that the setting is absent and retains checks for the pinned runtime launcher and other unit settings.

Changes: Systemd unit update

Layer / File(s) Summary
Remove the OOM preference directive
apps/server/src/cloud/bootService.ts, apps/server/src/cloud/bootService.test.ts
The unit template removes ManagedOOMPreference=avoid and its comment. The test checks that the setting is absent and retains checks for the pinned runtime launcher and other unit settings.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related PRs

  • pingdotgg/t3code#20968: This PR removes the same systemd directive and adds a regression assertion that it is absent.

Suggested reviewers: t3dotgg, juliusmarminge

Merge Risk: Minimal · up to dc3c1

Linux service units will no longer set an OOM preference, which restores systemd's default and avoids the reported user-manager CPU loop. Existing installs pick up the change on the next service install. No merge-blocking risk is identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the primary change: removing ManagedOOMPreference=avoid from Linux service units to fix the uresourced CPU loop. It follows conventional commit form and matches the raw summary, the PR…
Description check Passed The description covers the problem (ManagedOOMPreference=avoid triggers the uresourced 0.5.4 CPU loop, linked to issue #20931), the change (directive and comment removed, OOMPolicy and weights kept), …
Linked Issues check Passed Directly linked issue #20931 asks that the t3code.service unit not cause ongoing user-manager work. The cause is the ManagedOOMPreference=avoid directive, which the issue says triggers a uresourced …
Out of Scope Changes check Passed The PR changes two areas. In apps/server/src/cloud/bootService.ts, it removes only the ManagedOOMPreference=avoid line and its comment, which is the directive named by #20931. In apps/server/src/clo…
Full details: Title check

Explanation

The title names the primary change: removing ManagedOOMPreference=avoid from Linux service units to fix the uresourced CPU loop. It follows conventional commit form and matches the raw summary, the PR objectives, and the linked issue #20931. The title is 58 characters, which is acceptable.

Full details: Description check

Explanation

The description covers the problem (ManagedOOMPreference=avoid triggers the uresourced 0.5.4 CPU loop, linked to issue #20931), the change (directive and comment removed, OOMPolicy and weights kept), and the scope (a bug fix linked to an issue). Verification is detailed: the existing install test was described, systemd-analyze verify passed, and unverified areas are listed. The template's Problem, Change, Scope, and Verification sections are all present. The model and harness line is also present. Minor gap: the description does not give a full reproduction or the exact test results beyond the one failing test; the comments also describe more manual checks than the description states.

Full details: Linked Issues check

Explanation

Directly linked issue #20931 asks that the t3code.service unit not cause ongoing user-manager work. The cause is the ManagedOOMPreference=avoid directive, which the issue says triggers a uresourced feedback loop. The diff at the reviewed head removes that directive and its comment from the unit renderer in apps/server/src/cloud/bootService.ts (renderBootServiceUnit). The reviewed head keeps OOMScoreAdjust=100 and the other unit lines, so the diff does not remove them. The test change in apps/server/src/cloud/bootService.test.ts asserts that the generated unit omits ManagedOOMPreference, so the removal has automated coverage. The issue's suggested fix is the removal the code performs. The PR states that live Fedora/GNOME/uresourced behavior was not verified. That limit does not change the code-level compliance with the issue's coding requirement.

Full details: Out of Scope Changes check

Explanation

The PR changes two areas. In apps/server/src/cloud/bootService.ts, it removes only the ManagedOOMPreference=avoid line and its comment, which is the directive named by #20931. In apps/server/src/cloud/bootService.test.ts, the XDG configuration-home test reads the generated unit once and asserts that it omits ManagedOOMPreference. This test change supports the removal. No unrelated functional change was identified in the reviewed evidence.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 09a6216 and dc3c18e.

Files selected for processing: 2

  • apps/server/src/cloud/bootService.test.ts
  • apps/server/src/cloud/bootService.ts
Files with no reviewable changes: 1

  • apps/server/src/cloud/bootService.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@juliusmarminge juliusmarminge added bugfix Fix PR opened by an issue lead; enters the PR loop pr:needs-julius Needs Julius: route gate, unverifiable surface, or blocked labels Oct 27, 2026

Copy link
Copy Markdown
Collaborator Author

Note

Written by gpt-6.1-sol on behalf of Maria

Verified in a real dev environment

Verified unchanged head 3f1a472b72ec06dc3a920ff4ce959c68e0f45c83 on Ubuntu 24.04 with a real systemd user manager.

  • Created a separate github/main worktree at 4914cdd98f3d15ab2ddef34f0af70d3d68a2d8e5, ran vp i, and ran the actual node apps/server/src/bin.ts service install --base-dir /home/user/service-verification. The installed unit contained ManagedOOMPreference=avoid; the running manager reported avoid. This reproduces the problematic configuration, not the Fedora CPU loop.
  • Checked out the exact PR head, ran vp i, and installed through the same CLI. Main is version 0.1.3 and this head is 0.1.2: normal installation correctly refused the downgrade; explicit --allow-downgrade then regenerated the real unit. Systemd reported ManagedOOMPreference=none, OOMPolicy=continue, CPUWeight=1000, and IOWeight=1000. The unit passed systemd-analyze --user verify --man=no.
  • Neighbor checks passed: service status, restart with a new active main PID, idempotent reinstall, and uninstall. No fake service manager or substituted runtime was used; installation downloaded the pinned production runtime.
  • Started the PR's actual dev server detached, with an explicit isolated home and no seeded data (server 13773, web 5733). Used repository Playwright with /usr/bin/google-chrome, added a real git repository through the UI, selected Codex gpt-6.1-sol, and completed three real turns: pwd/git status, /proc/self/cgroup, and a follow-up response. The agent process lived in a separate app-t3code-agents slice. These turns exercised the PR dev web/server/provider path; they were not submitted to the downloaded service runtime.
  • Blacksmith: all 33 existing boot-service tests passed. vp run check:changed d9abfdd1cb595d186391ed5a28000554d425b5fc passed format, lint, both knip passes, GraphQL consistency, affected typechecks, and the focused tests. The explicit PR parent was needed because the worker snapshot had no origin/main history. Existing PR CI is green.

Full service command transcript

main: ManagedOOMPreference=avoid
PR:   ManagedOOMPreference=none
      OOMPolicy=continue
      CPUWeight=1000
      IOWeight=1000
      ActiveState=active

Real Codex gpt-6.1-sol turn completed in the PR dev web app

Real Codex follow-up through the running PR web client

No code changes were needed. Verification services were stopped and the temporary systemd unit was uninstalled.

Not verified

  • This VM has no Fedora uresourced 0.5.4. The original CPU loop, its disappearance under Fedora/GNOME, and behavior under actual memory pressure remain unverified. No CPU-performance claim is made.
  • No provider turn was run through the downloaded background-service runtime; provider turns used the PR's dev server. Boot activation after machine reboot, logout/linger recovery, remote/relay/tunnel connections, native desktop/mobile clients, and macOS/Windows behavior were not exercised.
  • This is a service-template change with no visual UI change, so matching visual before/after screenshots do not apply.

@maria-rcks
maria-rcks force-pushed the maria/takeover-pr-20968 branch from 3f1a472 to 09a6216 Compare November 6, 2026 04:34
# Conflicts:
#	apps/server/src/cloud/bootService.test.ts
@juliusmarminge

Copy link
Copy Markdown
Member

The automated pipeline stopped on this pull request and will not merge it:

the capture step could not show the problem on the base commit, so there is nothing to prove fixed.

Read the scenario documentation, fetched the requested commits, created the head worktree, computed the merge-base, inspected the service-unit change, and checked the operating system, systemd user manager, and uresourced availability. Uploaded the inspected environment transcript and its driver. Removed the worktree and scratch directory.

It stays open for a person to finish or close.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Fix PR opened by an issue lead; enters the PR loop pr:needs-julius Needs Julius: route gate, unverifiable surface, or blocked size:XS 0-9 changed lines (additions + deletions). vouch:trusted Author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: ManagedOOMPreference=avoid in t3code.service puts systemd --user into a 100% CPU loop with uresourced (Fedora/GNOME)

2 participants