fix(infra/restart): exclude ancestor pids from stale-gateway cleanup#68517
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d26748ccca
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Greptile SummaryThis PR fixes an infinite supervisor restart loop caused by The implementation is correct — bounded walk, cycle detection, graceful Confidence Score: 5/5Safe to merge; the only finding is a stale doc comment that does not affect runtime behaviour. All findings are P2. The logic is correct, the ancestor walk is bounded and cycle-safe, error paths degrade gracefully, tests cover all three filter code paths, and the public API is unchanged. No files require special attention beyond the minor comment update in src/infra/restart-stale-pids.ts. Prompt To Fix All With AIThis is a comment left during a code review.
Path: src/infra/restart-stale-pids.ts
Line: 147-150
Comment:
**Stale "Pure function — no I/O" comment**
The retained `Pure function — no I/O` claim is no longer accurate after this PR. `parsePidsFromLsofOutput` now calls `getSelfAndAncestorPidsSync()`, which reads up to 32 `/proc/<pid>/status` files on Linux. Future contributors relying on this comment to reason about testing, memoization, or call-site constraints will be misled.
```suggestion
/**
* Parse openclaw gateway PIDs from lsof -Fpc stdout.
* Excludes the current process and its ancestors
* (see `getSelfAndAncestorPidsSync` for the full rationale).
* Note: performs /proc reads on Linux via getSelfAndAncestorPidsSync.
*/
```
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "fix(infra/restart): exclude ancestor pid..." | Re-trigger Greptile |
bb1a8e1 to
0e79318
Compare
84eec58 to
5118a62
Compare
5118a62 to
fef3036
Compare
fef3036 to
7520a06
Compare
The stale-gateway cleanup filter already refused to kill process.pid — acknowledging the invariant that terminating a process whose death cascades into the caller is never safe. That invariant was applied only to the caller itself, not to its ancestors, which is why the openclaw-weixin sidecar triggered an unbounded restart loop: the sidecar's cleanup SIGTERM'd its parent gateway, the supervisor restarted the gateway, the gateway re-spawned the sidecar, the cleanup ran again. Complete the invariant by excluding the full self+ancestor PID set in both the lsof (Unix) and PowerShell/netstat (Windows) cleanup paths. Walk uses process.ppid unconditionally (Node built-in, no spawn) and /proc/<pid>/status on Linux for transitive ancestors, with graceful degradation where /proc is unavailable.
7520a06 to
45e0b44
Compare
Summary
Problem: The
openclaw-weixinsidecar invokescleanStaleGatewayProcessesSync()during its init path. The Unix filter insrc/infra/restart-stale-pids.tsexcluded onlyprocess.pid, and the two Windows filters did the same. Because the sidecar and its parent gateway share theopenclawcommand-name signature and the parent is listening on the configured gateway port, the parent's PID passed the filter and was SIGTERM'd. The supervisor restarted the gateway, the gateway re-spawned the sidecar, the cleanup ran again — an unbounded restart loop on ~45 s cadence, 100 % reproducible onmain(2026.4.14 and 2026.4.15).Root Cause: The
pid !== process.pidfilter encodes an invariant: "stale-gateway cleanup must never terminate a process whose death cascades into the caller." That invariant was applied only to the caller itself. Killing a direct ancestor is functionally identical to killing the caller — cgroup cleanup reaps the child, the supervisor (systemd / launchctl) respawns the ancestor, and the child re-enters the same cleanup call site. The same applies transitively to any ancestor whose death propagates via the supervisor. The existing guard was therefore not wrong, merely incomplete: it protected one node of the lineage where it needed to protect the whole chain.Fix: Complete the self-exclusion invariant at the filter layer so the cleanup primitive is safe regardless of caller context. A new helper
getSelfAndAncestorPidsSync()returns the set of PIDs comprising the current process, itsprocess.ppid(always via Node's syscall — no spawn, cross-platform), and on Linux the transitive ancestor chain read from/proc/<pid>/status.parsePidsFromLsofOutput(Unix) andfilterVerifiedWindowsGatewayPids/filterVerifiedWindowsGatewayPidsResult(Windows) both consult this set. The walk is bounded (MAX_ANCESTOR_WALK_DEPTH = 32), cycle-safe (early break onpids.has(parent)), and degrades silently on any/procread failure — a deliberate product decision for hardened containers (hidepid=2 / gVisor / AppArmor), documented in thereadParentPidFromProccatch-branch comment and locked by the "leaves the gateway grandparent in the kill list when /proc truncates the walk" regression test so the degraded outcome is explicit, not accidental. The fix is chosen over "detect misuse and refuse to run" or "patch the third-party sidecar call site" because (a) it repairs a design-level invariant rather than adding a detector, (b) it protects every present and future caller of the exported function, and (c) it has zero behavioural effect on the three legitimate supervisor-context callers (launchd.ts,restart.ts,gateway-cli/run.ts) whose ancestor chains are systemd/launchctl/shell and never listen on the gateway port.What changed:
src/infra/restart-stale-pids.ts:readFileSyncfromnode:fs.MAX_ANCESTOR_WALK_DEPTH = 32.readParentPidFromProc(pid)— Linux/proc/<pid>/statusparse, returnsnumber | null. The catch branch carries a comment documenting that a null return truncates the ancestor walk at the current hop; for the directgateway → sidecartopology this is safe becauseprocess.ppidis captured unconditionally without a/procread, but a deepergateway → plugin-host → sidecarchain on hidepid=2 / gVisor / AppArmor-locked namespaces leaves the gateway grandparent in the kill list. The degraded outcome is pinned by a regression test rather than silently allowed.getSelfAndAncestorPidsSync()— returnsSet<number>of self + direct parent + transitive ancestors (Linux). The check admits any positive ancestor PID including 1, because a containerised gateway is frequently the namespace entrypoint (PID 1); excluding 1 unconditionally would recreate the [Bug]: openclaw-weixin sidecar kills main gateway #68451 loop on every containerised install. Written in early-return style (no dead inner guards).parsePidsFromLsofOutputreplaces thepid !== process.pidfilter with!excluded.has(pid)whereexcluded = getSelfAndAncestorPidsSync(). Its JSDoc is updated to describe the Linux/procbehaviour (no more "pure function — no I/O" claim).filterVerifiedWindowsGatewayPidsandfilterVerifiedWindowsGatewayPidsResultapply the same exclusion set before argv verification, so the parent gateway PID is dropped beforereadWindowsProcessArgsSyncis even asked about it.__testingexport is unchanged — the existingsetSleepSyncOverride/setDateNowOverride/callSleepSyncRawremain; no new hooks were added. The ancestor walk is exercised by test-only stubbing ofprocess.ppidandvi.mock('node:fs'), keeping the runtime-reachable mutation surface at zero.src/infra/restart-stale-pids.test.ts:vi.mock('node:fs')interceptsreadFileSyncso the/procancestor lookup can be driven deterministically. A preciseas typeof actual.readFileSynccast satisfiestsgo's overload check without reaching forany.mockReadFileSynchoisted mock +beforeEachthat installs an ENOENT default (simulating a restricted/proc— the view a non-privileged process has under hidepid=2). Tests that need a specific/procpayload override withmockImplementation.withStubbedPpid(ppid, fn)helper —Object.defineProperty(process, 'ppid', ...)+ descriptor restore infinally, matching the pattern already used forprocess.platformstubbing in this file. No__testingmutation surface is introduced.it.skipIf(platform !== 'linux'), drives the real/procwalk with a mocked two-hop chain terminated byPPid:\t0.> 0guard (an earlier> 1draft would have reopened the loop for containerised installs).it.skipIf(platform !== 'linux'), pins the degraded behaviour so a future refactor cannot silently regress further.process.platformstub, asserting parent exclusion happens before the argv-verification step.What did NOT change (scope boundary):
launchd.ts,restart.ts,gateway-cli/run.ts) — the fix lives at the filter layer.lsofspawn logic,terminateStaleProcessesSync,waitForPortFreeSync,pollPortOnce, or any port/poll-budget timing constant.isGatewayArgvor the Windows argv-verification step; ancestor exclusion happens before that inspection, so argv lookups are never issued against caller-lineage PIDs.__testingsurface. No new runtime-reachable override — in particular, nosetAncestorPidProviderOverride(an earlier iteration proposed one and it was removed per review feedback so no in-process module can neutralise ancestor exclusion at runtime).cleanStaleGatewayProcessesSync(portOverride?)keeps its exact contract.readFileSyncis a Node built-in already transitively present in the module graph.anytypes introduced — new surfaces are typed (number | null,Set<number>, helpers) and the one unavoidable cast (as typeof actual.readFileSync) is a precise retype against the real module export, not a widening toany.Reproduction
2026.4.14or2026.4.15behind a supervisor (systemd, launchd, or equivalent withRestart=always).@tencent-weixin/openclaw-weixinv2.1.3 or v2.1.8 and enable it inplugins.allow.systemd-cglsshows the gateway and sidecar in a tight respawn cycle;journalctl -u openclawshows repeated EADDRINUSE / SIGTERM / exit-1 entries.With this patch applied, step 4 no longer loops: the sidecar's cleanup filter drops the parent gateway PID before any SIGTERM is sent, and the gateway remains stable.
Risk / Mitigation
getSelfAndAncestorPidsSync()is called fromparsePidsFromLsofOutput, which executes once perlsofpoll insidewaitForPortFreeSync(up to ~40 calls per cleanup budget). On Linux each call may read up to 32/proc/<pid>/statusfiles; on macOS/Windows it is in-memory only./procreads arereadFileSyncagainst a virtual filesystem — no disk I/O, no syscalls to external binaries. Worst-case arithmetic: 40 polls × ≤ 32 reads × few hundred bytes each ≈ well under 10 ms per cleanup, within noise of the existingspawnSync("lsof", …)overhead. Every read is wrapped intry/catchreturningnull; the walk also breaks on anynullresult, so a restricted/proc(user namespace, AppArmor, seccomp) degrades to "only self + ppid excluded" rather than failing. Existing tests pass unchanged because their syntheticstalePid = process.pid + Nvalues are never in the caller's real ancestor chain; new tests drive the walk deterministically viawithStubbedPpidandvi.mock('node:fs')./proc/${pid}/status) could be flagged by a security review for path injection.pidis always an integer produced byprocess.ppidor byNumber.parseIntof a/proc-sourced decimal field, filtered byNumber.isFinite(parsed) && parsed > 0. No user input, no string concatenation from untrusted data, no shell — the value is always a positive integer interpolated into a fixed path shape./procso the walk truncates atprocess.ppid, leaving a 3-levelgateway → plugin-host → sidecarchain with the gateway grandparent outside the exclusion set.readParentPidFromProccatch branch and pinned by the "leaves the gateway grandparent in the kill list when /proc truncates the walk" regression test. The reported [Bug]: openclaw-weixin sidecar kills main gateway #68451 case is direct-parent (gateway → sidecar) and is fully covered on every platform viaprocess.ppidwithout depending on/proc. A fuller fix for deeper chains on hardened hosts (macOS/Windows ancestor walk, pidfd-based Linux walk, or privileged cmdline probe) is tracked out of band.Change Type (select all)
Scope (select all touched areas)
src/infra/restart-stale-pids.ts)src/infra/restart-stale-pids.test.ts)Linked Issue/PR
Fixes #68451