Skip to content

[Bug]: [macOS] findGatewayPidsOnPortSync drops all PIDs due to lsof p_comm vs argv[0] mismatch #70664

Description

@mmartoccia

Bug type

Behavior bug (correctness) — silent no-op in a safety-critical cleanup path, observable as launchd respawn churn and log-file bloat.

Summary

On macOS, findGatewayPidsOnPortSync in src/infra/restart-stale-pids.ts never recognizes live openclaw gateway processes. parsePidsFromLsofOutput filters candidate PIDs by checking whether the lsof -Fpc command-name token contains "openclaw", but the openclaw gateway rewrites its argv[0] to "openclaw-gateway" after exec while the kernel's p_comm field (what lsof reports) stays "node". The filter therefore drops every real gateway PID, and cleanStaleGatewayProcessesSync silently no-ops even when there is a verifiable gateway listening on the port.

The result is a self-sustaining launchd KeepAlive respawn loop any time the managed PID changes: the new spawn can't detect the existing listener, fails EADDRINUSE, exits, and launchd spawns another one under ThrottleInterval. With the default plist written by openclaw doctor --repair (KeepAlive=true, RunAtLoad=true, ThrottleInterval=1), the cadence is one failed respawn every ~18 seconds, indefinitely.

Impact

  • Every macOS install running the gateway under launchd with KeepAlive=true is vulnerable. The bug is latent: the managed gateway itself keeps serving traffic, so functionality isn't visibly broken — the observable damage is respawn churn and log-file growth.
  • gateway.err.log grows quickly (a few MB/day under normal conditions, tens of MB/day once the loop is sustained). After a few days of uptime the file routinely hits tens to hundreds of MB, each EADDRINUSE cycle emitting ~7 lines.
  • The same filter is shared with pollPortOnceUnix, so waitForPortFreeSync returns {free: true} while the port is genuinely busy. This additionally masks the condition from any diagnostic that relies on that poll.
  • launchctl list reports runs=N but N is misleadingly low because failed spawns that exit in under MinimumRuntime=10 don't increment the counter. The orphan loop is therefore invisible to a casual launchctl print check — you have to watch ps directly, or see the log growth.

Root cause

In src/infra/restart-stale-pids.ts, parsePidsFromLsofOutput:

if (
  currentPid != null &&
  currentCmd &&
  normalizeLowercaseStringOrEmpty(currentCmd).includes("openclaw")
) {
  pids.push(currentPid);
}

currentCmd comes from lsof -Fpc output — the c field is the kernel p_comm / BSD_COMM field. On macOS (and Linux), that field is the basename of the exec'd binary as recorded at execve time, truncated to MAXCOMLEN. For a node-based gateway it is "node". The gateway sets its own process title via process.title = "openclaw-gateway" (or equivalent argv[0] rewrite) after startup — ps reads this rewritten argv[0] and prints "openclaw-gateway", but lsof does not; lsof stays with p_comm.

Because "node".includes("openclaw") is false, every real gateway PID is dropped. The outer filter pid !== process.pid is then irrelevant because the set is already empty.

Confirmed locally with a stock install:

$ /usr/sbin/lsof -nP -iTCP:18789 -sTCP:LISTEN -Fpc
p12345
cnode
f15

$ ps -p 12345 -o command=
openclaw-gateway

The Windows code path in the same file (filterVerifiedWindowsGatewayPids / isGatewayArgv) does not rely on a comm filter — it inspects the full process argv via PowerShell and matches against openclaw entry-point patterns. That's the correct abstraction; the Unix path was missing the equivalent step.

Reproduction

  1. Install openclaw on macOS.

  2. openclaw doctor --repair to install the default LaunchAgent with KeepAlive=true, RunAtLoad=true, ThrottleInterval=1.

  3. openclaw gateway start.

  4. Trigger any scenario where launchd's currently-tracked gateway PID exits briefly (e.g., the gateway's own in-process restart path, a /restart command, a direct kill <launchd-tracked-pid> where the actual listening PID is a different orphan).

  5. Tail ~/.openclaw/logs/gateway.err.log. Every ~18 seconds a fresh spawn reports:

    [gateway] ⚠️  Gateway is binding to a non-loopback address. …
    Gateway failed to start: another gateway instance is already listening on ws://0.0.0.0:18789 | listen EADDRINUSE: address already in use 0.0.0.0:18789
    If the gateway is supervised, stop it with: openclaw gateway stop
    Port 18789 is already in use.
    - pid <N> <user>: openclaw-gateway (*:18789)
    - Gateway already running locally. …
    
  6. Tail ~/.openclaw/logs/gateway.log for service-mode: cleared N stale gateway pid(s) before bind on portthe line never appears, because cleanStaleGatewayProcessesSync silently returned [].

  7. Optional definitive proof: launchctl bootout gui/$UID/ai.openclaw.gateway for ~60s and observe that no new orphan spawns occur. Re-bootstrap; the cycle resumes. This rules out any non-launchd source.

Proposed fix

Remove the lsof-comm-based filter and add a ps-based argv verifier on Unix, mirroring the Windows path's structure.

  1. parsePidsFromLsofOutput returns every listening PID (minus process.pid). No gateway-vs-other classification at the parse layer.
  2. New verifyGatewayPidByArgvSync(pid) helper: runs ps -ww -p <pid> -o command= and matches the result against the openclaw-gateway argv[0] rewrite and common entry-file patterns (/dist/index.js gateway, /openclaw.mjs gateway, /openclaw gateway, openclaw_repo…gateway for dev-mode invocations).
  3. findGatewayPidsOnPortSync on Unix runs lsof, then filters the returned PIDs through verifyGatewayPidByArgvSync.

Optional, separable hardening:

  • Bump PORT_FREE_TIMEOUT_MS from 2000 to 30000. Under load the kernel can take multiple seconds to release a TCP socket after SIGKILL (TIME_WAIT + teardown), and the current 2 s window doesn't leave useful slack.

I have a working patch against current main. Gateway.err.log growth halted, service-mode: cleared 1 stale gateway pid(s) began appearing correctly, and the EADDRINUSE respawn cycle was broken. Happy to open a PR.

Environment

  • macOS (Apple Silicon, Darwin 25.x)
  • openclaw 2026.4.x
  • Node 25.x
  • Default LaunchAgent as written by openclaw doctor --repair

Related (checked before filing)

This bug is orthogonal to all of the above. It's been silently present on every macOS install since the comm-filter was introduced, and no issue on the tracker names it. Filing this to get it on the record.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions