Skip to content

[Bug]: SIGUSR1/config.patch restart orphans gateway under custom supervisors → EADDRINUSE crash loop (workaround: OPENCLAW_NO_RESPAWN=1) #65668

Description

@mjamiv

Summary

When the gateway receives a SIGUSR1 restart signal (or any path that calls restartGatewayProcessWithFreshPid()) under a custom supervisor that is not one of the three hard-coded supervisor types (launchd, schtasks, systemd), the gateway spawns a detached child via spawn(..., { detached: true, stdio: "inherit" }), exits the current process, and the new child orphans to PPID 1. Any process tracking the original PID (e.g. a bash watchdog loop, a Kubernetes liveness probe with restartPolicy: Always, a custom Python supervisor) loses the gateway and tries to start a new one — which immediately hits EADDRINUSE because the orphaned child still owns the listening port.

The result is a permanent noisy crash loop after any config.patch, openclaw config set, or other config-edit-triggered SIGUSR1, until the orphan is manually killed.

openclaw doctor already surfaces an advisory line:

OPENCLAW_NO_RESPAWN is not set to 1; set it to avoid extra startup overhead from self-respawn.

(See prompt-select-styled-*.js:~1436 in the v2026.4.12-beta.1 bundle.) The advisory wording is misleading — under any custom supervisor, this is not "extra startup overhead", it's a permanent failure mode that persists across restarts.

Environment

  • OpenClaw: confirmed against installed bundles for v2026.4.9 (gateway-cli-CWoiyhEX.js), v2026.4.11-beta.1 (gateway-cli-CIrsDEzG.js), and v2026.4.12-beta.1 (gateway-cli-DwNao8Uk.js). Code shape is identical across all three.
  • Node: 22.x
  • OS: Linux (Ubuntu 24.04 host + sandbox container variants)
  • Trigger: any code path that fires restartGatewayProcessWithFreshPid() on a host where detectRespawnSupervisor(process.env) returns null.

Code-level trace (against v2026.4.12-beta.1)

gateway-cli-DwNao8Uk.js:770 defines restartGatewayProcessWithFreshPid():

function restartGatewayProcessWithFreshPid() {
  if (isTruthy(process.env.OPENCLAW_NO_RESPAWN)) return { mode: "disabled" };
  const supervisor = detectRespawnSupervisor(process.env);
  if (supervisor) {
    if (supervisor === "schtasks") {
      const restart = triggerOpenClawRestart();
      if (!restart.ok) return { mode: "failed", detail: ... };
    }
    return { mode: "supervised" };
  }
  if (process.platform === "win32") return {
    mode: "disabled",
    detail: "win32: detached respawn unsupported without Scheduled Task markers"
  };
  try {
    const args = [...process.execArgv, ...process.argv.slice(1)];
    const child = spawn(process.execPath, args, {
      env: process.env,
      detached: true,
      stdio: "inherit"
    });
    child.unref();
    return { mode: "spawned", pid: child.pid ?? void 0 };
  } catch (err) { ... }
}

detectRespawnSupervisor only recognizes:

  • launchd (macOS)
  • schtasks (Windows Task Scheduler)
  • systemd (via INVOCATION_ID / JOURNAL_STREAM / similar)

A bare bash watchdog loop like:

while true; do
  pkill -f "openclaw.*gateway"
  sleep 2
  /sandbox/openclaw gateway --port 18801 &
  PID=$!
  wait $PID
done

— or any equivalent in Python / Kubernetes init containers / Docker entrypoint — does not set any of those env vars, so detectRespawnSupervisor returns null and the code falls through to spawn(..., { detached: true }).

Reproducer

  1. Install OpenClaw inside any container or sandbox where the gateway is run under a custom supervisor (not systemd/launchd/schtasks). For example a bash loop:
    #!/bin/bash
    while true; do
      pkill -f "openclaw.*gateway" 2>/dev/null
      sleep 2
      openclaw gateway --port 18801 &
      wait $!
      echo "gateway exited, restarting..."
      sleep 1
    done
  2. Start the gateway. Confirm it binds port 18801 and serves health.
  3. Edit any value in ~/.openclaw/openclaw.json and run openclaw config patch <something> (or any other path that fires SIGUSR1, including some plugin install/uninstall flows).
  4. Observe in the log:
    [gateway] received SIGUSR1; restarting
    [gateway] restart mode: full process restart (spawned pid <N>)
    
  5. The original PID exits. The new spawned child reparents to PPID 1.
  6. The bash supervisor's wait $! returns; the loop runs pkill -f "openclaw.*gateway" — but pkill does match the orphaned child by name, so it gets killed. Loop spawns a new gateway. However, in many real supervisor implementations (including ours) the supervisor uses a tracked PID or a more specific match pattern that does NOT catch the orphaned child. In those cases the orphan keeps the port and the supervisor's new child loops on EADDRINUSE:
    Gateway failed to start: gateway already running (pid <orphan>); lock timeout after 5000ms
    Port 18801 is already in use.
    - pid ?: unknown (0.0.0.0:18801)
    
    This crash-loops every supervisor restart cycle (typically every 5–10 seconds) until the orphan is manually identified and killed.

Workaround

Setting OPENCLAW_NO_RESPAWN=1 in the gateway's environment makes restartGatewayProcessWithFreshPid() return { mode: "disabled" } immediately, falling through to handleRestartAfterServerClose()'s in-process restart path:

gatewayLog$1.info(`restart mode: in-process restart (${respawn.detail ?? "OPENCLAW_NO_RESPAWN"})`);
if (hadLock && !await reacquireLockForInProcessRestart()) return;
shuttingDown = false;
restartResolver?.();

The in-process restart releases the lock, drops the server, re-acquires the lock with the same PID, and re-runs runGatewayLoop. The supervisor's tracked PID stays valid and no orphan is created.

This works perfectly. We've been running on it for ~24h on one gateway with zero crash loops after config edits, vs ~7–10s of EADDRINUSE noise per config edit before the change.

Suggested fix

Three options, in order of increasing scope:

Option 1: re-word the openclaw doctor advisory (small, immediate)

Change the advisory text from:

OPENCLAW_NO_RESPAWN is not set to 1; set it to avoid extra startup overhead from self-respawn.

to:

OPENCLAW_NO_RESPAWN is not set to 1. Under any supervisor not recognized by detectRespawnSupervisor() (i.e. anything other than launchd, schtasks, or systemd), the SIGUSR1 restart path will detach-spawn an orphaned gateway and your supervisor will lose the PID. Set OPENCLAW_NO_RESPAWN=1 unless you specifically want detach-spawn behavior.

This alone would have saved us several hours of "why is the gateway crash-looping after a config edit".

Option 2: detect generic supervisor topology

In detectRespawnSupervisor, add a generic-supervisor check: if process.ppid !== 1 AND the parent process name is one of bash, sh, python, node, tini, etc., treat the parent as a supervisor and return "unknown-supervisor". The caller then uses the same mode: "supervised" path that systemd already takes — exit and let the parent re-spawn.

Risk: the parent may not actually be configured to restart on exit. But this is also the case for systemd users who didn't set Restart=on-failure — the existing detection assumes any of the three known supervisors will restart, which is itself a soft assumption. A bash while loop is at least as reliable as that.

Option 3: invert the default

Make in-process restart the default for runGatewayLoop, and make detached-spawn the explicit opt-in via something like OPENCLAW_DETACHED_RESPAWN=1. The advantages of detached-spawn (clearing leaked file descriptors, picking up new entry points, working around long-running native module state) are real but rare; in-process restart is what 99% of users want and is dramatically less footgunny.

Risk: this is a behavior change on a stable API and may break a small number of operators who rely on PID rotation for log rotation or similar.

Why this matters now

Two reasons:

  1. Trigger frequency is increasing. Issues like config.patch unconditionally schedules SIGUSR1 restart, ignoring hot-reload capability #46310 ("config.patch unconditionally schedules SIGUSR1 restart, ignoring hot-reload capability") mean that even routine config edits now fire SIGUSR1, where they previously triggered in-process hot-reload. So the population of users who hit this bug is growing.
  2. The advisory wording has lulled people into thinking it's cosmetic. "Extra startup overhead" sounds like a few hundred milliseconds. The actual failure mode is a permanent crash loop. We hit this twice in 48 hours before tracing it back to the SIGUSR1 detach-spawn behavior.

Related issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions