Skip to content

fix(gateway): write LaunchAgent stderr to a real log instead of /dev/null#87

Merged
Agent-Aurelius merged 2 commits into
mainfrom
fix/gateway-selfheal-logging
Jun 27, 2026
Merged

fix(gateway): write LaunchAgent stderr to a real log instead of /dev/null#87
Agent-Aurelius merged 2 commits into
mainfrom
fix/gateway-selfheal-logging

Conversation

@Agent-Aurelius

Copy link
Copy Markdown

What + why

When a gateway crash-loops on a customer box, there was no log to readStandardErrorPath in the LaunchAgent plist was hardcoded to /dev/null. A real incident (a customer harness) took break-glass SSH + triangulating the relay log, cloud readiness, and manual probes to find that the gateway was rejecting a retired config key. The harness-diagnostics review called this the core blind spot.

The fix is small because the infrastructure already exists: resolveGatewaySupervisorLogPaths already returns a sibling stderrPath (<prefix>.err.log, next to the stdout <prefix>.log in the same secured log dir) — the plist writer just threw it away and used /dev/null. This wires the real stderrPath through both plist-build sites (fresh install + restart) and removes the dead /dev/null constant. StandardInPath stays /dev/null.

Changes

  • src/daemon/launchd.ts — destructure + pass stderrPath from resolveGatewaySupervisorLogPaths at both writeLaunchAgentPlist and the restart path; drop the unused LAUNCH_AGENT_STDERR_PATH = "/dev/null" constant.
  • src/daemon/launchd.test.ts — the install-plist test now asserts StandardErrorPath.../Logs/openclaw/gateway.err.log (was /dev/null).

Not in this PR (explicit follow-up)

The full self-heal of the customer crash-loop has two more halves that involve sensitive boot/infra code and deserve their own review:

  1. In-process config self-heal — at gateway startup, when the config snapshot is invalid due to a recoverable legacy issue (e.g. retired channels.webchat, which has a doctor migration), auto-apply the migration + re-snapshot before throwing, instead of crash-looping. (Touches config-write-at-boot.)
  2. Formula-relink pre-flight — a LaunchAgent wrapper that relinks the unlinked formula CLI before launching the gateway. (Touches plist ProgramArguments restructuring; can't be done in-process since the process never starts when the CLI is unlinked.)

This PR makes #1 and #2 observable (the failure now lands in the err log), which is the prerequisite for fixing them safely.


Anvil handoff

  • Scope: gateway LaunchAgent plist generation (darwin). Routes stderr to the already-computed log path.
  • Touched surfaces: writeLaunchAgentPlist + the restart plist rebuild in src/daemon/launchd.ts. No behavior change beyond the stderr destination; no new files, no config schema, no API.
  • Firebase/deploy impact: none. Ships in the next openclaw release → brew tap bump. On upgrade, the gateway plist is rewritten with the new stderr path; the log dir is already ensured (stdout uses it).
  • Local gates: vitest src/daemon/launchd.test.ts 89/89 ✓ · oxlint 0 ✓ · oxfmt --check clean ✓.
  • Smoke evidence: generated plist now contains <key>StandardErrorPath</key><string>…/Library/Logs/openclaw/gateway.err.log</string>; StandardInPath remains /dev/null.
  • Blockers: none for merge.
  • Follow-ups: the two self-heal halves above (own PR).

🤖 Drafted by Aurelius — left draft for Anvil. Cory (LightDriverCS) approves from the human seat.

Agent-Aurelius and others added 2 commits June 26, 2026 17:34
…null

The gateway LaunchAgent sent StandardErrorPath to /dev/null, so when the gateway
crash-looped (e.g. a retired config key the current build rejects, or an unlinked
formula CLI) there was no log to read — diagnosing a live customer incident
required break-glass SSH and stitching the cause from the relay log, cloud
readiness, and manual probes.

`resolveGatewaySupervisorLogPaths` already computes a sibling `stderrPath`
(`<prefix>.err.log` next to the stdout `<prefix>.log`); the plist writer just
ignored it and hardcoded /dev/null. Wire the real stderr path through both
plist-build sites (install + restart) and drop the dead constant. stdin stays
/dev/null. Crash-loops are now observable from the log file.

Follow-up (separate PR): in-process self-heal that auto-applies recoverable
legacy-config migrations (e.g. retired channels.webchat) at gateway startup
before throwing, plus a LaunchAgent pre-flight that relinks the formula CLI —
the other two halves of the customer crash-loop, which involve config-write-at-
boot and plist restructuring and deserve their own focused review.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@LightDriverCS

Copy link
Copy Markdown

Anvil hardening pushed 5242278a49 on top of the plist fix.

What changed: macOS status/runtime hints and last-error diagnostics now point at/read gateway.err.log, and stderr error matches win over stale stdout matches. This keeps the new LaunchAgent stderr log discoverable from the same diagnostic surfaces that operators use during crash loops.

Local proof:

  • pnpm test src/daemon/launchd.test.ts src/daemon/diagnostics.test.ts src/daemon/runtime-hints.test.ts src/daemon/runtime-hints.windows-paths.test.ts src/cli/daemon-cli/status.print.test.ts -- --reporter=verbose
  • pnpm check
  • pnpm build
  • .agents/skills/autoreview/scripts/autoreview --mode local (clean after one accepted finding was fixed)

No Firebase/App Hosting, auth, tenant, data/schema, or new control-surface impact found.

@Agent-Aurelius
Agent-Aurelius marked this pull request as ready for review June 27, 2026 00:01

@LightDriverCS LightDriverCS left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anvil auto-approval — verdict READY, non-code-owned. Approved under the dedicated Anvil reviewer identity.

@Agent-Aurelius
Agent-Aurelius merged commit 28bda9f into main Jun 27, 2026
195 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants