Environment

• OpenClaw: 2026.6.9
• Backend: cliBackends.claude-cli (Anthropic Claude Code CLI)
• Channel: Telegram (long-polling); a DM lane and a group-topic lane both observed in the same incident
• Agent: agents.list[main], model claude-cli/claude-opus-4-7
• Host: single-VPS Linux, systemd --user (openclaw-gateway.service, Restart=always, StartLimitBurst=5, StartLimitIntervalSec=60)
• Workspace plugin in play: family-week (amplifier — see "Plugin attach cost" below; this issue is not about the plugin itself)

TL;DR

When a scheduled isolated cron agent fails to complete its setup phase within the configured timeout, onIsolatedAgentSetupTimeout (in server-cron-DfOwaY9F.js:3770) responds by calling requestSafeGatewayRestart. In every observed case the root cause of the setup timeout has been event-loop saturation in the parent gateway process — not corruption of the gateway state, not a stuck child process, not a leaked handle.

A full-process gateway restart is the wrong primitive for that root cause:

1. It doesn't fix the cause. Event-loop saturation is process-state. After restart, the same main-session embedded runs resume against the same workload, the loop re-saturates, and the next cron tick re-wedges. Observed cycle time ~10 minutes in the 2026-06-22 incident.
2. It actively destroys in-flight work. requestSafeGatewayRestart is called with delayMs: 0. The shutdown path then logs forced restart requested; skipping active work drain (drain timeout 0 ms). In the 2026-06-22 incident this nuked 3 active main-session embedded runs. Restart-recovery then failed for 2 of them with gateway timeout after 10000ms (the cross-boot retry-budget gap is tracked separately at [Bug]: Main-session restart-recovery has no cross-boot retry budget — wedged sessions death-loop the gateway across reboots (2026.6.9) #95750).
3. "Safe" in requestSafeGatewayRestart is misleading. It preserves pending emit hooks but does not preserve in-flight embedded runs, which is what users actually care about.
4. The restart is an amplifier, not a circuit breaker. Once systemd's Restart=always is in the loop, the gateway thrashes against the same saturation pattern. In the 2026-06-22 incident the systemd auto-restart counter reached 29 before the loop broke.

Production incident (2026-06-22, UTC)

Sequence reconstructed from gateway journal + stability bundle:

• 12:11:08 — first cron job stalls on phase runtime-plugins (setup, pre-execution).
• 12:15–12:21 — eventLoopUtilization sustained at 99–100% with eventLoopDelayP99Ms 8–12 s. Driver: 4 concurrent main-session embedded runs.
• 12:17:52, 12:17:57 — two cron jobs fail isolated agent setup, both on runtime-plugins phase.
• 12:21:40 — supervisor handoff file written: reason: "cron.isolated_agent_setup_timeout".
• 12:21:50 — gateway receives SIGUSR1; restarting.
• 12:22:14 — forced restart requested; skipping active work drain. 3 active embedded runs nuked.
• 12:22:16 — shutdown timed out; exiting without full cleanup → exit 1. Stability bundle written with reason: gateway.restart_shutdown_timeout. systemd auto-restart counter increments to 29.
• 12:22:18 — main-session-restart-recovery emits failed=2 recovered=0; the failure cause is gateway timeout after 10000ms on the embedded runs that had been nuked 4 seconds earlier.

Recovery: manual operator intervention to reset systemd state.

Code site

<workspace>/dist/server-cron-DfOwaY9F.js:3770 — onIsolatedAgentSetupTimeout → requestSafeGatewayRestart (with delayMs: 0). The decision is unconditional on setup timeout — no signal disambiguation between "gateway state is corrupt" and "this tick saturated the loop".

Plugin attach cost (why setup blows past the timeout under load)

Per isolated cron agent, runtime-plugin attach runs ~14 sequential register* calls (tools, channel handlers, command handlers, RPC, CLI, HTTP routes) synchronously on the gateway event loop. Under nominal load, this completes in ~3.3 s (journal line: agent runtime plugins pre-warmed in 3300ms). The cron setup timeout is 60 s default (configurable per job). Under sustained event-loop saturation (utilization near 1.0, p99Delay 8–12 s), the same 3.3 s of attached work elongates well past 60 s — not because anything is broken, just because the loop is starved.

This is the amplification path: any process-wide event-loop pressure converts cron setup into a restart trigger, and the restart re-enters the same pressure.

Expected behavior

onIsolatedAgentSetupTimeout should treat a single setup timeout as a tick-level failure, not a process-level one. Concretely:

• Default response: cancel the tick, back off this cron job's next fire (exponential), preserve in-flight non-cron work.
• Adaptive timeout: raise the cron setup timeout proportional to current event-loop p99Delay so a saturated loop doesn't get punished by a fixed-budget watchdog.
• Escalation criteria for a real restart: only restart the gateway when a stronger signal indicates actual plugin-runtime corruption — e.g. N consecutive setup timeouts across distinct cron jobs with the event loop idle between them. A saturated loop is evidence of workload pressure, not corruption.
• Never delayMs: 0 + drain-skip on a watchdog-triggered restart: if a restart is unavoidable, the drain budget should at minimum allow currently-streaming embedded runs to settle, or be marked for recovery with a non-trivial deadline (see [Bug]: Main-session restart-recovery has no cross-boot retry budget — wedged sessions death-loop the gateway across reboots (2026.6.9) #95750 for the recovery-side complement).

Actual behavior

Single isolated-cron setup timeout → unconditional requestSafeGatewayRestart(delayMs: 0) → forced restart with skipped work drain → in-flight embedded runs killed → recovery-side timeout failure (#95750) → systemd auto-restart → same loop saturation → next cron tick re-fires the same watchdog.

Reproducer

1. Run the gateway under systemd --user with Restart=always.
2. Register an isolated cron whose runtime-plugin set includes a moderately sized workspace plugin (the family-week workspace plugin in the observed incident registers ~14 routes synchronously; any plugin with comparable attach surface works).
3. Schedule it to fire every ~5 minutes.
4. Independently, start ≥3 concurrent main-session embedded turns with backends that hold the loop busy (large model deliberation, streaming-heavy SSE consumers, etc.) so eventLoopUtilization climbs to ≥0.95 and eventLoopDelayP99Ms exceeds the configured cron setup timeout's safety margin.
5. Observe: the next cron tick fails setup on phase runtime-plugins; onIsolatedAgentSetupTimeout fires; requestSafeGatewayRestart is called with delayMs: 0; the forced restart skips drain; in-flight embedded runs are killed; restart-recovery fails to recover them; the loop re-enters saturation and the next cron tick re-wedges.

The reproduction is deterministic given sustained event-loop pressure during a cron tick.

Evidence

• Stability bundle: <workspace>/logs/stability/openclaw-stability-2026-06-22T12-22-16-782Z-<pid>-gateway.restart_shutdown_timeout.json (reason gateway.restart_shutdown_timeout).
• Supervisor handoff file: <workspace>/gateway-supervisor-restart-handoff.json — recorded reason: "cron.isolated_agent_setup_timeout" at 12:21:40, ~10 s before SIGUSR1.
• Code site: <workspace>/dist/server-cron-DfOwaY9F.js:3770 — onIsolatedAgentSetupTimeout → requestSafeGatewayRestart.
• Journal sequence: as enumerated in "Production incident" above.
• Pre-warm baseline: agent runtime plugins pre-warmed in 3300ms (nominal load).
• Saturation snapshot at fault: eventLoopUtilization 99–100%, eventLoopDelayP99Ms 8–12 s, 4 concurrent main-session embedded runs.

Suggested fix

A staged remediation. Items (1) and (2) are the operator-facing win; (3) and (4) round out the contract.

1. Decouple setup-timeout from gateway-restart. onIsolatedAgentSetupTimeout should default to cancel this tick + back off this job's next fire. It should NOT call requestSafeGatewayRestart directly.
2. Adaptive setup timeout. Compute the effective setup deadline as baseSetupTimeoutMs + k * eventLoopDelayP99Ms (or a saturation-aware variant). Under nominal load this is a no-op; under saturation it prevents the false trigger.
3. Restart only on a corruption-class signal. If a restart is still needed (e.g. N consecutive setup timeouts across distinct jobs with idle event loop between them), gate it behind that stronger predicate, and use a non-zero drain budget that lets in-flight embedded runs settle.
4. Honest requestSafeGatewayRestart semantics. Either (a) rename to clarify it does not preserve in-flight embedded runs, or (b) extend it to actually preserve them via a longer drain plus the recovery-side fix at [Bug]: Main-session restart-recovery has no cross-boot retry budget — wedged sessions death-loop the gateway across reboots (2026.6.9) #95750. Today the name implies a guarantee the implementation doesn't provide.

Happy to scope a PR if maintainers prefer concrete code. The narrow change for (1) is a behavior flip in onIsolatedAgentSetupTimeout; the broader changes are larger.

Related upstream issues

• cron.sessionRetention does not prune isolated-target cron sessions — reaper targets :run: key shape which isolated jobs never generate #89666 — isolated-cron session reaper family. This issue is a sharper sibling: same incident family, but focused on the restart-as-response anti-pattern rather than the broader reaper concept.
• fix: restart gateway after isolated cron setup timeout #89055 — open PR; restart gateway after isolated cron setup timeout. This issue argues the inverse direction — that the restart-on-setup-timeout decision is itself wrong under the dominant failure mode. Worth coordinating before fix: restart gateway after isolated cron setup timeout #89055 merges.
• [Bug]: Main-session restart-recovery has no cross-boot retry budget — wedged sessions death-loop the gateway across reboots (2026.6.9) #95750 — main-session restart-recovery has no cross-boot retry budget. Complementary recovery-side fix: even when a restart is unavoidable, the recovery path needs to bound retries.
• [Feature]: Wall-clock deadline on channel-lane agent turns (no-settlement bound, dispatcher-level) #95630 — wall-clock deadline on channel-lane agent turns (trigger-layer fix for the saturation-driver workload).
• Long streaming model responses cause event loop starvation #89392, [Bug]: event-loop starvation delays fetch timeouts during gateway load #84835, [Bug]: 2026.5.22 Docker/WSL2 gateway event-loop starvation, 284s provider-auth prewarm, slow Telegram turn, and local RPC timeouts #86752, Gateway event loop starvation and HTTP/WS outage during sessions usage/cost under memory pressure #86718 — event-loop starvation family (the triggering condition for this issue).
• [Bug]: Isolated agent runs stall in runtime-plugins phase before execution start — recurring across hourly crons on 2026.5.22, no named-plugin diagnostic #87327 — isolated agent runs stall in runtime-plugins phase (the symptom this issue's restart trigger fires on).

Sanity questions

• Is there an existing config knob that biases onIsolatedAgentSetupTimeout toward "cancel tick" instead of "restart gateway"? I didn't find one; checked cron.* and recovery.*. If one exists, this issue collapses to a documentation fix.
• Is the requestSafeGatewayRestart name reserved for a future implementation that does preserve in-flight embedded runs? If so, the gap is the implementation, not the naming.