Observed
netclaw daemon stop waits 200 seconds for graceful exit and then force-kills the main PID. The daemon's own CoordinatedShutdown phase timeout for before-service-unbind (session drain) is also 200 seconds. The two budgets being equal means the daemon can never win the race when a drain times out: it needs 200s plus its remaining teardown time (actor system termination, ~100ms+), and the CLI kills it at exactly 200s.
Evidence from a production stop on 0.25.0-alpha.onnx.6 (timestamps UTC):
01:44:59.569 WARNING Coordinated shutdown phase [before-service-unbind] timed out after 00:03:20
01:44:59.574 remaining shutdown phases complete ([0] tasks each)
01:44:59.591 last daemon log line (EventStream shutting down)
01:44:59.694 Main process exited, code=killed, status=9/KILL ← CLI force-kill, ~103ms after the daemon's last log line
01:44:59.722 "Daemon stopped (was PID ...), but did not exit gracefully within 200s and had to be force-killed."
The daemon had already timed out its stuck drain task on its own and was ~100ms from a clean exit when the CLI SIGKILLed it. systemd's escalation never fired (TimeoutStopSec=230 — the unit already carries headroom; the CLI does not).
Expected
The CLI's force-kill budget must exceed the daemon's longest internal shutdown phase timeout plus a teardown margin (e.g., 220s against a 200s phase timeout, staying under the unit's TimeoutStopSec=230). Alternatively (better), after the graceful-wait budget elapses the CLI could poll for process exit for a short additional grace window before escalating to SIGKILL — that would have let this stop complete cleanly.
Impact
Every drain-phase timeout currently ends in SIGKILL rather than the daemon's own orderly late exit — turning a slow-but-recovering shutdown into an unclean kill (un-checkpointed WAL, Failed with result 'signal' unit state). Compounds with the companion issue where a pending interactive tool approval reliably causes the drain-phase timeout in the first place.
Observed on 0.25.0-alpha.onnx.6; both timeout values appear to be shared with dev.
Observed
netclaw daemon stopwaits 200 seconds for graceful exit and then force-kills the main PID. The daemon's ownCoordinatedShutdownphase timeout forbefore-service-unbind(session drain) is also 200 seconds. The two budgets being equal means the daemon can never win the race when a drain times out: it needs 200s plus its remaining teardown time (actor system termination, ~100ms+), and the CLI kills it at exactly 200s.Evidence from a production stop on
0.25.0-alpha.onnx.6(timestamps UTC):The daemon had already timed out its stuck drain task on its own and was ~100ms from a clean exit when the CLI SIGKILLed it. systemd's escalation never fired (
TimeoutStopSec=230— the unit already carries headroom; the CLI does not).Expected
The CLI's force-kill budget must exceed the daemon's longest internal shutdown phase timeout plus a teardown margin (e.g., 220s against a 200s phase timeout, staying under the unit's
TimeoutStopSec=230). Alternatively (better), after the graceful-wait budget elapses the CLI could poll for process exit for a short additional grace window before escalating to SIGKILL — that would have let this stop complete cleanly.Impact
Every drain-phase timeout currently ends in SIGKILL rather than the daemon's own orderly late exit — turning a slow-but-recovering shutdown into an unclean kill (un-checkpointed WAL,
Failed with result 'signal'unit state). Compounds with the companion issue where a pending interactive tool approval reliably causes the drain-phase timeout in the first place.Observed on
0.25.0-alpha.onnx.6; both timeout values appear to be shared withdev.