-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
Cron consecutiveErrors increments on gateway-restart interruptions, masking real agent-failure rate (post-#50414) #77298
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper-recovery-stuckclawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.staleMarked as stale due to inactivityMarked as stale due to inactivity
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper-recovery-stuckclawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.staleMarked as stale due to inactivityMarked as stale due to inactivity
Type
Fields
Priority
None yet
Summary
#50414 (closed completed 2026-04-25) fixed the original bug where cron jobs interrupted by gateway restart were silently marked completed. The current behavior in 2026.5.2 (8b2a6e5) flipped to the opposite problem: interrupted jobs are correctly marked failed, but
state.consecutiveErrorsincrements on every restart-induced interruption — making the metric useless for monitoring real agent health when the gateway gets restarted normally.Evidence
Job state observed today after a sequence of routine gateway restarts:
All four "errors" came from
openclaw config settriggering hot-reloads during config tuning, not from agent failures. The job ran successfully in every case where the gateway didn't restart mid-run.Expected behavior
gateway-restartinterruptions should:lastRunStatus: "interrupted", separatelastInterruptionReason: "gateway-restart").consecutiveErrors, or increment a separateconsecutiveInterruptionscounter.Suggested fix
In the cron runner, detect SIGUSR1/SIGTERM-driven shutdown and emit a distinct interruption status. Or expose a
--reset-errorsflag onopenclaw cron editfor operators to clear the counter manually.Environment
kind: every,everyMs: 600000,sessionTarget: isolated