Repository navigation
fix(cron): record designed heartbeat skips as successful no-ops - #145242
Lubaoshuai wants to merge 6 commits into
Conversation
A heartbeat that intentionally skips (empty heartbeat file, no pending event, disabled, ...) was recorded as a skipped cron run, which resolveCronCompletionStatus maps to failed: tasks list --status failed accrued a row per interval and failure alerts fired even though consecutiveErrors stayed 0 (openclaw#144959). Designed no-op skips now settle as ok runs whose summary carries the skip reason, matching the busy-deferral handoff path that already returns ok. The heartbeat runner's own state (system heartbeat last, consecutiveSkipped) keeps reporting the precise skipped status.
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
|
Codex review: needs real behavior proof before merge. What this changesRecords empty-heartbeat-file and no-pending-event skips as successful Cron runs while retaining other skipped outcomes. Example: A heartbeat monitor runs every 30 minutes with comments-only scratch.
Review scores
ProductKind: Bug fix · Worth it: Needs a maintainer decision · Fix scope: Complete Merge readiness⛔ Blocked before merge - 3 items remain Keep open: this PR contains a distinct Cron accounting change that current main does not implement, and its operator semantics need an owner decision. Priority: P2 Decision needed
Before merge
FindingsNone. Tests
Agent review detailsHow this fits togetherCron dispatches heartbeat wakes, waits for the heartbeat runner's terminal result, then persists run history, counters, completion status, and alert decisions. flowchart LR
A[Cron schedule] --> B[Heartbeat wake queue]
B --> C[Heartbeat preflight]
C --> D[Cron outcome mapping]
D --> E[Persisted run history]
D --> F[Counters and alerts]
Technical reviewBest possible solution: Resolve intentional no-op accounting in the existing Cron completion owner while preserving skip diagnostics and unsuccessful-skip visibility. Do we have a high-confidence way to reproduce the issue? Current-main source shows empty-scratch preflight returning skipped, Cron retaining that status, and completion accounting returning failed; the retired tasks list command cannot reproduce the original presentation on current main. Is this the best way to solve the issue? The patch uses the existing execution owner, but changing run status has a wider footprint than fixing completion accounting and needs an explicit Cron policy choice. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against d4babe8e8ba4. Provenance checked
TestingProof path: in-process harness. SecurityNone. EvidenceWhat I checked:
PR surfaceSource +29, Tests +158. Total +187 across 3 files. Added test files: 1. View PR surface stats
Root-cause clusterRelationship: Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. LabelsLabel changes: No label changes. Label justifications:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment HistoryReview history (8 earlier review cycles)
Reviewed October 16, 2026, 3:01 AM ET / 07:01 UTC (Revision 9). |
CI flagged that blanket-settling every heartbeat skip as ok erased the operator-visible skipped state for globally paused/disabled heartbeats. Restrict the ok settlement to the designed no-op reasons (empty-heartbeat-file, no-pending-event); every other skip reason, including disabled, keeps the previous skipped outcome.
…l task End-to-end case wiring the real heartbeat runner, a real comments-only scratch row in the sqlite state store, and the real cron service: the preflight skips the turn with empty-heartbeat-file and the run settles as ok/succeeded with no failed cron task rows in the ledger. Signed-off-by: Lubaoshuai <[email protected]>
|
Added the real-behavior proof requested by the review: |
Address the ClawSweeper review findings on the proof fixture: - seed and read the same cron partition (OPENCLAW_STATE_DIR points the default partition used by resolveHeartbeatPreflight at the sandbox) - supply the required requestHeartbeat dependency - assert the empty-heartbeat-file outcome and that the agent turn and delivery never run, so a plain successful heartbeat cannot pass Signed-off-by: Lubaoshuai <[email protected]>
…tions The previous commit accidentally removed the fixture safeguards that tie this regression test to the intended path: the OPENCLAW_STATE_DIR stub that points preflight at the sandbox partition, the required requestHeartbeat dependency from CronServiceDeps, the empty-heartbeat skip-summary assertion, and the reply/delivery spy checks that prove the agent turn never ran. Restore all of them (plus the state-db teardown and env unstubs they require), so the test cannot pass through ordinary heartbeat execution. Local vitest (cron config): 10/10 pass across service.heartbeat-payload, service.heartbeat-real-preflight and service.test.
|
6fb6ec6 restores the fixture safeguards that ffb0026 accidentally removed, addressing both P2 findings:
Local vitest run at 6fb6ec6 (cron config, redacted terminal output): Full local cron-suite run at 6fb6ec6 ( @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
Thank you for the detailed revision-7 review — both fixture findings are confirmed resolved, and the remaining items are now about product semantics rather than the patch itself. Two honest acknowledgments:
Given the reviewer recommendation and the required owner decision, I'll defer to the maintainers: if the projection approach in #145025 is preferred, I'm happy to close this PR in its favor (or rework this one onto the projection semantics if that's useful). Until then I'll leave the PR open so the decision has both options on the table. |
|
This pull request has been automatically marked as stale due to inactivity. |
|
Status update (the PR is awaiting a maintainer decision, not contributor action):
Happy to provide anything else that would help the decision. |
|
This pull request has been automatically marked as stale after 7 days without qualifying human activity. |
|
Acknowledging the stale mark — this is expected, since the PR has been sitting on the owner decision recorded in the ClawSweeper review (intentional no-op accounting here vs. the projection approach in #145025) for about a month, with findings=0 and an explicit "Keep open" from the reviewer. To be explicit about the endpoint rather than bumping indefinitely: if no owner decision lands by the time this stale window expires, I will close this PR myself — the reviewer's recommendation leans toward #145025's projection approach, and I don't want to keep an operator-semantics fork alive on life support. The issue #144959 it set out to fix remains open, and whichever direction is chosen, the failure-noise problem still deserves a fix. If any maintainer wants this kept alive past that, a single reply (or the |
|
This pull request has been automatically marked as stale after 7 days without qualifying human activity. |
|
Closing due to inactivity. |
What Problem This Solves
An agent whose heartbeat file is intentionally empty (or comments-only) skips every interval by design, but each skip was recorded as a
skippedcron run, whichresolveCronCompletionStatusmaps tofailed.openclaw tasks list --status failedaccrued one row per interval (~48/day) for the heartbeat cron and failure alerts fired, even thoughconsecutiveErrorsstayed 0 (#144959).Designed no-op skips now settle as
okruns whose summary carries the skip reason (heartbeat skipped: empty-heartbeat-file), matching the busy-deferral handoff path that already returnedok. The heartbeat runner's own state (system heartbeat last,consecutiveSkipped) keeps reporting the precise skipped status, so no signal is lost.Evidence
src/cron/service.heartbeat-real-preflight.test.tswires the real heartbeat runner (real preflight, no injected heartbeat results), a real comments-only scratch row in the sqlite state store (stock "keep this file empty" content), and the real cron service. The two review findings are addressed:OPENCLAW_STATE_DIRpoints the default partition (<state>/cron/jobs.json) at the seededcronStorePath, soresolveHeartbeatPreflightgenuinely consumes the seeded comments-only scratch instead of a different store.CronServiceDeps.requestHeartbeatmember is provided (non-waiting) alongsiderequestHeartbeatAndWait, matching the production contract.requestHeartbeatdependency, skip-summary + spy assertions), so the test again cannot pass through ordinary heartbeat execution.summary: "heartbeat skipped: empty-heartbeat-file", and the reply/telegram spies must stay unused — so an ordinary successful agent turn can no longer satisfy the test. Paused-reminder recovery stays covered: the operatordisabledskip keeps itsskipped/failedoutcome in the unit matrix.Runtime output (local vitest run at 6fb6ec6, cron config, redacted):
Full local cron-suite run at 6fb6ec6 (
service.heartbeat-payload.test.ts+service.heartbeat-real-preflight.test.ts+service.test.ts):tasks list --status failedcontains zero failed cron rows after the designed skip.service.heartbeat-payload.test.ts(10/10): designed skips settleok/succeededwithconsecutiveErrors: 0; the operatordisabledskip keeps itsskippedoutcome for visibility.resolveCronCompletionStatus, and the busy-deferral handoff path intimer-execution.tsthat already returnedok. The runner's own state (system heartbeat last,consecutiveSkipped) continues to report the precise skipped status.