fix(cron): clear pre-execution watchdog on any phase notification (#93530) by fsdwen · Pull Request #93591 · openclaw/openclaw

fsdwen · 2026-06-16T10:42:27Z

Summary

Cron isolated-agent jobs abort prematurely (~6 min) even with a 10 min
timeout. The 60s pre-execution watchdog fires because the
before_agent_reply phase notification is gated behind
hookRunner.hasHooks("before_agent_reply") — when no hook is registered,
the phase is never emitted to the watchdog, which never transitions to
the "executing" state.

Fix

Move notifyExecutionPhase("before_agent_reply", ...) outside the
hasHooks guard so the phase is always emitted for cron triggers.
Hook execution (runBeforeAgentReply) remains guarded by hasHooks.

Changes: 3 files (run.ts, cli-runner.ts — both runner-side guards)

Verification

All watchdog-related tests pass

✓ clears the pre-execution watchdog on explicit execution milestones (#80283)
✓ clears the pre-execution watchdog when isolated cron reaches attempt-dispatch (#81368)
✓ clears the pre-execution watchdog when isolated cron reaches context-assembled (#81368)
✓ clears the pre-execution watchdog when isolated cron reaches before-agent-reply (#81368)
✓ re-arms the pre-execution watchdog when before_agent_reply does not claim (#82811)
✓ times out isolated agent runs that stall before execution starts (#74803)

Test Files  2 passed (2)
     Tests  65 passed (65)

Fixes #93530

🤖 Generated with Claude Code

…enclaw#93530) When no before_agent_reply hook is registered, the phase notification is never emitted, causing the 60s pre-execution timeout to fire even though the runner is alive with other phases arriving. Change the pre-execution clear condition from 'stage === execution' to 'stage !== undefined' so any phase notification proves the runner is making progress and clears the pre-execution timeout. Co-Authored-By: Claude <[email protected]>

…enclaw#93530) When no before_agent_reply hook is registered, the phase notification is gated behind hookRunner.hasHooks(), so the cron pre-execution watchdog never receives the 'execution' stage signal and fires prematurely. Move notifyExecutionPhase('before_agent_reply') outside the hasHooks guard so the watchdog always learns the agent entered execution for cron triggers. Hook execution remains guarded. Also fix the same pattern in the CLI runner. Co-Authored-By: Claude <[email protected]>

clawsweeper · 2026-06-19T20:32:52Z

Codex review: needs real behavior proof before merge. Reviewed July 1, 2026, 2:01 AM ET / 06:01 UTC.

Summary
The PR moves cron before_agent_reply phase notifications outside hook-presence guards in the embedded and CLI runners, while hook execution stays guarded.

PR surface: Source +8. Total +8 across 2 files.

Reproducibility: yes. Source inspection shows the PR head emits before_agent_reply followed by runtime_plugins, and current watchdog/test code re-arms the pre-execution timeout on that sequence.

Review metrics: 1 noteworthy metric.

Cron phase emitters changed: 2 runner paths changed. Both embedded and CLI cron paths now synthesize watchdog phase notifications, so the shared timeout contract needs maintainer review before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #93530
Summary: This PR is an open candidate fix for the isolated cron no-hook watchdog issue; sibling PRs overlap but none is a merged safe superseder.

Members:

canonical: cron isolated-agent turn: pre-execution watchdog race with runEmbeddedAgent phase reporting #93530 - The PR body uses closing syntax for this open issue, and the issue describes the no-hook before_agent_reply phase gap this branch targets.
same_root_cause: fix(cron): emit before_agent_reply phase unconditionally for cron triggers #93535 - This open sibling implements the same targeted embedded-runner before_agent_reply phase-emission approach for the same issue.
same_root_cause: fix(cron): always emit before_agent_reply phase for cron triggers to prevent watchdog false-positive #94212 - This open sibling also implements unconditional embedded before_agent_reply emission for the same no-hook watchdog symptom.
partial_overlap: cron isolated agent pre-execution watchdog aborts runs despite setup progress #93912 - This open issue covers the broader setup-like phase progress behavior in the same pre-execution watchdog family.
partial_overlap: fix(cron): treat isolated setup phases as progress #93914 - This open PR targets the broader watchdog-layer setup-progress variant rather than only the runner hook gate.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Repair the no-hook phase sequence so it cannot re-enter waiting_for_execution before real execution starts.
[P1] Add redacted real cron or gateway proof showing the affected job shape running past the old 60-second pre-execution window.
Have maintainers choose one canonical behavior among the overlapping watchdog candidate PRs.

Proof guidance:

[P1] Needs real behavior proof before merge: The PR body supplies unit/watchdog test output only; it needs redacted real cron or gateway output, logs, recording, or a linked artifact showing the fixed path crossing the old 60-second cutoff. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] Merging as-is can leave no-hook cron runs back in waiting_for_execution after the synthetic before_agent_reply to runtime_plugins transition, so the false abort can still occur.
[P1] The PR body supplies watchdog/unit test output only, not redacted real cron or gateway proof showing the affected slow no-hook path survives the old 60-second pre-execution window.
[P1] Several open sibling PRs target adjacent cron watchdog behavior with different phase contracts, so maintainers should pick one canonical behavior before landing duplicate fixes.

Maintainer options:

Repair The Phase Contract Before Merge (recommended)
Distinguish absent-hook progress from unhandled-hook fallback so no-hook cron runs do not re-enter waiting_for_execution while true fallback stalls remain guarded.
Adopt A Watchdog-Layer Fix
Maintainers can choose the broader watchdog approach from the related candidate if that better expresses the intended setup-progress contract.
Pause For One Canonical Candidate
If this branch cannot supply the phase-contract repair and real proof, keep the linked issue open and close duplicate candidates only after a canonical fix lands.

Next step before merge

[P1] Contributor real-behavior proof and a maintainer phase-contract choice are still required, so this is not a safe automated repair lane from the PR branch.

Security
Cleared: The diff only changes in-process cron runner phase notifications and does not alter secrets, dependencies, workflows, package metadata, or external code execution surfaces.

Review findings

[P1] Preserve the no-hook watchdog clear in embedded runs — src/agents/embedded-agent-runner/run.ts:863
[P1] Preserve the no-hook watchdog clear in CLI runs — src/agents/cli-runner.ts:424-428

Review details

Best possible solution:

Settle one canonical cron phase contract that clears real no-hook/setup progress without hiding true startup or fallback stalls, then land it with focused regression coverage and redacted real cron proof.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows the PR head emits before_agent_reply followed by runtime_plugins, and current watchdog/test code re-arms the pre-execution timeout on that sequence.

Is this the best way to solve the issue?

No. Moving the phase notification outside hasHooks is only a plausible mitigation; as written, the follow-up runtime_plugins phase can undo the clear, so the safer fix must preserve true-stall detection.

Full review comments:

[P1] Preserve the no-hook watchdog clear in embedded runs — src/agents/embedded-agent-runner/run.ts:863
This branch emits before_agent_reply for every cron run, but then immediately emits runtime_plugins when the hook is absent or unhandled. The watchdog re-arms when before_agent_reply is followed by a pre-execution phase, so the reported no-hook embedded path can still be killed by the 60-second watchdog before real execution.
Confidence: 0.94
[P1] Preserve the no-hook watchdog clear in CLI runs — src/agents/cli-runner.ts:424-428
The CLI runner has the same phase sequence: it now emits before_agent_reply outside the hook guard and then emits runtime_plugins outside the guard too. That puts the CLI-backed cron path back into waiting_for_execution before the subprocess starts, leaving the linked watchdog bug unresolved for that runner.
Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.92

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against fc5ba0e58bb4.

Label changes

Label justifications:

P1: The PR targets a current isolated cron agent workflow where scheduled runs can abort before their configured timeout and suppress expected output.
merge-risk: 🚨 availability: The diff changes cron watchdog phase signaling in a way that can still leave scheduled agent runs falsely aborted before real execution.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body supplies unit/watchdog test output only; it needs redacted real cron or gateway output, logs, recording, or a linked artifact showing the fixed path crossing the old 60-second cutoff. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +8. Total +8 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	36	28	+8
Tests	0	0	0	0
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	36	28	+8

What I checked:

Repository policy read: Root AGENTS.md and the scoped src/agents/AGENTS.md were read fully; their PR review depth, proof, and agent-runtime guidance apply to this review. (AGENTS.md:1, fc5ba0e58bb4)
PR head embedded phase sequence: At the PR head, embedded cron emits before_agent_reply unconditionally and then emits runtime_plugins in the same cron block when no hook handles the turn. (src/agents/embedded-agent-runner/run.ts:841, ddd03887c51f)
PR head CLI phase sequence: At the PR head, CLI cron emits before_agent_reply unconditionally and then emits runtime_plugins before the subprocess path, creating the same watchdog sequence. (src/agents/cli-runner.ts:384, ddd03887c51f)
Current watchdog re-arms on that sequence: Current main maps runtime_plugins to pre_execution and re-arms the pre-execution watchdog when a pre-execution phase follows before_agent_reply. (src/cron/service/agent-watchdog.ts:121, fc5ba0e58bb4)
Regression coverage preserves fallback re-arm: Current tests emit before_agent_reply followed by runtime_plugins and expect a stalled before execution start timeout with runtime-plugins. (src/cron/service/timer.regression.test.ts:3198, fc5ba0e58bb4)
Current main still has the reported gate: Current main still emits embedded before_agent_reply only inside hookRunner?.hasHooks("before_agent_reply"), so the core bug is not already fixed on main. (src/agents/embedded-agent-runner/run.ts:1009, fc5ba0e58bb4)

Likely related people:

ghitafilali: Authored recent isolated cron setup-timeout work touching the watchdog, timer, and regression tests near this behavior. (role: recent adjacent cron watchdog contributor; confidence: medium; commits: b2c5e790b441; files: src/cron/service/agent-watchdog.ts, src/cron/service/timer.ts, src/cron/service/timer.regression.test.ts)
steipete: Authored the cron timer helper split that created the current agent-watchdog module and recent inline documentation in the same area. (role: cron service refactor contributor; confidence: medium; commits: 9d31cbbd6ad8, 1e7510ae103e; files: src/cron/service/agent-watchdog.ts, src/cron/service/timer.ts)
849261680: Authored a recent setup-timeout cron regression change in timer.regression.test.ts, adjacent to the watchdog behavior under review. (role: recent adjacent regression contributor; confidence: low; commits: 19707cce1d73; files: src/cron/service/timer.regression.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (1 earlier review cycle)

reviewed 2026-06-23T12:20:50.436Z sha ddd0388 :: needs real behavior proof before merge. :: [P1] Preserve the no-hook watchdog clear in embedded runs | [P1] Preserve the no-hook watchdog clear in CLI runs

openclaw-barnacle Bot added size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 16, 2026

openclaw-barnacle Bot added agents Agent runtime and tooling size: S and removed size: XS labels Jun 16, 2026

openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 19, 2026

clawsweeper Bot added the merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. label Jun 19, 2026

fsdwen closed this Jul 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(cron): clear pre-execution watchdog on any phase notification (#93530)#93591

fix(cron): clear pre-execution watchdog on any phase notification (#93530)#93591
fsdwen wants to merge 2 commits into
openclaw:mainfrom
fsdwen:fix/93530-cron-watchdog-race

fsdwen commented Jun 16, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 19, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

fsdwen commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Fix

Verification

All watchdog-related tests pass

Uh oh!

clawsweeper Bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

fsdwen commented Jun 16, 2026 •

edited

Loading

clawsweeper Bot commented Jun 19, 2026 •

edited

Loading