-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
pendingFinalDelivery heartbeat replay loops forever — no attempt cap or TTL on orphan sessions #85743
Copy link
Copy link
Open
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
pendingFinalDeliveryheartbeat replay loops forever — no attempt cap or TTL on orphan sessionsSummary
A session entry whose
pendingFinalDelivery: trueflag is set but never cleared (orphan / dead session, channel-state mismatch, etc.) causes the heartbeat replay branch insrc/auto-reply/reply/get-reply.tsto re-send the cachedpendingFinalDeliveryTextindefinitely. The same alert text is delivered to the user once per heartbeat tick, day after day. Observed in a 2026.5.18 install withpendingFinalDeliveryAttemptCount: 189andpendingFinalDeliveryCreatedAtfour days old.This is complementary to (not duplicated by) #83184 / #83187 (clear-after-heartbeat-success) and #84679 (
target === "none"clearing). Those fix the most common cause; this issue is about the fail-safe for everything they don't catch.Reproduction
pendingFinalDelivery: trueandpendingFinalDeliveryTextare set (e.g. process killed, channel disabled, telegram bot token rotated mid-delivery).sessionKey.~/.openclaw/agents/main/sessions/sessions.json→ the entry'spendingFinalDeliveryAttemptCountclimbs monotonically,pendingFinalDeliveryLastAttemptAtadvances,pendingFinalDeliveryTextis re-emitted verbatim each tick.In production we saw 189 attempts over 4 days, surfacing the same stale "CRITICAL ALERT" message to the user repeatedly.
Root cause
src/auto-reply/reply/get-reply.ts(lines ~432–458, heartbeat replay branch, insideif (sessionEntry?.pendingFinalDelivery && sessionEntry.pendingFinalDeliveryText) { if (opts?.isHeartbeat) { ... else { ... } } }):No bound check against
attemptCountorpendingFinalDeliveryCreatedAt. Clear only fires via the siblingheartbeatPending.shouldClearbranch (text stripped toHEARTBEAT_OK) or successful dispatch indispatch-from-config.ts:clearPendingFinalDeliveryAfterSuccess— neither triggers for stale orphan buffers.Proposed fix
Mirror the existing
suspendPendingFinalDelivery({ reason: "retry-limit" | "expiry" })vocabulary fromsrc/agents/subagent-registry-lifecycle.ts:500-548. Before incrementing at line ~434, check:attemptCount > PENDING_FINAL_DELIVERY_MAX_HEARTBEAT_ATTEMPTS(suggest10), orDate.now() - pendingFinalDeliveryCreatedAt > PENDING_FINAL_DELIVERY_TTL_MS(suggest24h)…and if either holds, route through the existing clear block (lines 406–431) with a warn log carrying the attempt count and age, then continue as if no pending delivery existed. This preserves the legitimate replay-after-restart behavior the buffer is designed for, while bounding the failure mode.
PR ready to follow — keeping the constants local so they can graduate to config if needed.
Workaround for users hit by this today
Until the upstream fix lands, a small janitor script that scans
sessions.jsonand clearspendingFinalDeliveryentries exceeding the same thresholds works. (Happy to share if useful.)