-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
Conversational agent lane wedges indefinitely: ingress events never reconcile (re-delivery storm on reconnect) + abort_embedded_run leaves owner lock unreleased (released=0) #97538
Copy link
Copy link
Open
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.ClawSweeper needs live local, crabbox, or manual validation to confirm this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.Good issue quality with a plausible reproduction path needing some confirmation.maturity:stableIssue affects a taxonomy feature currently scored M4/M5.Issue affects a taxonomy feature currently scored M4/M5.staleMarked as stale due to inactivityMarked as stale due to inactivity
Description
Metadata
Metadata
Assignees
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.ClawSweeper needs live local, crabbox, or manual validation to confirm this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.Channel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.Good issue quality with a plausible reproduction path needing some confirmation.maturity:stableIssue affects a taxonomy feature currently scored M4/M5.Issue affects a taxonomy feature currently scored M4/M5.staleMarked as stale due to inactivityMarked as stale due to inactivity
Type
Fields
Priority
None yet
Summary
Two related defects in the gateway's inbound-processing and stalled-run recovery paths combine to wedge a conversational agent's reply lane indefinitely, with no organic self-heal. Observed on a live WhatsApp deployment. The stall/abort machinery predates v6.5 (it was active on v5.20), so this does not appear to be a 6.5 regression.
Environment: OpenClaw gateway v2026.6.5, macOS (launchd-managed), WhatsApp + Slack channels.
Bug 1 —
abort_embedded_rundoes not release the session owner lock (released=0)When a conversational
embedded_runstalls (activeWorkKind=embedded_run,lastProgress=embedded_run:started, never advancing), the stuck-session recovery sweep fires:The abort reports
aborted=true drained=truebutreleased=0— the in-memory session active-owner lock is not freed. As a result:recovery_skipped_active_ownerfires every ~30s (the sweep sees a still-"active" owner and skips recovery);abort_embedded_runfires ~6 min later, againreleased=0;Only a gateway restart clears the held lock.
Expected: aborting a stalled embedded_run should release its owner lock as part of the abort, so the lane recovers without a restart.
Bug 2 — Ingress events never reconcile to
completed; re-deliver on every reconnectInbound events in
channel_ingress_eventsare processed but never markedcompleted— they remainpendingindefinitely. On every gateway reconnect/relink, the pending backlog re-delivers.Observed: ~25 pending events accumulated over 7 days, the oldest already processed a week earlier (e.g. media attachments whose derived artifacts were built at the time). Only a handful of events ever reached
completed. On each relink the gateway re-pushed the pending batch — confirmed via dedup (no new ingress rows created; the same events re-delivered).When the re-delivered batch includes heavy multimodal payloads (PDFs/images), the resulting
embedded_runstalls, triggering Bug 1.Critical operational interaction — restart makes it worse: a restart forces a reconnect, so restarting the gateway re-triggers a full re-delivery storm of the pending backlog, which immediately re-wedges the lane via Bug 1. The intuitive remediation (restart) therefore perpetuates the failure unless the pending queue is cleared first.
Expected: ingress events should reconcile to
completedafter successful processing (ack), so they are not re-delivered on reconnect.Combined user-visible impact
A conversational agent stops replying entirely on the affected channel. Inbound messages are received and broadcast but no reply is sent (run wedged at
:started); subsequent legitimate messages queue behind the wedge and are starved. There is no supported recovery path:channelssupports only add/remove/login/inspect).channel_ingress_eventsrowscompletedvia raw SQLite, then (2) restart — in that order, because restarting with a full queue re-triggers the storm.Repro (observed)
pending(never reconciled).embedded_runstalls at:started.released=0; the lock never frees; the loop persists until the queue is cleared and the gateway restarted.Suggested fixes / asks
abort_embedded_run(Bug 1).completedon successful processing so they do not re-deliver on reconnect (Bug 2).The two bugs are separable but interact: Bug 2 supplies the trigger (re-delivery storm), Bug 1 makes the resulting stall permanent. Happy to split into two issues if preferred.