You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Bug]: Stuck-session recovery never releases: status=aborted with released=0 on every event — wedges survive five correct-target aborts (~14h) and four ghost-target aborts (~16h) until an external restart or user /reset [2026.5.28] #92270
#71127 ("Stuck processing sessions are detected but never aborted — gateway requires external restart to recover") is closed with no linked fix version. 2026.5.28 does ship an auto-recovery path — and we have 11 logged recovery events from 36 hours on one host proving it engages on schedule, targets a session, logs action=abort_embedded_run aborted=true drained=true, outcome: status=aborted … and has never verifiably freed anything. Every single event reports released=0 forceCleared=false, and every wedge survived its "successful" aborts until an external gateway restart or a user /reset.
Two distinct defects compound:
Abort-without-release (primary). One session (8cfd5317-…) took five correct-target recoveries — 22:14 and 22:50 on June 10, then 10:46:54, 11:00:24, 11:58:25 the next morning, each status=aborted — while its wedge persisted ~14 hours, until the user's /reset at 11:58:46 cleared it (correct delivery resumed at 11:58:47, one second later). Correct target, claimed success, nothing released, five times.
Stale-target binding (secondary). In an earlier ~16-hour wedge on the same lane, every stalled-session WARN logged sessionId=unknown, and the four recoveries that fired (08:44–10:41) all aborted a different session id — one whose file had been retired by a /reset (renamed ….jsonl.reset.…) five minutes before the wedge began. The actually-wedged run kept executing tools (transcript-verified exec calls minutes after two of the "successful" aborts). A gateway restart was required.
In both cases the operator-visible result is #71127's original complaint: a session lane silently dead for many hours, recovery logs full of status=aborted, recovery achieved by nothing except restart/reset.
The recovery-event ledger (every event logged on this host, June 9–11)
#
time (PT)
target sessionId
status
released
forceCleared
outcome for the wedge
1–4
Jun 10 08:44, 08:50, 10:15, 10:41
14b2eb6b… (retired pre-reset session — not the wedged 0305949f…, whose WARNs said sessionId=unknown)
aborted √ó4
0
false
wedge survived; cleared only by gateway restart ~11:00
5–6
Jun 10 15:11, 15:22
70843510…
aborted √ó2
0
false
effectiveness unverified (session was /reset at 15:44)
7–11
Jun 10 22:14, 22:50; Jun 11 10:46:54, 11:00:24, 11:58:25
8cfd5317… (correct target)
aborted √ó5
0
false
wedge survived ~14h; cleared only by user /reset at 11:58:46
released=0 and forceCleared=false on all 11. No recovery event anywhere is followed by the stalled run resuming progress or the lane releasing.
Proof the wedged runs were alive (not stale bookkeeping)
The 16-hour wedge's transcript (agents/<agent>/sessions/0305949f-….jsonl) records exec tool calls at 06:20 UTC and 15:51–15:53 UTC — through the silent window and minutes after "successful" aborts #1–2. terminalProgressStale=true / classification=stalled_agent_run were accurate both times: detection is not the problem.
Two adjacent gaps that made this worse
Stall WARNs are queue-gated. A progress-frozen run emits no stalled-session WARNs while queueDepth=0. The 14-hour wedge was invisible in every log (and to our external watchdog) from ~23:00 until a user message queued at ~11:54 the next morning — WARNs began within seconds, 54 minutes of accumulated lastProgressAge already on the first line. A wedged-but-unqueued lane is currently undetectable by anyone.
/reset and the lane registry. The ghost-target case shows recovery resolving its target from somewhere that still held the pre-reset session (its file already renamed .reset.), while the stall detector simultaneously reported sessionId=unknown for the live run. Whatever registry recovery consults was not updated by the (queued, late-executing) /reset that created the wedged session.
Likely reproduction
On an agent DM lane, /reset while a turn is busy (so the reset queues and executes late).
Drive the successor session's turn into a no-progress stall, with at least one message queued behind it.
Observe recoveries every ≈380s of queued-work wait: status=aborted, released=0, and the stall surviving. Depending on registry state, the abort target is the live session (defect 1) or the retired pre-reset session with WARNs saying sessionId=unknown (defect 2). Only launchctl kickstart of the gateway, or a user /reset, clears the lane.
Make abort_embedded_run actually release the run/lane — or define and document what released=0 means, because today status=aborted is logged while the wedge demonstrably persists.
Verify after aborting (progress resumes / lane releases) before logging success; emit an explicit recovery=ineffective line when it didn't. Five silent false successes hid a 14-hour outage; four hid a 16-hour one.
Bind the abort target to the run that triggered the stall classification. If the owner can't be resolved (sessionId=unknown), escalate (lane-level force-clear) or log recovery=skipped reason=unresolvable_session — don't abort an independently-resolved (possibly retired) id.
Make /reset — including queued resets — atomically update the lane's session registry.
Emit stale-progress WARNs even when the queue is empty (low frequency is fine); queue-gated visibility hides wedges from operators and external watchdogs alike.
An out-of-process cron watchdog parses the diagnostics lines, treats platform recovery events as authoritative only until stall evidence postdates them, and pages a human with the launchctl kickstart command — i.e., it exists precisely because status=aborted cannot be trusted. Band-aid, not a fix.
Steps to reproduce
On an agent DM lane, /reset while a turn is busy so the reset queues and executes late.
Drive the successor session into a no-progress stall (classification=stalled_agent_run) with a message queued behind it.
Every ~380s, a "stuck session recovery" fires: action=abort_embedded_run, status=aborted, released=0, forceCleared=false — yet the stall persists until launchctl kickstart of the gateway or a user /reset.
Expected behavior
abort_embedded_run cancels the stalled run and frees the lane so the session resumes — or, if it can't, logs the failure (e.g. recovery=ineffective) instead of reporting status=aborted.
Actual behavior
Recovery reports status=aborted with released=0/forceCleared=false on every event while the run keeps executing (transcript shows tool calls minutes after "successful" aborts). Observed 5 correct-target aborts over ~14h, plus 4 ghost-target aborts against a /reset-retired session id (WARNs logged sessionId=unknown) over ~16h. The silent false successes masked multi-hour outages.
OpenClaw version
2026.5.28
Operating system
macOS (Apple Silicon), launchd-managed gateway
Install method
No response
Model
anthropic/claude-opus-4-8
Provider / routing chain
Anthropic primary; openai/gpt-5.5 fallback configured but not implicated (incidents began on interactive turns).
Bug type
Crash (process/app exits or hangs)
Beta release blocker
No
Summary
Summary
#71127 ("Stuck processing sessions are detected but never aborted — gateway requires external restart to recover") is closed with no linked fix version. 2026.5.28 does ship an auto-recovery path — and we have 11 logged recovery events from 36 hours on one host proving it engages on schedule, targets a session, logs
action=abort_embedded_run aborted=true drained=true,outcome: status=aborted… and has never verifiably freed anything. Every single event reportsreleased=0 forceCleared=false, and every wedge survived its "successful" aborts until an external gateway restart or a user/reset.Two distinct defects compound:
8cfd5317-…) took five correct-target recoveries — 22:14 and 22:50 on June 10, then 10:46:54, 11:00:24, 11:58:25 the next morning, eachstatus=aborted— while its wedge persisted ~14 hours, until the user's/resetat 11:58:46 cleared it (correct delivery resumed at 11:58:47, one second later). Correct target, claimed success, nothing released, five times.sessionId=unknown, and the four recoveries that fired (08:44–10:41) all aborted a different session id — one whose file had been retired by a/reset(renamed….jsonl.reset.…) five minutes before the wedge began. The actually-wedged run kept executing tools (transcript-verified exec calls minutes after two of the "successful" aborts). A gateway restart was required.In both cases the operator-visible result is #71127's original complaint: a session lane silently dead for many hours, recovery logs full of
status=aborted, recovery achieved by nothing except restart/reset.Environment
ai.openclaw.gateway)The recovery-event ledger (every event logged on this host, June 9–11)
14b2eb6b…(retired pre-reset session — not the wedged0305949f…, whose WARNs saidsessionId=unknown)70843510…/resetat 15:44)8cfd5317…(correct target)/resetat 11:58:46released=0andforceCleared=falseon all 11. No recovery event anywhere is followed by the stalled run resuming progress or the lane releasing.Proof the wedged runs were alive (not stale bookkeeping)
The 16-hour wedge's transcript (
agents/<agent>/sessions/0305949f-….jsonl) records exec tool calls at 06:20 UTC and 15:51–15:53 UTC — through the silent window and minutes after "successful" aborts #1–2.terminalProgressStale=true/classification=stalled_agent_runwere accurate both times: detection is not the problem.Two adjacent gaps that made this worse
queueDepth=0. The 14-hour wedge was invisible in every log (and to our external watchdog) from ~23:00 until a user message queued at ~11:54 the next morning — WARNs began within seconds, 54 minutes of accumulatedlastProgressAgealready on the first line. A wedged-but-unqueued lane is currently undetectable by anyone./resetand the lane registry. The ghost-target case shows recovery resolving its target from somewhere that still held the pre-reset session (its file already renamed.reset.), while the stall detector simultaneously reportedsessionId=unknownfor the live run. Whatever registry recovery consults was not updated by the (queued, late-executing)/resetthat created the wedged session.Likely reproduction
/resetwhile a turn is busy (so the reset queues and executes late).status=aborted,released=0, and the stall surviving. Depending on registry state, the abort target is the live session (defect 1) or the retired pre-reset session with WARNs sayingsessionId=unknown(defect 2). Onlylaunchctl kickstartof the gateway, or a user/reset, clears the lane.Asks
abort_embedded_runactually release the run/lane ‚Äî or define and document whatreleased=0means, because todaystatus=abortedis logged while the wedge demonstrably persists.recovery=ineffectiveline when it didn't. Five silent false successes hid a 14-hour outage; four hid a 16-hour one.sessionId=unknown), escalate (lane-level force-clear) or logrecovery=skipped reason=unresolvable_session‚Äî don't abort an independently-resolved (possibly retired) id./reset‚Äî including queued resets ‚Äî atomically update the lane's session registry.stuckSessionAbortMsproposed in [Bug]: Stuck processing sessions are detected but never aborted — gateway requires external restart to recover #71127; the current ‚âà380s queued-work gate is hardcoded and undocumented), and coordinate the release fix with the abort-cleanup family (Embedded run never exits after abort: waitForCompactionRetry() not tied to abort signal, session stays active and locked #12085, [Bug]: Embedded run timeout fails to clean up session/lane state when compaction is in-flight (waitForCompactionRetry not abort-aware) #13341, [Bug]: Embedded run timeout fails to clean up session/lane state when tool execution is in-flight #14228, Session lane stays stuck after embedded compaction run timeout #16331) ‚ÄîwaitForCompactionRetry()lane-locks suggest the same released-nothing class.Operator workaround (deployed)
An out-of-process cron watchdog parses the diagnostics lines, treats platform recovery events as authoritative only until stall evidence postdates them, and pages a human with the
launchctl kickstartcommand — i.e., it exists precisely becausestatus=abortedcannot be trusted. Band-aid, not a fix.Steps to reproduce
Expected behavior
abort_embedded_run cancels the stalled run and frees the lane so the session resumes — or, if it can't, logs the failure (e.g. recovery=ineffective) instead of reporting status=aborted.
Actual behavior
Recovery reports status=aborted with released=0/forceCleared=false on every event while the run keeps executing (transcript shows tool calls minutes after "successful" aborts). Observed 5 correct-target aborts over ~14h, plus 4 ghost-target aborts against a /reset-retired session id (WARNs logged sessionId=unknown) over ~16h. The silent false successes masked multi-hour outages.
OpenClaw version
2026.5.28
Operating system
macOS (Apple Silicon), launchd-managed gateway
Install method
No response
Model
anthropic/claude-opus-4-8
Provider / routing chain
Anthropic primary; openai/gpt-5.5 fallback configured but not implicated (incidents began on interactive turns).
Additional provider/model setup details
No response
Logs, screenshots, and evidence
Impact and severity
No response
Additional information
No response