fix: surface stalled-session interruptions by ZengWen-DT · Pull Request #103990 · openclaw/openclaw

ZengWen-DT · 2026-07-10T23:32:54Z

What Problem This Solves

Fixes an issue where a stuck-session recovery could remove a channel's streamed draft without sending a terminal notice. A non-cooperative tool could also keep running after the owning turn was aborted, leaving the caller blocked until it eventually settled.

Why This Change Was Made

The reply-operation terminal state now preserves run_stalled as a user-visible interruption while explicit user and restart cancellations keep their existing behavior. Tool execution races the existing cooperative AbortSignal, so an aborted run stops waiting while late tool settlement remains observed. Risk note: external side effects cannot be undone; the tool still receives the abort signal and its late promise is deliberately drained rather than delivered into the terminated run.

User Impact

Users whose stalled turn is recovered now receive a clear interruption message instead of losing all visible trace of the turn. Plugin tools that honor cancellation can stop promptly, and tools that do not no longer hold the aborted turn open.

Evidence

Focused regression proof drives the real reply-run registry expiry path into the production runner: expireStaleReplyOperation() records run_stalled and aborts the same operation before both terminal projections run. The tool abort proof exercises wrapToolWithAbortSignal with a non-cooperative handler. Negative control used the real pre-fix sources via git show HEAD^ -- <files>.

Before (pre-fix abort wrapper): caller stays blocked after run abort while the non-cooperative tool keeps running.

=== abort promptness ===
{
  "result": { "kind": "timeout", "ms": 523.063467 },
  "toolSettled": true,
  "signalAborted": true
}
Error: abort promptness FAIL: {"kind":"timeout","ms":523.063467}

Before (pre-fix reply runner): stalled recovery leaves an empty success outcome, so the channel-facing draft can disappear with no interruption notice.

{
  "result": { "kind": "failed", "code": "run_stalled" },
  "outcome": {
    "kind": "success",
    "runResult": { "payloads": [], "meta": {} }
  }
}
pre-fix stalled path: no interruption notice (bug reproduced)

After (fixed branch): abort returns in ~21ms, and stalled recovery surfaces the interruption payload.

=== abort promptness ===
{
  "result": {
    "kind": "rejected",
    "err": { "name": "AbortError" },
    "ms": 21.162068000000545
  },
  "toolSettled": true,
  "signalAborted": true
}
abort_promptness: PASS

=== stalled interruption ===
{
  "result": { "kind": "failed", "code": "run_stalled" },
  "outcome": {
    "kind": "final",
    "payload": {
      "text": "⚠️ This turn was interrupted because it stopped making progress. Please try again.",
      "isError": true
    }
  }
}
stalled_interruption: PASS
ALL PASS

Focused local verification:

node scripts/run-vitest.mjs src/auto-reply/reply/agent-runner-execution.test.ts -t "stalled reply operation|stalled heartbeat operation"
3 passed | 254 skipped

node scripts/run-vitest.mjs src/agents/agent-tools.abort.test.ts
1 passed

pnpm tsgo:core:test
passed

What was not tested: no live Slack Socket Mode round-trip of a 15-minute blocked tool. The proven boundary is the channel-facing reply projection and tool-wrapper abort race before outbound send, which is where the silent draft deletion and blocked caller originate.

AI-assisted: Codex/Cursor; focused proof run manually.

clawsweeper · 2026-07-10T23:48:28Z

Codex review: needs changes before merge. Reviewed July 18, 2026, 5:46 AM ET / 09:46 UTC.

Summary
The branch preserves run_stalled as a visible terminal reply outcome and races wrapped tool execution against the owning run’s abort signal.

PR surface: Source +67, Tests +151. Total +218 across 8 files.

Reproducibility: yes. at source level: the linked report and the PR’s negative-control output identify the stale-expiry and non-cooperative-tool paths, but this review did not execute the current-main reproduction.

Review metrics: 1 noteworthy metric.

Active abort-wrapper coverage: 1 outdated runtime target. The one tool-abort implementation change lands in the legacy wrapper rather than current main’s active pi-tools surface.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #103905
Summary: This PR is the direct candidate fix for the canonical silent stalled-recovery issue; the other linked stalled-run reports address distinct detection or recovery semantics.

Members:

canonical: Stuck-session recovery abort surfaces nothing to the user (surface_error with reason=none + draft deletion), and in-flight tool promises complete into a dead run #103905 - The PR uses closing syntax for this issue and implements both the visible terminal interruption and prompt tool-abort behavior it reports.
adjacent_distinct: fix(agent): make stuck recovery phase-safe #93655 - That PR changes phase-safe recovery and fallback ownership after a stall, whereas this PR changes terminal projection and late tool settlement after a terminal abort.
adjacent_distinct: Embedded agent runs intermittently stall at embedded_run:started (no progress, no hook execution) — v2026.6.9 through v2026.6.11 #103683 - That issue concerns identifying and recovering embedded-run startup stalls, not the terminal user notice and tool-promise behavior after recovery.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦞 diamond lobster
Patch quality: 🦪 silver shellfish
Result: blocked by patch quality or review findings.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Rebase on current main and move the abort race plus regression test to pi-tools.abort.ts.
Rerun focused Pi-tool abort and stalled-recovery coverage against the refreshed head.

Risk before merge

[P1] If merged without moving the race to the current pi-tools wrapper, an aborted turn can still wait for a non-cooperative tool handler to settle, leaving the availability half of the reported bug unresolved.

Maintainer options:

Port the abort race to the active wrapper (recommended)
Refresh the branch on current main, implement the prompt-abort race in pi-tools.abort.ts, and rerun the focused abort and stalled-recovery tests before merge.
Pause the PR
Do not merge the branch while its tool-abort change targets a wrapper that current runtime callers no longer use.

Next step before merge

[P2] A narrow mechanical repair is available: refresh the branch and move the abort race from the legacy wrapper to the active Pi-tool wrapper.

Security
Cleared: The diff adds no dependency, workflow, permission, secret, download, or publishing path; the abort race does not expand tool authority or attempt to undo external side effects.

Review findings

[P1] Move the abort race to the active Pi tool wrapper — src/agents/agent-tools.abort.ts:51-64

Review details

Best possible solution:

Rebase onto current main, apply the same abort race to the active pi-tools wrapper, and retain the run_stalled-before-generic-abort ordering with focused regression coverage.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: the linked report and the PR’s negative-control output identify the stale-expiry and non-cooperative-tool paths, but this review did not execute the current-main reproduction.

Is this the best way to solve the issue?

No; the terminal-outcome design is narrow and appropriate, but the abort race must be applied to the active pi-tools implementation rather than the migrated legacy wrapper.

Full review comments:

[P1] Move the abort race to the active Pi tool wrapper — src/agents/agent-tools.abort.ts:51-64
Current main routes wrapped tool execution through pi-tools.abort.ts, but this branch changes the legacy agent-tools.abort.ts surface. Rebase and port this race (with its regression test) to the active wrapper; otherwise non-cooperative tools still block aborted runs after merge.
Confidence: 0.86

Overall correctness: patch is incorrect
Overall confidence: 0.83

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a941d3ab727e.

Label changes

Label changes:

add merge-risk: 🚨 availability: The current branch can leave non-cooperative tools blocking an aborted run because its abort race does not cover the active wrapper on current main.
add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦞 diamond lobster and patch quality is 🦪 silver shellfish.
add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body provides before/after terminal evidence for both the stale-reply projection and non-cooperative-tool abort behavior, although the active-wrapper refresh still needs focused validation.
remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦪 silver shellfish, so this older rating label is no longer current.
remove merge-risk: 🚨 message-delivery: Current PR review merge-risk labels are merge-risk: 🚨 availability.
remove status: 👀 ready for maintainer look: Current PR status label is status: ⏳ waiting on author.

Label justifications:

P1: The PR targets a user-facing stalled-turn workflow that can leave a caller blocked and without an adequate terminal response.
merge-risk: 🚨 availability: The current branch can leave non-cooperative tools blocking an aborted run because its abort race does not cover the active wrapper on current main.
rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦞 diamond lobster and patch quality is 🦪 silver shellfish.
status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body provides before/after terminal evidence for both the stale-reply projection and non-cooperative-tool abort behavior, although the active-wrapper refresh still needs focused validation.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides before/after terminal evidence for both the stale-reply projection and non-cooperative-tool abort behavior, although the active-wrapper refresh still needs focused validation.

Evidence reviewed

PR surface:

Source +67, Tests +151. Total +218 across 8 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	6	69	2	+67
Tests	2	151	0	+151
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	8	220	2	+218

Acceptance criteria:

[P1] node scripts/run-vitest.mjs src/agents/pi-tools.abort.test.ts.
[P1] node scripts/run-vitest.mjs src/auto-reply/reply/agent-runner-execution-stalled-recovery.test.ts.
[P1] pnpm tsgo:core:test.

What I checked:

Canonical bug remains open: The PR is linked as the candidate fix for the reported silent stalled-recovery and late-tool-settlement behavior, and the canonical issue remains open. (200dea131a04)
Stale tool-wrapper target: The branch adds the abort race in the legacy wrapper, but the prior current-main review traced the active wrapper to src/agents/pi-tools.abort.ts; the proposed change at this location therefore does not establish runtime coverage on current main. (src/agents/agent-tools.abort.ts:51, 200dea131a04)
Current-main merge surface changed: The immediately preceding review recorded that current main had renamed the tool-wrapper surface to pi-tools, while this branch remained behind the base; the latest branch file list still changes agent-tools.abort.ts rather than the current wrapper. (src/agents/pi-tools.abort.ts:1, a941d3ab727e)
Terminal-outcome approach is supported: The PR checks the recorded run_stalled result before generic abort projection in both thrown-error and fallback-settlement paths, matching the issue’s requirement that stale recovery leave a visible interruption instead of a silent draft removal. (src/auto-reply/reply/agent-runner-error-handler.ts:232, 200dea131a04)
Real behavior evidence supplied: The PR body includes before/after terminal output for the stale-reply projection and non-cooperative tool abort, plus focused regression results; it is credible evidence for the intended behavior but predates the required current-wrapper refresh. (200dea131a04)

Likely related people:

steipete: Prior history review attributes the split reply-execution pipeline and its lifecycle/error-surfacing work to this contributor; the current repair belongs in that terminal-outcome boundary. (role: feature owner and recent area contributor; confidence: high; commits: 3f68402, 5f4bc6e, d0581ca; files: src/auto-reply/reply/agent-runner-error-handler.ts, src/auto-reply/reply/agent-runner-fallback-settlement.ts)
vincentkoc: Recent work in the reply execution surface covered error imports and provider/runtime normalization adjacent to this stalled-terminal handling. (role: recent adjacent contributor; confidence: medium; commits: 8a9ead6, aa27e27, f718ba6; files: src/auto-reply/reply/agent-runner-error-handler.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (10 earlier review cycles; latest 8 shown)

reviewed 2026-07-11T00:39:00.182Z sha e073f1f :: needs real behavior proof before merge. :: [P1] Check the stalled result before generic user abort
reviewed 2026-07-11T00:45:41.377Z sha e073f1f :: needs real behavior proof before merge. :: [P1] Check the stalled result before generic user abort
reviewed 2026-07-11T06:29:43.406Z sha 3525e7a :: needs maintainer review before merge. :: none
reviewed 2026-07-13T04:16:13.132Z sha 4aea3e6 :: needs maintainer review before merge. :: none
reviewed 2026-07-13T07:31:41.506Z sha 95e8af1 :: needs maintainer review before merge. :: none
reviewed 2026-07-16T03:29:31.344Z sha 55e24bc :: needs maintainer review before merge. :: none
reviewed 2026-07-18T09:25:53.770Z sha 3198858 :: needs maintainer review before merge. :: none
reviewed 2026-07-18T09:33:38.145Z sha 6a15996 :: needs maintainer review before merge. :: none

ZengWen-DT · 2026-07-11T00:04:46Z

@clawsweeper re-review

clawsweeper · 2026-07-11T00:04:49Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review published an exact durable review comment and queued an executing target-wide serialized router scan.
Run: https://github.com/openclaw/clawsweeper/actions/runs/29132024148
Updated: 2026-07-11T00:19:51.967Z

ZengWen-DT · 2026-07-11T00:23:23Z

@clawsweeper re-review

clawsweeper · 2026-07-11T00:23:26Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review published an exact durable review comment and queued an executing target-wide serialized router scan.
Run: https://github.com/openclaw/clawsweeper/actions/runs/29133101417
Updated: 2026-07-11T00:47:09.167Z

ZengWen-DT · 2026-07-11T06:13:41Z

@clawsweeper re-review

clawsweeper · 2026-07-11T06:13:43Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review published an exact durable review comment and queued an executing target-wide serialized router scan.
Run: https://github.com/openclaw/clawsweeper/actions/runs/29142729164
Updated: 2026-07-11T06:31:19.403Z

- wrapToolWithAbortSignal: race tool execution against run abort signal so non-cooperative tools no longer block an aborted turn - Stale reply-operation recovery now preserves run_stalled as a user-visible interruption instead of silently dropping the draft - Both the error handler and fallback-cycle settlement check for stalled operations before the broad user-abort predicate - isReplyOperationStalled and buildStalledRunReplyPayload helpers keep stalled handling explicit and separate from abort paths Closes openclaw#103905

openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Jul 10, 2026

ZengWen-DT force-pushed the fix/stuck-session-recovery-interruption branch from b984de4 to 31a8d1b Compare July 11, 2026 00:04

clawsweeper Bot removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 11, 2026

ZengWen-DT force-pushed the fix/stuck-session-recovery-interruption branch 2 times, most recently from 8eff5ce to 3525e7a Compare July 11, 2026 06:13

anyech mentioned this pull request Jul 12, 2026

Embedded agent runs intermittently stall at embedded_run:started (no progress, no hook execution) — v2026.6.9 through v2026.6.11 #103683

Closed

ZengWen-DT force-pushed the fix/stuck-session-recovery-interruption branch 2 times, most recently from 4aea3e6 to cd1bdd2 Compare July 13, 2026 06:35

openclaw-barnacle Bot added size: M and removed size: S labels Jul 13, 2026

ZengWen-DT force-pushed the fix/stuck-session-recovery-interruption branch 5 times, most recently from f2ad2fc to 95e8af1 Compare July 13, 2026 07:19

clawsweeper Bot mentioned this pull request Jul 16, 2026

fix(agent): make stuck recovery phase-safe #93655

Closed

ZengWen-DT force-pushed the fix/stuck-session-recovery-interruption branch from 55e24bc to 3198858 Compare July 18, 2026 09:22

ZengWen-DT requested a review from a team as a code owner July 18, 2026 09:30

clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jul 18, 2026

ZengWen-DT force-pushed the fix/stuck-session-recovery-interruption branch from 6a15996 to 200dea1 Compare July 18, 2026 09:42

ZengWen-DT closed this Jul 18, 2026

anyech mentioned this pull request Jul 19, 2026

fix: notify users when stalled-session recovery interrupts a turn #111092

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix: surface stalled-session interruptions#103990

fix: surface stalled-session interruptions#103990
ZengWen-DT wants to merge 1 commit into
openclaw:mainfrom
ZengWen-DT:fix/stuck-session-recovery-interruption

ZengWen-DT commented Jul 10, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 10, 2026 •

edited

Loading

Uh oh!

ZengWen-DT commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading

Uh oh!

ZengWen-DT commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading

Uh oh!

ZengWen-DT commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

ZengWen-DT commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZengWen-DT commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZengWen-DT commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZengWen-DT commented Jul 11, 2026

Uh oh!

clawsweeper Bot commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ZengWen-DT commented Jul 10, 2026 •

edited

Loading

clawsweeper Bot commented Jul 10, 2026 •

edited

Loading

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading

clawsweeper Bot commented Jul 11, 2026 •

edited

Loading