Version
v2026.6.11 (e085fa1a3ffd32d0ea6917e1e6fb4ecbffbb77d2), Node 24, Cloud Run (single instance), Slack channel via Socket Mode.
Summary
When the stuck-session recovery in src/logging/diagnostic-stuck-session-recovery.runtime.ts aborts an embedded run (action=abort_embedded_run), two behaviors combine so that the user's in-flight turn — and any result a long-running tool eventually produces — vanish silently:
1. The abort surfaces nothing to the user.
Observed sequence on a real session (timestamps trimmed):
[diagnostic] stalled session: ... state=processing age=916s reason=blocked_tool_call classification=blocked_tool_call activeWorkKind=tool_call lastProgress=tool:<name>:started lastProgressAge=916s recovery=checking
[diagnostic] stuck session recovery: ... age=916s action=abort_embedded_run aborted=true drained=true released=1
[agent/embedded] embedded abort settle timed out: runId=... timeoutMs=2000
[agent/embedded] embedded run failover decision: ... stage=assistant decision=surface_error reason=none from=anthropic/claude-sonnet-4-6
Despite decision=surface_error, no outbound payload is produced (reason=none), and the streamed draft message in the channel is then deleted (chat.delete on the partial reply). From the user's perspective, their message was being worked on for 15 minutes and then every visible trace of it disappeared — no error, no notice. Even for a genuinely wedged session, the abort should post an honest "this turn was interrupted" notice rather than deleting the only visible evidence.
2. The run abort neither cancels nor observes in-flight tool promises.
wrapToolWithAbortSignal (src/agents/agent-tools.abort.ts) passes the combined signal into execute but does not race the returned promise against the signal. A long-running tool handler (in our case a plugin tool that legitimately runs for ~20+ minutes) keeps executing after the abort, completes successfully — external side effects and all — and returns its result into a run that no longer exists. Nothing consumes it, and there is no hook or callback that tells the tool (or the plugin author) that the parent run died. The only workaround we found is threading the execute-arg AbortSignal into the handler manually and checking signal.aborted at completion to deliver the result through a side channel.
Repro
- Register a plugin tool whose
execute takes longer than diagnostics.stuckSessionAbortMs (default: max(5min, stuckSessionWarnMs × 3)) and emits no diagnostic activity on the session.
- Invoke it from a Slack (or any channel) session and send nothing else.
- At
warnMs, stalled session ... recovery=none warns begin; at the abort threshold, recovery fires abort_embedded_run.
- Observe: no user-facing error is posted, the streamed draft is deleted, and when the tool's promise later resolves, its result is dropped with no signal to the tool.
Suggested fixes
- On
abort_embedded_run, deliver an explicit interruption notice to the session's channel (the surface_error decision currently no-ops when reason=none), and don't delete the partial draft — or replace it with the notice instead.
- Either race tool executes against the run abort signal (with a documented cancellation contract), or expose the abort to the tool (e.g. invoke
onUpdate with a terminal event, or document that execute's signal is the only notification) so long-running tools can deliver or persist their results instead of completing into the void.
Happy to provide more log detail if useful.
Version
v2026.6.11 (
e085fa1a3ffd32d0ea6917e1e6fb4ecbffbb77d2), Node 24, Cloud Run (single instance), Slack channel via Socket Mode.Summary
When the stuck-session recovery in
src/logging/diagnostic-stuck-session-recovery.runtime.tsaborts an embedded run (action=abort_embedded_run), two behaviors combine so that the user's in-flight turn — and any result a long-running tool eventually produces — vanish silently:1. The abort surfaces nothing to the user.
Observed sequence on a real session (timestamps trimmed):
Despite
decision=surface_error, no outbound payload is produced (reason=none), and the streamed draft message in the channel is then deleted (chat.deleteon the partial reply). From the user's perspective, their message was being worked on for 15 minutes and then every visible trace of it disappeared — no error, no notice. Even for a genuinely wedged session, the abort should post an honest "this turn was interrupted" notice rather than deleting the only visible evidence.2. The run abort neither cancels nor observes in-flight tool promises.
wrapToolWithAbortSignal(src/agents/agent-tools.abort.ts) passes the combined signal intoexecutebut does not race the returned promise against the signal. A long-running tool handler (in our case a plugin tool that legitimately runs for ~20+ minutes) keeps executing after the abort, completes successfully — external side effects and all — and returns its result into a run that no longer exists. Nothing consumes it, and there is no hook or callback that tells the tool (or the plugin author) that the parent run died. The only workaround we found is threading the execute-argAbortSignalinto the handler manually and checkingsignal.abortedat completion to deliver the result through a side channel.Repro
executetakes longer thandiagnostics.stuckSessionAbortMs(default:max(5min, stuckSessionWarnMs × 3)) and emits no diagnostic activity on the session.warnMs,stalled session ... recovery=nonewarns begin; at the abort threshold, recovery firesabort_embedded_run.Suggested fixes
abort_embedded_run, deliver an explicit interruption notice to the session's channel (thesurface_errordecision currently no-ops whenreason=none), and don't delete the partial draft — or replace it with the notice instead.onUpdatewith a terminal event, or document thatexecute'ssignalis the only notification) so long-running tools can deliver or persist their results instead of completing into the void.Happy to provide more log detail if useful.