Skip to content

Diagnostic detects stalled_agent_run (recovery=none) after model_call:ended but never recovers #100213

Description

@pallaoro

Summary

An agent run wedged mid-turn: the last model call ended, but the run never advanced to the next tool call or to completion. The diagnostic subsystem correctly detects and logs the stall (classification=stalled_agent_run) but reports recovery=none and takes no action. The session stays state=processing indefinitely, with a queued message stuck behind it, until an operator intervenes.

Observed (production log)

[diagnostic] stalled session: sessionId=<id> sessionKey=agent:main:user:<uid>:chat:<key>
  state=processing age=216s queueDepth=1 reason=active_work_without_progress
  classification=stalled_agent_run activeWorkKind=embedded_run
  lastProgress=model_call:ended lastProgressAge=142s recovery=none

[diagnostic] stalled session: sessionId=<id> ...
  state=processing age=246s queueDepth=1 reason=active_work_without_progress
  classification=stalled_agent_run activeWorkKind=embedded_run
  lastProgress=model_call:ended lastProgressAge=172s recovery=none
  • age climbs each tick (216s → 246s → …); the run never self-clears.
  • lastProgress is stuck at model_call:ended — the model returned, but nothing advanced afterward.
  • recovery=none on every tick.
  • queueDepth=1 — a follow-up message is blocked behind the wedged run.

Impact

On Clawnify.com (managed OpenClaw platform) this stranded an in-progress task partway through. The only ways to clear it were an operator Stop or a gateway restart; the run would otherwise sit in processing indefinitely.

Questions / ask

  1. For classification=stalled_agent_run, should the detector attempt auto-recovery (re-drive / abort + requeue the pending message) rather than only observe? Today recovery=none means the detector names the problem but leaves the session wedged.
  2. If recovery=none is intentional for this class, what is the documented operator remediation?
  3. Root cause: what leaves a run at lastProgress=model_call:ended with no subsequent step — e.g. an empty/failed model response, or a tool-dispatch path that doesn't advance the state machine after the model returns?

Environment

  • OpenClaw 2026.6.5 (5181e4f), self-hosted gateway, activeWorkKind=embedded_run.
  • Possibly related to heartbeat / pending-run starvation reports.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.maturity:stableIssue affects a taxonomy feature currently scored M4/M5.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions