Skip to content

[Feature]: Optional session activity watchdog and alerts (blocked by #39127) #39141

Description

@sab-xan

Summary

Blocked by #39127

#39127

Add an optional gateway-side watchdog that consumes sessions.activity state and emits alerts (stale runs, waiting too long, child sessions stuck), so operators don’t have to build their own monitoring. Baseline should require no LLM tokens; optional tokenized summaries can be enabled.

Problem to solve

Even with a sessions.activity API, many users will not implement their own monitoring/control loop. Operators still need an easy way to detect and surface issues such as:

  • stale runs (busy forever, no progress)
  • waiting states that exceed thresholds (awaiting_user, awaiting_subagent, awaiting_approval)
  • runaway child spawns or large child trees that don’t converge
  • lack of progress signals for long-running work

Today, users must build polling scripts, manually watch UIs/logs, or miss these problems entirely.

This should be a built-in, opt-in capability that works out of the box once #39127 exists.

Proposed solution

Dependency

A) Watchdog engine (gateway-side, event-driven)

  • Subscribe internally to session activity state changes.,
  • Maintain timers per sessionKey.,
  • Detect conditions:
    • stale runs: session is busy but lastRunActivityAt does not advance for a configured time window,
    • blocked waiting too long: attention.state is awaiting_user or awaiting_subagent (or awaiting_approval) for longer than threshold,
    • optional: child tree issues (children.active above threshold for too long, or child failure counts),

B) Outputs (no-token baseline)
Provide alert outputs that do not require LLM calls:

  • Control UI notification/event (banner/toast/event log),
  • structured log line (machine-readable),
  • optional delivery to an operator target (Discord/Slack/Telegram/etc) using fixed templates,
  • optional webhook delivery with structured JSON,

Alerts should be deduped and rate-limited to avoid spam.

C) Optional tokenized summaries (opt-in)
If enabled, when an alert triggers, optionally call a cheap model to generate:

  • a short human-readable explanation
  • suggested next steps
    Requirements:
  • opt-in only, default off
  • configurable model and rate limits
  • must never block or slow the core gateway message processing

D) Configuration requirements

  • Watchdog is off by default or minimally noisy by default.
  • Thresholds are configurable per alert class.
  • Deduping and cooldown windows are required.
  • Allow scoping by agentId/sessionKey prefix if desired.

Security

  • No message content required for baseline alerts.
  • Follow existing sessions visibility rules; do not leak sensitive details in operator channels.

Test plan

  • Simulate sessions.activity transitions and verify alerts fire only after thresholds.
  • Verify dedupe and cooldown behavior.
  • Verify baseline works without any model/provider credentials.
  • Verify tokenized summaries are gated, rate-limited, and optional.

Alternatives considered

  • External polling scripts: everyone reinvents it; inconsistent and brittle.
  • Metrics-only approach: good for dashboards, but not a built-in operator alerting experience.
  • Manual log watching: time-consuming and unreliable.

Impact

Affected: operators running OpenClaw always-on (VPS/home server), multi-agent users, shared inbox setups.
Severity: reduces downtime and operator babysitting; prevents “silent stuck” states.
Frequency: continuous value for always-on deployments.
Consequence: without it, monitoring remains DIY and inconsistent.

Evidence/examples

Evidence/examples (optional)
Examples of desired alerts:

  • “Session X: stale run detected, no activity for 10m”
  • “Session Y: awaiting_subagent for 15m, blockedOn childSessionKey Z”
  • “Session A: awaiting_user for 2h”

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.enhancementNew feature or requestimpact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🌊 off-meta tidepoolIssue quality rating does not apply to this item.maturity:stableIssue affects a taxonomy feature currently scored M4/M5.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions