Skip to content

[Bug]: Telegram isolated ingress deletes spooled updates when the turn fails — errors are swallowed so handleUpdate never rejects (silent message loss) #92129

Description

@riseandshinefutures

Summary

In Telegram isolated polling ingress mode (default-on), a spooled inbound update is permanently deleted even when processing it failed, because the spool consumer keys deletion on bot.handleUpdate() resolving — and the Telegram turn pipeline swallows every error before it can reject. A gateway crash/recreate during an in-flight turn therefore loses the customer message silently and tracelessly: Telegram has already confirmed the offset (correctly — the spool write happens first, which is good), the spool entry is gone, and no session record exists.

The ingress side is doing exactly the right thing (writeTelegramSpooledUpdate before the offset advances). The durability contract breaks at the consume side.

The chain (v2026.5.18, verified unchanged in v2026.6.5)

  1. polling-session.ts#handleClaimedSpooledUpdate:
    • bot.handleUpdate() resolves → deleteTelegramSpooledUpdate() (durable copy gone)
    • bot.handleUpdate() rejects → #releaseFailedSpooledUpdate() (claim released for retry)
  2. But handleUpdate can essentially never reject:
    • bot-message.ts (processMessage) wraps dispatchTelegramMessage in a try/catch that logs telegram message processing failed, best-effort sends "Something went wrong…" (itself inside catch {}), and returns normally.
    • Anything that escapes the handler hits bot.catch(...) installed in bot-core.ts, which logs and consumes the error — grammY then resolves handleUpdate.
  3. Result: a turn that threw (LLM/network failure, abort during shutdown, any bug) is indistinguishable from a completed turn. The spool's release/retry path is effectively dead code for real processing failures, and deleteTelegramSpooledUpdate runs unconditionally.

The shutdown case is the sharpest edge: SIGTERM → in-flight turn work aborts → throw → swallowed → spool entry deleted → SIGKILL. The message ceases to exist with zero artifacts (pending_update_count: 0, empty spool dir, no session transcript entry, no reply, no error visible to the sender).

Observed in production

Docker-managed gateway (container recreate for a config change, docker stop -t 10). A user DM that arrived during the recreate window vanished: Telegram queue empty, ingress-spool-default/ empty, no session record, user got silence. Reconstructed against source as above; the swallowed-error log line was lost with the old container's stdout.

Repro sketch

  1. Run a gateway with Telegram isolated ingress (default).
  2. Make the turn fail mid-flight — e.g. point the model provider at a black-hole endpoint, or SIGTERM the gateway while a turn is processing (a sleep-in-tool works too).
  3. Send a DM; observe telegram message processing failed (or telegram bot error) in logs.
  4. Inspect the spool dir: the update's .json/.json.processing entry has been deleted, not released or tombstoned. Restart the gateway: nothing replays; the message is gone.

Expected: a failed turn releases the claim (pending retry) or dead-letters to a .failed tombstone (like the handler-timeout path already does) — never a clean delete.

Possible fix directions

  • Thread a per-update failure signal from the turn pipeline to the spool consumer (e.g. the update tracker already computes completed: boolean in its finishUpdate middleware — #handleClaimedSpooledUpdate could consult that instead of relying on promise rejection), or
  • Have processMessage's catch (and bot.catch) rethrow/flag when the update is a spooled isolated-ingress replay, so #releaseFailedSpooledUpdate can do its job, or
  • Dead-letter failed turns via failTelegramSpooledUpdateClaim with the error context (consistent with the existing handler-timeout tombstone semantics from fix(telegram): recover stalled isolated spool handlers #83505).

Related but distinct: #84674 tracks stale .processing claim recovery after recreate (PID-liveness false positive). Even with that fixed, this bug deletes the entry outright on swallowed failure, so there is nothing left to recover.

Environment

  • OpenClaw: 2026.5.18 (code paths verified identical at v2026.6.5)
  • Runtime: Docker, gateway via node openclaw.mjs gateway
  • Channel: Telegram, isolated polling ingress (default), messages.inbound.debounceMs unset (0)

Metadata

Metadata

Assignees

Labels

P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions