Skip to content

channel_ingress_events (SQLite): stale claims never recovered after session crash — Telegram DM deadlocks #90945

Description

@GDXbsv

channel_ingress_events (SQLite): stale claims never recovered — deadlocks Telegram sessions

OpenClaw version: 2026.6.1
Channel: Telegram DM
OS: Linux (CachyOS, kernel 7.0.11)


Summary

When a session dies mid-turn, channel_ingress_events rows stay in claimed state permanently. Every new message into that session gets claimed by a worker, which also fails, leaving another stuck claimed row. The channel becomes silently unresponsive until manually unblocked via direct SQLite write.

This happened twice in the same session within ~1 hour on 2026-06-06.


Steps to reproduce

  1. Active Telegram DM session running heavy work (parallel exec calls)
  2. Session dies mid-turn (gateway crash, kill -HUP on gateway PID, or unhandled rejection)
  3. Observe channel_ingress_events:
('claimed', 2)
('pending', 3)
  1. Gateway restarts — no recovery runs
  2. Send a new message to that session → new claimed row, never released
  3. Repeat — every new message adds another stuck claim

Why the existing recovery does NOT fire (dist analysis on v2026.6.1)

Issue #90940 was closed as 'already implemented'. After analysing /app/dist/ on v2026.6.1, the existing recovery covers a different code path — the file-based spool, not SQLite.

What IS implemented ✅

recoverStaleTelegramSpooledUpdateClaims in monitor-polling.runtime-C_Buepuh.js:

return await createTelegramIngressQueue(params.spoolDir).recoverStaleClaims({
  staleMs: params.staleMs ?? 216e5,  // 6 hours
  shouldRecover: async (claim) => !isTelegramSpooledUpdateClaimOwnedByOtherLiveProcess(claim)
});

This recovers stale claims in the file-based spool (~/.openclaw/telegram/ingress-spool-default/). Has both a TTL (6h) and a live-process guard. Works correctly.

What is NOT implemented ❌

The stuck claims are in channel_ingress_events (SQLite).

Evidence:

1. claimNext is never called externally

grep -rl 'claimNext' /app/dist/
# → only: /app/dist/ingress-queue-BphVF27i.js

Defined in the queue module, never imported anywhere else. The claimNext path (which calls recoverStaleClaims when staleMs is passed) is dead code.

2. openChannelIngressQueue has exactly one caller

grep -rl 'openChannelIngressQueue' /app/dist/
# → monitor-polling.runtime  (spool path only)
# → registry-BVye-IRt.js     (proxy definition)
# → plugins/runtime/index.js  (throws 'only available through plugin runtime proxy')

createTelegramIngressQueue in monitor-polling.runtime wraps the spool directory path, not the main SQLite ingress queue.

3. No startup sweep
On gateway boot, there is no code that iterates claimed rows in channel_ingress_events, checks if claim_owner PID is still alive, and resets dead claims to pending. The processExists() function exists in monitor-polling.runtime but is only wired to the spool path.

The gap

File-based spool (ingress-spool-default/)
  → recoverStaleTelegramSpooledUpdateClaims ✅ (6h TTL + live-process guard)

SQLite channel_ingress_events
  → recoverStaleClaims() exists in ingress-queue.js ✅
  → BUT never called on startup ❌
  → claimNext() never called with staleMs ❌
  → No startup sweep of dead claim_owner PIDs ❌

Proposed fix

Option A — Startup sweep (simplest)
On gateway start, iterate claimed rows in channel_ingress_events and reset those where claim_owner PID is no longer alive (same logic as processExists already used for spool).

Option B — Pass staleMs to claimNext
At the call site that drains channel_ingress_events, pass staleMs:

queue.claimNext({ staleMs: 10 * 60 * 1000 })

Option C — Periodic background reaper
A background timer (e.g. every 5 min) calls:

queue.recoverStaleClaims({ staleMs: 10 * 60 * 1000 })

Workaround

Script cron every 5 min, resets claims older than 10 minutes, notifies via Telegram.
Protecting the most recently claimed event in case it is actively being processed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions