OpenClaw version: 2026.6.10
Channel: Telegram (long-polling / isolated ingress mode)
Storage: channel_ingress_events table in state/openclaw.sqlite
Summary
Inbound messages stop being delivered to the agent after the gateway is restarted while a worker has an in-flight claimed row in channel_ingress_events. The claim is never released, so the stuck rows block all subsequent pending events on the same queue indefinitely. The agent keeps polling and reports healthy, but never processes or replies to any incoming message.
This is a classic stale-claim / orphaned-lease bug: the queue has no visibility-timeout or reaper to recover claims owned by a dead worker.
Impact
- Agent appears fully healthy:
/ready and /health return OK, Telegram polling logs show isolated polling ingress started, no errors.
- Telegram side shows
pending_update_count: 0 (updates are fetched), but the agent never emits an Inbound message log and never replies.
- Every message sent after the stuck claim is silently queued as
pending and never dispatched.
- Symptom is indistinguishable from "channel broken" at the surface;
openclaw doctor --lint does not detect it (config is valid; the failure is purely at the runtime queue layer).
Root cause
channel_ingress_events lifecycle: pending → claimed (worker takes it) → completed (purged) / failed.
When a worker claims an event (status='claimed', sets claim_token/claim_owner/claimed_at) and the gateway process is then killed mid-flight (SIGKILL, container restart, resource resize), the row stays claimed forever. On restart, a new worker does not reclaim these orphaned rows, and they sit ahead of newer pending rows in the queue ordering, blocking delivery.
The table schema already carries everything needed to detect this (claimed_at, attempts, last_attempt_at) — there is simply no logic that releases claims older than a timeout.
Reproduction (observed in production)
- Agent (Telegram, isolated polling ingress) is actively processing.
- Container is restarted abruptly (in our case: an orchestrator
stop → resize → start sequence, plus a few manual restarts during debugging) while several events were claimed.
- After restart: polling resumes cleanly, no conflict, healthy — but agent never processes any inbound message.
Observed DB state (40 stuck rows for one telegram queue):
status='claimed' count=31 (all claim_owner='7:239da3b5-...', claimed_at≈resize time, attempts=0, last_error=NULL)
status='pending' count=9 (every message the user sent afterwards, never dispatched)
All 31 claimed rows were owned by a single dead worker id; none ever advanced to completed. Telegram offset on the API side had advanced past them (updates fetched), but the agent never saw them because they were stuck in the local DB queue.
Workaround used
DELETE FROM channel_ingress_events
WHERE channel_id = 'telegram'
AND status IN ('claimed', 'pending');
After clearing the stuck rows and restarting, the new worker claimed → dispatched → completed normally (Inbound message logged, outbound send ok, queue back to empty).
Suggested fix
Add stale-claim recovery to the channel_ingress_events consumer, e.g. any of:
- Visibility timeout / claim TTL — on claim acquisition, reset rows where
status='claimed' AND claimed_at < now() - CLAIM_TTL_MS back to pending (clear claim_token/claim_owner), bump attempts.
- Startup reaper — on gateway boot, release all
claimed rows whose claim_owner doesn't match a live worker (workers are short-lived per-process, so any pre-restart claim is by definition orphaned).
- Worker heartbeat / lease renewal — require periodic claim renewal; expired leases auto-release.
Option (2) alone (release-on-boot) would have fully prevented this case, since a restart guarantees the previous claim owner is dead.
Optionally: have openclaw doctor surface a warning when channel_ingress_events has claimed rows with claimed_at older than N minutes — this would have made diagnosis immediate.
OpenClaw version: 2026.6.10
Channel: Telegram (long-polling / isolated ingress mode)
Storage:
channel_ingress_eventstable instate/openclaw.sqliteSummary
Inbound messages stop being delivered to the agent after the gateway is restarted while a worker has an in-flight
claimedrow inchannel_ingress_events. The claim is never released, so the stuck rows block all subsequentpendingevents on the same queue indefinitely. The agent keeps polling and reports healthy, but never processes or replies to any incoming message.This is a classic stale-claim / orphaned-lease bug: the queue has no visibility-timeout or reaper to recover claims owned by a dead worker.
Impact
/readyand/healthreturn OK, Telegram polling logs showisolated polling ingress started, no errors.pending_update_count: 0(updates are fetched), but the agent never emits anInbound messagelog and never replies.pendingand never dispatched.openclaw doctor --lintdoes not detect it (config is valid; the failure is purely at the runtime queue layer).Root cause
channel_ingress_eventslifecycle:pending→claimed(worker takes it) →completed(purged) /failed.When a worker claims an event (
status='claimed', setsclaim_token/claim_owner/claimed_at) and the gateway process is then killed mid-flight (SIGKILL, container restart, resource resize), the row staysclaimedforever. On restart, a new worker does not reclaim these orphaned rows, and they sit ahead of newerpendingrows in the queue ordering, blocking delivery.The table schema already carries everything needed to detect this (
claimed_at,attempts,last_attempt_at) — there is simply no logic that releases claims older than a timeout.Reproduction (observed in production)
stop → resize → startsequence, plus a few manual restarts during debugging) while several events wereclaimed.Observed DB state (40 stuck rows for one telegram queue):
All 31
claimedrows were owned by a single dead worker id; none ever advanced tocompleted. Telegram offset on the API side had advanced past them (updates fetched), but the agent never saw them because they were stuck in the local DB queue.Workaround used
After clearing the stuck rows and restarting, the new worker claimed → dispatched → completed normally (
Inbound messagelogged,outbound send ok, queue back to empty).Suggested fix
Add stale-claim recovery to the
channel_ingress_eventsconsumer, e.g. any of:status='claimed' AND claimed_at < now() - CLAIM_TTL_MSback topending(clearclaim_token/claim_owner), bumpattempts.claimedrows whoseclaim_ownerdoesn't match a live worker (workers are short-lived per-process, so any pre-restart claim is by definition orphaned).Option (2) alone (release-on-boot) would have fully prevented this case, since a restart guarantees the previous claim owner is dead.
Optionally: have
openclaw doctorsurface a warning whenchannel_ingress_eventshasclaimedrows withclaimed_atolder than N minutes — this would have made diagnosis immediate.