Skip to content

Telegram isolated polling ingress can deadlock when stale claim owner PID is reused #94546

Description

@billmaonew-lgtm

Summary

Telegram inbound delivery can become stuck even though Gateway, Telegram polling, and Bot API probe are healthy.

In this incident, Telegram updates were successfully written to OpenClaw's durable ingress queue, but one older update remained in claimed state. Its claim_owner PID no longer belonged to OpenClaw, but macOS had reused that numeric PID for another live system process. OpenClaw treated the stale claim as owned by another live handler and did not recover it, blocking later Telegram DM updates on the same lane.

Environment

  • OpenClaw: 2026.6.5 (5181e4f)
  • Gateway runtime: LaunchAgent
  • Gateway command: /opt/homebrew/opt/node/bin/node /opt/homebrew/lib/node_modules/openclaw/dist/index.js gateway --port 18789
  • Runtime: Node 25.5.0
  • OS: macOS
  • Telegram mode: polling, isolated ingress enabled
  • Gateway bind: 127.0.0.1:18789
  • State DB: ~/.openclaw/state/openclaw.sqlite

User-visible impact

Telegram DM messages stopped producing replies. From the user's perspective, Telegram messages appeared to be ignored.

However, Telegram itself was not broken. The messages were already inside OpenClaw's durable ingress queue.

Evidence

openclaw gateway status --deep --json showed Gateway healthy and RPC OK.

openclaw channels status --probe --json showed Telegram probe OK and Bot API reachable.

Queue state showed one claimed update and several later pending updates:

claimed | 1 | 2026-06-18 16:27:54
pending | 8 | 2026-06-18 17:51:14 - 2026-06-18 18:22:00

Representative rows:

event_id=0000000920256725 status=claimed received=2026-06-18 16:27:54
claim_owner=667:<uuid>

event_id=0000000920256726 status=pending received=2026-06-18 17:51:14
...
event_id=0000000920256733 status=pending received=2026-06-18 18:22:00

The claimed row owner PID was 667, but ps -p 667 showed a non-OpenClaw system process:

PID  PPID STAT COMMAND
667     1 Ss   /usr/sbin/filecoordinationd

The active OpenClaw Gateway process was a different PID:

PID    COMMAND
45027  node ... openclaw/dist/index.js gateway --port 18789

This means the stale claim owner PID had been reused by macOS for a process unrelated to OpenClaw.

Relevant implementation detail

In monitor-polling.runtime, stale Telegram spooled update recovery appears to use PID liveness:

function processPidFromOwnerId(ownerId) {
  const pid = Number.parseInt(ownerId.split(":", 1)[0] ?? "", 10);
  return Number.isSafeInteger(pid) && pid > 0 ? pid : -1;
}

function processExists(pid) {
  process.kill(pid, 0)
}

function isTelegramSpooledUpdateClaimOwnedByOtherLiveProcess(claim) {
  return Boolean(
    claim.claim &&
    claim.claim.processId !== TELEGRAM_SPOOLED_UPDATE_PROCESS_ID &&
    isFreshClaimOwner(claim.claim) &&
    processExists(claim.claim.processPid)
  );
}

Because PID 667 existed, OpenClaw treated the claim as owned by another live process, even though it was not an OpenClaw process.

The freshness window appears to be:

TELEGRAM_SPOOLED_UPDATE_PROCESSING_STALE_MS = 360 * 60 * 1000

This can block the lane for up to 6 hours if PID reuse occurs.

Actual behavior

A stale Telegram ingress claim was not recovered because the old numeric PID had been reused by another process. All later Telegram DM updates on the same lane remained pending and were not dispatched.

Expected behavior

OpenClaw should recover stale Telegram ingress claims when the original owner is no longer the same OpenClaw Gateway process, even if the numeric PID has been reused by the OS.

PID liveness alone is not enough to identify the original claim owner.

Workaround used

  1. Backed up SQLite state DB.
  2. Released only the stale claimed Telegram ingress row by setting it back to pending.
  3. Gateway then picked up the queue again.
  4. Logs confirmed inbound and outbound resumed:
Inbound message telegram:<redacted> -> <bot> (direct, 15 chars)
telegram outbound send ok ... messageId=<redacted>

Suggested fix

Do not rely only on numeric PID existence for claim_owner liveness.

Possible fixes:

  • Include process start time in claim_owner and verify PID + start time.
  • Include a Gateway boot/session UUID and verify it against active Gateway runtime.
  • Use a durable heartbeat lease per ingress handler rather than only process.kill(pid, 0).
  • If PID exists but the command is not OpenClaw Gateway/node runtime, treat the claim as stale.
  • Reduce the freshness window or add a safer recovery path for channel ingress queue deadlocks.

Severity

Medium to high for Telegram channel reliability.

A single stale claimed update can block all later messages in the same Telegram DM lane, while status/probe still reports Telegram as healthy. This makes the failure hard to detect from normal health checks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions