Skip to content

fix(telegram): recover stale ingress claims after restart#84690

Closed
zerone0x wants to merge 1 commit into
openclaw:mainfrom
zerone0x:fix/84674-telegram-spool-stale-claim
Closed

fix(telegram): recover stale ingress claims after restart#84690
zerone0x wants to merge 1 commit into
openclaw:mainfrom
zerone0x:fix/84674-telegram-spool-stale-claim

Conversation

@zerone0x

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Telegram isolated ingress could keep a fresh-looking .json.processing claim from a previous gateway identity when the PID was reused after recreate, blocking later same-lane updates.
  • Solution: Treat claims whose stored processPid equals the current process but whose unique processId differs as recoverable stale claims, not another live owner.
  • What changed: Updated Telegram spool ownership detection and regression tests for startup/drain recovery of restart-stale claims.
  • What did NOT change (scope boundary): No changes to Telegram delivery, handler timeout tombstoning, or cross-process live-claim protection for different live PIDs.

Motivation

  • Fixes a released Docker failure mode where a stale Telegram .processing file can survive gateway recreate and block later updates until manually moved aside.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: stale Telegram isolated-ingress .json.processing claim from an older gateway identity with the current PID no longer blocks later queued updates.
  • Real environment tested: local Linux checkout, Node v22.19.0, Telegram plugin spool/session tests.
  • Exact steps or command run after this patch: node scripts/run-vitest.mjs extensions/telegram/src/telegram-ingress-spool.test.ts extensions/telegram/src/polling-session.test.ts
  • Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output):
    ✓  extension-telegram  ../../extensions/telegram/src/polling-session.test.ts (42 tests) 9850ms
    ✓  extension-telegram  ../../extensions/telegram/src/telegram-ingress-spool.test.ts (7 tests) 14ms
    
    Test Files  2 passed (2)
         Tests  49 passed (49)
    
  • Observed result after fix: restart-stale current-PID/different-process-id claims are recovered back to pending updates, and focused Telegram spool/session tests pass.
  • What was not tested: live Docker/Telegram replay against a real bot.
  • Before evidence (optional but encouraged): existing regression test preserved the .processing file and left only the later update pending.

Root Cause (if applicable)

  • Root cause: live-claim detection considered any fresh claim with an existing PID to be owned by another live process, even when that PID was the current gateway process and the unique process id differed from the current gateway identity.
  • Missing detection / guardrail: no regression test covered gateway recreate / PID reuse where processPid === process.pid but processId belongs to an older gateway instance.
  • Contributing context (if known): isolated ingress drain uses this live-owner predicate before deciding whether to recover stale claims.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: extensions/telegram/src/telegram-ingress-spool.test.ts, extensions/telegram/src/polling-session.test.ts
  • Scenario the test should lock in: a .json.processing claim with the current PID but an older/different process identity is recoverable and returns the blocked update to the pending spool.
  • Why this is the smallest reliable guardrail: it exercises the exact plugin-local ownership predicate and drain recovery gate without needing live Telegram.
  • Existing test that already covers this (if any): N/A; existing coverage asserted the old blocking behavior.
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

Telegram isolated polling ingress can recover stale .processing claims left behind by a recreated gateway when the PID is reused, preventing later updates from staying blocked indefinitely.

Diagram (if applicable)

Before:
old gateway claim (.processing, pid=current, processId=old) -> treated live-owned -> later .json updates stay blocked

After:
old gateway claim (.processing, pid=current, processId=old) -> recovered to .json -> drain can process queued updates

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: Linux kvm16007 6.1.0-44-amd64 x86_64
  • Runtime/container: local source checkout, Node v22.19.0
  • Model/provider: N/A
  • Integration/channel (if any): Telegram isolated ingress spool
  • Relevant config (redacted): N/A

Steps

  1. Create a Telegram spool with update 42 claimed as .json.processing using processPid: process.pid and a different processId.
  2. Queue update 43 on the same lane.
  3. Run stale-claim recovery with the same live-owner gate used by isolated ingress drain.

Expected

  • Update 42 is recovered back to .json; updates 42 and 43 are pending in order.

Actual

  • After this patch, the focused regression test observes updates 42 and 43 pending and no lingering .json.processing file.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: focused Telegram spool/session regression tests pass; git diff --check passes.
  • Edge cases checked: existing stale-timeout recovery and handler/session tests still pass in the same focused run.
  • What you did not verify: live Docker/Telegram bot replay.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

  • Risk: accidentally replaying work still owned by a live second gateway.
    • Mitigation: the predicate only treats the current PID plus different process identity as restart-stale; different fresh live PIDs are still protected by the existing process-existence check.

@openclaw-barnacle openclaw-barnacle Bot added channel: telegram Channel integration: telegram size: XS triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI. labels May 20, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I did a careful shell check against current main, and this is already implemented.

Close: current main already implements the useful change through a broader merged Telegram ingress recovery fix, with regression coverage, maintainer acceptance, and beta release provenance.

Root-cause cluster
Relationship: superseded
Canonical: #97118
Summary: This PR is superseded by the merged maintainer Telegram ingress recovery PR that implemented the same current-PID recovery case plus broader queue-native drain safeguards.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

So I’m closing this as already implemented rather than keeping a duplicate issue open.

Review details

Best possible solution:

Keep the merged current-main implementation from #97118 and close this obsolete source PR.

Do we have a high-confidence way to reproduce the issue?

Yes. The stale-claim failure was source-reproducible from the live-owner predicate and is now covered by current-main Telegram spool/polling regression tests.

Is this the best way to solve the issue?

Yes. The merged current-main replacement is better than this stale PR because it includes the same PID-reuse recovery plus queue-native claiming, drain heartbeat behavior, focused tests, and live Telegram proof.

Security review:

Security review cleared: The source PR changes Telegram spool ownership logic and tests only; no dependency, workflow, permission, secret, package, or new code-execution surface is introduced.

AGENTS.md: found and applied where relevant.

What I checked:

Likely related people:

  • vincentkoc: Authored and landed the maintainer replacement fix in fix(telegram): recover stalled ingress spool claims #97118 and the beta backport in fix(telegram): recover stalled ingress spool claims #97543. (role: recent area contributor and merger; confidence: high; commits: b8e3de11608d, 8346e87808d8; files: extensions/telegram/src/telegram-ingress-spool.ts, extensions/telegram/src/polling-session.ts, extensions/telegram/src/polling-session.test.ts)
  • RomneyDa: The merged replacement PR body says it incorporated queue-native claim and PID-reuse work from the earlier superseded Telegram drain PR. (role: source queue-native claim contributor; confidence: medium; commits: b8e3de11608d; files: src/channels/message/ingress-queue.ts, extensions/telegram/src/polling-session.ts, extensions/telegram/src/telegram-ingress-spool.ts)
  • joshavant: Authored the merged Telegram spooled claim refresh work that the replacement recovery path builds on. (role: adjacent Telegram claim-refresh contributor; confidence: high; commits: db255b1154c1; files: extensions/telegram/src/polling-session.ts, extensions/telegram/src/telegram-ingress-spool.ts)
  • steipete: Authored the SQLite ingress queue refactor that backs the durable Telegram claim records involved in this recovery path. (role: shared queue storage contributor; confidence: medium; commits: b0679d1f13da; files: src/channels/message/ingress-queue.ts, extensions/telegram/src/telegram-ingress-spool.ts)

Codex review notes: model internal, reasoning high; reviewed against 561c713bb1f3; fix evidence: release v2026.6.11-beta.2, commit b8e3de11608d.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels May 20, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@martingarramon martingarramon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM-with-concerns. The core fix is correct — adding `claim.claim.processPid !== process.pid` short-circuits the live-owner check for restart-stale claims that recycled the current gateway PID, and the `polling-session.test.ts` change from `claimedAt: now - STALE_MS - 1` to `claimedAt: now` is important: the old test was passing because `isFreshClaimOwner` returned false (stale path), not because it was exercising the PID-reuse case. The new test covers the actual bug path.

Two actionable concerns:

1. Heading format — blocks CI gate.
`## Real behavior proof (required for external PRs)` should be `## Real behavior proof` (exact heading, no suffix). The CI gate matches the exact string; the `(required for external PRs)` suffix is causing the `Real behavior proof` check failure. Content is complete and correct — only the heading needs trimming.

2. Missing symmetric live-owner test.
The `polling-session.test.ts` change converts the existing blocked-case test into a recovery-case test. That's correct, but there's now no test asserting the symmetric path: "different PID + different processId + fresh + processExists → claim remains blocked." The old test was inadvertently covering this (via the stale path). A focused test for the still-blocked case would close the gap.

The other four CI failures (`check-additional-extension-bundled`, `check-lint`, `check-test-types`, `checks-node-core-runtime-infra-state`) are main-churn unrelated to this diff.

@clawsweeper

clawsweeper Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper applied the proposed close for this PR.

@clawsweeper clawsweeper Bot closed this Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: telegram Channel integration: telegram mantis: telegram-visible-proof Mantis should capture Telegram visible proof. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P1 High-priority user-facing bug, regression, or broken workflow. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telegram isolated ingress spool can remain blocked by stale .processing claim after gateway recreate

2 participants