Skip to content

Fix Telegram spooled claim refresh#96962

Merged
joshavant merged 1 commit into
mainfrom
fix/telegram-spooled-claim-refresh
Jun 26, 2026
Merged

Fix Telegram spooled claim refresh#96962
joshavant merged 1 commit into
mainfrom
fix/telegram-spooled-claim-refresh

Conversation

@joshavant

@joshavant joshavant commented Jun 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What Problem This Solves

Telegram polling can spool inbound updates into the shared ingress queue before dispatching them to the runtime. A long-running Telegram turn could hold a claimed ingress event past the stale-claim recovery window. Without refreshing the active claim, another poll loop can treat that still-active row as stale, reclaim it, and re-deliver or wedge the message while the first turn is still processing.

The maintainer screenshots described Telegram message delivery getting stuck. The live reproduction on main showed the same failure class: an active Telegram spooled ingress claim could age into stale recovery while its model turn was still running.

Why This Change Was Made

This PR makes the active Telegram spooled claim renewable while the message is being processed, and it hardens stale recovery so a worker only releases the exact stale claim it originally observed.

Key changes:

  • Add token-guarded optional claim refresh support to src/channels/message/ingress-queue.ts.
  • Add Telegram spool refresh plumbing in extensions/telegram/src/telegram-ingress-spool.ts.
  • Refresh active and deferred Telegram spooled claims from extensions/telegram/src/polling-session.ts while the runtime is processing the update.
  • Add a Telegram status/backlog diagnostic so operators can distinguish stuck backlog from normal in-flight work.
  • Add focused coverage for ingress stale-claim races, Telegram spool refresh behavior, and polling-session refresh lifecycle.

User Impact

Telegram users with working configurations should see the same normal delivery path for fast turns. Long-running turns should no longer be eligible for duplicate stale recovery while still active, which prevents the stuck/redispatch class seen in the screenshots.

The fix is bounded to claimed spooled Telegram ingress events and uses the existing queue database. It does not add new user configuration, change Telegram credentials, or alter non-spooled channel delivery behavior.

Evidence

Code and automated checks:

  • Commit: 88f37bcafef (Fix Telegram spooled claim refresh).
  • Focused tests passed for:
    • src/channels/message/ingress-queue.test.ts
    • extensions/telegram/src/telegram-ingress-spool.test.ts
    • extensions/telegram/src/polling-session.test.ts
  • Broader regression test set passed for the touched Telegram/message queue surface.
  • pnpm build passed.
  • pnpm check:changed passed on Testbox run tbx_01kw16av103z2f395vvtggxzfs.
  • $autoreview passed with no accepted/actionable findings.

Live Telegram HITL proof with 1Password-injected Telegram credentials:

  • /status delivered through a live Telegram bot.
  • Fast Telegram model turn delivered OPENCLAW_E2E_OK_1; confirmed visible by HITL.
  • Slow 330s Telegram model turn delivered OPENCLAW_E2E_OK_SLOW; confirmed visible by HITL.
  • During the slow turn, the same ingress row remained claimed and its claim timestamp refreshed from 1782453619721 to 1782453919724 instead of aging into stale recovery.
  • After the slow turn completed, both queues drained:
    • channel_ingress_events: 0
    • delivery_queue_entries: 0
  • The runtime transcript contains the slow assistant response plus Telegram final-delivery mirror telegram-final:agent:main:main:7676134290:337:0.

Local proof artifacts:

  • .artifacts/qa-e2e/telegram-hitl/spooled-claim-refresh/gateway.log
  • .artifacts/qa-e2e/telegram-hitl/spooled-claim-refresh/slow-mock-openai-requests.ndjson
  • .artifacts/qa-e2e/telegram-hitl/spooled-claim-refresh/run/state/state/openclaw.sqlite

@openclaw-barnacle openclaw-barnacle Bot added channel: telegram Channel integration: telegram size: L maintainer Maintainer-authored PR labels Jun 26, 2026
@clawsweeper

clawsweeper Bot commented Jun 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 26, 2026, 2:30 AM ET / 06:30 UTC.

Summary
The PR adds token-guarded claim refresh to the shared channel ingress queue, wires Telegram spooled update processing to refresh active/deferred claims, hardens stale recovery, and adds focused queue/Telegram tests.

PR surface: Source +171, Tests +356. Total +527 across 6 files.

Reproducibility: yes. at source level. Current main can decide stale recovery from an old claimedAt snapshot, while the PR body reports a live Telegram slow-turn proof where the active claim timestamp refreshed instead of aging into recovery.

Review metrics: 2 noteworthy metrics.

  • Plugin-Facing Queue API: 1 optional method added. ChannelIngressQueue is exported through the plugin SDK barrel, so even an optional method is a maintainer-visible contract change.
  • Persisted Claim Semantics: 2 behaviors changed. The PR refreshes stored claim timestamps and changes stale recovery to release only the exact still-stale claim token.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] The protected maintainer label means explicit maintainer handling remains required before merge.
  • [P1] The PR adds an optional method to an SDK-exported queue type; maintainers should accept that plugin-facing API surface or ask for it to be internalized.
  • [P1] The diff updates persisted ingress claim timestamps and stale-release behavior used by Telegram delivery, so mistakes could duplicate, wedge, or suppress inbound messages even with green CI.
  • [P1] Native Telegram Desktop proof is currently in progress; the PR body already has sufficient live HITL/log proof, but maintainers may still want the visual proof before landing.

Maintainer options:

  1. Land With Maintainer Acceptance (recommended)
    Maintainers can merge with the current proof if they accept the optional queue API addition and persisted claim-refresh semantics as the right bug-fix boundary.
  2. Internalize The Refresh Seam
    If refreshClaim should not become plugin-facing API, move the refresh behavior behind a private Telegram/core seam before merge.
  3. Coordinate With Broader Recovery Work
    Pause this PR only if maintainers want active-claim refresh folded into the broader stale channel_ingress_events recovery direction tracked by channel_ingress_events (SQLite): stale claims never recovered after session crash — Telegram DM deadlocks #90945.

Next step before merge

  • [P2] Manual review is required because the PR has the protected maintainer label and changes persisted Telegram/message-delivery state plus an SDK-exported queue type.

Security
Cleared: The diff changes TypeScript queue/session logic and tests only; it adds no dependencies, workflows, secrets handling, downloads, or permission changes.

Review details

Best possible solution:

Land the bounded token-guarded refresh and stale-release guard after maintainer review accepts the SDK surface and the persisted Telegram queue semantics.

Do we have a high-confidence way to reproduce the issue?

Yes at source level. Current main can decide stale recovery from an old claimedAt snapshot, while the PR body reports a live Telegram slow-turn proof where the active claim timestamp refreshed instead of aging into recovery.

Is this the best way to solve the issue?

Yes, with maintainer acceptance of the small API addition. Refreshing the exact active claim token and rechecking token plus claimed_at before stale release is the narrowest maintainable fix for this active-processing race; broader crash/startup recovery remains tracked separately.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4fc504d321b6.

Label changes

Label changes:

  • add merge-risk: 🚨 compatibility: The diff adds an optional method to the SDK-exported ChannelIngressQueue type.

Label justifications:

  • P1: The PR targets a real Telegram delivery stall/duplicate-redelivery class affecting long-running channel turns.
  • merge-risk: 🚨 compatibility: The diff adds an optional method to the SDK-exported ChannelIngressQueue type.
  • merge-risk: 🚨 message-delivery: A regression in Telegram spooled claim refresh or stale release could duplicate, wedge, or suppress inbound Telegram messages.
  • merge-risk: 🚨 session-state: The change updates persisted ingress claim timestamps and recovery behavior that preserve active Telegram processing state.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (logs): The PR body provides after-fix live Telegram HITL/log proof for fast and slow turns, claim timestamp refresh, and drained queues; no contributor proof action is needed.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides after-fix live Telegram HITL/log proof for fast and slow turns, claim timestamp refresh, and drained queues; no contributor proof action is needed.
  • mantis: telegram-visible-proof: Mantis should capture Telegram visible proof. The PR changes visible Telegram message delivery behavior for long-running turns, so a short Telegram Desktop proof is useful and already being pursued.
Evidence reviewed

PR surface:

Source +171, Tests +356. Total +527 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 3 191 20 +171
Tests 3 356 0 +356
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 6 547 20 +527

What I checked:

  • Repository policy applied: Root AGENTS.md, extensions/AGENTS.md, src/channels/AGENTS.md, and Telegram maintainer notes were read; their guidance requires whole-path Telegram review, persisted-state caution, and real Telegram proof for Telegram behavior changes. (AGENTS.md:13, 4fc504d321b6)
  • Current-main stale recovery race: On current main, recoverStaleClaims snapshots claimed rows by claimedAt and then releases the claim later through the normal release path, so a concurrent refresh/reclaim race is possible without the PR's token-and-cutoff recheck. (src/channels/message/ingress-queue.ts:443, 4fc504d321b6)
  • PR queue fix: The PR head adds optional refreshClaim guarded by queue name, event id, claimed status, and claim token, and changes stale release to require the same token and still-stale claimed_at cutoff. (src/channels/message/ingress-queue.ts:548, 88f37bcafefa)
  • PR Telegram lifecycle fix: The PR head starts a periodic claim refresh for claimed spooled updates, carries its stop callback through active and deferred processing, and stops refreshing on completion, failure, deferred timeout, or handler timeout. (extensions/telegram/src/polling-session.ts:580, 88f37bcafefa)
  • Plugin-facing API surface: ChannelIngressQueue is exported through the plugin SDK channel-outbound barrel, so adding refreshClaim is an additive plugin-facing type surface that needs maintainer acceptance even though it is optional. (src/plugin-sdk/channel-outbound.ts:149, 4fc504d321b6)
  • Focused regression coverage: The PR adds tests for token-guarded refresh, stale-recovery snapshot races, active Telegram claim refresh, deferred claim refresh, and backlog health diagnostics. (src/channels/message/ingress-queue.test.ts:280, 88f37bcafefa)

Likely related people:

  • zhang-guiping: git blame attributes the current shared ingress queue, Telegram spool helper, and polling drain implementation to bead84f. (role: introduced behavior; confidence: high; commits: bead84f0ee6e; files: src/channels/message/ingress-queue.ts, extensions/telegram/src/polling-session.ts, extensions/telegram/src/telegram-ingress-spool.ts)
  • Ayaan Zaidi: Git commit metadata lists Ayaan Zaidi as committer for bead84f, the current-main commit carrying the central queue and Telegram spool files. (role: recent committer; confidence: medium; commits: bead84f0ee6e; files: src/channels/message/ingress-queue.ts, extensions/telegram/src/polling-session.ts, extensions/telegram/src/telegram-ingress-spool.ts)
  • VACInc: Commit 13ecca5 recently changed Telegram session-init spool retry behavior in the same polling/spool files. (role: recent adjacent contributor; confidence: medium; commits: 13ecca5408cb; files: extensions/telegram/src/polling-session.ts, extensions/telegram/src/telegram-ingress-spool.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. mantis: telegram-visible-proof Mantis should capture Telegram visible proof. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 26, 2026
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. label Jun 26, 2026
@joshavant
joshavant merged commit db255b1 into main Jun 26, 2026
178 of 194 checks passed
@joshavant
joshavant deleted the fix/telegram-spooled-claim-refresh branch June 26, 2026 06:31
@github-actions github-actions Bot mentioned this pull request Jun 26, 2026
Open
@RomneyDa RomneyDa mentioned this pull request Jun 26, 2026
Closed
This was referenced Jun 26, 2026
Merged
Closed
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 27, 2026
@joshavant
Fix Telegram spooled claim refresh (openclaw#96962)
5565758
This was referenced Jun 28, 2026
Merged
Closed
Merged
QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026
@joshavant @QiuYuang
Fix Telegram spooled claim refresh (openclaw#96962)
fef7a5b
@amittell amittell mentioned this pull request Jul 5, 2026
Closed
@obviyus obviyus mentioned this pull request Jul 9, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

channel: telegram Channel integration: telegram maintainer Maintainer-authored PR mantis: telegram-visible-proof Mantis should capture Telegram visible proof. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: L status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@joshavant