fix(telegram): move network fallback to resolver-scoped dispatchers#40740
Conversation
Greptile SummaryThis PR reworks Telegram networking from process-global mutation to resolver-scoped undici dispatchers, eliminating the shared-state contention that caused intermittent fallback thrash in multi-account or mixed-network setups. All Telegram send, probe, audit, bot polling, and media download paths now share the same transport resolution logic, and sticky IPv4 fallback is contained within each resolver's closure rather than affecting the whole process. Key changes:
The architectural refactor is sound: resolver-scoped dispatchers correctly replace process-global mutation, and the sticky IPv4 fallback closure semantics are well-tested across multiple accounts and configurations. Confidence Score: 5/5
Last reviewed commit: f8a8461 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a5913d5e5e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@greptile-apps Thanks for the review, this has been addressed. Please re-review. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 494207a9a3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8238157d0e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 930ea6bd36
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@greptile-apps @codex Updated based on comments, can you please re-review 🙂 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a3cc6d9c61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@greptile-apps Thanks, those minor issues have been addressed. Please re-review? |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 99b6ce947b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@greptile-apps Reviewed and addressed. Please re-review? For @codex reviews, our NO_PROXY implementation intentionally matches undici semantics and bypass logic.
Preference is to match established undici behaviour here to avoid drift and unexpected handling. |
|
Codex Review: Didn't find any major issues. Delightful! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@greptile-apps Thanks - last (🤞🏼) minor style issue addressed. Can I get another review please? 😊 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e42f8aa31b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@greptile-app Thanks, cleared up those tests, are we looking good now? 🙂 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f3e11d4b37
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@greptile-app thanks for the thoughtful review. I'd appreciate a re-assessment. We've aligned those cache keys. Regarding your concern on "manually replicating undici's NO_PROXY logic", we need a classification before dispatch to decide whether sticky IPv4 fallback can safely arm. EnvHttpProxyAgent does not expose route decisions (proxy vs direct NO_PROXY bypass), so this is something we need to determine, and undici's NO_PROXY logic is a logical source. This is less about "manually replicating" their implementation as it is using it as an established, proven solution. I do not believe this choice significantly increases the complexity, and there is not a significant risk posed by Undici's proxy bypass rules changing in future. |
|
Applied follow-up for the env-proxy review points.
Verified with targeted Telegram tests plus |
ce7b17d to
e47ef6d
Compare
e47ef6d to
a4456d4
Compare
|
Merged via squash.
Thanks @sircrumpet! |
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
* main: (43 commits) docs: add openclaw#42173 to CHANGELOG — strip leaked model control tokens (openclaw#42216) Agents: align onPayload callback and OAuth imports docs: add Tengji (George) Zhang to maintainer table (openclaw#42190) fix: strip leaked model control tokens from user-facing text (openclaw#42173) Changelog: add unreleased March 9 entries chore: add .dev-state to .gitignore (openclaw#41848) fix(agents): avoid duplicate same-provider cooldown probes in fallback runs (openclaw#41711) fix(mattermost): preserve markdown formatting and native tables (openclaw#18655) feat(acp): add resumeSessionId to sessions_spawn for ACP session resume (openclaw#41847) ACPX: bump bundled acpx to 0.1.16 (openclaw#41975) mattermost: fix DM media upload for unprefixed user IDs (openclaw#29925) fix(msteams): use General channel conversation ID as team key for Bot Framework compatibility (openclaw#41838) fix(mattermost): read replyTo param in plugin handleAction send (openclaw#41176) fix(sandbox): pass real workspace to sessions_spawn when workspaceAccess is ro (openclaw#40757) fix(ui): replace Manual RPC text input with sorted method dropdown (openclaw#14967) CI: select Swift 6.2 toolchain for CodeQL (openclaw#41787) fix(agents): forward memory flush write path (openclaw#41761) fix(telegram): move network fallback to resolver-scoped dispatchers (openclaw#40740) fix(security): harden replaceMarkers() to catch space/underscore boundary marker variants (openclaw#35983) fix(web-search): recover OpenRouter Perplexity citations from message annotations (openclaw#40881) ...
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus
|
I guess this pull request broke my bot on a Hetzer server. Telegram is super spotty since it and using an older version it works fine. |
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus (cherry picked from commit 45b74fb)
…penclaw#40740) Merged via squash. Prepared head SHA: a4456d4 Co-authored-by: sircrumpet <[email protected]> Co-authored-by: obviyus <[email protected]> Reviewed-by: @obviyus (cherry picked from commit 45b74fb)
Over the last week I saw a noticeable regression in Telegram reliability, with bot sends intermittently failing and logs filling up with
Network request for 'sendMessage' failedand repeated fallback messages.After digging into it, the core issue turned out not to be a single failing request path, but how Telegram networking state was being managed. Several parts of the Telegram stack could update shared network settings, which on my system led to
autoSelectFamilyrepeatedly flipping between true and false and caused fallback behaviour to be retried far more often than expected.That investigation surfaced three related problems:
This PR takes a more durable approach: it moves Telegram networking to resolver-scoped dispatchers, makes fallback behaviour consistent across all Telegram network paths, and adds a range of new tests to ensure consistent handling across Telegram network calls.
Summary
Describe the problem and fix in 2–5 bullets:
autoSelectFamily, DNS order, and undici global dispatcher), which made transport behavior vulnerable to reapplication across unrelated Telegram flows.ETIMEDOUT,EHOSTUNREACH), repeated fallback warnings, and inconsistent behaviour between polling, sends, probes, audits, and media downloads.src/telegram/fetch.tsto build resolver-scoped undici dispatchers instead of mutating process-global Telegram network stateinit.dispatcheracross retries, and prevented caller-owned dispatcher failures from arming sticky IPv4 fallbacktimeoutMsbudgetHistory / Context
This section documents why this PR is a core transport fix
Previous Telegram network handling (before this PR)
From late January through early March 2026, Telegram fetch behavior evolved into a process-global transport model:
2026-01-26(b861a0bd73): Telegram network hardening introduced process-levelautoSelectFamilyhandling.2026-02-16(c762bf71f6): Node 22 default moved toautoSelectFamily=true.2026-02-22(53adae9cec): added defaultdnsResultOrder=ipv4firstbehavior on Node 22+.2026-02-25(0078070680): added global undici dispatcher refresh to align withautoSelectFamily.2026-03-01(e6049345db) and2026-03-02(666a4763ee): added proxy-preservation behavior around global dispatcher replacement.2026-03-02(9c03f8be08, then493ebb915b): added retry path that forced IPv4 fallback on qualifying network errors.2026-03-02(c973b053a5): proxy-env plumbing refactor (same fallback semantics).2026-03-05(05fb16d151): global undici timeout hardening added another path that can reapply global dispatcher state.Why this became a concern
autoSelectFamily=true (default-node22)interleaved withautoSelectFamily=false (config), plus repeated fallback warnings) were consistent with that shared-state contention pattern.What this PR changes architecturally
net,dns, and undici dispatcher state.This is intended to supersede symptom-level fallback tweaks by removing the underlying shared-global-state coupling in Telegram fetch handling.
Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
User-visible / Behavior Changes
NO_PROXY-bypassed traffic.timeoutMsas an overall budget.Security Impact (required)
No)No)No)No)No)Yes, explain risk + mitigation: N/ARepro + Verification
Environment
Steps
Expected
Actual
Evidence
Attach at least one:
Before (production logs showing transport thrash):
After (this branch):
Focused regression checks (this branch):
Human Verification (required)
What you personally verified (not just CI), and how:
bunx vitest run src/telegram/fetch.test.ts src/telegram/send.proxy.test.ts src/telegram/proxy.test.ts src/telegram/network-config.test.tspassed.pnpm tsgopassed.pnpm format:checkpassed.pnpm buildpassed.init.dispatcher.ETIMEDOUT,EHOSTUNREACH) with repeated fallback warnings when not applied, and confirmed these were resolved once the patch was applied.Review Conversations
Compatibility / Migration
Yes)No)No)Failure Recovery (if this breaks)
src/telegram/fetch.ts,src/telegram/send.ts,src/telegram/probe.ts,src/telegram/bot.ts,src/telegram/bot-handlers.ts,src/telegram/bot/delivery.resolve-media.ts,src/telegram/audit-membership-runtime.ts, andextensions/telegram/src/channel.ts.Risks and Mitigations
AI Assisted: yes, Codex investigation, root cause analysis, tests, review, edge-case discovery