-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
[Bug]: event-loop starvation delays fetch timeouts during gateway load #84835
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.bugSomething isn't workingSomething isn't workingclawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.maintainerMaintainer-authored PRMaintainer-authored PRmaturity:stableIssue affects a taxonomy feature currently scored M4/M5.Issue affects a taxonomy feature currently scored M4/M5.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.bugSomething isn't workingSomething isn't workingclawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.maintainerMaintainer-authored PRMaintainer-authored PRmaturity:stableIssue affects a taxonomy feature currently scored M4/M5.Issue affects a taxonomy feature currently scored M4/M5.
Type
Fields
Priority
None yet
Bug type
Behavior bug (incorrect output/state without crash)
Beta release blocker
No
Summary
A fetch timeout configured for 10s fired after 17.175s while the gateway process was CPU-saturated, so timeout drift from event-loop starvation was only visible as an unstructured log line.
Steps to reproduce
fetchWithTimeoutwith a 10s timeout.Expected behavior
When a fetch timeout fires late enough to indicate event-loop starvation, the gateway should surface a structured diagnostic event with timeout, elapsed, and timer-delay fields so diagnostics/stability consumers can correlate timeout drift with gateway pressure.
Actual behavior
The live log reported a 10s timeout firing after 17.175s and included
timerDelayMs=7175, but this was only attached to the fetch-timeout log record. There was no dedicated structured diagnostic event for the delayed timeout.OpenClaw version
Development checkout at
79be9401306f9e6c18937850b32629f0c3029fa4.Operating system
Linux environment with
pidstat; exact distribution/version: NOT_ENOUGH_INFOInstall method
Development gateway; exact launch method: NOT_ENOUGH_INFO
Model
NOT_ENOUGH_INFO
Provider / routing chain
Telegram API request through
fetchWithTimeout; full provider/model route: NOT_ENOUGH_INFOAdditional provider/model setup details
NOT_ENOUGH_INFO
Logs, screenshots, and evidence
Impact and severity
Affected: Gateway network requests and channel/provider health checks that rely on timer-based fetch timeouts.
Severity: High; configured timeout budgets cannot be enforced reliably when the event loop is saturated.
Frequency: Observed once in the 2026-05-21 05:07-05:08 pidstat window.
Consequence: user-visible failures, reconnects, and health checks can appear several seconds later than intended, and support diagnostics cannot query delayed timeout events directly.
Additional information
Related historical issue: #78695 reported larger event-loop starvation symptoms. This report is for the narrower missing structured diagnostic event when
fetchWithTimeoutitself detects timeout timer drift.