Skip to content

[Bug]: 2026.4.29 embedded agent reply latency ~40-47s before stream-ready; packaged channel deps missing #75650

Description

@jackiehuangtw

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

After upgrading to OpenClaw 2026.4.29, Telegram and Discord replies consistently wait ~39-47s before stream-ready while logs show repeated embedded-run prep delays and event-loop/cpu liveness warnings.

Steps to reproduce

  1. Upgrade to OpenClaw 2026.4.29.
  2. Run Gateway via systemd user service.
  3. Enable Telegram and Discord channels.
  4. Send normal Telegram/Discord direct messages.
  5. Observe user-visible delay and logs showing trace:embedded-run prep stages around 39-47s before stream-ready.
  6. Optionally run openclaw status --deep; before manual hotfix it may print missing grammy / discord-api-types/v10 warnings.

Expected behavior

After a user message is received, the agent should begin streaming/replying without a consistent 40+ second OpenClaw-side preparation delay. 2026.4.26 did not show the same user-visible delay in this setup.

Actual behavior

After upgrade, replies frequently wait tens of seconds before the model response begins. Representative logs repeatedly show embedded-run prep taking about 39-47s before phase=stream-ready, with recurrent Gateway event_loop_delay / cpu liveness warnings. Telegram direct and Discord direct/channel conversations are both affected.

OpenClaw version

2026.4.29 (a448042)

Operating system

Linux 6.17.0-22-generic x64; Node v22.22.2

Install method

Stable channel, pnpm/global package; Gateway running as systemd user service

Model

Default model openai-codex/gpt-5.5; Discord test also observed slow behavior with x-ai/grok-4.1-fast

Provider / routing chain

Local OpenClaw Gateway on 127.0.0.1:18789; Telegram + Discord channels enabled; no external reverse proxy in this test

Additional provider/model setup details

Channels enabled: Telegram + Discord. Telegram network config already uses autoSelectFamily=false and dnsResultOrder=verbatim from a prior Telegram IPv6/IPv4 workaround. The latency appears in both Telegram and Discord, so it does not look channel-specific.

Related public reports/PRs found while triaging:

Logs, screenshots, and evidence

Representative logs:


[trace:embedded-run] prep stages: phase=stream-ready totalMs=44202 stages=workspace-sandbox:2ms@2ms,skills:0ms@2ms,core-plugin-tools:15451ms@15453ms,bootstrap-context:11ms@15464ms,bundle-tools:2132ms@17596ms,system-prompt:10914ms@28510ms,session-resource-loader:5737ms@34247ms,agent-session:3ms@34250ms,stream-setup:9952ms@44202ms

[trace:embedded-run] prep stages: phase=stream-ready totalMs=47121 stages=workspace-sandbox:836ms@836ms,skills:0ms@836ms,core-plugin-tools:15702ms@16538ms,bootstrap-context:2278ms@18816ms,bundle-tools:2059ms@20875ms,system-prompt:10532ms@31407ms,session-resource-loader:5447ms@36854ms,agent-session:1ms@36855ms,stream-setup:10266ms@47121ms

liveness warning: reasons=event_loop_delay,event_loop_utilization,cpu interval=30s eventLoopDelayP99Ms=14294.2 eventLoopDelayMaxMs=14294.2 eventLoopUtilization=0.999 cpuCoreRatio=1.035 active=1 waiting=0 queued=0


Recent sample summary:


prep_total: n=28 min=39169 median=43098 max=47322 avg=43106
startup_total: n=27 min=14318 median=17234 max=34723 avg=20519
core_plugin_tools: n=28 min=15018 median=15405 max=15880 avg=15414
system_prompt: n=28 min=7250 median=10068 max=11549 avg=9499
stream_setup: n=28 min=8679 median=9718 max=10266 avg=9517
model_resolution: n=27 min=3648 median=4164 max=14914 avg=7156
auth: n=27 min=5333 median=6805 max=7381 avg=6720


Packaged dependency warnings observed before manual hotfix:


[channels] failed to load bundled channel discord: Cannot find module 'discord-api-types/v10'
Require stack:
- $GLOBAL_OPENCLAW/dist/extensions/discord/discord-*.js

[channels] failed to load bundled channel telegram: Cannot find module 'grammy'
Require stack:
- $GLOBAL_OPENCLAW/dist/extensions/telegram/send-*.js


Manual hotfix applied:


cd $GLOBAL_OPENCLAW
npm install --no-save [email protected] [email protected]


After the hotfix, module resolution succeeds:


grammy OK -> $GLOBAL_OPENCLAW/node_modules/grammy/out/mod.js
discord-api-types/v10 OK -> $GLOBAL_OPENCLAW/node_modules/discord-api-types/v10.js
json5 OK -> $GLOBAL_OPENCLAW/node_modules/json5/lib/index.js


However, reply latency persisted afterward; post-hotfix samples still show prep stages such as `totalMs=40597`, `44202`, and `47121`.

Impact and severity

Affected users/systems/channels: this setup's Telegram direct messages and Discord direct/channel messages.

Severity: high usability impact; normal chat replies feel delayed by ~40s before model output begins.

Frequency: repeated across many recent runs after the 2026.4.29 upgrade.

Consequence: messaging channels feel unreliable/slow, and the delay appears to be in OpenClaw/Gateway prep before provider streaming rather than only model latency.

Additional information

Other observations:

  • Gateway CPU is often high during/after runs. A status snapshot showed the Gateway node process around 40% CPU while idle-ish.
  • Recurrent liveness warnings include high event loop delay and event loop utilization.
  • There are Discord API fetch timeout / native slash command deploy 429 warnings, but the same slow response is visible on Telegram too, so this does not look like a single Discord-only bottleneck.
  • No session-write-lock / sessions.json.lock warnings were observed locally, so this may be adjacent to [Bug]: Windows startup and reply latency, session lock held for 191s on sessions.json.lock #70857 but not the exact same lock symptom.
  • Local sanitized evidence was collected as status.txt and relevant-log-summary.txt; I can provide additional redacted lines if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions