Skip to content

[Bug]: 2026.4.29 gateway instability: Active Memory timeouts, embedded-run prep latency, event-loop pressure #76212

Description

@harleymdsavage

Bug type

Regression (worked better before, now unstable/slow)

Summary

After upgrading to OpenClaw 2026.4.29, the gateway became noticeably unstable and slow. The main symptoms are high per-turn latency, Active Memory timeouts, repeated embedded-run startup/prep overhead, event-loop delay warnings, and slow control/session RPCs. Temporarily disabling Active Memory did not resolve the overall instability, so Active Memory appears to be affected by the broader embedded-run/gateway performance regression rather than being the only root cause.

This looks related to the current 2026.4.29 regression cluster, especially #76123, #76166, #76047, #76048, and the superseded Active Memory report #76043.

Environment

  • OpenClaw: 2026.4.29 (a448042)
  • Install/update channel: stable / npm latest
  • OS: Ubuntu Linux x64
  • Node: v22.22.2
  • Gateway: systemd service, local loopback gateway
  • Primary model route: openai-codex/gpt-5.5
  • Active Memory model route: zai/glm-5.1
  • Telegram direct channel is enabled and generally reachable

Observed behavior

  1. Normal direct-message turns are slow or appear stuck.
  2. Active Memory runs time out during prompt preparation / hidden embedded runs.
  3. Gateway logs show large embedded-run startup/prep costs before model output begins.
  4. Gateway logs show event-loop/liveness warnings under normal use.
  5. sessions.list and node.list calls are repeatedly slow enough to show up in logs and likely contribute to gateway pressure.
  6. Disabling Active Memory alone did not fix the broader instability.
  7. Telegram health reports OK, but there are transient outbound Bot API failures such as sendChatAction network failures.

Local evidence / log excerpts

Representative gateway log excerpts, sanitized:

[plugins] active-memory: agent=main session=<direct-session> start timeoutMs=30000 model=zai/glm-5.1
[plugins] [hooks] before_prompt_build handler from active-memory failed: timed out after 60000ms
[diagnostic] lane task error: lane=main durationMs=60719 error="CommandLaneTaskTimeoutError: Command lane \"main\" task timed out after 60000ms"
[plugins] active-memory: agent=main session=<direct-session> done status=timeout elapsedMs=60825 summaryChars=0
[agent/embedded] embedded run failover decision: stage=assistant decision=surface_error reason=timeout from=zai/glm-5.1

Embedded-run startup/prep examples:

[trace:embedded-run] startup stages: phase=attempt-dispatch totalMs=16889 stages=runtime-plugins:9907ms,model-resolution:1763ms,auth:2627ms,attempt-dispatch:2591ms
[trace:embedded-run] prep stages: phase=stream-ready totalMs=20049 stages=core-plugin-tools:9575ms,bundle-tools:877ms,system-prompt:4323ms,session-resource-loader:894ms,stream-setup:4309ms

Event-loop/liveness example:

[diagnostic] liveness warning: reasons=event_loop_delay,event_loop_utilization,cpu eventLoopDelayP99Ms=9412 eventLoopUtilization=0.995 cpuCoreRatio=1.088

Slow RPC examples:

[ws] res sessions.list ~2180ms-2210ms
[ws] res node.list ~2180ms-2640ms

Telegram transient outbound failure example:

[telegram] sendChatAction failed: Network request for 'sendChatAction' failed

Active Memory config details

Active Memory is enabled only for the main agent and direct chats. The local config is already reduced/conservative:

{
  "enabled": true,
  "agents": ["main"],
  "allowedChatTypes": ["direct"],
  "model": "zai/glm-5.1",
  "modelFallback": "zai/glm-5.1",
  "queryMode": "message",
  "promptStyle": "preference-only",
  "timeoutMs": 30000,
  "maxSummaryChars": 120,
  "circuitBreakerMaxTimeouts": 1,
  "circuitBreakerCooldownMs": 600000,
  "recentUserTurns": 0,
  "recentAssistantTurns": 0,
  "recentUserChars": 300,
  "recentAssistantChars": 40,
  "cacheTtlMs": 120000
}

Expected behavior

  • Simple direct-message turns should not spend tens of seconds in embedded-run prep before reaching the model.
  • Active Memory should either return bounded recall quickly or degrade gracefully without blocking the user-visible turn for ~60s.
  • sessions.list / node.list should not materially contribute to gateway event-loop pressure during normal use.
  • Transient Telegram typing/send failures should not leave the system feeling wedged.

Related issues / possible overlap

Notes

This report is intentionally sanitized and omits hostnames, domains, usernames, chat identifiers, local filesystem paths, and tokens.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions