[Bug]: Active Memory timeoutMs clock starts at plugin level, not at LLM call — embedded run setup overhead causes 100% timeout

Summary

Active Memory's timeoutMs timer starts when the plugin begins its pre-reply hook, but the embedded agent run initialization (context building, workspace setup, skill resolution) consistently takes 18–19 seconds before the LLM call even begins. With the default timeoutMs: 15000, the abort signal fires before the embedded run starts, causing 100% timeout rate even though the actual LLM responds in ~1.5 seconds.

Environment

• OpenClaw: 2026.4.24
• OS: Linux 5.15.0-173-generic (x64)
• Runtime: Node v22.22.2, npm-global install
• Provider: amazon-bedrock (Bedrock Converse Stream API, auth: aws-sdk)
• Active Memory model: amazon-bedrock/amazon.nova-micro-v1:0
• Active Memory config: timeoutMs: 15000, queryMode: recent, promptStyle: contextual, bootstrapContextMode: lightweight (hardcoded in plugin)
• Skills: 50+ skills loaded (including 20+ lark-* symlink skills that trigger symlink-escape warnings)
• Agents with active-memory: 3 (main, cbz001, clawdoctor)

Root Cause Analysis

The timing problem

12:03:25.482  [plugins] active-memory: start timeoutMs=15000
                ↓ 19.0s — embedded run initialization
12:03:40.482  ⏰ 15s timeout expires, abort signal fires
                ↓ but embedded run is still initializing...
12:03:44.528  Embedded run session actually starts (4s after timeout!)
12:03:44.572  Prompt submitted to Nova Micro
12:03:46.351  Nova Micro responds in 1.8s, calls memory_search
12:03:46.352  memory_search returns "Aborted" (abort signal already fired)
12:03:47.768  Model returns "NONE" (can't search memory)
              → Plugin reports: status=timeout, elapsedMs=22328, summaryChars=0

Direct API test proves model is fine

Tested Nova Micro directly via AWS SDK (@aws-sdk/client-bedrock-runtime Converse API):

The model responds in 1–1.5 seconds. The 19-second overhead is entirely in embedded run initialization.

Evidence from trajectory files

The persisted trajectory (active-memory-*.trajectory.jsonl) confirms the embedded run itself is fast once it starts:

session.started:  04:03:44.528Z
context.compiled: 04:03:44.572Z  (44ms)
prompt.submitted: 04:03:44.572Z  (0ms)
model.completed:  04:03:44.670Z  (98ms for first LLM turn)

But the session JSONL shows a openclaw:prompt-error at 04:03:44.574Z with message "active-memory timeout after 15000ms" — the abort signal was already pending when the run started.

Statistics

24/24 active-memory runs timed out today (100% failure rate):

Total runs:     24
Timeouts:       24
Successes:      0
Timeout rate:   100%
Config timeout: 15,000ms
Actual elapsed: 19,062–31,207ms (avg ~21,227ms)
Setup overhead: 18,000–19,000ms (consistent)
Actual LLM:     ~1,500–3,200ms (when it gets to run)

Expected Behavior

timeoutMs should govern the LLM call duration, not include the embedded run initialization overhead. If initialization takes 19 seconds, a timeoutMs: 15000 should mean 19s setup + 15s LLM budget = 34s total, not timeout before the LLM call starts.

Alternatively, the initialization overhead should be dramatically reduced so it doesn't dominate the timeout window.

Observed Behavior

• timeoutMs timer starts at plugin hook entry
• Embedded run initialization takes 18–19 seconds
• By the time the LLM call begins, the abort signal has already fired
• memory_search tool calls return "Aborted"
• Model returns "NONE" (no useful memory recall)
• Plugin reports status=timeout with summaryChars=0

Contributing Factors

1. Skill resolution overhead: 20+ lark-* symlink skills trigger symlink-escape path checks during each embedded run, even though bootstrapContextMode: "lightweight" is set.

2. Blocking embedded run: The runEmbeddedPiAgent call appears to have significant setup overhead (context compilation, workspace resolution, agent directory setup) that runs synchronously before the LLM call.

Suggested Fix

One or more of:

1. Start timeoutMs clock at LLM call time, not plugin hook entry — the most impactful fix. The timeout should measure "how long we wait for the model," not "how long the entire pre-reply path takes."

2. Exclude skill resolution from embedded run initialization — Active Memory only uses memory_search and memory_get (via toolsAllow), so full skill catalog resolution is unnecessary.

3. Cache/reuse embedded run context across calls — if the same agent/session triggers active-memory repeatedly, the context compilation result could be cached.

4. Report setup overhead separately in logs — add setupMs to the done log line so users can distinguish initialization time from LLM time:

   [plugins] active-memory: done status=ok elapsedMs=22000 setupMs=19000 llmMs=3000 summaryChars=42
   

Workaround

Setting timeoutMs: 45000 (or higher) allows the embedded run to complete, but adds ~20 seconds of blocking latency to every reply, which defeats the purpose of active memory as a lightweight pre-reply enrichment.

Related Issues

• Bug: 2026.4.14 upgrade makes existing conversations fail broadly after repeated active-memory pre-reply timeouts #66849 — 2026.4.14 upgrade causes active-memory timeouts (closed)
• Active Memory plugin + qmd update chain times out at 120s on 4.15 #68825 — Active Memory + qmd chain timeout on 4.15 (closed as duplicate of Bug: 2026.4.14 upgrade makes existing conversations fail broadly after repeated active-memory pre-reply timeouts #66849)
• [Bug]: Active-memory embedded sub-agent run blocks event loop, starving Telegram polling — agent goes permanently unresponsive #65517 — Active-memory embedded sub-agent blocks event loop

Log Excerpts

Plugin start → timeout pattern (repeats for all 24 runs)

2026-04-27T12:03:25.482+08:00 [plugins] active-memory: agent=main session=agent:main:telegram:default:direct:REDACTED activeProvider=amazon-bedrock activeModel=amazon.nova-micro-v1:0 start timeoutMs=15000 queryChars=897
2026-04-27T12:03:47.808+08:00 [agent/embedded] embedded run failover decision: runId=active-memory-mogo9mzv-5b8ff721 stage=assistant decision=surface_error reason=timeout from=amazon-bedrock/amazon.nova-micro-v1:0 profile=-
2026-04-27T12:03:47.810+08:00 [plugins] active-memory: agent=main session=agent:main:telegram:default:direct:REDACTED activeProvider=amazon-bedrock activeModel=amazon.nova-micro-v1:0 done status=timeout elapsedMs=22328 summaryChars=0

Skill symlink-escape warnings (20+ per run, contributes to setup overhead)

2026-04-27T12:05:56.766+08:00 [skills] Skipping escaped skill path outside its configured root: source=openclaw-managed root=~/.openclaw/skills reason=symlink-escape requested=~/.openclaw/skills/lark-doc resolved=~/.agents/skills/lark-doc
(... 20+ similar lines for lark-attendance, lark-base, lark-calendar, etc.)