perf: skip packageManager.resolve() by loading extensionFactories directly by sunvember · Pull Request #82536 · openclaw/openclaw

sunvember · 2026-05-16T10:26:46Z

Summary

This PR eliminates the 5-9 second packageManager.resolve() overhead on every embedded run by skipping reload() and loading inline extensionFactories directly.

Problem

The previous approach (PR #82523, now closed) attempted to cache DefaultResourceLoader instances, but was fundamentally flawed:

Bottleneck misdiagnosis: The overhead is NOT in the constructor (which only assigns properties in milliseconds), but in reload() → packageManager.resolve()
Caching ineffective: Callers always invoke await loader.reload(), so caching doesn't avoid the resolve() overhead
Stale closure bug: Cached loader's extensionFactories closure captured stale values from the first run

Root Cause Analysis

packageManager.resolve() scans:

~/.pi/extensions/skills/prompts/themes
<cwd>/.pi/extensions/skills/prompts/themes
~/.agents/skills
All ancestor .agents/skills directories up to git repo root

When OpenClaw's embedded runs use all no* flags true (noExtensions, noSkills, noPromptTemplates, noThemes, noContextFiles):

resolve() executes all filesystem scans (expensive, 5-9 seconds)
Results are discarded → empty arrays for skills/extensions/prompts/themes
Only loadExtensionFactories() produces useful output (inline extensions for compaction safeguards)

Key insight: The DefaultResourceLoader constructor already initializes the state that matches reload() output when no* flags are true:

extensionsResult = { extensions: [], errors: [], runtime: createExtensionRuntime() }
skills = [], prompts = [], themes = [], agentsFiles = []

The only difference is systemPrompt = undefined, but OpenClaw overrides this via applySystemPromptOverrideToSession() anyway.

Solution

Instead of caching loaders, we:

Create a new loader with no* flags true (milliseconds)
Call loadExtensionFactories(runtime) directly (public method, async but fast)
Skip reload() entirely

// Before (5-9s overhead)
const loader = new DefaultResourceLoader({...});
await loader.reload(); // ← resolve() scans many directories

// After (~milliseconds)
const loader = new DefaultResourceLoader({...});
const runtime = loader.extensionsResult.runtime;
const inline = await loader.loadExtensionFactories(runtime); // ← direct call
loader.extensionsResult.extensions.push(...inline.extensions);

Changes

File	Change
`resource-loader.ts`	`createEmbeddedPiResourceLoader` is now async, loads extensionFactories directly, no caching
`attempt.ts`	`await createEmbeddedPiResourceLoader()` + removed `await loader.reload()`
`compact.ts`	Same pattern
`resource-loader.test.ts`	Rewritten for new behavior

Performance Impact

Before: 5-9 seconds overhead per embedded run (filesystem scans)
After: ~milliseconds (only async extension factory loading)

Safety

systemPrompt being undefined is safe because OpenClaw overrides it via applySystemPromptOverrideToSession()
Compaction guards (applyPiCompactionSettingsFromConfig, applyPiAutoCompactionGuard) still execute after loader creation
Inline extensions for compaction safeguards are loaded correctly via loadExtensionFactories()

…oad overhead DefaultResourceLoader construction triggers packageManager.resolve() which scans skills directories and blocks the event loop for 5-9 seconds on every embedded run. This is pure overhead when the same workspace/agent is reused across consecutive runs (normal session flow). Solution: - Add in-memory cache keyed by (cwd, agentDir) in resource-loader.ts - Cache TTL defaults to 60s, configurable via OPENCLAW_RESOURCE_LOADER_CACHE_TTL_MS - add markResourceLoaderReloaded() to track reload completion - Add pruneResourceLoaderCache() for periodic cleanup - Add invalidateResourceLoaderCache() for targeted or full cache invalidation - Call markResourceLoaderReloaded in attempt.ts and compact.ts after reload() Files changed: - resource-loader.ts: cache logic - resource-loader.test.ts: 10 new test cases - attempt.ts: call markResourceLoaderReloaded after reload - compact.ts: call markResourceLoaderReloaded after reload AI-assisted development

…ectly ## Problem The previous PR (openclaw#82523) attempted to cache DefaultResourceLoader instances, but this approach was fundamentally flawed because: 1. The bottleneck is NOT in the constructor (which only assigns properties) 2. The bottleneck is in `reload()` → `packageManager.resolve()` 3. `resolve()` scans many directories (~/.pi/*, ~/.agents/skills, ancestor .agents/skills up to git repo root) 4. Caching the loader doesn't help because callers always call `await loader.reload()` 5. The cached loader's extensionFactories closure captured stale values ## Root Cause Analysis When all `no*` flags are true: - `resolve()` still executes all filesystem scans (expensive) - But the results are discarded (empty arrays for skills/extensions/prompts/themes) - Only `loadExtensionFactories()` produces useful output The `DefaultResourceLoader` constructor initializes: - `extensionsResult = { extensions: [], errors: [], runtime: createExtensionRuntime() }` - `skills = [], prompts = [], themes = [], agentsFiles = []` - `systemPrompt = undefined` These match `reload()` output when `no*` flags are true, except `systemPrompt`. OpenClaw overrides `systemPrompt` via `applySystemPromptOverrideToSession()`, so the undefined value is harmless. ## Solution Instead of caching loaders, we: 1. Create a new loader with `no*` flags true 2. Call `loadExtensionFactories()` directly (public method) 3. Skip `reload()` entirely This eliminates the 5-9 second `resolve()` overhead while still loading inline extensions needed for compaction safeguards and tool middleware. ## Changes - `resource-loader.ts`: Changed to async function, loads extensionFactories directly without calling reload() - `attempt.ts`: Use `await createEmbeddedPiResourceLoader()` + remove reload() - `compact.ts`: Same pattern - `resource-loader.test.ts`: Rewritten to match new behavior ## Performance Impact Before: 5-9 seconds overhead per embedded run After: ~milliseconds (only extension factory loading, no filesystem scans) Files changed: - src/agents/pi-embedded-runner/resource-loader.ts - src/agents/pi-embedded-runner/resource-loader.test.ts - src/agents/pi-embedded-runner/compact.ts - src/agents/pi-embedded-runner/run/attempt.ts

clawsweeper · 2026-05-16T10:32:15Z

Thanks for the contribution. I reviewed the branch, and this PR is not a good landing base for OpenClaw.

Close: the performance problem is still real, but this branch is no longer a viable landing candidate because it patches the retired Pi runner path, bypasses private loader internals, and still lacks after-fix real behavior proof.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #79899
Summary: This PR is a candidate fix for the canonical DefaultResourceLoader.reload() event-loop stall issue, but the branch is stale and not a safe landing path.

Members:

canonical: DefaultResourceLoader.reload() blocks event loop for 12-15s on Windows due to synchronous filesystem scanning #79899 - The open issue specifically tracks DefaultResourceLoader.reload() blocking due to synchronous resource discovery and links this PR as related.
superseded: perf: cache resourceLoader across embedded runs to eliminate 5-9s reload overhead #82523 - The prior closed PR attempted a cache-based fix for the same loader overhead and was explicitly superseded by this direct-load attempt.
duplicate: session-resource-loader blocks event loop 5-7s on every agent run (synchronous file I/O) #82384 - The closed issue reports the same per-run session-resource-loader synchronous file-I/O overhead.
partial_overlap: fix(sessions): cache warm transcript reads to avoid per-turn re-parse #90412 - The merged PR fixed warm transcript re-parse growth in the same prep-stage family but explicitly did not address the constant reload() baseline cost.
adjacent_distinct: [2026.6.1 regression] bundle-tools prep stage 5–7s → 21–27s (3–4×) with 3 stdio MCP servers #91375 - The open issue is another prep-stage latency regression, but its reported bottleneck is bundle-tools with MCP servers rather than resource-loader reload.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

So I’m closing this PR rather than keeping an unmergeable branch open. A new narrow PR that carries only the useful part is welcome.

Review details

Best possible solution:

Keep #79899 as the canonical work item and replace this stale branch with a fresh current-runner PR that adds a loader-owned fast path with real timing proof.

Do we have a high-confidence way to reproduce the issue?

No, not for the exact Windows or large-tree timing magnitude. Source inspection does show the current and latest-release path still calls resourceLoader.reload() and packageManager.resolve().

Is this the best way to solve the issue?

No. The performance direction is plausible, but this PR patches retired files and bypasses private loader internals instead of adding a supported fast path on the current runner.

Security review:

Security review cleared: The diff changes agent runtime TypeScript and tests only; no dependency, workflow, secret, install, package, or supply-chain surface is changed.

AGENTS.md: found and applied where relevant.

What I checked:

Repository policy read: Root AGENTS.md and scoped src/agents/AGENTS.md / src/agents/embedded-agent-runner/run/AGENTS.md were read fully; the review applied the repo guidance to inspect current code, sibling paths, dependency/source contracts, and performance proof before verdict. (AGENTS.md:25, 6c2c43d63f4c)
PR diff targets retired path: The live PR diff changes four src/agents/pi-embedded-runner/** files, while current main no longer has that subtree. (src/agents/pi-embedded-runner/run/attempt.ts:1842, 00816c0eed71)
Current main uses embedded-agent runner: Current main has src/agents/embedded-agent-runner/{resource-loader.ts,compact.ts,run/attempt.ts} and no src/agents/pi-embedded-runner tree, so the submitted files are not the live runtime surface. (src/agents/embedded-agent-runner/resource-loader.ts:24, 6c2c43d63f4c)
Current attempt path still reloads: The live embedded attempt path constructs createEmbeddedAgentResourceLoader and still awaits resourceLoader.reload(), so the central issue remains but this branch does not touch the current code path. (src/agents/embedded-agent-runner/run/attempt.ts:2301, 6c2c43d63f4c)
Current compaction path still reloads: The embedded compaction path uses the same current resource-loader helper and awaits resourceLoader.reload(), so a real fix must cover both normal attempts and compaction. (src/agents/embedded-agent-runner/compact.ts:1219, 6c2c43d63f4c)
Reload contract is broader than inline factories: DefaultResourceLoader.reload() reloads settings, calls packageManager.resolve(), resolves CLI extension sources, loads inline factories, detects extension conflicts, applies source info, and builds skills/prompts/themes/context state; the PR shortcut bypasses that contract through private members. (src/agents/sessions/resource-loader.ts:353, 6c2c43d63f4c)

Likely related people:

vincentkoc: Current blame for the live embedded-agent resource-loader, attempt reload call, compaction reload call, and session loader implementation points to the recent current-main path refresh commit. (role: recent current-main contributor; confidence: medium; commits: ab165d119cc7; files: src/agents/embedded-agent-runner/resource-loader.ts, src/agents/embedded-agent-runner/run/attempt.ts, src/agents/embedded-agent-runner/compact.ts)
steipete: Prior ClawSweeper history in the PR discussion ties the earlier embedded no-discovery loader shape and the Pi-to-embedded-agent rename to this account's commits, making them relevant for owner routing. (role: feature-history owner; confidence: medium; commits: a3c7fea5126e, bb46b79d3c14; files: src/agents/pi-embedded-runner/resource-loader.ts, src/agents/pi-embedded-runner/run/attempt.ts, src/agents/pi-embedded-runner/compact.ts)
ly85206559: The related merged compaction PR preserved compaction settings across resource-loader reload, which is one semantic any supported fast path must keep. (role: adjacent reload-preservation contributor; confidence: medium; commits: 55d9b6c2a600, c696718fc46c; files: src/agents/pi-embedded-runner/run/attempt.ts, src/agents/pi-embedded-runner/compact.ts)
Alix-007: The merged warm transcript cache PR fixed another session-resource-loader subpath and explicitly left the constant loader reload() baseline cost for the canonical issue/PR thread. (role: adjacent performance fix contributor; confidence: medium; commits: e4473025adb3, a0b16f37e835; files: src/agents/sessions/session-manager.ts, src/agents/embedded-agent-runner/session-manager-init.ts)

Codex review notes: model internal, reasoning high; reviewed against 6c2c43d63f4c.

clawsweeper · 2026-06-21T01:34:02Z

ClawSweeper applied the proposed close for this PR.

Action: closed this PR.
Close reason: low-signal unmergeable PR.
Evidence: durable ClawSweeper review.

sunvember added 2 commits May 16, 2026 17:18

openclaw-barnacle Bot added agents Agent runtime and tooling size: M triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 16, 2026

Pukerud mentioned this pull request May 17, 2026

DefaultResourceLoader.reload() blocks event loop for 12-15s on Windows due to synchronous filesystem scanning #79899

Closed

Alix-007 mentioned this pull request Jun 4, 2026

fix(sessions): cache warm transcript reads to avoid per-turn re-parse #90412

Merged

clawsweeper Bot mentioned this pull request Jun 14, 2026

Windows + Feishu DM becomes very slow after upgrade to 2026.5.2; latency appears in agent/session processing, likely amplified by large session context #76702

Closed

clawsweeper Bot closed this Jun 21, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf: skip packageManager.resolve() by loading extensionFactories directly#82536

perf: skip packageManager.resolve() by loading extensionFactories directly#82536
sunvember wants to merge 2 commits into
openclaw:mainfrom
sunvember:perf/skip-resolve-direct-load-extensions

sunvember commented May 16, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 16, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

sunvember commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Root Cause Analysis

Solution

Changes

Performance Impact

Safety

Related

Uh oh!

clawsweeper Bot commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jun 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sunvember commented May 16, 2026 •

edited

Loading

clawsweeper Bot commented May 16, 2026 •

edited

Loading