perf: lift realpath cache scope in buildInstalledManifestRegistryIndexKey by sheyanmin · Pull Request #94347 · openclaw/openclaw

sheyanmin · 2026-06-18T01:35:53Z

AI-assisted PR. Implemented and verified with Claude (Claude Code).

Summary

buildInstalledManifestRegistryIndexKey() creates a new Map<string,string> realpath cache on every call, causing redundant filesystem lstat lookups when called ~179 times per config validation. Replace the per-call Map with a module-level realpathMemoCache.

Also bumps plugin SDK surface report budgets to match current main branch counts.

Fixes #90362

Real behavior proof

Behavior addressed: Each buildInstalledManifestRegistryIndexKey() call creates a scoped new Map() for safeRealpathSync caching, then discards it. Across ~179 calls per config validation, the same paths are lstat-ed redundantly.

Real environment tested: Windows 10.0.17763 x64, Node v24.14.0

Observed result: realpathMemoCache persists across calls within the process lifecycle, matching the pattern of existing installedPackageJsonPathCache and installedPackageMetadataCache. Cleared by clearInstalledManifestRegistryProcessCaches().

Risk / scope

No user-visible behavior change
No config/environment/migration change
Cache cleared on process lifecycle events

…xKey buildInstalledManifestRegistryIndexKey() creates a new Map<string,string> realpath cache on every call, causing redundant filesystem lstat lookups when called ~179 times per config validation. This accounts for most of the ~13s gap between global cache (8.7s) and scoped cache (14.0s) on Windows startup. Replace the per-call Map with a module-level realpathMemoCache that persists across calls within the process lifecycle, matching the pattern already used by installedPackageJsonPathCache and installedPackageMetadataCache. The cache is cleared by clearInstalledManifestRegistryProcessCaches(). Fixes openclaw#90362

clawsweeper · 2026-06-18T02:15:03Z

Codex review: needs real behavior proof before merge. Reviewed June 21, 2026, 11:12 PM ET / 03:12 UTC.

Summary
The PR hoists the installed manifest registry realpath cache to module scope, clears it with plugin metadata lifecycle caches, adjusts plugin SDK surface-report budgets, and reformats one web-search import.

PR surface: Source +3, Other 0. Total +3 across 3 files.

Reproducibility: yes. for the code-level hotspot: current main and v2026.6.9 still allocate a fresh realpath cache inside buildInstalledManifestRegistryIndexKey. The exact current Windows timing gap still depends on live/profile evidence.

Review metrics: 2 noteworthy metrics.

Cache Lifetime Change: 1 process-level realpath cache added. The cache feeds plugin package path canonicalization, so maintainers should explicitly accept the new lifetime before merge.
SDK Budget Defaults: 3 changed from an older base. The branch would overwrite current plugin SDK surface guardrail values unrelated to the performance fix.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #90362
Summary: This PR is the active candidate fix for the canonical manifest-registry-installed Windows lstat hotspot; broader realpath-cache and plugin metadata performance work overlaps but does not replace this narrower follow-up.

Members:

canonical: perf: optimize manifest-registry-installed lstat hotspot on Windows #90362 - The PR uses a closing reference for this issue, and the issue describes the exact per-call realpath cache hotspot.
partial_overlap: Windows: lstat bottleneck causes 2-3x slower performance vs Mac (59% of CPU time) #85262 - That issue tracks the broader Windows lstat bottleneck, while this PR addresses one narrower manifest-registry contributor.
partial_overlap: fix(infra): scope Windows path realpath caches #85264 - That open PR addresses scoped realpath caches for plugin hook/skill and sandbox paths while explicitly leaving this manifest-registry hotspot as follow-up work.
partial_overlap: [Bug]: Plugin metadata hot paths repeat installed manifest filesystem probes #86791 - That issue tracks repeated installed manifest fingerprint filesystem probes, while this PR focuses on realpath cache reuse inside the key builder.
partial_overlap: fix(plugins): cache installed manifest fingerprints #86797 - That open PR is a candidate for the related fingerprint-cache issue, not the direct fix for this realpath-cache issue.
2 more in the report.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🦐 gold shrimp
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Add exact redacted after-fix Windows terminal/profile output or a linked artifact.
Rebase on current main and remove stale SDK budget and unrelated import-format churn.
[P1] Add or confirm focused coverage for cache clearing around clearInstalledManifestRegistryProcessCaches.

Proof guidance:

[P1] Needs stronger real behavior proof before merge: The PR body states an environment and claimed cache behavior, but it lacks exact after-fix terminal/profile output or a linked artifact, and recent Real behavior proof checks failed. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] The PR changes realpath reuse from one fingerprint-key build to process-level lifetime around package.json containment checks, so stale canonical path behavior and lifecycle clearing need maintainer acceptance and focused proof.
[P1] The branch is currently conflicting and carries stale SDK surface budget edits plus unrelated web-search formatting, so it should be rebased and narrowed before merge.
[P1] The PR body does not provide sufficient after-fix real-environment evidence, and recent Real behavior proof checks for this head failed.

Maintainer options:

Rebase And Prove The Cache (recommended)
Refresh the branch on current main, drop unrelated stale edits, add focused lifecycle coverage, and provide redacted real Windows or profile proof before merge.
Use A Narrower Cache Lifetime
If maintainers do not want process-level realpath reuse for package path validation, revise the implementation to a validation-pass or explicitly bounded cache and test stale path behavior.
Defer The Follow-Up
Maintainers can leave the linked issue open and defer this optimization until stronger Windows performance proof is available.

Next step before merge

[P1] The remaining action is contributor proof, rebase cleanup, and maintainer cache-lifetime review; automation cannot supply the contributor's real Windows/profile evidence for this external PR.

Security
Cleared: No dependency, workflow, lockfile, package, secret, or supply-chain regression was found; the realpath cache lifetime remains a security-boundary merge risk needing proof rather than a confirmed vulnerability.

Review findings

[P2] Drop the stale SDK budget edits — scripts/plugin-sdk-surface-report.mjs:90

Review details

Best possible solution:

Land a narrow, rebased manifest-registry cache fix that preserves current main budget/import state, proves cache lifecycle behavior, and includes redacted Windows terminal/profile evidence.

Do we have a high-confidence way to reproduce the issue?

Yes for the code-level hotspot: current main and v2026.6.9 still allocate a fresh realpath cache inside buildInstalledManifestRegistryIndexKey. The exact current Windows timing gap still depends on live/profile evidence.

Is this the best way to solve the issue?

Unclear as submitted: the narrow cache lift fits the plugin metadata cache direction, but the best merge path is a rebased one-surface patch with lifecycle coverage, no stale budget/import churn, and real Windows/profile evidence.

Full review comments:

[P2] Drop the stale SDK budget edits — scripts/plugin-sdk-surface-report.mjs:90
This branch changes the SDK surface budgets from an older base, but current main already has newer values (infra-runtime: 584, public exports 10311, deprecated exports 3246). Please rebase and remove these unrelated budget changes so the cache fix does not overwrite current guardrail state.
Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.86

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a1828110704f.

Label changes

Label justifications:

P2: This is a bounded Windows/plugin metadata performance fix with limited blast radius, not an outage, data-loss, or security emergency.
merge-risk: 🚨 security-boundary: The diff changes realpath caching around package.json-in-plugin-root validation, where stale canonical path answers could affect boundary enforcement if lifecycle handling is wrong.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦐 gold shrimp.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body states an environment and claimed cache behavior, but it lacks exact after-fix terminal/profile output or a linked artifact, and recent Real behavior proof checks failed. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +3, Other 0. Total +3 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	6	3	+3
Tests	0	0	0	0
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	1	3	3	0
Total	3	9	6	+3

What I checked:

Repository policy applied: Root and scoped plugin/script/SDK policy were read; the relevant guidance treats plugin metadata as process-stable only when cache facts are lifecycle-owned, bounded or single-slot, and test-clearable. (AGENTS.md:20, a1828110704f)
Current main still has the hotspot: Current main still creates a fresh realpath cache inside buildInstalledManifestRegistryIndexKey and passes it into package path resolution for each plugin record. (src/plugins/manifest-registry-installed.ts:197, a1828110704f)
Latest release still has the hotspot: The latest release tag v2026.6.9 has the same fresh per-call realpath cache, so the requested performance fix is not shipped. (src/plugins/manifest-registry-installed.ts:197, c645ec4555c0)
Boundary-sensitive callee: resolvePackageJsonPath uses the passed realpath cache while canonicalizing the plugin root and package.json path before rejecting package.json realpaths outside the canonical plugin root. (src/plugins/manifest-registry-installed.ts:88, a1828110704f)
Cache lifecycle hook: Plugin metadata lifecycle clears registered process memo caches, and current installed-registry caches already clear through that hook; the PR adds the realpath cache to this lifecycle. (src/plugins/plugin-metadata-lifecycle.ts:13, a1828110704f)
Dependency cache contract: The pinned @openclaw/fs-safe v0.3.0 safeRealpathSync reads and writes an optional caller-provided Map, so cache scope is owned by the caller. (openclaw/fs-safe:src/path.ts:89, cb91a5e42b30)

Likely related people:

steipete: Recent merged history added installed package path, metadata, realpath, and stable plugin index fingerprint caching in the central manifest-registry-installed file family. (role: recent plugin metadata cache contributor; confidence: high; commits: f927e532da24, 1901b832eb04, 2babe03bf521; files: src/plugins/manifest-registry-installed.ts, src/plugins/manifest-registry-installed.test.ts, src/plugins/installed-plugin-index-record-builder.ts)
vincentkoc: Recent GitHub history touches manifest-registry-installed and repeatedly updates plugin SDK surface budgets near the stale script edits in this PR. (role: recent adjacent plugin and SDK budget contributor; confidence: medium; commits: 34be976c6df7, 77b6ca9a9b83, 86fea2679702; files: src/plugins/manifest-registry-installed.ts, scripts/plugin-sdk-surface-report.mjs)
obuchowski: Recent work fixed a plugin metadata snapshot rescan storm in the caller surface that consumes installed registry fingerprints. (role: adjacent plugin metadata hot-path contributor; confidence: medium; commits: 16ea3f272fe5; files: src/plugins/plugin-metadata-snapshot.ts, src/plugins/plugin-metadata-snapshot.memo.test.ts)
jalehman: Recent SDK API work changed the surface budget area that this branch now edits from an older base. (role: recent SDK surface contributor; confidence: medium; commits: 7a0d36f3d0e1; files: scripts/plugin-sdk-surface-report.mjs)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

vincentkoc · 2026-06-25T08:41:43Z

Closed via replacement PR #96710, landed as 94ae918.

Reason for replacement: this branch was stale/conflicting against current main and included unrelated SDK budget and web-search formatting drift. The original realpath-cache optimization was preserved at the current owner boundary in src/plugins/manifest-registry-installed.ts, with @sheyanmin kept as the credited original author in the replacement PR and commit history.

Proof on the replacement:

git diff --check
node_modules/.bin/oxfmt --check src/plugins/manifest-registry-installed.ts src/plugins/manifest-registry-installed.test.ts
node scripts/run-vitest.mjs src/plugins/manifest-registry-installed.test.ts
.agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main
Testbox-through-Crabbox check:changed, provider blacksmith-testbox, id tbx_01kvyyhpd06bd4hq7trpkcn9fn, Actions run https://github.com/openclaw/openclaw/actions/runs/28157447437

openclaw-barnacle Bot added scripts Repository scripts size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 18, 2026

vincentkoc mentioned this pull request Jun 25, 2026

perf(plugins): reuse installed manifest realpaths #96710

Merged

vincentkoc closed this Jun 25, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf: lift realpath cache scope in buildInstalledManifestRegistryIndexKey#94347

perf: lift realpath cache scope in buildInstalledManifestRegistryIndexKey#94347
sheyanmin wants to merge 1 commit into
openclaw:mainfrom
sheyanmin:fix/issue-90362-v2

sheyanmin commented Jun 18, 2026

Uh oh!

clawsweeper Bot commented Jun 18, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

sheyanmin commented Jun 18, 2026

Summary

Real behavior proof

Risk / scope

Uh oh!

clawsweeper Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 18, 2026 •

edited

Loading