fix(infra): scope Windows path realpath caches#85264
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 6, 2026, 4:09 AM ET / 08:09 UTC. Summary PR surface: Source +77, Tests +127. Total +204 across 10 files. Reproducibility: yes. source-reproducible with supplied live evidence: current main still uses uncached helper/call-site shapes, and the issue/PR include Windows CPU-profile and Crabbox evidence for the lstat-heavy path. I did not rerun native Windows profiling in this read-only review. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Maintainer decision needed
Security Review detailsBest possible solution: Land this PR, or a maintainer replacement, after explicit security-boundary acceptance of the operation-scoped cache lifetime while keeping the manifest-registry optimization separate as already handled by #96710. Do we have a high-confidence way to reproduce the issue? Yes, source-reproducible with supplied live evidence: current main still uses uncached helper/call-site shapes, and the issue/PR include Windows CPU-profile and Crabbox evidence for the lstat-heavy path. I did not rerun native Windows profiling in this read-only review. Is this the best way to solve the issue? Yes, with maintainer sign-off: operation-scoped caches are the narrow maintainable alternative to the original process-wide cache proposal, and the remaining manifest-registry hotspot is already handled by a separate merged PR. The unresolved choice is security-boundary acceptance, not a code defect I can support as a finding. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 535ab05dcd88. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +77, Tests +127. Total +204 across 10 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (2 earlier review cycles) |
|
ClawSweeper PR egg 🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat. Where did the egg go?
|
a8f8cd9 to
dd1631b
Compare
|
Heads up: this PR needs to be updated against current |
0517140 to
807dd1c
Compare
807dd1c to
e6a2e83
Compare
|
@vincentkoc Hi! The latest force-push introduced 9 CI failures (check-prod-types, check-lint, check-test-types, check-guards, etc.). It looks like the re-export change in |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
51d9c02 to
2c3031b
Compare
|
@steipete Hi Peter! I noticed you recently merged #86517 which caches plugin package realpaths in Our PR here takes a complementary approach: scoping realpath/existing-ancestor caches per-operation in the plugin hook/skill resolution and sandbox host-path validation paths. Since you're familiar with this area and the security-boundary tradeoffs, would you be able to take a look when you have a moment? CI is all green now (131 checks passing), and ClawSweeper rates it 🐚 platinum hermit with 🦞 diamond lobster proof. The main blocker is explicit maintainer/security acceptance for the operation-scoped cache lifetime. Thanks! |
|
This assigned pull request has been automatically marked as stale after being open for 27 days. |
|
Still relevant — this fix is a prerequisite for #90362. Keeping open while review is in progress. |
|
This assigned pull request has been automatically marked as stale after being open for 27 days. |
1 similar comment
|
This assigned pull request has been automatically marked as stale after being open for 27 days. |
Move realpath cache boundary from process-wide to caller-owned scopes to prevent stale symlink/ancestor answers in security-sensitive containment checks while maintaining performance. Changes: - Add hostPathCache to fs-paths.ts mount resolution - Accept cache parameter in resolvePathViaExistingAncestorSync - Create per-operation caches in plugin hooks and skills resolution - Use scoped caches in sandbox host-path validation - Fix boundary-path.ts export conflict (remove duplicate export) - Add regression tests for stale symlink/junction detection Performance: 55% reduction in lstat time on Windows (21,376ms vs 47,834ms baseline) Fixes openclaw#85262
|
This assigned pull request has been automatically marked as stale after being open for 27 days. |
|
Closing due to inactivity. |
Summary
Windows
lstat/realpath calls are expensive enough that repeated path-boundary checks can dominate startup and agent-path flows. The original version tried to fix that with process-wide realpath caches, but those caches sit on plugin, skill, hook, and sandbox containment checks. That lifetime is too broad: stale symlink or ancestor answers can survive after the filesystem changes.This revision keeps the performance direction while moving the cache boundary to caller-owned operation scopes:
src/infra/path-guards.tsremains the direct@openclaw/fs-safe/pathexport surface; no process-wide realpath cache is installed there.src/infra/boundary-path.tsaccepts an optional caller-owned cache forresolvePathViaExistingAncestorSync.Closes #85262
Real behavior proof
Behavior addressed: reduces repeated Windows path canonicalization in plugin hook/skill and sandbox host-path operations without adding stale process-wide path answers to security-sensitive containment checks.
Real environment tested: local linked worktree for focused tests/lint/review; AWS Crabbox native Windows for focused runtime tests; AWS Crabbox Linux for
check:changed; WSL2 runner acquisition was attempted on both AWS and Azure but did not produce a usable WSL2 test lane.Exact steps or command run after this patch:
Evidence after fix:
483fc336ad: 5 shards passed, 114 tests.483fc336ad: passed.483fc336ad: clean, no accepted/actionable findings.provider=aws,lease=cbx_dc20a584615f,run=run_b4c210ac8166, hydrate run https://github.com/openclaw/openclaw/actions/runs/26719083996. Focused Vitest passed 5 shards: 111 passed, 3 skipped.provider=aws,lease=cbx_ab9a2c3903d5,run=run_4137bbc8c85d,pnpm check:changedpassed.Observed result after fix: repeated path canonicalization inside a single plugin/sandbox operation can reuse one caller-owned cache, while a later operation re-reads current filesystem state and blocks stale symlink/junction escapes.
What was not tested: full end-to-end Windows startup profiling at the current rebased head
483fc336ad(performance data was collected at prior heads; the rebase was conflict-resolution only with no logic changes). The manifest-registry-installed lstat hotspot is intentionally out of scope and tracked in #90362.Remaining performance gap
The largest remaining
lstatgap is inmanifest-registry-installed.ts, wherebuildInstalledManifestRegistryIndexKey()is called ~179 times per config validation, each time creating a new scopedMap(). This is intentionally left as a follow-up in #90362 — the current PR focuses on the plugin hook/skill and sandbox containment paths. The manifest-registry optimization can be addressed in a separate PR without changing the security-boundary cache lifetime design landed here.Runner notes
node_modules;pnpm install --frozen-lockfilehit WindowsEMFILEtwice near finalization. Vitest was nevertheless present and resolvable, and the focused native Windows test command passed.483fc336ad;origin/maincontinued moving during validation, so the final broad gate is recorded against the rebased head before that later churn.Windows Performance Proof
Environment
Method
CPU profiling via Node.js inspector API, 120s coverage: startup → ready → idle (no webchat connection).
Scoped Cache vs Global Cache vs No Cache
lstattotalgateway-1779357547869.cpuprofilegateway-1779440772524.cpuprofilegateway-1780375536440.cpuprofileWhy scoped cache is slower than global cache (+13s)
The gap is not from "other callers not passing a cache" — it is from the largest single caller
manifest-registry-installed.ts:The scoped-cache approach still delivers a 55% reduction in lstat time and 3s faster startup compared to no cache, while avoiding stale process-wide cache answers in security-sensitive containment checks.
Cache Behavior (instrumented dist, global-cache version, call counting)
safeRealpathSynctotal callsisPathInsideWithRealpathcallsPer-phase breakdown:
Cache entries = 3 per plugin (root dir + package.json + entry file).
292 entries × ~200 bytes = ~58KB memory. Bounded by plugin count.
Cache growth test (after installing @openclaw/feishu)
+1 plugin = +3 cache entries (+~600 bytes). Cache is bounded.
Reproducing
Rebase note (2026-07-06): Rebased onto latest
origin/main(e5a605424e) — conflict-resolution only, no logic changes. New head:483fc336ad.