fix(memory-wiki): don't prune unsafe-local pages when a source path is transiently unavailable#97560
fix(memory-wiki): don't prune unsafe-local pages when a source path is transiently unavailable#97560LEXES7 wants to merge 4 commits into
Conversation
…s transiently unavailable
|
Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 5:27 PM ET / 21:27 UTC. Summary PR surface: Source +59, Tests +187. Total +246 across 2 files. Reproducibility: yes. source-backed. The canonical issue and PR body provide a production-function repro, and current main plus v2026.6.10 still drop missing configured paths before the destructive prune runs. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land the scoped unreadable-path preservation fix once maintainers accept conservative explicit-file retention, then close the canonical issue and supersede the weaker competing PR. Do we have a high-confidence way to reproduce the issue? Yes, source-backed. The canonical issue and PR body provide a production-function repro, and current main plus v2026.6.10 still drop missing configured paths before the destructive prune runs. Is this the best way to solve the issue? Yes, with maintainer acceptance of the explicit-file retention tradeoff. The latest head preserves only exact unreadable paths before pruning, which is the narrow owner-boundary fix for the data-loss path. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against a54919dcbc95. Label changesLabel justifications:
Evidence reviewedPR surface: Source +59, Tests +187. Total +246 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
Addressed the review:
No config/state migration: this only changes which imported pages are pruned |
|
@clawsweeper re-review Added real before/after behavior proof to the PR body: on pristine base 7ac8b48 the transient outage prunes/deletes the page and human-notes (4 cases fail), and on this branch the same real fs.rename/chmod outages preserve them (7 pass). These drive the real syncMemoryWikiUnsafeLocalSources against a real on-disk vault; nothing in the collect/prune path is stubbed. Also implemented the recommended scope-aware prune: unreadable roots preserve their entries while readable roots still clean up genuinely deleted files (proven by the multi-path partial-outage case). No state/data-model migration — same source-sync state shape. Note on the cluster: this is distinct from the merged #95614. That fix preserves notes on page re-ingest, but it only runs while the page still exists; this bug is the prune-delete path that removes the page file outright during an outage, before any re-ingest preservation can run. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Added a standalone NON-TEST real-workflow log to the PR body (Node 24, real on-disk vault, real fs.rename outage), driving the real syncMemoryWikiUnsafeLocalSources outside the test harness:
This is in addition to the fs-backed regression suite. The standalone run uses the in-memory source-sync fallback store (the SQLite store is unrelated to the prune logic). Remaining explicit-file retention is the intentional data-safety tradeoff under "Accept Conservative Explicit-File Retention". |
…ning unsafe-local
|
@clawsweeper re-review Addressed the nested-outage scope finding: the prune now preserves only the EXACT unreadable subtree, not the whole configured root. collectUnsafeLocalArtifacts reports each unreadable directory individually, and only entries under those exact paths are kept active. Added the requested regression "prunes a deleted readable sibling while a nested directory is unreadable": with a nested dir offline AND a readable sibling file deleted under the same root, the sibling's stale page is still pruned (removedCount 1) while the unavailable subtree is preserved. 8/8 unsafe-local tests pass; before/after on pristine base 7ac8b48 is in the PR body (5 fail on base, 8 pass here). Remaining explicit-file retention is the intentional data-safety tradeoff (Accept Conservative Explicit-File Retention). |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
This pull request has been automatically marked as stale due to inactivity. |
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: fix(memory-wiki): don't prune unsafe-local pages when a source path is transiently unavailable This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
|
Closing due to inactivity. |
Closes #97523
What Problem This Solves
Fixes an issue where memory-wiki users running
vaultMode: "unsafe-local"would permanently lose imported pages and their hand-written human-notes blocks when a configured source path became transiently unavailable (an undocked external drive, a rebooting NAS, or a cloud-synced folder that is not mounted yet). Anywiki_*tool that triggered a sync while the path was gone deleted every imported page for that path; when the path came back, pages were recreated empty and the user's notes were unrecoverable.Why This Change Was Made
A transiently unreadable source collects zero artifacts in exactly the same way a real deletion does, and the
unsafe-localsync pruned unconditionally on that signal. The fix makes the prune aware of the exact unreadable subtree:collectUnsafeLocalArtifactsnow records the precise paths that could not be read —fs.statfailing on a configured path, a missing explicit file, orfs.readdirfailing on the configured directory or any nested subdirectory (each failing directory is reported individually).syncMemoryWikiUnsafeLocalSourcespreserves only the imported entries that live under those unreadable paths (keeps their keys active), then prunes normally. So a transient outage never deletes a page or its human-notes, while readable siblings — even a deleted file in a readable directory under the same configured root as an unreadable nested mount, or another configured source entirely — still get their stale pages cleaned up.This is the narrowed, source-scope-aware behavior ClawSweeper recommended. An explicit configured file that vanishes is indistinguishable from an unmounted one, so its page is conservatively kept (data-safety first) until it returns.
User Impact
Users with
unsafe-localnotes on removable, network, or cloud-synced storage no longer lose imported pages or their manually authored human-notes when a source path (or a nested mount inside it) is briefly offline. Healthy sources keep cleaning up deleted files normally. After the source returns, the next sync reconciles cleanly with notes intact. No config changes required.Evidence
Real workflow run (standalone, non-test) — Node 24
Drove the real
syncMemoryWikiUnsafeLocalSourcesfrom a standalone script (node --import tsx) against a real on-disk vault — not the test harness. The outage is a real directoryfs.rename(ENOENT), the way an undocked drive / unmounted NAS presents. Steps per run: (1) sync imports the source page, (2) write a hand-written note into the page's human-notes block on disk, (3) take the source dir offline and sync, (4) remount and sync.BEFORE — pristine base
7ac8b48a08(currentmain):AFTER — this branch, identical steps:
Regression tests (real on-disk vault, pristine base vs this branch)
These drive the real
syncMemoryWikiUnsafeLocalSourcesagainst a real on-disk vault, with the outage produced by a realfs.rename/fs.chmodon the source path (nothing in the collect/prune path is stubbed; only the plugin state store is the in-memory test store, which is incidental to the prune logic).BEFORE — pristine base
7ac8b48a08, same tests, the bug reproduces:AFTER — this branch, identical inputs:
What the new/updated tests prove:
keeps imported pages and human notes when a configured directory is transiently unreadable— imports a page, writes a real user note into its human-notes block, takes the source directory offline with a realfs.renameoutage, syncs (assertsremovedCount0 and page + note survive), then re-mounts and syncs again (clean, note intact).does not prune when a nested subdirectory is transiently unreadable— a nested subdirectory made unreadable mid-sync no longer prunes its page (removedCount0).prunes a deleted readable sibling while a nested directory is unreadable— proves the precise-subtree scope: only the unavailable subtree is preserved; a genuinely deleted readable sibling under the same configured root still prunes (removedCount1).keeps the imported page when an explicit configured file is transiently unavailable— explicit-file (unsafeLocal.pathsfile shape) coverage: a vanished explicit file is preserved (removedCount0, page still on disk).prunes a readable source even when another configured source is unreadable— one source offline, the other deletes a file → the healthy source still prunes (removedCount1) while the offline source's page is preserved.oxlint and
oxfmt --checkare clean on both changed files.AI-assisted (Claude Code); change reviewed and understood by the author.