Skip to content

memory-wiki unsafe-local sync deletes imported pages and user human-notes blocks when a configured source path is transiently unavailable #97523

Description

@yetval

Summary

In memory-wiki vaultMode: "unsafe-local", a transiently unavailable configured source path (undocked external drive, NAS reboot, cloud-synced folder not mounted yet) causes the next wiki sync to delete every imported page for that path AND destroy the user-authored human-notes block inside each page. The loss is permanent: on the next successful sync the page is recreated empty, with no prior content to restore the notes from.

Environment

  • Commit: 27c1685 (origin/main at time of filing)
  • Surface: memory-wiki plugin, unsafe-local vault mode (extensions/memory-wiki)
  • Affected users: anyone running vaultMode: "unsafe-local" with a configured note path on removable/network/cloud-synced storage. Opt-in mode, but for those users the loss is irreversible and triggered by an everyday event.

Steps to reproduce

  1. Configure memory-wiki in unsafe-local mode with unsafeLocal.allowPrivateMemoryCoreAccess: true and unsafeLocal.paths pointing at a directory on a removable/network/cloud-synced volume.
  2. Sync once (any wiki_* tool routes through syncMemoryWikiUnsafeLocalSources). Pages are imported.
  3. Hand-edit a page's human-notes block (the <!-- openclaw:human:start --> ... <!-- openclaw:human:end --> region) with your own notes.
  4. The configured path becomes transiently unavailable (drive undocked, NAS rebooting, mount not ready). Any wiki tool triggers a sync while it is gone.
  5. Re-dock / remount and sync again.

Expected

A transiently unreadable source must not be treated as deleted. Imported pages and their human-notes blocks survive the temporary outage; the next successful sync re-reconciles cleanly.

Actual

The sync during the outage collects zero artifacts (the missing path is indistinguishable from "all sources deleted"), so pruneImportedSourceEntries removes every unsafe-local page file and its state entry. When the path returns, pages are recreated fresh with empty human-notes blocks. The user's hand-written notes and any manual edits are gone and unrecoverable.

Root cause (if known)

extensions/memory-wiki/src/unsafe-local.ts collectUnsafeLocalArtifacts conflates "transiently absent" with "empty" and returns zero artifacts on ENOENT:

const stat = await fs.stat(absoluteConfiguredPath).catch(() => null);
if (!stat) {
  continue;
}

listAllowedFilesRecursive has the same conflation for an empty/unreadable mountpoint:

const entries = await fs.readdir(rootDir, { withFileTypes: true }).catch(() => []);

syncMemoryWikiUnsafeLocalSources then calls the prune unconditionally, with no "zero artifacts collected -> skip prune" guard:

const removedCount = await pruneImportedSourceEntries({
  vaultRoot: config.vault.path,
  group: "unsafe-local",
  activeKeys,
  state,
});

pruneImportedSourceEntries (extensions/memory-wiki/src/source-sync-state.ts:261-277) hard-deletes every state entry whose syncKey is not in activeKeys:

const pageAbsPath = path.join(params.vaultRoot, entry.pagePath);
await fs.rm(pageAbsPath, { force: true }).catch(() => undefined);
delete params.state.entries[syncKey];

The deleted page carries the user-editable human-notes block. Normal re-syncs preserve that block (added by #95614), but the prune-delete path bypasses preservation entirely; once the page file and its state entry are gone, the next successful sync has no existing content to restore from, so the block is regenerated empty.

Sibling surfaces

The bridge sync path was already hardened against this exact "collection produced zero results -> do not prune" conflation. extensions/memory-wiki/src/bridge.ts gates the prune on capability availability (comment cites #68373):

const memoryCapability = getMemoryCapabilityRegistration();
const removedCount = memoryCapability
  ? await pruneImportedSourceEntries({ ... group: "bridge", ... })
  : 0;

The unsafe-local path has no equivalent guard. Distinct from #67658 / #67711 (the bridge-mode version of this conflation, different trigger and vault mode) and from #95614 (which added human-notes preservation only on the normal re-ingest path, not the prune-delete path). Both are cited above.

Real behavior proof

Behavior addressed: unsafe-local sync prunes all imported pages and destroys user human-notes blocks when a configured source path is transiently unavailable.
Real environment tested: drove the real production function syncMemoryWikiUnsafeLocalSources (imported directly from extensions/memory-wiki/src/unsafe-local.ts) end to end against a temp vault, at commit 27c1685. The transient outage is simulated by renaming the source directory aside between syncs and restoring it (a real ENOENT, the way an undocked drive or unmounted NAS presents). No internal function was stubbed.
Exact steps or command run after this patch: node --import tsx /tmp/openclaw-bug-repros/wiki-unsafe-local-prune/repro.mjs (step 1 creates a page and writes a human note; step 2 syncs while the source dir is renamed away; step 3 restores the dir and syncs again).
Evidence after fix:

# OBSERVED (buggy, origin/main 27c1685f10)
Step1 imported: 1 page: unsafe-local-notes-9c70662b-ideas-md-069c9403.md
Step1b user notes written into human block.
Step2 removedCount: 1 | page still on disk: false
Step3 imported: 1 | human notes survived: false

--- ASSERTION ---
FAIL (bug present): transient unreadability pruned the page and DESTROYED the user's human notes.

# EXPECTED (correct behavior on identical inputs; produced by skipping the prune when zero artifacts were collected)
Step1 imported: 1 page: unsafe-local-notes-67e7428e-ideas-md-8f75b87e.md
Step1b user notes written into human block.
Step2 removedCount: 0 | page still on disk: true
Step3 imported: 0 | human notes survived: true

--- ASSERTION ---
PASS (no bug): page + notes survived a transient read failure.

Observed result after fix: on current main the transient outage deletes the page (removedCount: 1, page gone) and the restored page loses the user's human-notes block (human notes survived: false); when the prune is skipped on a zero-artifact sync, the page and notes both survive.
What was not tested: only the single-configured-path case was exercised (which prunes all pages); multi-path configs where one of several paths drops were not run, nor was the live gateway/wiki_* tool entry that routes into this function (the production function itself was driven directly). No fix is proposed here; the EXPECTED column was produced by a throwaway local edit, reverted, purely to show the corrected behavior on identical inputs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Emergency: data loss, security bypass, crash loop, or unusable core runtime.clawsweeper:current-main-reproClawSweeper found a high-confidence current-main issue reproduction.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.impact:data-lossCan lose, corrupt, or silently drop user/session/config data.issue-rating: 🦀 challenger crabExceptional issue quality: high-confidence current-main reproduction and actionable evidence.no-staleExclude from stale automation

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions