Skip to content

fix(memory-wiki): preserve pages when source paths are unavailable#111034

Merged
steipete merged 1 commit into
mainfrom
fix/memory-wiki-transient-source-prune
Jul 18, 2026
Merged

fix(memory-wiki): preserve pages when source paths are unavailable#111034
steipete merged 1 commit into
mainfrom
fix/memory-wiki-transient-source-prune

Conversation

@steipete

Copy link
Copy Markdown
Contributor

Closes #97523

Additional instructions

MUST: Keep Allow edits from maintainers enabled for this PR so maintainers
can help update the branch when needed.

What Problem This Solves

Fixes an issue where users importing private files with Memory Wiki's unsafe-local mode could lose imported pages and their human annotations when a configured directory or mount was temporarily unavailable during sync.

Why This Change Was Made

The sync now treats each configured source path as its own authoritative scope. Files missing from a readable directory are still pruned, while entries belonging to an unreadable scope are retained until that scope becomes readable again or is removed from configuration. Capacity accounting uses the union of newly collected and retained keys so overlapping configured paths cannot be double-counted.

User Impact

Temporary mount, permission, or filesystem outages no longer delete Memory Wiki pages or human notes. Genuine deletions inside reachable configured directories continue to be reflected on the next sync.

Evidence

  • OPENCLAW_HEAVY_CHECK_LOCK_SCOPE=worktree node scripts/run-vitest.mjs extensions/memory-wiki/src/unsafe-local.test.ts extensions/memory-wiki/src/source-sync-state.test.ts extensions/memory-wiki/src/source-page-shared.test.ts — 3 files passed, 15 tests passed.
  • OPENCLAW_HEAVY_CHECK_LOCK_SCOPE=worktree node scripts/check-changed.mjs -- extensions/memory-wiki/src/unsafe-local.ts extensions/memory-wiki/src/unsafe-local.test.ts — Testbox exit 0; extension production/test typechecks, formatting, lint, boundary, dependency, database-first, and import-cycle gates passed: https://github.com/openclaw/openclaw/actions/runs/29665173619
  • Fresh autoreview after the capacity fix: no accepted/actionable findings; patch assessed correct.

Preserve imported unsafe-local wiki pages and human annotations while configured source scopes are temporarily unreadable.
@steipete
steipete marked this pull request as ready for review July 18, 2026 23:32
@steipete

Copy link
Copy Markdown
Contributor Author

Land-ready proof for head 96fafc68d3f752d3b5beba97338ac8df4e65ade9:

  • Focused Memory Wiki regression suite: 3 files passed, 15 tests passed.
  • Hosted changed gate: Testbox run 29665173619, exit 0; extension typechecks, formatting, lint, boundary, dependency, database-first, and import-cycle checks passed.
  • Exact-head PR CI: run 29665401772, successful including openclaw/ci-gate.
  • Fresh autoreview after the accepted capacity-accounting repair: no actionable findings.

Behavior proof covers both sides of the invariant: unavailable configured scopes retain imported pages and human notes, while missing files in readable sibling scopes are still pruned.

@clawsweeper

clawsweeper Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 18, 2026, 7:36 PM ET / 23:36 UTC.

Summary
The PR changes Memory Wiki unsafe-local sync to retain imported pages from temporarily unreadable configured paths while continuing to prune genuinely deleted files from readable paths.

PR surface: Source +30, Tests +64. Total +94 across 2 files.

Reproducibility: yes. from source inspection: current main converts failed stat or readdir calls into no artifacts and unconditionally prunes inactive unsafe-local state; the linked report supplies a concrete rename/unmount reproduction path.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: serialized state: extensions/memory-wiki/src/unsafe-local.test.ts, unknown-data-model-change: extensions/memory-wiki/src/unsafe-local.test.ts, unknown-data-model-change: extensions/memory-wiki/src/unsafe-local.ts, vector/embedding metadata: extensions/memory-wiki/src/unsafe-local.test.ts, vector/embedding metadata: extensions/memory-wiki/src/unsafe-local.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add a redacted terminal or live-runtime transcript covering initial import, edited human notes, temporary source unavailability, successful retention, restoration, and a true deletion in a reachable scope.
  • After adding proof, update the PR body for a fresh ClawSweeper review; if it does not trigger, ask a maintainer to comment @clawsweeper re-review.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The body reports focused tests and CI checks, but no inspectable after-fix live run, terminal transcript, or redacted runtime log demonstrates that a real unavailable path preserves human notes while a reachable deletion still prunes; add that proof and redact private paths or content. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The PR body reports focused tests and changed-file checks, but it does not yet include inspectable after-fix real-behavior evidence showing an unavailable source retains an edited human-notes block while a reachable deletion still prunes.

Maintainer options:

  1. Decide the mitigation before merge
    Merge a scoped-unavailability repair after maintainers confirm the retention boundary and the contributor adds a redacted terminal or live-runtime transcript demonstrating both preservation during an outage and pruning after a reachable deletion.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] The repair appears mechanically sound, but the protected maintainer label and missing contributor real-behavior proof require human PR review rather than an automated repair lane.

Security
Cleared: The diff is confined to local Memory Wiki collection and pruning behavior, with no new dependencies, workflows, permissions, secrets handling, or supply-chain surface.

Review details

Best possible solution:

Merge a scoped-unavailability repair after maintainers confirm the retention boundary and the contributor adds a redacted terminal or live-runtime transcript demonstrating both preservation during an outage and pruning after a reachable deletion.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection: current main converts failed stat or readdir calls into no artifacts and unconditionally prunes inactive unsafe-local state; the linked report supplies a concrete rename/unmount reproduction path.

Is this the best way to solve the issue?

Yes. Retaining only entries belonging to unavailable configured scopes is narrower than globally skipping pruning, because it preserves outage-affected pages while retaining cleanup for files missing from reachable scopes.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4362e1a4fe39.

Label changes

Label changes:

  • add P0: This PR addresses an established path that can permanently remove imported pages and user-authored human notes when an everyday mount or permission outage occurs.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.

Label justifications:

  • P0: This PR addresses an established path that can permanently remove imported pages and user-authored human notes when an everyday mount or permission outage occurs.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
Evidence reviewed

PR surface:

Source +30, Tests +64. Total +94 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 55 25 +30
Tests 1 66 2 +64
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 121 27 +94

What I checked:

  • Current-main data-loss path: Current main suppresses failed directory reads as an empty file list and suppresses failed stats as absent paths; the unsafe-local sync then builds active keys only from collected artifacts and unconditionally calls the state-prune helper, which deletes every unsafe-local entry not in that set. (extensions/memory-wiki/src/unsafe-local.ts:48, 4362e1a4fe39)
  • Destructive state contract: The shared pruning helper removes the stored page file and deletes its sync-state entry for every inactive key, so a transiently unavailable source can destroy the page that carries the human-notes block. (extensions/memory-wiki/src/source-sync-state.ts:261, 4362e1a4fe39)
  • Branch fixes at the scope boundary: The proposed collection result records unavailable configured paths rather than treating them as empty; the branch retains their matching state keys and computes capacity from the collected-plus-retained key union, avoiding double-counting for overlapping configured paths. (extensions/memory-wiki/src/unsafe-local.ts:30, 96fafc68d3f7)
  • Regression coverage: The new regression test sets up one unavailable and one available configured directory, verifies preservation of the unavailable page and its human notes, and still exercises deletion pruning for the reachable directory. (extensions/memory-wiki/src/unsafe-local.test.ts:108, 96fafc68d3f7)
  • Canonical tracker: The PR explicitly targets the open report whose reproduction describes the same unsafe-local transient-mount deletion and human-notes loss; it is an implementation candidate, not a reason to close the issue before merge. (96fafc68d3f7)

Likely related people:

  • steipete: Authored the focused source-sync repair and the accompanying regression coverage for the affected Memory Wiki path; repository history and release metadata also associate this account with active memory-wiki work. (role: recent Memory Wiki contributor; confidence: medium; commits: 96fafc68d3f7; files: extensions/memory-wiki/src/unsafe-local.ts, extensions/memory-wiki/src/unsafe-local.test.ts)
  • yetval: Provided the concrete current-main reproduction, identified the collection-to-prune path, and documented the irreversible human-notes loss that this PR targets. (role: behavior reporter; confidence: high; files: extensions/memory-wiki/src/unsafe-local.ts, extensions/memory-wiki/src/source-sync-state.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit 941ed4f into main Jul 18, 2026
152 of 164 checks passed
@steipete
steipete deleted the fix/memory-wiki-transient-source-prune branch July 18, 2026 23:39
@steipete

Copy link
Copy Markdown
Contributor Author

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 19, 2026
Preserve imported unsafe-local wiki pages and human annotations while configured source scopes are temporarily unreadable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory-wiki unsafe-local sync deletes imported pages and user human-notes blocks when a configured source path is transiently unavailable

1 participant