Skip to content

fix(memory-wiki): stop duplicate bridge imports under polling#91828

Merged
steipete merged 2 commits into
openclaw:mainfrom
mushuiyu886:repair/issue-91154
Jul 11, 2026
Merged

fix(memory-wiki): stop duplicate bridge imports under polling#91828
steipete merged 2 commits into
openclaw:mainfrom
mushuiyu886:repair/issue-91154

Conversation

@mushuiyu886

@mushuiyu886 mushuiyu886 commented Jun 10, 2026 •
edited by steipete
Loading

Copy link
Copy Markdown
Contributor

Closes #91154

What Problem This Solves

Fixes an issue where users with large bridge-mode Memory Wiki vaults could see wiki.palace and wiki.importInsights stall, consume excessive memory, or fail while the UI issued overlapping polls. At the reported scale, each poll independently rescanned roughly 1,900 memory files and rewrote the same sync state.

Why This Change Was Made

Equivalent imported-source requests now share one complete source-and-index flight, while different runtime snapshots serialize through the existing per-vault mutation coordinator. Warm imports skip page verification/rendering before opening the guarded vault, and source-sync state persists only changed SQLite rows. The already-landed bounded path-mismatch retry remains the single write-recovery owner; this PR does not add a second retry layer or change config/API surfaces.

User Impact

Concurrent Memory Wiki polls now observe one consistent import result instead of launching duplicate scans. Large warm bridge imports avoid thousands of unnecessary guarded filesystem operations and SQLite rewrites, while changed or removed sources still update normally.

Evidence

  • Isolated loopback Gateway canary on Node 24 with 1,914 Markdown sources across seven workspaces:
    • Initial import: 1,914 sources and 1,914 SQLite state rows in 48.7 seconds.
    • Warm poll: 3.9 seconds, no ingest/state changes.
    • After changing one source, eight concurrent RPCs completed in 9.6 seconds; both wiki.bridge.import callers reported the same updatedCount: 1, skippedCount: 1913 result.
    • Exactly one generated source page and one SQLite row changed; one ingest record was added; no path-mismatch or internal error appeared.
  • Focused Testbox regression lane: 5 files, 47 tests passed.
  • Full Memory Wiki Testbox shard: 36 files, 304 tests passed.
  • pnpm check:changed: passed on Blacksmith Testbox tbx_01kx7cbz836gaprgcsmajshany.
  • Fresh local autoreview: clean after addressing the capacity-ordering finding.

AI-assisted; verified against the current source, callers, sibling mutation paths, tests, and the @openclaw/fs-safe contract.

@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-wiki size: M proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 10, 2026
@clawsweeper

clawsweeper Bot commented Jun 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: stale review; fresh review needed.

Summary
The latest durable ClawSweeper review was for head 376ae3773ee778401b27f3fdca5420d9468233fe, but the PR head is now 8688904b37695d093445672838401b947eb56af8. Its old verdict and PR readiness labels are no longer current.

Next step
Run or wait for a fresh ClawSweeper review on the current PR head.

Review history (4 earlier review cycles)
  • reviewed 2026-07-04T03:27:52.986Z sha b1f006b :: needs maintainer review before merge. :: none
  • reviewed 2026-07-04T14:22:33.462Z sha b1f006b :: needs maintainer review before merge. :: none
  • reviewed 2026-07-04T17:42:03.386Z sha b1f006b :: needs maintainer review before merge. :: none
  • reviewed 2026-07-10T12:43:36.996Z sha 8258ba2 :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 10, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 10, 2026
@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 10, 2026
@mushuiyu886

mushuiyu886 commented Jun 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

The retry accounting finding has been addressed in the latest head: recovered path-mismatch writes now preserve changed / created classification, and focused regression coverage verifies the bridge import is counted as importedCount=1 / skippedCount=0 after recovery. Checks and real behavior proof are green on the latest head.

@clawsweeper

clawsweeper Bot commented Jun 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review. labels Jun 10, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 10, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 10, 2026
@mushuiyu886

mushuiyu886 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Checks are green on head 454bf0b. The patch now coalesces full memory-wiki imported-source sync through index refresh, and the PR body includes gateway-load proof: 60 concurrent wiki.palace / wiki.importInsights / wiki.status calls, 13 bridge artifacts, syncBridgeArtifactListings=1, ingestLogCount=1, compileLogCount=1, statusBridgeSourceCount=13.

@clawsweeper

clawsweeper Bot commented Jun 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 10, 2026
@mushuiyu886

mushuiyu886 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

The patch was tightened after the last review: bridge direct-call coalescing was removed, leaving one process-local in-flight map at the imported-source sync boundary that covers bridge import plus index refresh. The updated PR body includes the rerun local proofs: 17 focused tests, imported-source local proof with concurrentImportedSourceListings=1, and gateway-load proof with 60 concurrent wiki.palace / wiki.importInsights / wiki.status calls, syncBridgeArtifactListings=1, ingestLogCount=1, compileLogCount=1.

@clawsweeper

clawsweeper Bot commented Jun 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 10, 2026
@clawsweeper clawsweeper Bot mentioned this pull request Jun 20, 2026
Closed
@mushuiyu886 mushuiyu886 changed the title Harden memory wiki bridge imports fix(memory-wiki): harden bridge imports under concurrent syncs Jun 22, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 23, 2026
@mushuiyu886

mushuiyu886 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 24, 2026
@clawsweeper clawsweeper Bot mentioned this pull request Jul 1, 2026
Merged
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. proof: sufficient ClawSweeper judged the real behavior proof convincing. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jul 2, 2026
This was referenced Jul 10, 2026
Closed
Merged
@steipete

steipete commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

@github-actions github-actions Bot mentioned this pull request Jul 11, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@steipete steipete

Labels

extensions: memory-wiki P1 High-priority user-facing bug, regression, or broken workflow. proof: supplied External PR includes structured after-fix real behavior proof. size: L

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

memory-wiki: wiki.palace fails with (path-mismatch) and wiki.importInsights takes 100s+ under load

2 participants

@mushuiyu886 @steipete