fix(memory-core): strip dreaming managed blocks from daily memory reads#68408
fix(memory-core): strip dreaming managed blocks from daily memory reads#68408jasonmakr wants to merge 3 commits into
Conversation
Light sleep and REM dreaming phases write managed blocks into daily memory
files (memory/YYYY-MM-DD.md) when using inline storage mode. These blocks
contain session-agnostic candidate memories that leak cross-session context
when other sessions read the same daily file via memory_get.
This commit:
- Adds stripDreamingManagedBlocks() to memory-host-markdown (plugin-sdk)
which removes <!-- openclaw:dreaming:{light,rem}:{start,end} --> blocks
and their optional headings from arbitrary markdown content.
- Applies the stripping in QmdMemoryManager.readFile() for daily memory
file paths (matching memory/YYYY-MM-DD.md pattern), so memory_get
callers no longer see dreaming staging data from other sessions.
- Adds comprehensive tests for the new stripping function.
The dreaming blocks are preserved on disk for the dreaming subsystem's
own use — only the read path is filtered.
Fixes openclaw#68367
Documents the known limitation that partial reads (with from/lines params) of daily memory files still expose dreaming blocks, and outlines potential future approaches.
Greptile SummaryThis PR strips dreaming managed blocks (Light Sleep / REM Sleep) from daily memory files on the read path to prevent cross-session context leakage. The core logic in
Confidence Score: 3/5Not safe to merge — two P1 issues: a CI-breaking missing baseline update and an incomplete fix that leaves partial reads unfiltered. Two P1 findings: (1) the plugin SDK API baseline hash was not updated, causing
|
| export function stripDreamingManagedBlocks(content: string): string { | ||
| let result = content; | ||
| for (const block of DREAMING_MANAGED_BLOCKS) { | ||
| const pattern = new RegExp( | ||
| `(?:${escapeRegex(block.heading)}\\n)?${escapeRegex(block.startMarker)}[\\s\\S]*?${escapeRegex(block.endMarker)}\\n?`, | ||
| "gm", | ||
| ); | ||
| result = result.replace(pattern, ""); | ||
| } | ||
| // Collapse runs of 3+ blank lines into 2. | ||
| result = result.replace(/\n{3,}/g, "\n\n"); | ||
| return result; | ||
| } |
There was a problem hiding this comment.
Plugin SDK API baseline not updated
memory-host-markdown is a published package.json export (./plugin-sdk/memory-host-markdown) and is listed in scripts/lib/plugin-sdk-entrypoints.json. Adding a new export (stripDreamingManagedBlocks) to it is a public Plugin SDK surface change, but pnpm plugin-sdk:api:gen was not run and docs/.generated/plugin-sdk-api-baseline.sha256 was not updated — the hash in the file is identical before and after this PR. pnpm plugin-sdk:api:check will fail in CI until the hash is regenerated and committed.
Run pnpm plugin-sdk:api:gen and commit the updated .sha256 file alongside this change.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/plugin-sdk/memory-host-markdown.ts
Line: 65-77
Comment:
**Plugin SDK API baseline not updated**
`memory-host-markdown` is a published `package.json` export (`./plugin-sdk/memory-host-markdown`) and is listed in `scripts/lib/plugin-sdk-entrypoints.json`. Adding a new export (`stripDreamingManagedBlocks`) to it is a public Plugin SDK surface change, but `pnpm plugin-sdk:api:gen` was not run and `docs/.generated/plugin-sdk-api-baseline.sha256` was not updated — the hash in the file is identical before and after this PR. `pnpm plugin-sdk:api:check` will fail in CI until the hash is regenerated and committed.
Run `pnpm plugin-sdk:api:gen` and commit the updated `.sha256` file alongside this change.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83df36e3c7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // Strip managed dreaming blocks from daily memory files so that | ||
| // light-sleep / REM staging data from other sessions is not leaked | ||
| // into memory_get results. See openclaw/openclaw#68367. | ||
| const fullText = isDailyMemoryPath(relPath) ? stripDreamingManagedBlocks(full.text) : full.text; |
There was a problem hiding this comment.
Strip dreaming blocks for paginated daily reads
The new filtering only runs on the full-read path, so requests that set from or lines still return raw daily file content. In practice, memory_get pagination commonly follows nextFrom from an initial truncated result, which takes this partial-read branch and can still surface managed light/REM blocks from other sessions. This leaves the cross-session leakage bug unfixed for continuation reads.
Useful? React with 👍 / 👎.
| result = result.replace(pattern, ""); | ||
| } | ||
| // Collapse runs of 3+ blank lines into 2. | ||
| result = result.replace(/\n{3,}/g, "\n\n"); |
There was a problem hiding this comment.
Preserve blank lines when no dreaming block is removed
This helper always rewrites \n{3,} to \n\n, even when no dreaming markers are present, so daily-memory reads are no longer a no-op on ordinary content. That mutates user-authored formatting and can shift line positions in returned excerpts, which breaks the expectation that memory_get returns exact file excerpts unless a managed block was actually stripped.
Useful? React with 👍 / 👎.
…iles Previously, memory_get calls with from/lines parameters on daily memory files (memory/YYYY-MM-DD.md) bypassed the dreaming block filter, leaking cross-session staging data from Light Sleep and REM Sleep phases. The fix redirects partial reads of daily files through the full-read path: read the entire file, strip dreaming managed blocks, then slice the requested line range from the cleaned content. This ensures line numbers are consistent with a full read and no staging data is ever exposed. Non-daily files continue to use the efficient streaming partial read path, since they never contain dreaming blocks. Daily memory files are small (a few KB) so the overhead of a full read is negligible. Resolves the TODO(openclaw#68367) left in the initial implementation. Addresses review feedback on openclaw#68408.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b20b72e61d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| function isDailyMemoryPath(relPath: string): boolean { | ||
| return DAILY_MEMORY_PATH_RE.test(relPath); |
There was a problem hiding this comment.
Normalize path before deciding whether to strip dreaming blocks
resolveReadPath() canonicalizes user input, so requests like memory/./2026-04-18.md (or qmd/<collection>/memory/tmp/../2026-04-18.md) still resolve to the same daily file, but isDailyMemoryPath() matches the raw relPath string. Those normalized-equivalent paths fail DAILY_MEMORY_PATH_RE, so the new stripping logic is skipped and memory_get can still expose managed dreaming block content; this keeps the cross-session leakage fix bypassable via path syntax alone.
Useful? React with 👍 / 👎.
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Keep open. The branch addresses a real memory-core cross-session leakage bug, but it is not merge-ready because the filter can still be bypassed through normalized-equivalent paths, the PR exposes a memory-core-specific helper as public Plugin SDK API without an intentional SDK contract path, and external real behavior proof is still missing. Canonical path: Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review. So I’m closing this here because the remaining work is already tracked in the canonical issue. Review detailsBest possible solution: Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review. Do we have a high-confidence way to reproduce the issue? Yes. Source inspection shows current main can write inline light/REM blocks into memory/YYYY-MM-DD.md and QmdMemoryManager.readFile returns raw full or partial content; the PR's remaining bypass is reproducible in source with a canonical-equivalent path such as memory/./YYYY-MM-DD.md. Is this the best way to solve the issue? No. The read-path filtering is a plausible mitigation, but the current implementation checks the wrong path shape and promotes a memory-core-specific helper into public SDK API instead of keeping the ownership boundary narrow. Security review: Security review needs attention: The PR targets a sensitive cross-session memory exposure, but the raw-path predicate leaves a concrete bypass in the security-sensitive read path.
AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 1d2bebbb41bf. |
|
Codex review: found issues before merge. What this changes: This PR adds a public Plugin SDK markdown helper for stripping managed Light Sleep and REM dreaming blocks, wires it into memory-core QMD daily-memory full and partial reads, and adds focused helper/read-path tests. Maintainer follow-up before merge: This is an active contributor implementation PR for a real memory isolation bug, but the remaining work is normal maintainer review/request-changes on the source branch rather than a duplicate automated replacement PR by default. Review findings:
Review detailsBest possible solution: Land a corrected memory-core read-path filter that classifies daily files from the resolved canonical path, strips only managed Light/REM blocks from both full and paginated memory_get reads, preserves ordinary daily-note text byte-for-byte when no managed block is removed, keeps dreaming internals reading raw files, and updates the Plugin SDK API baseline if the helper remains public. Full review comments:
Overall correctness: patch is incorrect Acceptance criteria:
What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 3215ab6de5db. |
|
Thanks for the detailed reviews! I understand the three main issues: P1: Path normalization bypass P2: Plugin SDK API baseline P2: Blank line preservation Working on fixes now. |
|
ClawSweeper PR egg 🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat. Where did the egg go?
|
|
This pull request has been automatically marked as stale due to inactivity. |
|
ClawSweeper applied the proposed close for this PR.
|
Problem
Light sleep and REM dreaming phases write managed blocks into daily memory files (
memory/YYYY-MM-DD.md) when using inline storage mode. These blocks contain session-agnostic candidate memories that leak cross-session context when other sessions read the same daily file viamemory_get.While the default storage mode was changed to
"separate"(writing tomemory/dreaming/light/instead), users who:"inline"or"both"storage mode"inline"to"separate"but have old blocks in daily files...still experience cross-session contamination when reading daily memory files.
Solution
1. New utility:
stripDreamingManagedBlocks()(memory-host-markdown.ts)Strips all managed dreaming blocks (
<!-- openclaw:dreaming:{light,rem}:{start,end} -->) and their optional headings (## Light Sleep,## REM Sleep) from markdown content. Collapses excessive blank lines after stripping.This function is in the plugin-sdk so it can be reused by other extensions if needed.
2. Apply stripping in
QmdMemoryManager.readFile()When reading daily memory files (matching
memory/YYYY-MM-DD.mdpattern), the dreaming blocks are stripped before returning content to the caller. This ensuresmemory_getresults are clean of staging data from other sessions.Important: The dreaming blocks are preserved on disk — only the read path is filtered. The dreaming subsystem continues to read the raw file content through its own code path.
3. Comprehensive tests
6 test cases covering:
Files changed
src/plugin-sdk/memory-host-markdown.ts— AddedstripDreamingManagedBlocks()src/plugin-sdk/memory-host-markdown.test.ts— Added 6 tests for the new functionextensions/memory-core/src/memory/qmd-manager.ts— Applied stripping inreadFile()for daily memory pathsFixes #68367