fix(memory-core): suppress raw dreaming inline dumps on fallback (Fixes #70509)#70523
fix(memory-core): suppress raw dreaming inline dumps on fallback (Fixes #70509)#70523deepujain wants to merge 9 commits into
Conversation
Greptile SummaryThis PR suppresses raw dreaming metadata dumps from being written to daily
Confidence Score: 3/5Not safe to merge as-is — the refactor introduces a regression on the normal write path that affects every successful dreaming write, not just fallback cases. One P1 finding:
Prompt To Fix All With AIThis is a comment left during a code review.
Path: extensions/memory-core/src/dreaming-markdown.ts
Line: 105-122
Comment:
**Missing `withTrailingNewline` on the normal write path**
The old code always called `withTrailingNewline(updated)` before writing, regardless of which branch produced `updated`. The new code applies `withTrailingNewline` inside `removeManagedMarkdownBlock` (the suppression branch), but the `replaceManagedMarkdownBlock` branch no longer has it. Every successful non-fallback inline dreaming write will now drop the trailing newline from the daily memory file.
```suggestion
const updated =
inlineBody.length > 0
? replaceManagedMarkdownBlock({
original,
heading: DAILY_PHASE_HEADINGS[params.phase],
startMarker: markers.start,
endMarker: markers.end,
body: inlineBody,
})
: removeManagedMarkdownBlock({
original,
heading: DAILY_PHASE_HEADINGS[params.phase],
startMarker: markers.start,
endMarker: markers.end,
});
const finalContent = inlineBody.length > 0 ? withTrailingNewline(updated) : updated;
if (finalContent !== original) {
await fs.writeFile(inlinePath, finalContent, "utf-8");
}
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: extensions/memory-core/src/dreaming-narrative.ts
Line: 916-921
Comment:
**Outer catch shadows request-scoped fallback on append failure**
If the `appendNarrativeEntry` call inside `startNarrativeRunOrFallback`'s catch block throws (e.g. a transient write error), that exception propagates through the outer `try` and is caught here, returning `{ fallbackUsed: false }`. The subagent runtime WAS request-scoped, so the daily memory file will still contain the raw dump — but suppression never fires. The window is small, but it is the exact scenario this PR aims to guard against.
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "fix(memory-core): suppress raw dreaming ..." | Re-trigger Greptile |
| const updated = | ||
| inlineBody.length > 0 | ||
| ? replaceManagedMarkdownBlock({ | ||
| original, | ||
| heading: DAILY_PHASE_HEADINGS[params.phase], | ||
| startMarker: markers.start, | ||
| endMarker: markers.end, | ||
| body: inlineBody, | ||
| }) | ||
| : removeManagedMarkdownBlock({ | ||
| original, | ||
| heading: DAILY_PHASE_HEADINGS[params.phase], | ||
| startMarker: markers.start, | ||
| endMarker: markers.end, | ||
| }); | ||
| if (updated !== original) { | ||
| await fs.writeFile(inlinePath, updated, "utf-8"); | ||
| } |
There was a problem hiding this comment.
Missing
withTrailingNewline on the normal write path
The old code always called withTrailingNewline(updated) before writing, regardless of which branch produced updated. The new code applies withTrailingNewline inside removeManagedMarkdownBlock (the suppression branch), but the replaceManagedMarkdownBlock branch no longer has it. Every successful non-fallback inline dreaming write will now drop the trailing newline from the daily memory file.
| const updated = | |
| inlineBody.length > 0 | |
| ? replaceManagedMarkdownBlock({ | |
| original, | |
| heading: DAILY_PHASE_HEADINGS[params.phase], | |
| startMarker: markers.start, | |
| endMarker: markers.end, | |
| body: inlineBody, | |
| }) | |
| : removeManagedMarkdownBlock({ | |
| original, | |
| heading: DAILY_PHASE_HEADINGS[params.phase], | |
| startMarker: markers.start, | |
| endMarker: markers.end, | |
| }); | |
| if (updated !== original) { | |
| await fs.writeFile(inlinePath, updated, "utf-8"); | |
| } | |
| const updated = | |
| inlineBody.length > 0 | |
| ? replaceManagedMarkdownBlock({ | |
| original, | |
| heading: DAILY_PHASE_HEADINGS[params.phase], | |
| startMarker: markers.start, | |
| endMarker: markers.end, | |
| body: inlineBody, | |
| }) | |
| : removeManagedMarkdownBlock({ | |
| original, | |
| heading: DAILY_PHASE_HEADINGS[params.phase], | |
| startMarker: markers.start, | |
| endMarker: markers.end, | |
| }); | |
| const finalContent = inlineBody.length > 0 ? withTrailingNewline(updated) : updated; | |
| if (finalContent !== original) { | |
| await fs.writeFile(inlinePath, finalContent, "utf-8"); | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/memory-core/src/dreaming-markdown.ts
Line: 105-122
Comment:
**Missing `withTrailingNewline` on the normal write path**
The old code always called `withTrailingNewline(updated)` before writing, regardless of which branch produced `updated`. The new code applies `withTrailingNewline` inside `removeManagedMarkdownBlock` (the suppression branch), but the `replaceManagedMarkdownBlock` branch no longer has it. Every successful non-fallback inline dreaming write will now drop the trailing newline from the daily memory file.
```suggestion
const updated =
inlineBody.length > 0
? replaceManagedMarkdownBlock({
original,
heading: DAILY_PHASE_HEADINGS[params.phase],
startMarker: markers.start,
endMarker: markers.end,
body: inlineBody,
})
: removeManagedMarkdownBlock({
original,
heading: DAILY_PHASE_HEADINGS[params.phase],
startMarker: markers.start,
endMarker: markers.end,
});
const finalContent = inlineBody.length > 0 ? withTrailingNewline(updated) : updated;
if (finalContent !== original) {
await fs.writeFile(inlinePath, finalContent, "utf-8");
}
```
How can I resolve this? If you propose a fix, please make it concise.| } catch (err) { | ||
| // Narrative generation is best-effort — never fail the parent phase. | ||
| params.logger.warn( | ||
| `memory-core: narrative generation failed for ${params.data.phase} phase: ${formatErrorMessage(err)}`, | ||
| ); | ||
| return { fallbackUsed: false }; |
There was a problem hiding this comment.
Outer catch shadows request-scoped fallback on append failure
If the appendNarrativeEntry call inside startNarrativeRunOrFallback's catch block throws (e.g. a transient write error), that exception propagates through the outer try and is caught here, returning { fallbackUsed: false }. The subagent runtime WAS request-scoped, so the daily memory file will still contain the raw dump — but suppression never fires. The window is small, but it is the exact scenario this PR aims to guard against.
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/memory-core/src/dreaming-narrative.ts
Line: 916-921
Comment:
**Outer catch shadows request-scoped fallback on append failure**
If the `appendNarrativeEntry` call inside `startNarrativeRunOrFallback`'s catch block throws (e.g. a transient write error), that exception propagates through the outer `try` and is caught here, returning `{ fallbackUsed: false }`. The subagent runtime WAS request-scoped, so the daily memory file will still contain the raw dump — but suppression never fires. The window is small, but it is the exact scenario this PR aims to guard against.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0ef2a0e533
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!runId) { | ||
| return; | ||
| return { fallbackUsed: true }; |
There was a problem hiding this comment.
Distinguish failed fallback from successful fallback
Treating !runId as fallbackUsed: true conflates a successful request-scoped fallback with a failed fallback write, because startNarrativeRunOrFallback returns null in both cases. runLightDreaming/runRemDreaming now use this flag to suppress inline dreaming blocks, so if appendNarrativeEntry fails during request-scoped fallback (for example, transient DREAMS.md write errors), the raw inline block is removed and no diary entry is written, losing the phase output entirely. Return whether the fallback append actually succeeded (or use a tri-state result) before suppressing inline content.
Useful? React with 👍 / 👎.
|
Related work from PRtags group Title: Open PR candidate: Dream Diary duplicate/raw fallback cleanup
|
|
Tightened the dreaming follow-up: inline daily-memory writes keep their trailing newline again, and request-scoped narrative fallback now reports whether the diary append actually succeeded before suppressing inline output. Added regression coverage for the fallback-write-failed path too. Local targeted testing is blocked in this environment by the repo non-isolated Vitest runner issue, so I am leaning on CI for the final pass. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3457e37e93
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!existingPattern.test(params.original)) { | ||
| return params.original; | ||
| } | ||
| const stripped = params.original.replace(existingPattern, "").replace(/\n{3,}/g, "\n\n").trim(); |
There was a problem hiding this comment.
Preserve surrounding note formatting when deleting inline block
When fallback suppression removes a managed dreaming block, removeManagedMarkdownBlock rewrites the entire daily note with .replace(/\n{3,}/g, "\n\n").trim(), which collapses any 3+ blank-line runs and strips leading/trailing whitespace outside the managed section. This means an unrelated user-authored note (for example, intentional spacing in prose or code snippets) is silently reformatted whenever a fallback-triggered cleanup runs, so the patch can mutate content beyond the targeted dreaming block.
Useful? React with 👍 / 👎.
|
Picked up the daily-note formatting follow-up. Removing a managed dreaming block now only removes the targeted block instead of collapsing unrelated blank lines or trimming the rest of the note, and I added a regression test for that cleanup path. Local Vitest is still blocked in this environment by the repo's current non-isolated-runner issue on Node 22.12.0, so CI is doing the real verification. |
|
Picked up the last daily-note cleanup edge case. Removing the managed dreaming block now trims only the seam it creates, so the surrounding note spacing stays intact without ballooning the gap. Local Vitest is still blocked here by the current non-isolated runner issue, but I verified the exact removal case and the normal trailing-newline path with a direct targeted runtime check. |
|
Pushed an empty |
|
Pushed |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b3cfd6505
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const narrativeResult = await generateAndAppendDreamNarrative({ | ||
| subagent: params.subagent, | ||
| workspaceDir: params.workspaceDir, | ||
| data, | ||
| nowMs, |
There was a problem hiding this comment.
Persist phase outputs before optional narrative call
runLightDreaming now awaits generateAndAppendDreamNarrative before calling writeDailyDreamingPhaseBlock/recordDreamingPhaseSignals (same ordering change in runRemDreaming). Because generateAndAppendDreamNarrative waits on subagent.run without a hard timeout, a stalled subagent request can block this path indefinitely and prevent any light/REM phase artifacts from being written. Before this commit, those core phase writes happened first, so narrative stalls did not suppress primary dreaming output.
Useful? React with 👍 / 👎.
8b3cfd6 to
40a2e96
Compare
|
Rebased on current main to pick up the talk silence-timeout docs baseline that was failing the runtime-config shard. I verified the expected 700 ms/900 ms docs and help text are present on this head; GitHub checks are rerunning now. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 40a2e9652a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| `memory-core: narrative fallback failed for ${params.data.phase} phase (${formatFallbackWriteFailure(fallbackErr)})`, | ||
| ); | ||
| } | ||
| return null; | ||
| return { runId: null, fallbackUsed: true }; |
There was a problem hiding this comment.
Report fallback only after fallback entry is persisted
This path marks fallbackUsed: true even when the request-scoped fallback write fails, because the appendNarrativeEntry error is swallowed and the function still returns fallback as used. In that failure mode (for example, DREAMS.md is not writable), runLightDreaming/runRemDreaming will suppress inline dreaming output based on this flag, so the phase output is lost from both the daily note and DREAMS.md.
Useful? React with 👍 / 👎.
|
Synced the MiniMax FAQ troubleshooting model refs that were failing the agentic-agents docs-sync shard. Verified the expected MiniMax M2.7 refs are present in docs/help/faq.md and git diff --check is clean; GitHub checks are rerunning. |
|
Added the stale internal docs link fixes that were failing check-docs after the previous push. Verified docs-link-audit now reports broken_links=0; GitHub checks are rerunning. |
109c95c to
2f0577f
Compare
|
Rebased this branch onto current main and tightened the dreaming fallback flow: phase output is persisted before narrative generation, request-scoped fallback only reports fallbackUsed after the fallback entry is written, and fallback suppression still removes raw inline dumps after a successful fallback. Local check: pnpm test -- extensions/memory-core/src/dreaming-markdown.test.ts extensions/memory-core/src/dreaming-narrative.test.ts extensions/memory-core/src/dreaming-phases.test.ts passes. |
|
Swept this PR again. Existing CI is green, and the inline Greptile/Codex comments are addressed on the current head: newline preservation, targeted managed-block removal, fallback success reporting, and phase-write ordering are covered. Focused tests passed locally: |
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Keep this PR open but not merge-ready: current main solved the default daily-memory pollution by shipping separate storage by default, yet explicit inline mode remains an opt-in path and this branch carries unique inline cleanup. The patch still misses the detached cron/system-event path and can double-record dream completion state when it removes the inline block, and it still needs real behavior proof plus a rebase. Canonical path: Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review. So I’m closing this here because the remaining work is already tracked in the canonical issue. Review detailsBest possible solution: Close this stale PR. The latest review rated it F, the branch still lacks merge-ready proof, and there has been no human follow-up after the durable review. Do we have a high-confidence way to reproduce the issue? Yes, source inspection gives a high-confidence inline-mode path: current main writes the light/REM daily block before narrative fallback, and explicit inline storage remains supported. I did not run a live fallback scenario because this review was read-only. Is this the best way to solve the issue? No. The PR is a plausible mitigation, but the best fix must cover detached cron fallback and avoid using the completion writer as a removal helper that emits another completion event. Security review: Security review cleared: The diff does not add dependencies, workflow execution, secret handling, or a new code-execution path; the intended memory change reduces raw metadata exposure but still needs functional fixes. AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model gpt-5.5, reasoning high; reviewed against fa614d0907e8. |
Keep request-scoped dreaming fallbacks from writing light/REM metadata blocks into daily memory files while still allowing the fallback dream diary entry. Fixes openclaw#70509
2f0577f to
d2b530a
Compare
|
Rebased this branch onto current upstream/main and resolved the memory-core narrative conflict. The request-scoped fallback success path keeps its info-level log, and the focused memory-core tests pass locally: |
|
ClawSweeper PR egg 🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat. Where did the egg go?
|
|
This pull request has been automatically marked as stale due to inactivity. |
|
ClawSweeper applied the proposed close for this PR.
|
Summary
Problem
When dreaming narrative generation falls back because the subagent runtime is request-scoped, the inline
memory/YYYY-MM-DD.mddreaming blocks still write the raw light/REM candidate dump. That leaves daily memory files full of structured dreaming metadata instead of normal human-readable notes.Why it matters
What changed
What did NOT change
DREAMS.mdwrites.MEMORY.mdapplication.Change Type
Scope
extensions/memory-core/src/dreaming-phases.tsextensions/memory-core/src/dreaming-markdown.tsextensions/memory-core/src/dreaming-narrative.tsextensions/memory-core/src/short-term-promotion.tsextensions/memory-core/src/dreaming-phases.test.tsLinked Issue
Closes #70509
User-visible / Behavior Changes
memory/YYYY-MM-DD.md.DREAMS.md, so the dreaming run still leaves a human-readable artifact.Security Impact
Low risk, positive direction. This reduces the chance that raw dreaming metadata and evidence-style snippets get copied into user-facing daily memory files during fallback runs.
Repro + Verification
Environment
upstream/mainSteps
RequestScopedSubagentRuntimeError.memory/YYYY-MM-DD.mdandDREAMS.md.Expected
## Light Sleep/## REM Sleepmetadata dump.DREAMS.mdstill receives the fallback narrative entry.Actual before this patch
Evidence
git diff --checkextensions/memory-core/src/dreaming-phases.test.tsHuman Verification
test/vitest/vitest.contracts-channel-surface.config.ts.tsc -p tsconfig.extensions.test.json --noEmitalso fails in this environment on many unrelated missing extension dependencies, but none of the reported errors were in the touched dreaming files.Compatibility / Migration
None.
Failure Recovery
Revert this PR. The change is isolated to dreaming inline-write suppression during fallback and does not alter stored data formats.
Risks and Mitigations