Skip to content

Commit 0c4e0d7

Browse files
gumadeirasgumadeiras
authored
memory: block dreaming self-ingestion (#66852)
Merged via squash. Prepared head SHA: 4742656 Co-authored-by: gumadeiras <[email protected]> Co-authored-by: gumadeiras <[email protected]> Reviewed-by: @gumadeiras
1 parent 5702ab6 commit 0c4e0d7

8 files changed

Lines changed: 477 additions & 9 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ Docs: https://docs.openclaw.ai
3535
- Agents/context engines: preserve prompt-only token counts, not full request totals, when deferred maintenance reuses after-turn runtime context so background compaction bookkeeping matches the active prompt window. (#66820) thanks @jalehman.
3636
- BlueBubbles/inbound: add a persistent file-backed GUID dedupe so MessagePoller webhook replays after BB Server restart or reconnect no longer cause the agent to re-reply to already-handled messages. (#19176, #12053, #66816) Thanks @omarshahine.
3737
- Secrets/plugins/status: align SecretRef inspect-vs-strict handling across plugin preload, read-only status/agents surfaces, and runtime auth paths so unresolved refs no longer crash read-only CLI flows while runtime-required non-env refs stay strict. (#66818) Thanks @joshavant.
38+
- Memory/dreaming: stop ordinary transcripts that merely quote the dream-diary prompt from being classified as internal dreaming runs and silently dropped from session recall ingestion. (#66852) Thanks @gumadeiras.
3839

3940
## 2026.4.14
4041

‎docs/concepts/dreaming.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,9 @@ After each phase has enough material, `memory-core` runs a best-effort backgroun
8080
subagent turn (using the default runtime model) and appends a short diary entry.
8181

8282
This diary is for human reading in the Dreams UI, not a promotion source.
83+
Dreaming-generated diary/report artifacts are excluded from short-term
84+
promotion. Only grounded memory snippets are eligible to promote into
85+
`MEMORY.md`.
8386

8487
There is also a grounded historical backfill lane for review and recovery work:
8588

‎extensions/memory-core/src/dreaming-phases.test.ts‎

Lines changed: 90 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -713,13 +713,101 @@ describe("memory-core dreaming phases", () => {
713713
expect(Object.keys(sessionIngestion.files)).toHaveLength(1);
714714
expect(Object.values(sessionIngestion.files)).toEqual([
715715
expect.objectContaining({
716-
lineCount: 2,
717-
lastContentLine: 2,
716+
lineCount: 0,
717+
lastContentLine: 0,
718718
contentHash: expect.any(String),
719719
}),
720720
]);
721721
});
722722

723+
it("does not reread unchanged dreaming-generated transcripts after checkpointing skip state", async () => {
724+
const workspaceDir = await createDreamingWorkspace();
725+
vi.stubEnv("OPENCLAW_TEST_FAST", "1");
726+
vi.stubEnv("OPENCLAW_STATE_DIR", path.join(workspaceDir, ".state"));
727+
const sessionsDir = resolveSessionTranscriptsDirForAgent("main");
728+
await fs.mkdir(sessionsDir, { recursive: true });
729+
const transcriptPath = path.join(sessionsDir, "dreaming-narrative.jsonl");
730+
await fs.writeFile(
731+
transcriptPath,
732+
[
733+
JSON.stringify({
734+
type: "custom",
735+
customType: "openclaw:bootstrap-context:full",
736+
data: {
737+
runId: "dreaming-narrative-light-1775894400455",
738+
sessionId: "dream-session-1",
739+
},
740+
}),
741+
JSON.stringify({
742+
type: "message",
743+
message: {
744+
role: "user",
745+
timestamp: "2026-04-05T18:01:00.000Z",
746+
content: [
747+
{ type: "text", text: "Write a dream diary entry from these memory fragments." },
748+
],
749+
},
750+
}),
751+
].join("\n") + "\n",
752+
"utf-8",
753+
);
754+
const mtime = new Date("2026-04-05T18:05:00.000Z");
755+
await fs.utimes(transcriptPath, mtime, mtime);
756+
757+
const { beforeAgentReply } = createHarness(
758+
{
759+
agents: {
760+
defaults: {
761+
workspace: workspaceDir,
762+
},
763+
list: [{ id: "main", workspace: workspaceDir }],
764+
},
765+
plugins: {
766+
entries: {
767+
"memory-core": {
768+
config: {
769+
dreaming: {
770+
enabled: true,
771+
phases: {
772+
light: {
773+
enabled: true,
774+
limit: 20,
775+
lookbackDays: 7,
776+
},
777+
},
778+
},
779+
},
780+
},
781+
},
782+
},
783+
},
784+
workspaceDir,
785+
);
786+
787+
try {
788+
await beforeAgentReply(
789+
{ cleanedBody: "__openclaw_memory_core_light_sleep__" },
790+
{ trigger: "heartbeat", workspaceDir },
791+
);
792+
793+
const readFileSpy = vi.spyOn(fs, "readFile");
794+
await beforeAgentReply(
795+
{ cleanedBody: "__openclaw_memory_core_light_sleep__" },
796+
{ trigger: "heartbeat", workspaceDir },
797+
);
798+
799+
expect(
800+
readFileSpy.mock.calls.some(
801+
([target]) => typeof target === "string" && target === transcriptPath,
802+
),
803+
).toBe(false);
804+
readFileSpy.mockRestore();
805+
} finally {
806+
vi.restoreAllMocks();
807+
vi.unstubAllEnvs();
808+
}
809+
});
810+
723811
it("dedupes reset/deleted session archives instead of double-ingesting", async () => {
724812
const workspaceDir = await createDreamingWorkspace();
725813
vi.stubEnv("OPENCLAW_TEST_FAST", "1");

‎extensions/memory-core/src/dreaming-phases.ts‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -739,10 +739,7 @@ async function collectSessionIngestionBatches(params: {
739739
mtimeMs: Math.floor(Math.max(0, stat.mtimeMs)),
740740
size: Math.floor(Math.max(0, stat.size)),
741741
};
742-
const cursorAtEnd =
743-
previous !== undefined &&
744-
previous.lineCount > 0 &&
745-
previous.lastContentLine >= previous.lineCount;
742+
const cursorAtEnd = previous !== undefined && previous.lastContentLine >= previous.lineCount;
746743
const unchanged =
747744
Boolean(previous) &&
748745
previous.mtimeMs === fingerprint.mtimeMs &&

‎extensions/memory-core/src/short-term-promotion.test.ts‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,95 @@ describe("short-term promotion", () => {
203203
});
204204
});
205205

206+
it("ignores contaminated dreaming snippets when recording short-term recalls", async () => {
207+
await withTempWorkspace(async (workspaceDir) => {
208+
await recordShortTermRecalls({
209+
workspaceDir,
210+
query: "action preference",
211+
results: [
212+
{
213+
path: "memory/2026-04-03.md",
214+
source: "memory",
215+
startLine: 1,
216+
endLine: 1,
217+
score: 0.92,
218+
snippet:
219+
"Candidate: Default to action. confidence: 0.76 evidence: memory/.dreams/session-corpus/2026-04-08.txt:1-1 recalls: 3 status: staged",
220+
},
221+
],
222+
});
223+
224+
expect(
225+
JSON.parse(await fs.readFile(resolveShortTermRecallStorePath(workspaceDir), "utf-8")),
226+
).toMatchObject({
227+
version: 1,
228+
entries: {},
229+
});
230+
});
231+
});
232+
233+
it("ignores bullet-prefixed dreaming snippets when recording short-term recalls", async () => {
234+
await withTempWorkspace(async (workspaceDir) => {
235+
await recordShortTermRecalls({
236+
workspaceDir,
237+
query: "action preference",
238+
results: [
239+
{
240+
path: "memory/2026-04-03.md",
241+
source: "memory",
242+
startLine: 1,
243+
endLine: 5,
244+
score: 0.92,
245+
snippet: [
246+
"- Candidate: Default to action.",
247+
" - confidence: 0.76",
248+
" - evidence: memory/.dreams/session-corpus/2026-04-08.txt:1-1",
249+
" - recalls: 3",
250+
" - status: staged",
251+
].join("\n"),
252+
},
253+
],
254+
});
255+
256+
expect(
257+
JSON.parse(await fs.readFile(resolveShortTermRecallStorePath(workspaceDir), "utf-8")),
258+
).toMatchObject({
259+
version: 1,
260+
entries: {},
261+
});
262+
});
263+
});
264+
265+
it("keeps ordinary snippets that only quote dreaming prompt markers", async () => {
266+
await withTempWorkspace(async (workspaceDir) => {
267+
await recordShortTermRecalls({
268+
workspaceDir,
269+
query: "debug note",
270+
results: [
271+
{
272+
path: "memory/2026-04-03.md",
273+
source: "memory",
274+
startLine: 1,
275+
endLine: 1,
276+
score: 0.75,
277+
snippet:
278+
"Debug note: quote Write a dream diary entry from these memory fragments for docs, but do not use dreaming-narrative-like labels in production.",
279+
},
280+
],
281+
});
282+
283+
const store = JSON.parse(
284+
await fs.readFile(resolveShortTermRecallStorePath(workspaceDir), "utf-8"),
285+
) as { entries: Record<string, { snippet: string }> };
286+
expect(Object.values(store.entries)).toEqual([
287+
expect.objectContaining({
288+
snippet:
289+
"Debug note: quote Write a dream diary entry from these memory fragments for docs, but do not use dreaming-narrative-like labels in production.",
290+
}),
291+
]);
292+
});
293+
});
294+
206295
it("records recalls and ranks candidates with weighted scores", async () => {
207296
await withTempWorkspace(async (workspaceDir) => {
208297
await recordShortTermRecalls({
@@ -940,6 +1029,86 @@ describe("short-term promotion", () => {
9401029
});
9411030
});
9421031

1032+
it("does not rank contaminated dreaming snippets from an existing short-term store", async () => {
1033+
await withTempWorkspace(async (workspaceDir) => {
1034+
const storePath = resolveShortTermRecallStorePath(workspaceDir);
1035+
await fs.writeFile(
1036+
storePath,
1037+
JSON.stringify(
1038+
{
1039+
version: 1,
1040+
updatedAt: "2026-04-04T00:00:00.000Z",
1041+
entries: {
1042+
contaminated: {
1043+
key: "contaminated",
1044+
path: "memory/2026-04-03.md",
1045+
startLine: 1,
1046+
endLine: 1,
1047+
source: "memory",
1048+
snippet:
1049+
"Reflections: Theme: assistant. confidence: 1.00 evidence: memory/.dreams/session-corpus/2026-04-08.txt:2-2 recalls: 4 status: staged",
1050+
recallCount: 4,
1051+
dailyCount: 0,
1052+
groundedCount: 0,
1053+
totalScore: 3.6,
1054+
maxScore: 0.95,
1055+
firstRecalledAt: "2026-04-03T00:00:00.000Z",
1056+
lastRecalledAt: "2026-04-04T00:00:00.000Z",
1057+
queryHashes: ["a", "b"],
1058+
recallDays: ["2026-04-03", "2026-04-04"],
1059+
conceptTags: ["assistant"],
1060+
},
1061+
},
1062+
},
1063+
null,
1064+
2,
1065+
),
1066+
"utf-8",
1067+
);
1068+
1069+
const ranked = await rankShortTermPromotionCandidates({
1070+
workspaceDir,
1071+
minScore: 0,
1072+
minRecallCount: 0,
1073+
minUniqueQueries: 0,
1074+
});
1075+
1076+
expect(ranked).toEqual([]);
1077+
});
1078+
});
1079+
1080+
it("treats diff-prefixed dreaming snippets as contaminated", () => {
1081+
expect(
1082+
__testing.isContaminatedDreamingSnippet(
1083+
"@@ -1,1 - Candidate: Default to action. confidence: 0.76 evidence: memory/.dreams/session-corpus/2026-04-08.txt:1-1 recalls: 3 status: staged",
1084+
),
1085+
).toBe(true);
1086+
});
1087+
1088+
it("treats bracket-prefixed dreaming snippets as contaminated", () => {
1089+
expect(
1090+
__testing.isContaminatedDreamingSnippet(
1091+
"([ Candidate: Default to action. confidence: 0.76 evidence: memory/.dreams/session-corpus/2026-04-08.txt:1-1 recalls: 3 status: staged",
1092+
),
1093+
).toBe(true);
1094+
});
1095+
1096+
it("does not treat ordinary candidate notes with daily-memory evidence as contaminated", () => {
1097+
expect(
1098+
__testing.isContaminatedDreamingSnippet(
1099+
"Candidate: move backups weekly. confidence: 0.76 evidence: memory/2026-04-08.md:1-1",
1100+
),
1101+
).toBe(false);
1102+
});
1103+
1104+
it("treats transcript-style dreaming prompt echoes as contaminated", () => {
1105+
expect(
1106+
__testing.isContaminatedDreamingSnippet(
1107+
"[main/dreaming-narrative-light.jsonl#L1] User: Write a dream diary entry from these memory fragments:",
1108+
),
1109+
).toBe(true);
1110+
});
1111+
9431112
it("skips direct candidates that exceed maxAgeDays during apply", async () => {
9441113
await withTempWorkspace(async (workspaceDir) => {
9451114
const applied = await applyShortTermPromotions({
@@ -987,6 +1156,53 @@ describe("short-term promotion", () => {
9871156
});
9881157
});
9891158

1159+
it("does not append contaminated dreaming snippets during direct apply", async () => {
1160+
await withTempWorkspace(async (workspaceDir) => {
1161+
const applied = await applyShortTermPromotions({
1162+
workspaceDir,
1163+
minScore: 0,
1164+
minRecallCount: 0,
1165+
minUniqueQueries: 0,
1166+
candidates: [
1167+
{
1168+
key: "memory:memory/2026-04-03.md:1:1",
1169+
path: "memory/2026-04-03.md",
1170+
startLine: 1,
1171+
endLine: 1,
1172+
source: "memory",
1173+
snippet:
1174+
"Candidate: Default to action. confidence: 0.76 evidence: memory/.dreams/session-corpus/2026-04-08.txt:1-1 recalls: 3 status: staged",
1175+
recallCount: 4,
1176+
avgScore: 0.97,
1177+
maxScore: 0.97,
1178+
uniqueQueries: 2,
1179+
firstRecalledAt: "2026-04-03T00:00:00.000Z",
1180+
lastRecalledAt: "2026-04-04T00:00:00.000Z",
1181+
ageDays: 0,
1182+
score: 0.99,
1183+
recallDays: ["2026-04-03", "2026-04-04"],
1184+
conceptTags: ["assistant"],
1185+
components: {
1186+
frequency: 1,
1187+
relevance: 1,
1188+
diversity: 1,
1189+
recency: 1,
1190+
consolidation: 1,
1191+
conceptual: 1,
1192+
},
1193+
},
1194+
],
1195+
});
1196+
1197+
expect(applied.applied).toBe(0);
1198+
await expect(
1199+
fs.readFile(path.join(workspaceDir, "MEMORY.md"), "utf-8"),
1200+
).rejects.toMatchObject({
1201+
code: "ENOENT",
1202+
});
1203+
});
1204+
});
1205+
9901206
it("applies promotion candidates to MEMORY.md and marks them promoted", async () => {
9911207
await withTempWorkspace(async (workspaceDir) => {
9921208
await writeDailyMemoryNote(workspaceDir, "2026-04-01", [

0 commit comments

Comments
 (0)