Skip to content

fix(memory-core): route dreaming corpus through session metadata#96517

Merged
jalehman merged 1 commit into
openclaw:mainfrom
jalehman:josh/path3-memory-host-session-corpus
Jun 24, 2026
Merged

fix(memory-core): route dreaming corpus through session metadata#96517
jalehman merged 1 commit into
openclaw:mainfrom
jalehman:josh/path3-memory-host-session-corpus

Conversation

@jalehman

@jalehman jalehman commented Jun 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What Problem This Solves

This PR closes the remaining Path 3 3.1b memory-host session corpus follow-up for the Dreaming corpus defects reported in #90313. It keeps retained reset/delete archive artifacts available to the shared memory-host transcript corpus for QMD and memory_search, while preventing Dreaming from re-ingesting those archives, and it classifies cron-descended child transcripts through existing session metadata.

Why This Change Was Made

#93187 found the right bug class, but it pulled the fix through legacy helper surfaces and SDK export churn. This maintainer-owned slice keeps the owner boundary on the existing session transcript corpus seam in packages/memory-host-sdk, avoids a new SDK API or SQLite contract, and adds the ratchet coverage needed for the Path 3 tracker.

User Impact

Dreaming session-corpus files stop filling with reset/delete archive duplicates and cron-spawned child transcript noise. Shared corpus consumers still see archive artifacts where they are expected, so historical transcript search behavior is preserved.

Evidence

  • node scripts/run-vitest.mjs test/scripts/plugin-sdk-surface-report.test.ts packages/memory-host-sdk/src/host/session-files.test.ts extensions/memory-core/src/dreaming-phases.test.ts test/scripts/check-session-accessor-boundary.test.ts - passed on rebased head cc7fac26ee9 (72 unit/script tests + 48 memory-core extension tests).
  • node scripts/check-session-accessor-boundary.mjs - passed.
  • node scripts/plugin-sdk-surface-report.mjs --check - passed.
  • pnpm exec oxfmt --check --threads=1 ... - passed for all changed files.
  • node scripts/run-oxlint.mjs ... - passed for all changed files.
  • node scripts/run-tsgo.mjs -p tsconfig.core.json ... - passed.
  • node scripts/run-tsgo.mjs -p tsconfig.extensions.json ... - passed.
  • node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.test.packages.json ... - passed.
  • node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json ... - passed.
  • node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.test.src.json ... - passed.
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base upstream/main ... - clean, no accepted/actionable findings; overall patch correct (0.84).
  • CI attribution: old head 3d88a034986 failed checks-node-compact-small-whole-2 in test/scripts/plugin-sdk-surface-report.test.ts with expected +0 to be 1 at line 103. That check was unrelated to this PR's touched files and was fixed upstream by d01c2906015 test(sdk): assert surface budget growth guard and 5eec2158ea0 fix(sdk): restore post-build surface budgets; this branch was rebased to inherit those fixes and pushed as cc7fac26ee9. GitHub CI is now green on cc7fac26ee9 with merge state CLEAN (69 successful checks, 29 skipped, CodeQL neutral, no pending or failing checks).

Real behavior proof

Behavior addressed: Dreaming learns only from live non-cron session transcripts while the shared memory-host session corpus still exposes retained reset/delete archive artifacts for QMD and memory_search.

Real environment tested: Local live OpenClaw gateway from /Users/phaedrus/Projects/clawdbot, temporarily loaded with pre-rebase proof SHA 7e3590a3f897a536e98905774ad948e49bfddbdd, OpenClaw 2026.6.10 (7e3590a), then restored to the original detached checkout 352c4a40d838d7deb0bfdf38fa33665aef00be72. Rebased PR head is cc7fac26ee980af61c53cb2501c2e4a301ad18be; focused post-rebase tests/checks above passed after the conflict-free code rebase.

Exact steps or command run after this patch:

cd /Users/phaedrus/Projects/clawdbot
git status -sb
git rev-parse HEAD
pnpm openclaw gateway status --deep
pnpm openclaw gateway call health --json --timeout 10000
curl -fsS http://127.0.0.1:18789/readyz
curl -fsS http://127.0.0.1:18789/healthz
git fetch /Users/phaedrus/.codex/worktrees/405b/clawdbot josh/path3-memory-host-session-corpus:refs/heads/josh/path3-memory-host-session-corpus-live-proof
git switch josh/path3-memory-host-session-corpus-live-proof
pnpm build
pnpm openclaw gateway restart
pnpm openclaw --version
pnpm openclaw gateway status --deep
pnpm openclaw gateway call health --json --timeout 10000
curl -fsS http://127.0.0.1:18789/readyz
curl -fsS http://127.0.0.1:18789/healthz
pnpm exec tsx /tmp/path3-memory-host-session-corpus-live-proof.ts
wc -l /tmp/openclaw-path3-memory-host-corpus-live-proof/workspace/memory/.dreams/session-corpus/2026-04-05.txt
grep -E "Live primary session insight|Normal child should be dreamed|Archived session should not be dreamed|Cron parent should not be dreamed|Cron child should not be dreamed|Cron grandchild should not be dreamed|Orphan cron child should not be dreamed" /tmp/openclaw-path3-memory-host-corpus-live-proof/workspace/memory/.dreams/session-corpus/2026-04-05.txt
find /tmp/openclaw-path3-memory-host-corpus-live-proof/state -maxdepth 4 -type f | sort
rm -rf /tmp/openclaw-path3-memory-host-corpus-live-proof /tmp/path3-memory-host-session-corpus-live-proof.ts
git switch --detach 352c4a40d838d7deb0bfdf38fa33665aef00be72
pnpm build
pnpm openclaw gateway restart
pnpm openclaw --version
pnpm openclaw gateway status --deep
pnpm openclaw gateway call health --json --timeout 10000
curl -fsS http://127.0.0.1:18789/readyz
curl -fsS http://127.0.0.1:18789/healthz

Evidence after fix: The isolated live-proof script seeded temp OPENCLAW_STATE_DIR and OPENCLAW_CONFIG_PATH with active, archive-only, cron parent, cron child, cron grandchild, orphan cron child, and normal child transcripts. It reported verdict: "PASS", corpusEntryCount: 7, archiveArtifactVisibleToCorpus: true, cron-classified paths cron-parent.jsonl, cron-child.jsonl, cron-grandchild.jsonl, and orphan-child.jsonl, and a persisted Dreaming corpus file with 28 lines. The grep proof showed only Live primary session insight and Normal child should be dreamed lines in memory/.dreams/session-corpus/2026-04-05.txt; archive and cron phrases were absent. Persisted temp state included state/state/openclaw.sqlite, proving the Dreaming plugin-state seam wrote through SQLite. Cleanup removed both the temp proof root and temp proof script. The restored live gateway later reported OpenClaw 2026.6.10 (352c4a4), ok: true, memoryCoreLoaded: true, eventLoopDegraded: false, /readyz ready, and /healthz live.

Observed result after fix: Shared corpus enumeration still includes retained archive artifacts, but Dreaming skips archive artifacts before parsing and marks cron-lineage transcripts as seen without appending their content to the Dreaming session-corpus. Live/normal session content is appended as expected.

What was not tested: The future SQLite transcript flip, doctor import/migration behavior, new SDK APIs, and defense-in-depth prompt-stem filtering were not tested because they are out of scope for this 3.1b seam slice. The real live proof was run before the rebase from 7e3590a3f897a536e98905774ad948e49bfddbdd to cc7fac26ee980af61c53cb2501c2e4a301ad18be; the rebased head received the focused test/guard/type/autoreview proof listed above.

Related

@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-core Extension: memory-core scripts Repository scripts size: M maintainer Maintainer-authored PR labels Jun 24, 2026
@jalehman jalehman mentioned this pull request Jun 24, 2026
Closed
@clawsweeper

clawsweeper Bot commented Jun 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 24, 2026, 2:28 PM ET / 18:28 UTC.

Summary
The PR filters Dreaming session ingestion away from archive-artifact corpus entries, computes cron-descended session lineage in the memory-host transcript corpus metadata path, and updates focused regression and boundary tests.

PR surface: Source +73, Tests +158, Other +1. Total +232 across 6 files.

Reproducibility: yes. Current main is source-reproducible: Dreaming sends every memory-host corpus entry except checkpoints into ingestion, while reset/deleted archive artifacts remain part of the shared usage-counted corpus and cron classification is direct-key only.

Review metrics: 1 noteworthy metric.

  • Corpus Eligibility Gates: 2 changed. The PR adds a Dreaming archive-artifact skip and a cron-lineage closure, both affecting which persisted transcript content can enter memory state.

Stored data model
Persistent data-model change detected: persistent cache schema: packages/memory-host-sdk/src/host/session-transcript-corpus.ts, serialized state: extensions/memory-core/src/dreaming-phases.test.ts, serialized state: packages/memory-host-sdk/src/host/session-files.test.ts, serialized state: packages/memory-host-sdk/src/host/session-transcript-corpus.ts, unknown-data-model-change: extensions/memory-core/src/dreaming-phases.test.ts, unknown-data-model-change: packages/memory-host-sdk/src/host/session-files.test.ts, and 2 more. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #90313
Summary: This PR is the active candidate fix for the linked Dreaming archive-ingestion and cron-lineage defects.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] The PR intentionally changes which persisted transcript artifacts and cron-descended session metadata lineages can enter Dreaming memory state, so maintainers should confirm the session-state behavior before merge.
  • [P1] The live gateway proof predates the final rebase, although current-head focused tests, guards, type/lint checks, SDK-surface proof, autoreview, Real behavior proof, and CI are green.

Maintainer options:

  1. Accept Session-State Policy After Owner Review (recommended)
    Proceed once the memory/session owner agrees that Dreaming should skip archive artifacts and cron-descended children while shared corpus consumers still see retained archives.
  2. Repeat Current-Head Live Proof
    Ask for a current-head live gateway run only if maintainers do not want to rely on the pre-rebase live proof plus current-head focused checks and green CI.
  3. Pause For Broader Migration Stack
    Pause this PR if maintainers want the Dreaming corpus change to wait for the broader Path 3 session/transcript migration queue.

Next step before merge

  • [P2] No repair lane is needed; the remaining action is maintainer session-state review and merge/landing judgment on a clean, proof-sufficient PR.

Security
Cleared: No concrete security or supply-chain concern was found; the diff does not change dependencies, workflows, secrets, permissions, package resolution, or downloaded code execution.

Review details

Best possible solution:

Land this seam-owned fix after memory/session owners accept the intentional Dreaming session-state behavior, use it to resolve #90313, and leave the broader SQLite migration work tracked by #88838.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main is source-reproducible: Dreaming sends every memory-host corpus entry except checkpoints into ingestion, while reset/deleted archive artifacts remain part of the shared usage-counted corpus and cron classification is direct-key only.

Is this the best way to solve the issue?

Yes. Filtering archive artifacts at the Dreaming collector preserves QMD and memory_search archive indexing, and computing cron lineage in the memory-host corpus metadata seam avoids new SDK, config, or SQLite surface.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against c2d2f7fef9bd.

Label changes

Label justifications:

  • P2: This is a normal-priority memory correctness fix for Dreaming corpus quality with limited blast radius.
  • merge-risk: 🚨 session-state: The PR changes how persisted transcript artifacts and session metadata lineages are classified before they enter Dreaming memory state.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes live gateway terminal proof for the behavior plus current-head focused test, guard, type, format, lint, SDK-surface, autoreview, and CI proof after rebase.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes live gateway terminal proof for the behavior plus current-head focused test, guard, type, format, lint, SDK-surface, autoreview, and CI proof after rebase.
Evidence reviewed

PR surface:

Source +73, Tests +158, Other +1. Total +232 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 2 83 10 +73
Tests 3 158 0 +158
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 1 0 +1
Total 6 242 10 +232

What I checked:

Likely related people:

  • jalehman: Authored merged PR 96162 that introduced the accessor-backed memory transcript corpus seam and has prior Dreaming session-store work in this area, so this PR is connected to prior merged feature history beyond only opening the branch. (role: recent area contributor; confidence: high; commits: c24d266b2d09, a1b01f0281d0, cc7fac26ee98; files: packages/memory-host-sdk/src/host/session-transcript-corpus.ts, packages/memory-host-sdk/src/host/session-files.ts, extensions/memory-core/src/dreaming-phases.ts)
  • Cameron Beeley: Current-main blame on the Dreaming corpus loop and memory-host classifier points to commit 9aea104, which carried the current accessor-backed files into main. (role: recent area contributor; confidence: medium; commits: 9aea104cc88a; files: extensions/memory-core/src/dreaming-phases.ts, packages/memory-host-sdk/src/host/session-transcript-corpus.ts, packages/memory-host-sdk/src/host/session-files.ts)
  • vignesh07: History search ties the session transcript Dreaming ingestion and checkpointing path to commit 6951765. (role: introduced Dreaming ingestion behavior; confidence: medium; commits: 695176542fcf; files: extensions/memory-core/src/dreaming-phases.ts)
  • Rick_Xu: Commit 2fe1ff8 introduced reset/deleted session archives into the usage-counted predicate that this PR deliberately preserves for shared consumers. (role: introduced archive usage behavior; confidence: high; commits: 2fe1ff8ea8ab; files: src/config/sessions/artifacts.ts)
  • vincentkoc: Commit b7d59f7 added archived QMD session transcript export behavior, which is the sibling shared-corpus behavior this PR avoids breaking. (role: adjacent archive corpus contributor; confidence: medium; commits: b7d59f783152; files: extensions/memory-core/src/memory/qmd-manager.ts, src/config/sessions/artifacts.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 24, 2026
@clawsweeper clawsweeper Bot added P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 24, 2026
@jalehman
jalehman force-pushed the josh/path3-memory-host-session-corpus branch from 3d88a03 to cc7fac2 Compare June 24, 2026 18:16
@jalehman
jalehman merged commit da50a45 into openclaw:main Jun 24, 2026
105 checks passed
@jalehman
jalehman deleted the josh/path3-memory-host-session-corpus branch June 24, 2026 18:29
@jalehman jalehman mentioned this pull request Jun 24, 2026
Merged
@github-actions github-actions Bot mentioned this pull request Jun 24, 2026
Open
This was referenced Jun 24, 2026
Merged
Merged
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 25, 2026
@jalehman
fix(memory-core): route dreaming corpus through session corpus metada…
2304088 
…ta (openclaw#96517)
This was referenced Jun 26, 2026
Open
Closed
Closed
QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026
@jalehman @QiuYuang
fix(memory-core): route dreaming corpus through session corpus metada…
ad6d0b1 
…ta (openclaw#96517)
@xialonglee xialonglee mentioned this pull request Jul 6, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

extensions: memory-core Extension: memory-core maintainer Maintainer-authored PR merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Dreaming session-corpus: cron classification doesn't follow subagent parentage; archived (.deleted/.reset) transcripts re-ingested

1 participant

@jalehman