Skip to content

fix(memory-wiki): honor durable: true frontmatter in stale pages report#94509

Closed
chenweicong736 wants to merge 3 commits into
openclaw:mainfrom
chenweicong736:fix/wiki-stale-pages-durable-frontmatter-93485
Closed

fix(memory-wiki): honor durable: true frontmatter in stale pages report#94509
chenweicong736 wants to merge 3 commits into
openclaw:mainfrom
chenweicong736:fix/wiki-stale-pages-durable-frontmatter-93485

Conversation

@chenweicong736

Copy link
Copy Markdown

Summary

The memory-wiki Stale Pages report flags intentionally durable reference pages (concepts, syntheses) as aging or stale, drowning out real staleness signal in bridge-mode vaults. This PR introduces an opt-in durable: true frontmatter marker so authors can flag pages whose value comes from durability rather than recency.

  • Problem: In an 853-source / 5-concept / 7-synthesis vault, every durable reference page shows up in reports/stale-pages.md because the report only excluded report kind, not the durable kinds. Signal-to-noise drops to zero.
  • Solution: Honor a strict boolean durable frontmatter field at the stale-pages filter step. Pages with durable: true are excluded regardless of updatedAt.
  • What changed:
    • extensions/memory-wiki/src/markdown.tsWikiPageSummary.durable field + parser using asBoolean.
    • extensions/memory-wiki/src/compile.ts — One-line filter addition page.durable !== true.
    • extensions/memory-wiki/src/markdown.test.ts — Strict parser coverage (true/false/string/missing/no-frontmatter).
    • extensions/memory-wiki/src/compile.test.ts — End-to-end compile coverage (durable concept + durable synthesis + control).
    • docs/plugins/memory-wiki.md — New Opting pages out of the Stale Pages report section.
  • What did NOT change: No config surface, no threshold constants, no other dashboard report, no lint path, no agent-digest path. The marker is strict (real boolean only — string "true" is intentionally ignored to avoid accidental activation) and entirely opt-in.

Linked Issue

Fixes #93485

Changes

  • extensions/memory-wiki/src/markdown.ts — Adds durable?: boolean to WikiPageSummary and parses it via asBoolean(parsed.frontmatter.durable) in toWikiPageSummary; strict boolean only, so sloppy durable: "true" YAML is silently ignored. Honors the user's explicit request in memory-wiki: Stale Pages report flags intentionally durable references (concepts, syntheses) as aging #93485 to make durable a recognized frontmatter field.
  • extensions/memory-wiki/src/compile.ts — Adds page.durable !== true to the Stale Pages dashboard filter; a single-line addition that scopes the opt-in to the one report the issue describes.
  • extensions/memory-wiki/src/markdown.test.ts — New parses the durable opt-in flag from frontmatter strictly test covering true, false, string literal, missing field, and no-frontmatter cases.
  • extensions/memory-wiki/src/compile.test.ts — New excludes durable pages from the Stale Pages report test that writes three vault pages (durable concept + durable synthesis + non-durable concept), runs compileMemoryWikiVault, and asserts the report excludes durable pages but includes the control.
  • docs/plugins/memory-wiki.md — New subsection Opting pages out of the Stale Pages report under Dashboards and health reports so operators discover the marker.

Real behavior proof

  • Behavior addressed: Pages with durable: true frontmatter are excluded from reports/stale-pages.md regardless of updatedAt. Pages without the marker behave exactly as before.

  • Real environment tested: Linux 4.19.112 (x86_64), Node v24.13.1, branch fix/wiki-stale-pages-durable-frontmatter-93485 rebased on origin/main (commit 6d640a6556).

  • Exact steps or command run after this patch:

    node --import tsx --input-type=module <<'NODE'
    import { toWikiPageSummary, renderWikiMarkdown } from "./extensions/memory-wiki/src/markdown.ts";
    // ... (4 fixtures: true / false / "true" string / missing)
    NODE
    node scripts/run-vitest.mjs run extensions/memory-wiki/src/markdown.test.ts --maxWorkers=1
    node scripts/run-vitest.mjs run extensions/memory-wiki/src/compile.test.ts --maxWorkers=1
    
  • Evidence after fix:

    $ node --import tsx --input-type=module <<'NODE'
    ... (production parse proof, 4 cases)
    NODE
    durable: true (boolean) -> true
    durable: false (boolean) -> false
    durable: 'true' (string literal) -> undefined
    durable: missing field -> undefined
    
    $ node scripts/run-vitest.mjs run extensions/memory-wiki/src/markdown.test.ts --maxWorkers=1
     Test Files  1 passed (1)
          Tests  8 passed (8)
    
    $ node scripts/run-vitest.mjs run extensions/memory-wiki/src/compile.test.ts --maxWorkers=1
     Test Files  1 passed (1)
          Tests  14 passed (14)

    The new compile test writes three vault pages: concepts/alpha-concept.md (durable: true), syntheses/alpha-synthesis.md (durable: true), and concepts/beta-concept.md (no marker). All three carry updatedAt: 2024-01-01 (>2 years old, well past the 90-day stale threshold). After compileMemoryWikiVault, the test asserts that Alpha Concept and Alpha Synthesis are absent from reports/stale-pages.md while Beta Concept is present.

  • Observed result after fix:

    1. Pages with durable: true frontmatter are skipped at the stale-pages filter step regardless of updatedAt.
    2. Pages without the marker behave exactly as before the patch.
    3. String literals like durable: "true" are deliberately not honored — only real booleans activate the marker.
    4. No other dashboard report (open-questions, contradictions, claim-health, person-agent-directory, relationship-graph, provenance-coverage, privacy-review) is affected.
    5. The lint path's stale-page warning code and the agent-digest.json cache shape are unchanged.
  • What was not tested: Live openclaw wiki compile against a real bridge-mode vault (the existing test harness covers the same path through compileMemoryWikiVault); multi-vault compile; durable: true interaction with non-concept/synthesis kinds beyond entity.

AI-Assisted

This PR was prepared with assistance from Claude Code.
Co-Authored-By: Claude Opus 4.8 [email protected]


Fixes #93485

The Stale Pages report previously flagged every non-report page whose
updatedAt was older than 30 days, including intentionally durable
references such as concept and synthesis pages. Bridge-mode vaults
typically have many such pages, drowning out real staleness signal.

This change adds a strict boolean `durable` frontmatter marker. Pages
with `durable: true` are excluded from the stale-pages filter at
compile time. The marker is strict (asBoolean only accepts real booleans,
not string literals), is opt-in, and does not touch updatedAt, so other
freshness signals remain accurate. Lint and agent-digest callers are
unaffected; the change is scoped to the dashboard report filter.

Fixes openclaw#93485

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation extensions: memory-wiki size: S proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 18, 2026
@clawsweeper

clawsweeper Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 22, 2026, 4:02 AM ET / 08:02 UTC.

Summary
The branch adds a strict durable: true memory-wiki frontmatter marker, excludes those pages from reports/stale-pages.md, and documents/tests the parser and report behavior.

PR surface: Source +4, Tests +129, Docs +18. Total +151 across 5 files.

Reproducibility: yes. Current main source sends old concept and synthesis pages through assessPageFreshness into Stale Pages output when older than the 30-day aging threshold; I did not run compile because this review is read-only and compile writes artifacts.

Review metrics: 1 noteworthy metric.

  • Wiki frontmatter contract: 1 added. The PR documents and activates a persistent page marker that changes generated report output for existing vault files.

Stored data model
Persistent data-model change detected: serialized state: extensions/memory-wiki/src/compile.test.ts, unknown-data-model-change: extensions/memory-wiki/src/markdown.test.ts, vector/embedding metadata: extensions/memory-wiki/src/compile.test.ts, vector/embedding metadata: extensions/memory-wiki/src/markdown.test.ts, vector/embedding metadata: extensions/memory-wiki/src/markdown.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #93485
Summary: This PR is one candidate fix for the canonical memory-wiki Stale Pages durable-reference policy issue; another open PR proposes different default-exclusion semantics.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🐚 platinum hermit
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted generated reports/stale-pages.md or openclaw wiki compile output to the PR body; updating the body should trigger ClawSweeper re-review, or a maintainer can comment @clawsweeper re-review.
  • Get maintainer confirmation on opt-in durable: true markers versus default concept/synthesis exclusions.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: The PR body has parser terminal output and Vitest results, but not generated Stale Pages report output from the changed report path; add redacted terminal output, logs, screenshot, or linked artifact and update the PR body for re-review.

Risk before merge

  • [P1] Merging this PR creates a documented persistent durable: true page marker, so existing upgraded vault pages with that boolean frontmatter would stop appearing in reports/stale-pages.md.
  • [P1] The canonical issue has a competing open PR at fix(memory-wiki): exclude durable reference pages from stale report #94369 that chooses default concept/synthesis exclusions instead of opt-in page markers.
  • [P1] Contributor proof still does not show generated reports/stale-pages.md output or an openclaw wiki compile/small-vault artifact from the changed report path.

Maintainer options:

  1. Confirm opt-in durable markers
    Land this branch only if maintainers want authors to explicitly mark durable pages and accept that existing boolean durable: true pages will be hidden from Stale Pages.
  2. Prefer default durable page kinds
    Pause or close this PR in favor of fix(memory-wiki): exclude durable reference pages from stale report #94369 if concept and synthesis pages should be excluded by default.
  3. Require generated report proof
    Ask for redacted reports/stale-pages.md output from a small vault or openclaw wiki compile before considering merge.

Next step before merge

  • [P1] Needs maintainer policy choice and contributor generated-report proof; automation cannot safely choose between the two open durable-page semantics.

Security
Cleared: The diff only changes in-process memory-wiki parsing/report code, tests, and docs; it adds no dependency, workflow, permission, secret, network, or execution surface.

Review details

Best possible solution:

Choose one durable-page freshness policy, align docs/tests and any lint follow-through, and land one PR with generated Stale Pages output proof.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main source sends old concept and synthesis pages through assessPageFreshness into Stale Pages output when older than the 30-day aging threshold; I did not run compile because this review is read-only and compile writes artifacts.

Is this the best way to solve the issue?

Unclear. The code path is narrow, but maintainers need to choose opt-in durable markers versus default concept/synthesis exclusions and decide whether lint should align.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d3f7f7d1fc91.

Label changes

Label justifications:

  • P2: The PR targets a normal-priority bundled memory-wiki report-noise problem with limited blast radius and no outage, data-loss, or security signal.
  • merge-risk: 🚨 compatibility: The new documented durable: true frontmatter semantics can change generated Stale Pages output for existing upgraded vaults.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body has parser terminal output and Vitest results, but not generated Stale Pages report output from the changed report path; add redacted terminal output, logs, screenshot, or linked artifact and update the PR body for re-review.
Evidence reviewed

PR surface:

Source +4, Tests +129, Docs +18. Total +151 across 5 files.

View PR surface stats
Area Files Added Removed Net
Source 2 4 0 +4
Tests 2 129 0 +129
Docs 1 18 0 +18
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 5 151 0 +151

What I checked:

Likely related people:

  • vincentkoc: Git history ties the Stale Pages dashboard, freshness thresholds, lint stale-page path, and recent carried-forward memory-wiki code to Vincent Koc's commits. (role: introduced behavior / recent area contributor; confidence: high; commits: c73aeed929d1, 44fd8b0d6e38, 536c8a840bda; files: extensions/memory-wiki/src/compile.ts, extensions/memory-wiki/src/claim-health.ts, extensions/memory-wiki/src/lint.ts)
  • steipete: Path history shows Peter Steinberger authored memory-wiki docs and related memory helper refactors relevant to deciding whether durable becomes a documented page contract. (role: adjacent docs/refactor contributor; confidence: medium; commits: 9eacd29138a7, dffa88f39615; files: docs/plugins/memory-wiki.md, src/utils/boolean.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 18, 2026
@chenweicong736
chenweicong736 deleted the fix/wiki-stale-pages-durable-frontmatter-93485 branch July 7, 2026 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation extensions: memory-wiki merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory-wiki: Stale Pages report flags intentionally durable references (concepts, syntheses) as aging

1 participant