Skip to content

feat: soft-archive wings — exclude from search without deleting data (#332)#336

Open
matrix9neonebuchadnezzar2199-sketch wants to merge 2 commits intoMemPalace:developfrom
matrix9neonebuchadnezzar2199-sketch:feat/soft-archive-wings
Open

feat: soft-archive wings — exclude from search without deleting data (#332)#336
matrix9neonebuchadnezzar2199-sketch wants to merge 2 commits intoMemPalace:developfrom
matrix9neonebuchadnezzar2199-sketch:feat/soft-archive-wings

Conversation

@matrix9neonebuchadnezzar2199-sketch
Copy link
Copy Markdown

Closes #332

Summary

Add the ability to archive/unarchive wings so they are excluded from search results without deleting any data.

Problem

When a project ends or becomes inactive, its memories still appear in search results. The only current option is delete-wing (#310), which permanently destroys data. Users need a non-destructive way to hide stale wings from active search.

Solution

Config layer (config.py)

  • archive_wing(name) / unarchive_wing(name): toggle "archived": true flag in wing_config.json
  • get_archived_wings(): return set of archived wing names
  • load_wing_config() / save_wing_config(): public accessors for wing_config.json

Search layer (searcher.py)

  • Both search() and search_memories() now exclude archived wings by default using {"wing": {"$ne": ...}} in the ChromaDB where filter
  • When a specific --wing is requested, archive filtering is skipped (explicit intent)
  • search_memories() accepts include_archived=True to override exclusion

MCP layer (mcp_server.py)

  • mempalace_archive_wing / mempalace_unarchive_wing: new tools
  • mempalace_search: new include_archived parameter
  • mempalace_status: shows archived wings separately

Data safety

  • Zero data deletion — one metadata flag in wing_config.json
  • Fully reversible with unarchive
  • Archived wings remain accessible via include_archived=True

Testing

  • 11 new tests in tests/test_archive.py covering:
    • Archive/unarchive state transitions
    • Idempotency (double archive, unarchive non-archived)
    • Preservation of existing wing config fields
    • wing_config.json I/O including corrupt file handling
  • All existing tests pass (99 passed, 2 pre-existing Windows-only failures unrelated to this change)

Related: #331 (time-decay scoring — the other half of time-aware memory management)

Copy link
Copy Markdown

@web3guru888 web3guru888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved — Clean, well-structured implementation. Zero data deletion, fully reversible archiving. Tests are solid (11 tests covering idempotency, corrupt JSON recovery, config preservation). This directly addresses a real need from Issue #332.

Two architectural observations (not blockers — fine for a follow-up):

  1. search() config instantiation: In searcher.py, the non-MCP search() path creates a new MempalaceConfig() on every call to read archived wings. This means a filesystem read per search invocation. Consider either passing the config from the caller or caching it. Also, search() lacks the include_archived parameter that search_memories() has — might be worth adding for API consistency.

  2. $ne scaling: With many archived wings, the $and filter grows linearly (one $ne per archived wing). For repositories with 50+ wings this could become unwieldy. A future optimization could store an archived metadata field on each drawer entry and filter with a single {"archived": {"$ne": true}}, but that is a bigger change and not needed now.

We plan to use this feature in the MemPalace-AGI integration for archiving completed research cycles, so happy to see it land. Nice work! 👍

@matrix9neonebuchadnezzar2199-sketch
Copy link
Copy Markdown
Author

Thanks for the review and approval @web3guru888!

Both points are noted:

  1. Config caching — agreed, instantiating on every call is wasteful. I'll address this in a follow-up (either pass config from the caller or add a module-level cache with a short TTL).

  2. $ne scaling — good catch. For now the linear filter is fine for typical usage, but storing archived as drawer-level metadata would be the right long-term fix. Will track this separately.

Appreciate the detailed feedback!

- archive_room() / unarchive_room() / get_archived_rooms() in config.py
- searcher.py excludes archived rooms from where clause
- mempalace_archive_room / mempalace_unarchive_room MCP tools
- tool_status now reports archived_rooms per wing
- 9 new tests for room-level archiving
- Addresses @web3guru888 feedback on PR MemPalace#336

Made-with: Cursor
@matrix9neonebuchadnezzar2199-sketch
Copy link
Copy Markdown
Author

@web3guru888 Great suggestion — implemented room-level archiving in the latest push (9526e73).

What's added:

  • archive_room(wing, room) / unarchive_room(wing, room) / get_archived_rooms(wing) in config.py
  • wing_config.json schema now supports "archived_rooms": ["room_a", "room_b"] per wing, coexisting with the wing-level "archived" flag
  • searcher.py excludes archived rooms via $ne in the where clause (both search() and search_memories())
  • mempalace_archive_room / mempalace_unarchive_room MCP tools
  • tool_status reports archived_rooms per wing
  • include_archived=True bypasses both wing and room exclusions
  • 9 new tests (all passing alongside the original 11)

Your ASTRA-dev use case (archive evidence/ rooms while keeping hypotheses and decisions visible within the same wing) should now work directly:
mempalace_archive_room(wing="astrophysics", room="evidence")

CLI commands (mempalace archive --room) left for a follow-up to keep this PR focused.

@web3guru888
Copy link
Copy Markdown

This is exactly what we needed — thank you. The API surface is clean: archive_room(wing, room) + get_archived_rooms(wing) maps directly to our use case.

Our test case:

mempalace_archive_room(wing="astrophysics", room="evidence")
# Now search("gravitational wave detection") only hits hypotheses/ and decisions/ rooms
# get_archived_rooms("astrophysics") → ["evidence"]
# include_archived=True still works for cross-cycle audits

The $ne exclusion in the where clause is the right approach — avoids post-filter overhead. Will integrate against this and report back after running our 208-discovery suite through it.

One note: we track 31 rooms across 5 wings in production. The per-wing archived_rooms list should scale fine, but worth keeping an eye on that wing_config.json if it grows — not an issue yet, just flagging for the future.

@matrix9neonebuchadnezzar2199-sketch
Copy link
Copy Markdown
Author

Good point on scalability — at 31 rooms across 5 wings the per-wing list approach should be fine, but agreed it's worth watching. If it grows significantly, migrating to a per-drawer archived metadata flag (as you suggested in your earlier review) would collapse the filter to a single condition. Happy to revisit if real-world usage hits that threshold.

matrix9neonebuchadnezzar2199-sketch pushed a commit to matrix9neonebuchadnezzar2199-sketch/mempalace that referenced this pull request Apr 10, 2026
…rchive nudges

- Chrom metadata synapse_mark=new on new drawers (MCP + miner)
- build_soft_archive_proposal for MemPalace#336-style archive wing suggestions
- mempalace_status: enriched consolidation_details, phase3 block, tagging-window count
- Config: consolidation_inactive_days, soft_archive_suggestions, target_wing

Made-with: Cursor
@web3guru888
Copy link
Copy Markdown

Agreed — 31 rooms across 5 wings is well within comfortable range for the per-wing list approach. We're at similar scale and haven't felt any pressure there either. Good call keeping it simple now and revisiting if usage grows; no need to over-engineer the filter path prematurely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/mcp MCP server and tools area/search Search and retrieval enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: soft-archive wings — exclude from search without deleting data

3 participants