Skip to content

Closets not backfilled for non-mined palaces — mempalace compress writes to wrong collection (_compressed vs _closets) #1244

@maurist10

Description

@maurist10

Summary

Palaces that were built via MCP mempalace_add_drawer calls (no mempalace mine runs) ship with zero closets after upgrading to 3.3.x. Hybrid search returns closet_boost: 0.0 and matched_via: "drawer" for every result — the +38% R@1 improvement advertised in the 3.3.0 release notes never kicks in.

The natural-looking remedy — mempalace compress — is a different feature that writes to a separate collection, and the naming is confusable enough that users (and at least one diligent agent) reach for it expecting a closet build.

Repro

  1. Build a palace through MCP only:
    # via MCP, repeated:
    mempalace_add_drawer(wing="wing_x", room="status", text="...")
    
  2. Upgrade pip install --upgrade mempalace (3.0.0 → 3.3.3 in our case).
  3. Call mempalace_reconnect, then any mempalace_search. All results return:
    "closet_boost": 0.0,
    "matched_via": "drawer"
  4. Inspect collections:
    import chromadb
    c = chromadb.PersistentClient(path="~/.mempalace/palace")
    [col.name for col in c.list_collections()]
    # ['mempalace_drawers']   <-- no mempalace_closets

Where the confusion compounds

mempalace compress (cli.py:519) writes to mempalace_compressed:

comp_col = backend.get_or_create_collection(palace_path, "mempalace_compressed")
...
print(f"  Stored {len(compressed_entries)} compressed drawers in 'mempalace_compressed' collection.")

But the searcher (palace.py:66–68) reads from mempalace_closets:

def get_closets_collection(palace_path: str, create: bool = True):
    return get_collection(palace_path, collection_name="mempalace_closets", create=create)

Both produce AAAK-style compact pointers, both walk the drawer set, both call AAAK code paths — but they don't share a collection. The closet build is currently only invoked from miner.py:750 + :766 inside process_file().

Impact

  • Any palace that grew via MCP / external scripts / migration from another tool stays at the pre-3.3.0 search quality unless the user re-mines source files into the existing palace (which would write new drawers that they don't want).
  • mempalace compress looks like the right command from its help text (Compress drawers using AAAK Dialect (~30x reduction)) but does not feed the search-boost layer.

Suggested fixes (any one of these would resolve it)

  1. Add an explicit closet-build CLI, e.g. mempalace closets build [--wing X], that walks mempalace_drawers, groups by source_file, and calls build_closet_lines + upsert_closet_lines per group. Existing miner code already does this — extracting the loop into a standalone command is mechanical.
  2. Have mempalace compress also write closets (or rename it to make the distinction explicit). Right now the docstring + help text don't separate "AAAK summary for wake-up context" from "search-boost index".
  3. Document the gap. Even just a release note or migration-guide entry: "If your palace was not populated via mempalace mine, run X to backfill closets."

Workaround used here

For anyone hitting this in the meantime — a ~50-line backfill script importing build_closet_lines + upsert_closet_lines + purge_file_closets from mempalace.palace works. Walks mempalace_drawers, groups by source_file (skips empty/"?" source), writes to mempalace_closets, calls mempalace_reconnect, done. After backfill, closet_boost > 0 shows up on relevant queries and matched_via: "drawer+closet" triggers as expected.

Environment

  • mempalace 3.3.3
  • chromadb 1.5.6
  • Windows 11, Python 3.14
  • Palace size: 593 drawers, ~9 MB

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/cliCLI commandsbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions