SELECT c.name, COUNT(*)
FROM embeddings e
JOIN segments s ON e.segment_id = s.id
JOIN collections c ON s.collection = c.id
GROUP BY c.name;
-- mempalace_closets : 499
-- mempalace_drawers : 67580   <-- before repair
-- mempalace_drawers : 10000   <-- after repair

Expected Behavior

Either:
- Repair must process all drawers (paginated collection.get(limit=N, offset=K) loop), or
- Repair must fail loudly when the source set exceeds the extraction cap, refusing to overwrite.

Silently dropping 85% of memories with a "Repair complete" success message is the worst possible outcome for a tool whose entire job is data preservation.

Root Cause (suspected)

collection.get() in ChromaDB defaults to a 10,000-row limit. The repair extraction path likely calls .get() once without paginating. Same issue would affect any palace > 10K drawers.

Trigger of the Underlying HNSW Corruption

Pre-existing palace state — exact cause unknown. mempalace status, mempalace repair, and chromadb.Collection.count() on mempalace_drawers all segfault when the corrupt HNSW is loaded. Closets collection
unaffected. The 3.3.2 #1000 "Quarantine stale HNSW" fix did not auto-trigger here; manual quarantine of HNSW files was required just to make repair runnable — at which point the 10K cap surfaced.

Mitigation Used

Restored the pre-repair backup (~/.mempalace.bak-<date>) and recovered the lost 57,580 drawers manually by extracting embedding vectors directly from chroma.sqlite3 and rebuilding the HNSW index without going
through mempalace repair.

Suggested Fix

In the repair extraction loop, paginate:

BATCH = 5000
offset = 0
while True:
    batch = src.get(limit=BATCH, offset=offset, include=["documents","metadatas","embeddings"])
    if not batch["ids"]:
        break
    dst.add(ids=batch["ids"], documents=batch["documents"],
            metadatas=batch["metadatas"], embeddings=batch["embeddings"])
    offset += BATCH

Plus a sanity-check assertion: assert len(dst) == src_total_count before declaring success and before overwriting the live palace.

Severity

High. Silent data loss in a tool sold as "verbatim memory, 96.6% R@5". The user's only signal something went wrong was the post-repair status showing a number that looked too round.