feat(lifespan): auto-migrate Stop-hook checkpoints to recovery collection (Phase E)

jphein · claude · jphein · commit 034023c8dec5 · 2026-04-26T10:19:54.000-07:00
Calls mempalace.migrate.migrate_checkpoints_to_recovery() during the
daemon's lifespan startup, after the HNSW quarantine pass and before
the ChromaDB warmup ping. Idempotent — re-runs return 0 once the
canonical palace has reorganized; the only one-time cost is the first
restart after deploy.

Closes Phase E of the checkpoint-collection-split spec
(docs/superpowers/specs/2026-04-25-checkpoint-collection-split.md
on jphein/mempalace). Phases A–D shipped on the mempalace fork on
2026-04-25/26 — the migrate function exists and is wired into
`mempalace repair --mode reorganize` for explicit operator runs.
This commit makes the daemon do it automatically on first startup
post-upgrade, so operators don't have to know about the manual command.

Gated behind PALACE_AUTO_MIGRATE_CHECKPOINTS=1 (default). Set =0
to skip auto-migrate in environments where the one-time cost is
unwanted; the manual `mempalace repair --mode reorganize` path
remains.

Defensive: ImportError if the mempalace fork-side function isn't
present (upstream-shaped installs); other exceptions logged as
non-fatal warnings so the daemon still starts.

After this lands and the canonical 151K-drawer palace migrates,
the predicted Cat 9 A/B convergence (kind=all ≈ kind=content
tokens-per-question) becomes testable — that's the empirical proof
the structural fix worked.

Co-Authored-By: Claude Opus 4.7 (1M context) &lt;noreply@anthropic.com&gt;
diff --git a/main.py b/main.py
@@ -321,6 +321,35 @@ async def lifespan(app: FastAPI):
             len(moved), moved,
         )
 
+    # Migrate Stop-hook auto-save checkpoints from the main searchable
+    # collection into the dedicated mempalace_session_recovery collection
+    # so they don't dominate vector top-N. Idempotent — re-runs return 0
+    # once the canonical palace has reorganized. Gated behind
+    # PALACE_AUTO_MIGRATE_CHECKPOINTS so operators can disable in
+    # environments where the one-time migration cost is unwanted.
+    # See mempalace docs/superpowers/specs/2026-04-25-checkpoint-collection-split.md.
+    if os.environ.get("PALACE_AUTO_MIGRATE_CHECKPOINTS", "1") != "0":
+        try:
+            from mempalace.migrate import migrate_checkpoints_to_recovery
+
+            loop = asyncio.get_running_loop()
+            moved_checkpoints = await loop.run_in_executor(
+                None, migrate_checkpoints_to_recovery, _mp._config.palace_path
+            )
+            if moved_checkpoints:
+                logger.info(
+                    "Migrated %d checkpoint drawer(s) from main → mempalace_session_recovery; "
+                    "mempalace_search now queries content-only.",
+                    moved_checkpoints,
+                )
+        except ImportError:
+            # mempalace.migrate.migrate_checkpoints_to_recovery is fork-side
+            # only at the moment; on upstream-shaped installs without it the
+            # daemon should still start cleanly.
+            logger.debug("migrate_checkpoints_to_recovery not available; skipping auto-migrate.")
+        except Exception as e:
+            logger.warning("Auto-migrate of checkpoints failed (non-fatal): %s", e)
+
     # Warm the ChromaDB client before accepting traffic. The Rust HNSW binding
     # occasionally segfaults on the very first request if opened cold; opening
     # it here (before yield) ensures the PersistentClient is fully initialized.
