Bug
mempalace status crashes with:
sqlite3.OperationalError: too many SQL variables
Root cause
miner.py calls col.get(limit=total, include=["metadatas"]) without pagination. SQLite binds one host variable per metadata item, hitting the compile-time limit (~32766) when the palace exceeds ~30k drawers.
Reproduction threshold
Any palace with more than ~30k drawers. Confirmed on a palace with 50k+ drawers.
Workaround
sqlite3 ~/.mempalace/palace/chroma.sqlite3 "SELECT COUNT(*) FROM embeddings;"
mempalace search continues to work correctly.
Fix
Paginate the col.get() call in batches of 5000. ChromaDB's get() accepts an offset parameter, so this is straightforward.
See PR from srgvg/mempalace for the one-function fix.
Bug
mempalace statuscrashes with:Root cause
miner.pycallscol.get(limit=total, include=["metadatas"])without pagination. SQLite binds one host variable per metadata item, hitting the compile-time limit (~32766) when the palace exceeds ~30k drawers.Reproduction threshold
Any palace with more than ~30k drawers. Confirmed on a palace with 50k+ drawers.
Workaround
mempalace searchcontinues to work correctly.Fix
Paginate the
col.get()call in batches of 5000. ChromaDB'sget()accepts anoffsetparameter, so this is straightforward.See PR from
srgvg/mempalacefor the one-function fix.