Skip to content

mempalace status CLI fails on palaces with >32k drawers: too many SQL variables (miner.py:852) #1073

@sha2fiddy

Description

@sha2fiddy

On a 465k-drawer palace, mempalace status dies:

File ".../mempalace/miner.py", line 852, in status
    r = col.get(limit=total, include=["metadatas"]) if total else {"metadatas": []}
File ".../chromadb/api/rust.py", line 440, in _get
chromadb.errors.InternalError: Error executing plan: Internal error: error returned from database: (code: 1) too many SQL variables

col.get(limit=total) becomes WHERE id IN (?, ?, ...) server-side and blows SQLite's 32,766 bound-parameter limit. Any palace past that size kills the CLI.

Env: mempalace develop @ 1b00f93, chromadb 0.6.3, Python 3.12, macOS.

Repro:

mempalace --palace <palace-with->32k-drawers> status

Fix: chunk the get, e.g.

BATCH = 10_000
metas = []
for offset in range(0, total, BATCH):
    metas.extend(col.get(limit=BATCH, offset=offset, include=["metadatas"])["metadatas"])

Or aggregate counts in the backend — the MCP mempalace_status tool already does this without fetching metadata and doesn't hit the limit. The two paths could share an implementation.

MCP tool unaffected. Low severity, but blocks the CLI once a palace reaches active-use scale (~32k drawers = a few months of mining).

Can open a PR with the chunked-get fix if that's the preferred direction.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/cliCLI commandsbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions