chore(deps): pin chromadb to >=1.0,<2#426
Conversation
The previous constraint of chromadb>=0.5.0,<0.7 forces installs onto the 0.5/0.6 line, which uses an older on-disk format. Palaces created with chromadb 1.x (the current major) can't be opened by 0.6 — get_collection fails with KeyError: '_type' when reading collection config json. Pin to the 1.x line so: - editable installs keep working with palaces built on 1.x - new installs land on the same major as the current chromadb release (1.5.x) - we have a single supported major instead of straddling two on-disk formats Note: existing 0.5/0.6 palaces are auto-migrated by chromadb 1.x on first open. The migration is one-way — once migrated, the palace cannot be reopened by chromadb 0.5/0.6. All chromadb APIs used by mempalace (PersistentClient, get_collection, create_collection, get_or_create_collection, collection.get, collection.count, collection.query, collection.add, collection.delete) are stable across the 0.5 → 1.x boundary. No code changes required.
|
Will this also solve this problem? |
web3guru888
left a comment
web3guru888
left a comment
There was a problem hiding this comment.
This goes in the right direction — issue #445 reports that users who installed ChromaDB 1.x (which pip resolves by default now) can't open palaces created under the current >=0.5.0,<0.7 pin, because ChromaDB 1.0 changed its storage format.
But bumping straight to >=1.0,<2 without a migration path will break anyone running 0.5.x or 0.6.x who upgrades mempalace:
- Existing palaces won't open — ChromaDB 1.0 can't read 0.5/0.6 SQLite storage directly
- No
mempalace repairguidance — the CLI repair command re-indexes vectors but doesn't handle the underlying Chroma storage format migration - The current test suite runs against 0.5/0.6 — has anyone verified all tests pass against chromadb 1.x?
Suggestion: consider >=0.5.0,<2 (widen the range rather than bump the floor), and add a note in the upgrade docs that users on Chroma 1.x should run mempalace repair after upgrading. The repair command already does a full re-index which should handle the format gap — but it needs to be tested against the 0.5→1.0 transition.
@iSapozhnik re your question — yes, this should solve #445, but only if mempalace's own code is compatible with the ChromaDB 1.x API. The pin change alone isn't enough if there are breaking API changes upstream.
|
Thanks @web3guru888 — you're right that my original PR body overclaimed on migration. I said "auto-migrates on first open" but after checking the chromadb 1.0 release notes, that's not actually the case — there's a migration guide, not an automatic migration. I'll strike that line. On the constraint choice: widening to The core problem is that there's no way to both (a) fix #445 for 1.x users and (b) avoid breaking 0.6 users without a code-level compatibility layer or migration command. A constraint change alone can only pick one side. Given that:
I think the right scope for this PR is:
A proper migration story (detect old format, run On your other points:
Let me know if you'd rather I close this and wait for a proper migration PR, or merge this as-is with a release-notes callout. |
|
Adding weight to this: on Python 3.14, chromadb 0.6.x doesn't just cause data incompatibility, it fails to import entirely due to pydantic.v1 incompatibility (#487). The <0.7 pin has no viable target on 3.14. Widening to >=1.0 is the only path forward for current Python versions. |
|
You've thought this through carefully, and I agree with the analysis. The narrow-window argument is right: widening to Recommendation: merge as >=1.0,<2 with clear release notes. The case for >=1.0 is:
The migration command gap is worth a dedicated issue rather than blocking this PR. A proper One thing to add to your release notes callout: recommend users run |
5 participants
What does this PR do?
Fixes installs for users whose palaces were created with chromadb 1.x (see #445). The current constraint
chromadb>=0.5.0,<0.7forces installs onto the 0.5/0.6 line, which uses an older on-disk format. Palaces built on chromadb 1.x can't be opened by 0.6 —get_collectionfails withKeyError: '_type'when reading collection config JSON, and the CLI surfaces it as a misleading "No palace found" message.Pin to the 1.x line so:
Known break and migration note
This change does break users with existing chromadb 0.5/0.6 palaces. chromadb 1.x does not auto-migrate the 0.6 on-disk format — the chromadb 1.0 migration guide covers a dedicated process. For mempalace specifically, the simplest recovery is to back up
~/.mempalace/palace/and re-mine the source files into a fresh 1.x palace.A proper in-tool migration command (
mempalace migrate-chromaor similar) is a separate concern and belongs in a follow-up PR. This PR is intentionally scoped to the dependency pin.API compatibility audit
All chromadb APIs used by mempalace are part of the stable core interface and work identically on 0.5 → 1.x:
chromadb.PersistentClient(path=...)palace.py,mcp_server.py,layers.py,searcher.py,cli.pyclient.get_collection(name)palace.py,mcp_server.py,layers.py,searcher.py,cli.pyclient.create_collection(name)palace.pyclient.get_or_create_collection(name)mcp_server.pycollection.get(where=..., limit=...)palace.py,miner.py,mcp_server.pycollection.count()mcp_server.py,cli.py,layers.pycollection.query(...)searcher.py,layers.pycollection.add(...)miner.py,convo_miner.pycollection.delete(...)mcp_server.pyNo code changes needed.
How to test
pip install -e .python -c "import chromadb; print(chromadb.__version__)"→1.5.xpython -m pytest tests/ -v(passes locally against chromadb 1.5.7)mempalace statusopens it without aKeyError: '_type'.Replaces
This replaces #365, which accumulated stale diffs from being rebased off an older main. This PR is a single clean commit off current
main.Related
Checklist