Skip to content

fix: pin chromadb to <1.0 to avoid 1.5.x queue-stall regression (#1006)#1008

Closed
bensig wants to merge 1 commit intodevelopfrom
fix/chromadb-pin-below-1-0
Closed

fix: pin chromadb to <1.0 to avoid 1.5.x queue-stall regression (#1006)#1008
bensig wants to merge 1 commit intodevelopfrom
fix/chromadb-pin-below-1-0

Conversation

@bensig
Copy link
Copy Markdown
Collaborator

@bensig bensig commented Apr 18, 2026

Summary

Fixes #1006.

chromadb>=0.5.0 with no upper bound lets fresh pip install mempalace pull the latest chromadb==1.5.8, which has a queue-processor regression that makes writes invisible. Mine reports success; mempalace status / search show 0 drawers. chromadb==0.6.3 verified working.

Change

One line in pyproject.toml:

- "chromadb>=0.5.0",
+ "chromadb>=0.5.0,<1.0",

Test plan

  • Fresh venv + pip install from this branch — verify chromadb resolves below 1.0
  • mempalace mine <dir> --mode convos reports filed drawers
  • mempalace status shows the filed drawers (not 0)
  • mempalace search returns real content with non-None metadata

Root cause

The 1.x architecture introduced async segment materialization via embeddings_queue. In 1.5.x that processor isn't running from PersistentClient alone — writes durably queue but never reach the embeddings / embedding_metadata tables. SQLite-level evidence in #1006.

Upstream fix not investigated in this PR. Long-term: contribute to ChromaDB or raise the ceiling once a 1.x version ships with the processor restored.

Note for affected users

Users who already pulled 1.5.x can recover without losing data:

pip install "chromadb<1.0"
# re-mine — deterministic IDs, safe upsert

@bensig
fix: pin chromadb to <1.0 to avoid 1.5.x queue-stall regression (#1006)
8ad5772 
chromadb 1.5.x has a regression where the embeddings_queue to embeddings
processor never runs from PersistentClient. Writes land durably in the
queue but segment materialization never happens, so mempalace status
and search show 0 drawers after a mine that reported hundreds filed.
Confirmed broken on 1.5.8 and 1.5.6; chromadb 0.6.3 works immediately.

pyproject.toml previously pinned chromadb>=0.5.0 with no upper bound,
so every fresh pip install mempalace today pulls broken 1.5.8. Pinning
to <1.0 restores working behavior for new installs while keeping the
0.5.0 floor so existing users on older chromadb are unaffected.

See #1006 for full diagnosis with SQLite-level evidence and version
test matrix.
@bensig
Copy link
Copy Markdown
Collaborator Author

bensig commented Apr 18, 2026

Code review

Found 1 issue:

  1. Pinning chromadb<1.0 forces installation back to the 0.6.x line, which imports from pydantic.v1 — this fails to import on Python 3.13+ and is documented as a hard regression in #487. PR #302 was specifically written to widen the chromadb ceiling so that mempalace works on modern Python; this PR reverses that fix. Two related PRs are also going in the opposite direction for the same reason: #426 (>=1.0,<2) and #581 (>=1.5.4,<2, already APPROVED with 50K-drawer migration validation per jphein's testing). Consider a narrower workaround for the 1.5.6/1.5.8 queue-stall instead — e.g., excluding the specific broken versions (!=1.5.6,!=1.5.8) or raising the floor to a known-good 1.x release — so modern-Python users aren't hard-blocked.

mempalace/pyproject.toml

Lines 26 to 31 in 8ad5772

]
dependencies = [
"chromadb>=0.5.0,<1.0",
"pyyaml>=6.0,<7",
]

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@bensig
Copy link
Copy Markdown
Collaborator Author

bensig commented Apr 18, 2026

Closing in favor of #581 (fix: upgrade chromadb to >=1.5.4 for python 3.13/3.14 compatibility).

Per the review on this PR, the <1.0 cap reintroduces a documented breaking incompatibility on Python 3.13+: chromadb 0.6.x imports from `pydantic.v1`, which fails on modern Python — see #487 for the exact ConfigError. This reverses the intent of #302, which deliberately widened the chromadb ceiling to fix the same regression.

#581 is already APPROVED with a 50K-drawer auto-migration validation (per @jphein's testing — 648 tests pass, palace migrates 0.6.x → 1.5.x cleanly). It keeps mempalace on a supported-Python path and plausibly excludes the broken 1.5.x versions via its >=1.5.4 floor.

Issue #1006 stays open. The queue-stall regression is real; the right fix is a narrow floor in the 1.5.x line, not a blanket cap below 1.0. If #581 merges and 1.5.4 itself exhibits the queue-stall, a follow-up bump to the floor is simpler than capping below the whole major.

@bensig bensig closed this Apr 18, 2026
This was referenced Apr 18, 2026
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@milla-jovovich milla-jovovich Awaiting requested review from milla-jovovich milla-jovovich is a code owner

@igorls igorls Awaiting requested review from igorls igorls is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

bug: fresh pip install mempalace silently broken on chromadb 1.5.x — writes queued but never flush, search returns ghosts

1 participant

@bensig