Skip to content

fix(memory): detect and recover from Qdrant collection dimension mismatch (#1951)#1954

Merged
bug-ops merged 1 commit intomainfrom
fix-1951-qdrant-collection-dims
Mar 17, 2026
Merged

fix(memory): detect and recover from Qdrant collection dimension mismatch (#1951)#1954
bug-ops merged 1 commit intomainfrom
fix-1951-qdrant-collection-dims

Conversation

@bug-ops
Copy link
Copy Markdown
Owner

@bug-ops bug-ops commented Mar 17, 2026

Summary

  • QdrantOps::ensure_collection and ensure_collection_with_quantization previously returned Ok(()) when a collection existed without checking its vector dimensions
  • On embedding model change (e.g. 4096-dim qwen3-embedding → 1536-dim text-embedding-3-small), all collections backed by these methods would silently fail at insert/search time with dimension-mismatch errors
  • Collections affected: zeph_conversations, zeph_session_summaries, zeph_key_facts, zeph_corrections, zeph_graph_entities, and code-index collections in zeph-index
  • Fix: added a private get_collection_vector_size helper using collection_info; both ensure methods now verify dimensions and recreate the collection (with a WARN log) when there is a mismatch

Changes

  • crates/zeph-memory/src/qdrant_ops.rs: dimension check + auto-recreation in ensure_collection and ensure_collection_with_quantization; new get_collection_vector_size helper; four #[ignore] integration tests
  • CHANGELOG.md: entry in [Unreleased] section

Notes

  • EmbeddingRegistry (skills, MCP) already had its own inline dimension detection. That logic remains and still works correctly; it now also benefits from the fix in the underlying QdrantOps call, but the redundancy is harmless.
  • Data loss on recreation is intentional and documented in the doc comment and the warning log.

Test plan

  • cargo +nightly fmt --check passes
  • cargo clippy --workspace --features full -- -D warnings passes
  • cargo nextest run --config-file .github/nextest.toml --workspace --features full --lib --bins — 6127 tests pass
  • Integration tests (#[ignore]) can be verified manually with a live Qdrant instance at :6334 by running cargo nextest run -p zeph-memory -- --ignored

Closes #1951

…atch

QdrantOps::ensure_collection previously returned Ok(()) if a collection
already existed, without verifying that its vector dimensions matched the
required size. When the embedding model changed (e.g. 4096-dim →
1536-dim), all collections backed by QdrantOps continued silently until
the first insert or search produced a dimension-mismatch error.

This was a latent bug for every collection that goes through QdrantOps
directly:
  - zeph_conversations
  - zeph_session_summaries
  - zeph_key_facts
  - zeph_corrections
  - zeph_graph_entities
  - zeph-index code collections

EmbeddingRegistry (used by skills/MCP) already had its own inline
detection, but all other collections bypassed it.

Fix: add dimension verification in QdrantOps::ensure_collection and
ensure_collection_with_quantization via a shared private helper
get_collection_vector_size. On mismatch, the collection is deleted and
recreated, with a WARN log listing the old and new dimension. Data loss
is expected and documented.

Also add four #[ignore] integration tests (require live Qdrant at
:6334) that cover idempotency for same-size and recreation on mismatch
for both ensure_collection variants.

Closes #1951
@github-actions github-actions bot added bug Something isn't working size/M Medium PR (51-200 lines) documentation Improvements or additions to documentation memory zeph-memory crate (SQLite) rust Rust code changes and removed size/M Medium PR (51-200 lines) labels Mar 17, 2026
@bug-ops bug-ops enabled auto-merge (squash) March 17, 2026 23:01
@bug-ops bug-ops merged commit c18ee21 into main Mar 17, 2026
20 checks passed
@bug-ops bug-ops deleted the fix-1951-qdrant-collection-dims branch March 17, 2026 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation memory zeph-memory crate (SQLite) rust Rust code changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(memory): Qdrant collection recreation incomplete for main conversation store

1 participant