-
Notifications
You must be signed in to change notification settings - Fork 2
bug(qdrant): dimension mismatch crash when switching embedding models with existing collections #1815
Description
Bug
When switching embedding models (e.g. from qwen3-embedding 4096-dim to text-embedding-3-small 1536-dim), Qdrant collections that already exist with the old dimensions are not recreated, causing Vector dimension error on search and context preparation failed in the agent loop.
Reproduction
- Run Zeph once with
embedding_model = "qwen3-embedding"andvector_backend = "qdrant". Qdrant collections are created with 4096 dimensions. - Run Zeph again with
embedding_model = "text-embedding-3-small". Collections are NOT recreated. - Observe errors:
WARN zeph_mcp::registry: Qdrant MCP tool search failed: vector store error: search error:
Vector dimension error: expected dim: 4096, got 1536
WARN zeph_core::agent: context preparation failed: vector store error: search error:
Vector dimension error: expected dim: 4096, got 1536
WARN zeph_memory::semantic::recall: Failed to store embedding:
Vector dimension error: expected dim: 4096, got 1536
Root Cause
EmbeddingRegistry::sync() detects model changes by checking the embedding_model metadata field on existing points. When a collection exists but has 0 points (e.g. zeph_mcp_tools when no MCP servers are configured), scroll_all() returns empty and model_changed = false — no recreation fires.
The zeph_skills collection IS recreated because it contains points with embedding_model metadata from the previous sync. Collections with 0 points are not detected.
Additionally, the search path has no recovery logic for DimensionMismatch errors — they propagate as context preparation failed WARN, silently degrading agent quality.
Impact
- Agent context preparation fails silently (WARN, not error) — memory recall is disabled
- New memories cannot be stored in Qdrant
- Affects all collections with 0 points:
zeph_mcp_tools, potentiallyzeph_key_facts,zeph_corrections,zeph_session_summaries
Fix Suggestion
In ensure_collection(), compare the existing collection's configured vector dimension against the expected dimension from the embedding model. If mismatch is detected, recreate the collection. This is dimension-aware rather than metadata-aware and handles the empty-collection case.
// Pseudocode in ensure_collection():
let existing_dim = ops.get_collection_dim(collection_name).await?;
if existing_dim != expected_dim {
warn!("dimension mismatch ({existing_dim} vs {expected_dim}), recreating collection");
ops.delete_collection(collection_name).await?;
}Files
crates/zeph-memory/src/embedding_registry.rs—sync()model_changed check (line ~145)crates/zeph-memory/src/embedding_registry.rs—ensure_collection()