Skip to content

bug(qdrant): dimension mismatch crash when switching embedding models with existing collections #1815

@bug-ops

Description

@bug-ops

Bug

When switching embedding models (e.g. from qwen3-embedding 4096-dim to text-embedding-3-small 1536-dim), Qdrant collections that already exist with the old dimensions are not recreated, causing Vector dimension error on search and context preparation failed in the agent loop.

Reproduction

  1. Run Zeph once with embedding_model = "qwen3-embedding" and vector_backend = "qdrant". Qdrant collections are created with 4096 dimensions.
  2. Run Zeph again with embedding_model = "text-embedding-3-small". Collections are NOT recreated.
  3. Observe errors:
WARN zeph_mcp::registry: Qdrant MCP tool search failed: vector store error: search error:
  Vector dimension error: expected dim: 4096, got 1536

WARN zeph_core::agent: context preparation failed: vector store error: search error:
  Vector dimension error: expected dim: 4096, got 1536

WARN zeph_memory::semantic::recall: Failed to store embedding:
  Vector dimension error: expected dim: 4096, got 1536

Root Cause

EmbeddingRegistry::sync() detects model changes by checking the embedding_model metadata field on existing points. When a collection exists but has 0 points (e.g. zeph_mcp_tools when no MCP servers are configured), scroll_all() returns empty and model_changed = false — no recreation fires.

The zeph_skills collection IS recreated because it contains points with embedding_model metadata from the previous sync. Collections with 0 points are not detected.

Additionally, the search path has no recovery logic for DimensionMismatch errors — they propagate as context preparation failed WARN, silently degrading agent quality.

Impact

  • Agent context preparation fails silently (WARN, not error) — memory recall is disabled
  • New memories cannot be stored in Qdrant
  • Affects all collections with 0 points: zeph_mcp_tools, potentially zeph_key_facts, zeph_corrections, zeph_session_summaries

Fix Suggestion

In ensure_collection(), compare the existing collection's configured vector dimension against the expected dimension from the embedding model. If mismatch is detected, recreate the collection. This is dimension-aware rather than metadata-aware and handles the empty-collection case.

// Pseudocode in ensure_collection():
let existing_dim = ops.get_collection_dim(collection_name).await?;
if existing_dim != expected_dim {
    warn!("dimension mismatch ({existing_dim} vs {expected_dim}), recreating collection");
    ops.delete_collection(collection_name).await?;
}

Files

  • crates/zeph-memory/src/embedding_registry.rssync() model_changed check (line ~145)
  • crates/zeph-memory/src/embedding_registry.rsensure_collection()

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions