-
-
Notifications
You must be signed in to change notification settings - Fork 69.6k
[Bug] Memory indexer doesn't detect embedding dimension mismatch between stored vectors and active provider #32277
Copy link
Copy link
Open
Description
Problem
When switching memory search providers (e.g., from local embeddinggemma-300M fallback to Gemini gemini-embedding-001), the existing vector database contains embeddings with a different dimension (300 vs 3072). The indexer:
- Doesn't detect the dimension mismatch
- Doesn't warn the user
- Returns zero or garbage results silently —
memory_searchappears to work but finds nothing relevant
This is a silent data integrity failure that's very hard to diagnose.
Expected Behavior
When the active embedding provider produces vectors with a different dimension than those stored in the database, the system should:
- Detect the mismatch on startup or first query
- Warn with a clear log message:
[memory] Dimension mismatch: stored vectors are 300-dim but active provider (gemini-embedding-001) produces 3072-dim. Re-index required. - Optionally auto-re-index or provide a CLI command:
clawdbot memory reindex
Steps to Reproduce
- Configure
memorySearch.provider: 'gemini'withremote.apiKey - Let the system fall back to local provider (e.g., API unavailable)
- Accumulate memory entries with local embedding dimensions
- Restore Gemini access — queries now use 3072-dim vectors against 300-dim stored vectors
memory_searchreturns results with very low scores or mismatched content
Environment
- Clawdbot v2026.1.24-3
- SQLite memory backend (
main.sqlite) - Gemini
gemini-embedding-001(3072-dim) vs localembeddinggemma-300M(300-dim) - Discovered running 5 agents in production, 3 of which had stale indexes
Workaround
Delete main.sqlite and let the system re-index all memory files from scratch. This works but loses any entries not backed by source files.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Type
Fields
Give feedbackNo fields configured for issues without a type.