SQLite backend runs without vector search: sqlite-vec is never installed into the plugin deps dir #634

Summary

On the default SQLite backend, the plugin's MCP server runs with vector search disabled. It does not say so.

The cause is that sqlite-vec is missing from the packages the launcher installs into deps/. The launcher also removes the user site-packages from sys.path, so a user-level pip install sqlite-vec is never seen either.

It gets worse: the store still stamps embedding_model = 'neural' on memories it wrote no vector for. Installing sqlite-vec later does not fix them, because the re-embed worklist only selects rows labelled 'fallback'.

Environment

Defect 1 (root cause): sqlite-vec is not in the launcher's package lists

scripts/launcher_pins.py defines what ensure_deps / ensure_all_deps install into deps/:

BASE_PACKAGES: list[tuple[str, str]] = [
    ("mcp", "mcp==2.1.1"),
    ("pydantic", "pydantic==2.13.5"),
    ("pydantic_settings", "pydantic-settings==2.15.0"),
    ("numpy", f"numpy=={numpy_version(sys.version_info[:2])}"),
    ("psycopg", "psycopg[binary]==3.3.5"),
    ("psycopg_pool", "psycopg_pool==3.3.1"),
    ("pgvector", "pgvector==0.5.0"),
]

ML_PACKAGES holds sentence-transformers, flashrank and torch. Neither list contains sqlite-vec.

The PostgreSQL drivers are installed on every machine. The SQLite extension is installed on none, although SQLite is the zero-config default. pyproject.toml declares it only in the [sqlite] extra, which the plugin install path does not use.

Defect 2: the launcher removes the only place it could come from

scripts/launcher.py drops the user site-packages on purpose. The torch mismatch is a real problem, so the removal itself is reasonable:

_site.addsitedir(deps_dir)
...
if _user_site:
    _dropped = os.path.normcase(os.path.abspath(_user_site))
    sys.path[:] = [q for q in sys.path
                   if os.path.normcase(os.path.abspath(q)) != _dropped]

The consequence: _try_load_vec() in sqlite_store.py hits ImportError, logs sqlite-vec unavailable … vector search disabled at INFO level, and sets _has_vec = False. Nothing reaches the user.

Defect 3: the model label is written even when no vector is

SqliteMemoryStore._insert_memory_rows stamps the model before checking _has_vec:

if embedding is not None:
    self._stamp_embedding_model(memory_id, data.get("embedding_model"))
    if self._has_vec:
        ...INSERT INTO memories_vec...

reembed_memory does the same: the vec write is guarded by _has_vec, but _stamp_embedding_model runs anyway.

The _stamp_embedding_model docstring states the precondition "memory_id refers to a just-written vec row". Both callers break it when _has_vec is false.

Result: a row says neural and has no vector. select_fallback_embeddings filters on embedding_model = 'fallback', so this row is never picked up again, even after sqlite-vec becomes available.

Defect 4: nothing surfaces the degraded mode

• check_setup / doctor return ready: true. The SQLite branch only checks that the store opens.
• consolidate reports "embedding_upgrade": {"upgraded": 86} while 0 vectors were written.

Reproduction

1. SQLite backend, sqlite-vec present only in the user site-packages (or nowhere).
2. Mark rows for re-embedding: UPDATE memories SET embedding_model='fallback'.
3. Run consolidate.
4. Observe: embedding_upgrade.upgraded = row count, and every row is now neural.
5. SELECT count(*) FROM memories_vec_rowids → 0.

With sqlite-vec==0.1.9 added to deps/ (pip install --no-deps --target <deps> sqlite-vec==0.1.9), rows re-marked fallback, and the cycle run again: 86 vectors written, and KNN queries return sensible neighbours.

Suggested fixes

1. Add ("sqlite_vec", "sqlite-vec==0.1.9") to BASE_PACKAGES. It is small and has no dependencies.
2. Stamp embedding_model only after the vec row is written. That is the precondition the docstring already states.
3. Base the re-embed worklist on the vector's absence, not only on the label. For example: current_memories rows with no entry in memories_vec_rowids. This also repairs stores already mislabelled by defect 3.
4. On the SQLite backend, make doctor report whether sqlite-vec loads, with the fix command. It can stay optional if vector-less operation is a supported mode.
5. Have embedding_upgrade count vectors actually written, not rows processed.

Scope

Observed only on Windows. Defect 1 does not depend on the platform: the pin lists are the same everywhere. I have not checked whether sqlite-vec arrives transitively on macOS or Linux.

Workaround

python -m pip install --no-deps --target "$CLAUDE_PLUGIN_DATA/deps" sqlite-vec==0.1.9

Then re-mark affected rows fallback and run consolidate.