-
Notifications
You must be signed in to change notification settings - Fork 2
provider switch silently degrades semantic tool/skill selection when new provider lacks embedding support #2225
Description
Summary
When switching providers via /provider <name>, if the new provider has no embedding_model configured, semantic tool schema filtering and skill matching silently fall back to all tools/all skills without notifying the user.
Reproduction
Config with two providers — one with embedding_model, one without:
[[llm.providers]]
name = "openai-stt"
model = "gpt-4o-mini"
embedding_model = "text-embedding-3-small"
[[llm.providers]]
name = "openai-quality"
model = "gpt-4o"
# no embedding_modelSession:
/provider openai-quality
What is 23 times 8?
Expected
Either:
a) Keep using the original embedding provider for query embedding regardless of active conversational provider, OR
b) Emit a visible INFO to the user: "Semantic tool filtering unavailable — provider does not support embeddings. Using all tools."
Actual
WARN failed to embed query: embedding not supported by openai
WARN skill matcher returned no results, falling back to all skills
WARN tool filter: query embed failed, using all tools: embedding not supported by openai
Tool selection degrades from semantic top_k=5 to all 23 tools. No user-visible notification.
Impact
- MEDIUM: tool filtering efficiency drops after provider switch
- User has no indication of degraded behaviour
- Discovered in CI-209 live session (2026-03-27)
Fix Direction
Decouple embedding provider from active conversational provider. Tool schema filter and skill matcher should use a dedicated embedding provider (configurable via embedding_provider field or defaulting to whichever provider in the pool has embedding_model set), independent of /provider switch.