[Feature Request]: Semantic tool auto-filtering @ MCP Client / Proxy #845
Replies: 23 comments 15 replies
-
Feature Proposal: Enable Dynamic Tool Discovery Based on User RequestProblem SummaryCurrently, the MCP specification does not allow sending the original user request to the MCP Server or Proxy before retrieving the list of available tools as part of the tool discovery. This significantly limits the ability to implement semantic tool filtering or dynamic discovery based on the user’s intent. Limitations of Current Design
Proposed SolutionModify the MCP specification (Server and Client) to allow the original user message to be included in requests for available tools for dynamic discovery per each individual user request. This enables MCP Proxies to perform dynamic, context-aware tool filtering based on the actual and each individual request. Without tool filteringLLM selects from a list of hundreds (or potentially thousands) of tools, gets confused. graph TD
UQ[User Question] --> MC[MCP Client]
MC --> MP[MCP Server or Proxy - no context]
MP --> TL[Returns All Tools]
TL --> LLM[LLM selects from list of hundreds of tools]
Current workaround:Always requires first calling one tool that searches for other tools explicitly. Makes LLM guess if tools are needed in the first place, potentially making one extra call anyway every time. It makes LLM guess which tools are available. Returns suboptimal results: graph TD
UQ[User Question] --> MC[MCP Client]
MC --> LLM1[LLM guesses if tool is needed and available]
MC --> LTT[Call list_tools tool]
LTT --> TL[Returns Tool List]
TL --> LLM2[LLM selects a tool from the list]
Proposed solution: Semantic filteringRequires modification of the MCP specification for both MCP Clients to send the original user question as part of new dynamic tool discovery, and the MCP Server specification to be able to handle dynamic tool discovery using the provided context of the original user question for each individual request. LLM gets only a limited list of the most relevant available tools for each request. graph TD
UQ[User Question] --> MC[MCP Client with Updated specification]
MC --> MP[MCP Server or Proxy with Updated specification - sends user question BEFORE dynamic tool discovery for each query]
MP --> SF[Semantic Filter using Question]
SF --> TL[Returns Relevant Tools for each individual user request]
TL --> LLM[LLM selects relevant tool from a limited list of tools that are the most relevant to the user request]
Benefits of the Proposed Change
Implementation SuggestionUpdate the MCP spec to allow:
|
Beta Was this translation helpful? Give feedback.
-
|
This suggestion injects application-specific complexity into the MCP protocol which is not a good idea. A better solution is for the MCP servers to return an updated tool list along with the response, if the tool list has changed. That's it! If this parameter is not returned, it could mean the tool list is unchanged. |
Beta Was this translation helpful? Give feedback.
-
|
That carries some heavy inefficiencies, including being more expensive, slower and at the same time less effective. |
Beta Was this translation helpful? Give feedback.
-
|
Perhaps you are right. May be just a notification that the tool list has changed is enough |
Beta Was this translation helpful? Give feedback.
-
|
Though dynamic tool discovery for each individual user request needs the MCP specification updated for both the client & the server (though it can be optional). Currently, when the client requests the list of tools from the server, it does not send the original user question along with the tool discovery request. The current MCP specification creates a need to explicitly request tool discovery. As I was saying that is suboptimal, because the first model needs to guess if a tool is needed to make that step. Second, the LLM needs to guess which tools you might have. Both are simply not logical, and will create many unnecessary requests (eating your money) and at the same time provide worse results. Therefore I believe mcp specification update is in order. |
Beta Was this translation helpful? Give feedback.
-
|
in my server implementations, it would be easy to sent the "tool_list_has_changed" hint. I also expose a tool to get the latest tool info. So no guessing there. It is suboptimal in the sense that the client has to then call my "get_tool_list" tool. But in my case, tool lists don't change all that often. So the additional call cost is bearable in a trade-off vs. additional protocol complexity But perhaps your use case is different |
Beta Was this translation helpful? Give feedback.
-
|
Dear @qdrddr |
Beta Was this translation helpful? Give feedback.
-
|
The problem actually is a bit more complicated. |
Beta Was this translation helpful? Give feedback.
-
|
Another elegant solution would be to allow MCP Clients to utilize the |
Beta Was this translation helpful? Give feedback.
-
|
Here is a paper that may be relevant |
Beta Was this translation helpful? Give feedback.
-
|
I think this is relevant. Addressing LLMs limitations in generating sophisticated long-form outputs. Survey analysis of over 1400 research papers:
https://github.com/Meirtz/Awesome-Context-Engineering |
Beta Was this translation helpful? Give feedback.
-
|
I believe that MCP Client should be able to do RAG with MCP to improve quality and scalability. |
Beta Was this translation helpful? Give feedback.
-
|
Context Rot Affects LLM Performance Longer input does not guarantee consistent results 🔍 Chroma researchers tested 18 LLMs on simple tasks
Research results https://github.com/chroma-core/context-rot 
|
Beta Was this translation helpful? Give feedback.
-
|
TypeScript Implementation of MCP Tool Semantic Search |
Beta Was this translation helpful? Give feedback.
-
|
Reranker with graphs |
Beta Was this translation helpful? Give feedback.
-
|
Suggested Embedding model: Codestral-Embed from mistral and Mxbai-rerank-v2 to improve performance for, code, MCP, and tool retrieval. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Hi Qdrddr -- this existing PR seems relevant to the discussion: #322. There has also been an active discussion in the MCP Contributor Discord if you wish to share ideas further. |
Beta Was this translation helpful? Give feedback.
-
@evalstate where can I find a join link to the MCP Contributor Discord server? |
Beta Was this translation helpful? Give feedback.
-
|
Didn't mean to send you on a treasure hunt - we are revamping the site to make it easier to signpost :) |
Beta Was this translation helpful? Give feedback.
-
|
Hi @qdrddr! While semantic search as you've outlined it is an interesting idea, it's not something we can bake into the protocol. We can't dictate how a client should be built or whether it should include a RAG system. We do want to provide mitigations for the tool overload problem in the protocol though and are working on a filtering SEP that hopefully can help. |
Beta Was this translation helpful? Give feedback.
-
|
My point was not to force but to 1. Guide/Recommend. 2. Provide necessary means to implement and Standardize. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
It seems to me that the goal you’re aiming for is to address context window bloat—sometimes referred to as Context Rot. This issue impacts both response quality, as the LLM can become overwhelmed when too many tools are included, and cost-efficiency, since listing all tools and their descriptions significantly increases token usage. @cliffhall That said, if the main goal is to address context bloat/rot, I don't believe SEP or the group/tag-based solutions will solve this problem. If that’s not the objective, then I may be missing the intended purpose behind SEP, tags, or groups—and would appreciate any clarification. |
Beta Was this translation helpful? Give feedback.
-
|
Thread for discussing this idea: |
Beta Was this translation helpful? Give feedback.
-
|
@cliffhall - How about my suggestion of "a hint that the tool list has changed"? This is light-weight and minimal effort. MCP servers can send a standardized hint along with the response and clients that recognize this hint can refresh the tool list before the next iteration of thought and action |
Beta Was this translation helpful? Give feedback.
-
|
Why hinting (extra step) if we can directly inject for each user request semantically close pre-filtered (or an empty) list of tools before calling for LLM? |
Beta Was this translation helpful? Give feedback.
-
We already have a |
Beta Was this translation helpful? Give feedback.
-
|
A few recent papers supporting the idea of RAG and semantic search for MCP Tools: Both papers conducted experiments demonstrating that both RAG techniques decrease the number of consumed tokens while at the same time increasing task completeness score with Vector Search + Reranker. We basically improve quality and making it cheaper.
|
Beta Was this translation helpful? Give feedback.
-
|
Would appreciate if you vote for this improvement idea to standardize MCP Specification with Delegated Advanced Tool Search in my comment feature here. |
Beta Was this translation helpful? Give feedback.
-
|
That works! Ty!
…________________________________
From: Cliff Hall ***@***.***>
Sent: Friday, August 1, 2025 1:09 PM
To: modelcontextprotocol/modelcontextprotocol ***@***.***>
Cc: Dhar Rawal ***@***.***>; Comment ***@***.***>
Subject: Re: [modelcontextprotocol/modelcontextprotocol] [Feature Request]: Semantic tool auto-filtering @ MCP Client / Proxy (Discussion #845)
How about my suggestion of "a hint that the tool list has changed"? This is light-weight and minimal effort. MCP servers can send a standardized hint along with the response and clients that recognize this hint can refresh the tool list before the next iteration of thought and action.
We already have a listChanged<https://modelcontextprotocol.io/specification/2025-06-18/server/tools#list-changed-notification> notification<https://modelcontextprotocol.io/specification/2025-06-18/server/tools#list-changed-notification> as part of the tools capability<https://modelcontextprotocol.io/specification/2025-06-18/server/tools#capabilities>. Not sure we want to include such a hint additionally as part of a normal tool response. It would mean extra client side code to inspect every tool response and execute the same handler code that responds to listChanged. Unless I'm misinterpreting your suggestion.
—
Reply to this email directly, view it on GitHub<#845 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3F2UHAYPZP4HQWGJAV4Q733LOUPDAVCNFSM6AAAAACBNBHW7GVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGOJWGE4DMNY>.
You are receiving this because you commented.
|
Beta Was this translation helpful? Give feedback.
-
|
JWhy hinting (extra step) if we can simply always include pre-filtered list of relevant tools (if any) before MCP Client calls for its LLM? |
Beta Was this translation helpful? Give feedback.
-
|
MCP-use implemented a tool semantic search mechanism |
Beta Was this translation helpful? Give feedback.
-
|
@qdrddr That's awesome. They have Langchain as a dependency and are a purely Python project. It's easy for them. However, MCP is a protocol specification, and to add semantic search / tool-filtering at that level, would be dictating that all official SDKs in all languages must support it, and if there is no off-the-shelf solution in that language, it would have to be built by the SDK maintainer. Individual SDKs must implement the spec, but seem to be experimenting with different extracurricular features, such as the SessionGroups in the Python SDK, which doesn't appear in other SDKs, as it is not a spec feature. I was initially against that idea, but I've since come to think it's good to have the SDKs act as laboratories for new features that might eventually become part of the spec. I think your best bet is to direct this feature request toward your favorite SDK, not the protocol itself. I don't think it will get much traction at the specification level. |
Beta Was this translation helpful? Give feedback.
-
|
We already answered that concern. We don't dictate. It's just an example. Protocol still needs a search capability standardized @cliffhall |
Beta Was this translation helpful? Give feedback.
-
|
MCP-Agent with EmbeddingRouter for semantic tool search and filtering. |
Beta Was this translation helpful? Give feedback.
-
|
MCP-Universe: Benchmarking Large Language Models with Key findings: Token count increases rapidly with interaction steps, often leading to context overflow and degraded performance in multi-step tasks requiring extensive reasoning.” This proves how MCP tool pre-filtering (prior LLM) is important, and as I was saying, especially manifests in multi-steps. https://mcp-universe.github.io |
Beta Was this translation helpful? Give feedback.
-
|
Cloudflare Turns MCP Tools into TypeScript APIs Cloudflare proposes converting MCP tools into TypeScript APIs so that LLMs can generate code using them. 🔧 Improves handling of many complex tools
Though I still believe Semantic Search can be complimentary for this design. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Is your feature request related to a problem? Please describe.
LLMs receive an unfiltered list of 20–50+ tools, which causes cognitive overload and reduces their ability to accurately choose and use the correct tools for a given prompt. This leads to inefficient context usage and degraded performance. Complimentary with #278
Describe the solution you'd like
Introduce semantic filtering that uses embedding-based similarity to select only the most relevant tools per prompt. Instead of selecting a fixed number (top-k), apply a semantic distance threshold to include only those tools whose meaning closely matches the user's intent.
Describe alternatives you've considered
Additional context
🧭 Type of Feature
Please select the most appropriate category:
🧭 Epic
Title: Semantic Auto-Filtering of MCP Tools
Goal: Dynamically select only the most relevant tools based on the user prompt using semantic understanding.
Why now: With 20–50+ tools available, current static lists overload the LLM model and reduce efficiency. Tool selection should be optimized for better precision and performance.
🙋♂️ User Story 1
As a: developer using the MCP ecosystem
I want: the system to semantically filter and display only relevant tools
So that: the LLM isn't overloaded by irrelevant tools and performs more accurate reasoning
✅ Acceptance Criteria
🙋♂️ User Story 2
As a: model integrator or AI engineer
I want: to reduce tool overload during tool selection phase
So that: inference becomes more efficient and the model response quality improves
✅ Acceptance Criteria
📐 Design Sketch (optional)
flowchart TD subgraph Tool Embedding Index X[Precomputed Tool Embeddings] end A[User Prompt] --> B[Generate Prompt Embedding] B --> C[Semantic Search Against Tool Index] X --> C F[Filtering Criterion: top-k or similarity threshold] C --> D[Filtered Tool List] F --> D D --> E[Prompt Assembly]🔗 MCP Standards Check
[ ] Change adheres to current MCP specifications will need to send original User Prompt to the MCP Proxy / MCP Client for semantic search.
[X] No breaking changes to existing MCP-compliant integrations
[ ] If deviations exist, please describe them below:
🔄 Alternatives Considered
📓 Additional Context
Issue: Current tool injection results in cognitive overload for LLMs
Related discussion: Internal conversations on prompt-to-tool alignment and context-window optimization
Suggested embedding providers: OpenAI, OpenAI-compatible API endpoints (such as Nebius Studio, LiteLLM Proxy, DeepInfra), Mistral, Cohere, Jina, or local BERT models
Suggested VectorDB: PostgreSQL with pgvector extension (simply replace container with pgvector), Lance DB, In-Proces: pg_embed + pgvector, Chroma DB
Beta Was this translation helpful? Give feedback.
All reactions