Guidance for building servers that act as proxy clients to many servers #94
Replies: 12 comments 14 replies
-
|
Thanks for flagging this! These are great points to think about. Regarding your specific list:
Can you share more about the concern you think exists with (2)? If "instructions" refer to "how to use all the tools that the server provides" (for example), a server-of-servers seems like it could get most of the way by just naively concatenating the instructions from all its child servers. I'm curious if that seems inadequate, though!
I think it's probably reasonable for clients to automatically namespace everything per-server (and this could apply to the server-of-servers with its children servers as well). There might be problems with this I'm not seeing, though. We certainly need to give more thought here.
We probably want to extend these notifications to support delta updates regardless ("these things were added, these were deleted"), which might solve this problem too. |
Beta Was this translation helpful? Give feedback.
-
|
With respect to |
Beta Was this translation helpful? Give feedback.
-
@jspahrsummers, I think it's not adequate because I envision the set of configured servers changing over time. Even within the life-cycle of a single session. Maybe the tool allows servers to be dynamically installed? In this case, the instructions will be invalidated and the server will have no mechanism to inform clients. |
Beta Was this translation helpful? Give feedback.
-
|
Servers acting as aggregators can work with the existing protocol by implementing their own namespacing. Some aggregators may not even want to expose their child servers directly; they can combine multiple low-level servers (e.g., filesystem, database) into a high-level, domain-specific API. The idea of having a server with the main purpose of reducing repeated logic in client applications is interesting, but I'd argue that libraries/SDKs are the ones that should take care of that. Very possible I'm missing something. That said, building tree-like MCP applications does raise many questions. For example, imagine a leaf server requesting sampling from a root LLM through a few middle layers. It might help to have standard guidelines for forwarding requests and handling notifications, so each node doesn’t need to invent its own pass-through logic. Same goes for the other direction, i.e. notifying a downstream branch. It's hard to speculate at this stage, we'll know more in a month or two when different patterns emerge. |
Beta Was this translation helpful? Give feedback.
-
|
This sounds to me like a problem to solve at the discovery level. That is, say you have an server that exports a discovery endpoint like https://github.com/orgs/modelcontextprotocol/discussions/84 you can define separate MCP server endpoints for serving. That is, you can use http urls as the namespace. I think you'll continue to run into a set of things that break when trying to treat separate MCP servers as a single server (e.g. oauth with a proxy of numerous MCP subservers with their own oauth endpoints for each subset of the server sounds complicated). This may be fine for aggregating simple public servers -- but when trying to generalize further there will probably continue to be a number of things like this where trying to add more isolation within the MCP protocol will be complex for clients to adopt. |
Beta Was this translation helpful? Give feedback.
-
|
@ggoodman I'm working on a similar problem, I think you're spot on with how you're thinking about the general flow here. One question - could this all be as simple as introducing the notion of hierarchy to the various name fields? So, a prompt name like "code_review" becomes "@my-code-reviewer/code_review" or "my-code-reviewer.com/code_review" Myself and @justinwilaby are working on a standardized search capability for servers with high volumes of tools and resources, and I think this fits in nicely with what we're working on: We're specifically thinking about the enterprise use case of a proxy that proxies to numerous servers downstream from it and might have thousands or tens of thousands of tools exposed, way too many to simply list, and where you absolutely will have name collisions. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @ggoodman , I had similar feelings and built something you might find helpful. It sort of takes the approach @jspahrsummers mentioned, where it acts as a server of servers. It's essentially a central ingress/load balancer that can route traffic to different mcp servers based on things like tool call names. link if you are curious. It's completely open-source; hope it helps with your use case. |
Beta Was this translation helpful? Give feedback.
-
|
@mclenhard just to confirm. Lets say you have many MCP servers (with different urls). This proxy gateway would:
Does that make sense? Has anyone built anything like this>? |
Beta Was this translation helpful? Give feedback.
-
That is correct. As for your other points, three and four are fully handled. For OAuth, right now, it just forwards the request. It does maintain a cache of sessions to ensure that requests get forwarded to whatever server was previously responsible for that session, but there's definitely more work to be done here. Limiting scope is a good idea, and I've been thinking about it myself. I just haven't settled on the proper way to handle this yet. How much should go into my gateway vs. the application itself? I would love your thoughts if you have any here. Ideas and PR's are always welcome! |
Beta Was this translation helpful? Give feedback.
-
|
What you are describing is essentially an API gateway. Always keep in mind that we have two identities on the client side. The identity of the actual client application and the one of the user on whose behalf the calls are made. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, it's an API gateway. Good call out. I didn't even think about the identity of the client itself, just the user. Probably some interesting tooling to be built there as well. |
Beta Was this translation helpful? Give feedback.
-
|
It's the same set of problems like in API management. Seems natural that we use the same or at least similar solutions. Of course my opinion might be biased coming from the APIM side of things :D |
Beta Was this translation helpful? Give feedback.
-
|
Regarding client (not user) identity: there are many use cases that are enabled by this, including anti-abuse mechanisms, quality of service mechanisms, "peering" agreements, auditability, etc. There's already significant momentum behind using HTTP Message Signatures to identify Agents on the web, as well as other non-AI use cases (crawlers, aggregators, proxy operators). See this recent blogpost by CloudFlare, which mentions OpenAI's adoption of this standard. I know similar conversations have been happening across other anti-abuse vendors and agent operators. There's also some conversation around using "optiona mTLS" as an alternative mechanism (one is much more "web developer persona" friendly, the other is more "network admin persona" friendly). Since MCP servers may be proxies or aggregators of traffic themselves, I think it's going to be important for the MCP protocol to add identity mechanisms internally. Both when initialization a connection (for receiving and validating the identity of the entity making the inbound request), and also for outbound requests. I can also imagine wanting to chain these identities such that downstream entities would be able to say "I'm receiving this request from MCP client X who received a request from Agent Y". Making the chain of operators verifiable would enable many downstream use cases. It's probably in the interest of MCP folks to participate in these conversations, and also to adopt these standards. For this to work well I also think we're going to want any A2A protocols to participate as well. |
Beta Was this translation helpful? Give feedback.
-
|
I'm currently solving the issue of namespacing by using slash separation. For context, I'm building a MCP registry+Proxy (exactly what @sterankin describes above) focused on enterprises. Every MCP server registered in my proxy is its own namespace. Tools can only be called by using the format But I've already run into several issues here:
So it seems like currently Anthropic is very restrictive on the whole namespacing concept. The only good solution I found is to use double underscore Do people here have opinions on this? Would be great to have some guidance on how this should be handled properly |
Beta Was this translation helpful? Give feedback.
-
|
It looks like a lot of people are running into issues with that regex. Having no spec-defined limitations on tool names (beyond "a string") means strict validations are entirely left up to client application implementers when limitations of some sort are needed for security or other reasons. Claude Code requiring alphanumeric+dash+underscore tool names is much more restrictive than the spec allows for, though. Might be worth having a spec-defined limitation here for clients to use where necessary. |
Beta Was this translation helpful? Give feedback.
-
|
I agree. Tool naming format should be defined as part of MCP spec. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, sharing some related notes that seem relevant here from the Technical Steering Committee meeting at the MCP Dev Summit on 5/23/25: More context from the discussion:The suggestion below emerged during discussions about:
The committee consensus was that namespacing should be:
Plan 9-inspired examples:In Plan 9, everything is a file and namespaces are per-process. You can "mount" remote resources into your local namespace at any path, creating a unified hierarchical view. How this concept could be applied to MCP Servers and Tools: Hierarchical Organization Just like Plan 9's file system, tools would have full paths: Remote Server Mounting A proxy server could dynamically mount remote MCP servers, eg: # Conceptual example
proxy.mount("https://api.github.com/mcp", "/github/")
proxy.mount("https://slack-mcp.com", "/slack/")
proxy.mount("local-file-server", "/files/")Advantages mentioned at the meeting:
Current allowed character limitationsAs noted in the meeting (and by @duaraghav8 above), MCP tool names must match Other approaches discussed for avoiding Tool name collisions and naming: |
Beta Was this translation helpful? Give feedback.
-
|
@olaservo Thanks for writing this up - the main thing that this misses vs this PR is the ability to scope down tool lists by namespace. So, while this handle the naming collision issue, it does NOT handle namespaces as a way to implement interfaces or "workspaces" where a small set of tools are composed for better grokability by agents (ie smaller sets of tools to select from rather than huge lists). @evalstate did some tests that current LLM context max out remarkably early with tools, something in the range of 100 tools I believe (please correct me if I'm wrong). So, a major component of #334 is ways to scope the tool sets down. That DOES require a small protocol change, and I think is an incredibly important because it allows scoping down of lists. So, a client could conceivably get or search namespaces, or present namespaces to the user to select, then have a constrained set of tools. A bit of information that I found really good - this isn't JUST for proxying, it's important for companies with large api surface areas. Microsoft's office api (Graph) has 26,000 endpoints - if 100 tool is the max, that's 260 servers they have to maintain, expose, and teach the models how to orchestrate across (I have no idea how they do that tbh), or teach their business analyst users how to connect to different servers. This is as opposed to, with namespaces, where you could namespace by feature area, then expose some targeted namespaces like "send email" that expose tools draft email, send email, spell check. @cliffhall ^ you might like this use case too for similar reasons. So, without a protocol change, you are still limited in the number of tools on a server because you can't list tools from just a single namespace, so we haven't really accomplished anything with "guidance." |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @patwhite, you're absolutely right that the mounting approach I described from the Dev Summit doesn't by itself address the tool discovery/listing problem for large servers. Apologies for maybe combining topics too much here. To clarify a little more: I think the 'guidance vs prescriptive' comment was specifically focused on the name collision aspect, not so much the scale/filtering challenge. (Edited) |
Beta Was this translation helpful? Give feedback.
-
|
My only small reservation here would be that the function names can impact LLM behavior, so a system where the function names can change easily might impact re-use of prompts or agent reliability. If user A mounts Slack endpoints at "slack/" and user B mounts them at "comms/" then you could get pretty divergent behavior. I suspect people end up promoting a "recommended" prefix anyway... |
Beta Was this translation helpful? Give feedback.
-
|
Part of this also comes down to having high-quality tool descriptions - if I mount Slack tools at That puts the onus strongly on server authors to write good tool descriptions, which isn't always the case, however. |
Beta Was this translation helpful? Give feedback.
-
|
I think this will also be less of an issue as clients hopefully gain more awareness of the server I have tried to use this in my own MCP servers, but unfortunately, at least Claude seems unaware of it. I haven't used other clients. |
Beta Was this translation helpful? Give feedback.
-
|
Sharing my project MetaMCP, a MCP proxy + aggregator. You can group MCP servers into namespaces then assign endpoints to them. You can also filter tools out, search feature WIP. https://github.com/metatool-ai/metamcp Btw I think there are 2 dimensions logical group may consider: like typical blog posts, there are categories and tags concepts, where a blog can only fall into one category, but can have many tags, and the filter by tag can be both "AND" or "OR" logic. Similarly from a tool search perspective, a category is a elasticsearch like "index", then within the index you can filter by tags, apply search dsls, or have vector search combined in hybrid mode to take in user intent from natural language. |
Beta Was this translation helpful? Give feedback.
-
|
Ya, you and I are working on similar projects - if you check out #322 you'll see a full proposal around search, I'm 100% in agreement we need a protocol level mechanism for search |
Beta Was this translation helpful? Give feedback.
-
|
I don't see this explicitly mentioned here but In an MCP gateway, the tool, prompt, resource names are used for routing to the right MCP server implementing that specific resource. It seems unlikely to happen, but implementations should consider that a tool name may include the used separator string (like ___). MCP gateways can't just assume that the last part of the tool name after splitting using the separator is the actual resource name and anything else is the server name. |
Beta Was this translation helpful? Give feedback.
-
|
I actually thought about this a lot. As of today, given the lack of naming convention in MCP spec and the strict RegExp in Claude code & desktop, the double/triple underscore is the best way to create separation between tool name & server name within a Gateway for routing. But I would definitely love to see:
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Pre-submission Checklist
Your Idea
I think there is lots of potential for an MCP server to act as a single point of configuration for many other MCP servers. This would benefit apps looking to act as MCP clients as they could thereby offload all of this management logic to an MCP proxy app.
Here is a straw man for a CLI experience:
mcp setup claude-- Installmcp run --scope claudeas a binary inclaude.json.mcp server search github-- Search the list of known MCP servers for ones that have the keyword,GitHub.mcp server install --scope claude @namespace/mcp-server-- Install an MCP server locally, scoped toclaude.mcp server install @namespace2/mcp-server-2-- Install another MCP server locally for all scopes (clients).mcp run --scope <scope>-- The entrypoint for the binary to act as astdioMCP server.Note
In a world where some form of local service discovery is added to the protocol (or becomes a community convention), step 1 might even be skipped. It would be enough to install the binary and have it start on login and listen in whatever way works with local service discovery.
This sort of UX would allow many clients to benefit from a single local experience for configuring LLM context providers. It would be a perfect place for users to authenticate themselves to 3rd party services without every MCP client needing to re-implement this logic over and over. It might even allow the protocol to remain agnostic to this sort of bearer credential. The protocol could limit itself to the mutual authentication between client and server.
Spec changes
To help support this sort of hypothetical setup, I think the spec might benefit from some extra level of namespacing. Here are some area that I can think of based on the parts of the spec I've interacted with:
list_changednotifications -- These notifications would invalidate the whole namespace when finer-grained invalidation might be more efficient. If a namespacing concept were to be introduced, these notifications could be augmented to limit their applicability to zero or more namespaces.As I haven't implemented anywhere close to 100% of the spec, the above is almost certainly not an exhaustive list. But hopefully it offers some inspiration for some potential future directions in the spec.
Scope
Beta Was this translation helpful? Give feedback.
All reactions