Repository navigation
Description
Summary
Load MCP tool schemas on demand instead of eagerly, reducing system prompt size and context waste across all providers
Problem statement
Every configured MCP server currently loads ALL its tools eagerly at startup. Full tool schemas (name, description, JSON parameter schema) are serialized into every LLM API call and system prompt. With multiple MCP servers, this adds dozens or hundreds of tool definitions, wasting context window tokens on tools that may never be used in a given conversation. For example, a setup with 3 MCP servers exposing 50 tools total could add ~77K tokens of tool schema to every turn, even when only 2-3 tools are actually needed. This degrades both cost efficiency and model accuracy (more noise in context = worse tool selection).
Proposed solution
Implement Claude Code-style deferred tool loading with two tiers:
- Deferred: MCP tools are listed by name and short description only in the system prompt inside an
<available-deferred-tools>section. They are NOT sent as native tool definitions. - Activated: When the LLM needs an MCP tool, it first calls a built-in
tool_searchtool (always present when deferred MCP tools exist). This fetches the full schema, registers the tool in a per-conversationActivatedToolSet, and returns the full definition so the LLM can call it on the next turn.
Controlled by mcp.deferred_loading config key (default true). Setting false preserves current eager behavior exactly.
Key components:
DeferredMcpToolStub/DeferredMcpToolSet: lightweight name+description stubs for system prompt listing and searchActivatedToolSet: per-conversation mutable state tracking tools activated during a conversationToolSearchTool: built-in tool implementingTooltrait, supportsselect:name1,name2for exact match and keyword search- Per-iteration
tool_specsrecomputation in the agent loop to pick up newly activated tools - Helpful error message when LLM tries to call a deferred tool without activating it first
Reference: Anthropic Advanced Tool Use describes this pattern and reports context reduction from ~77K to ~8.7K tokens with accuracy improvements.
Non-goals / out of scope
- Deferred loading for native (non-MCP) tools. These are always eagerly loaded as today.
- Persistent activation across conversations. Activated tools are scoped to a single conversation turn.
- Automatic activation (LLM must explicitly call
tool_search). - Changes to MCP server connection lifecycle. Servers are still connected at startup; only schema delivery to the LLM is deferred.
Alternatives considered
- Signature-based approach: Adding
deferred_toolsandactivated_toolsparameters directly torun_tool_call_loop. Rejected because the function already has 15 parameters and this would require updating ~20+ test call sites. Task-local variables (consistent with existingTOOL_LOOP_*pattern) achieve the same result with zero signature churn. - Combined registry rebuild: Building a new
Vec<Box<dyn Tool>>each iteration by cloning the entire static registry plus activated tools. Rejected becauseBox<dyn Tool>is not cloneable. Instead, execution functions accept anextra_toolsslice that is searched alongside the static registry. - Automatic activation on first call: Transparently activating deferred tools when the LLM calls them without
tool_search. Rejected because it hides the activation step from the LLM, preventing it from seeing the full schema before crafting arguments.
Acceptance criteria
mcp.deferred_loading = true(default): MCP tools appear only as names in the system prompt,tool_searchis registered as a built-in tool, callingtool_searchactivates tools and makes them callable on subsequent turns.mcp.deferred_loading = false: Current eager behavior preserved exactly. Notool_searchtool, no<available-deferred-tools>section.- Calling a deferred tool without activation returns a helpful error directing the LLM to use
tool_search. - All existing tests pass with no regressions. Non-MCP tool behavior completely unchanged.
docs/config-reference.mddocumentsmcp.deferred_loading.
Architecture impact
src/tools/— newmcp_deferredandtool_searchmodules,McpRegistry::empty()constructor, module registrationsrc/config/schema.rs—McpConfig.deferred_loadingfieldsrc/agent/loop_.rs— task-localDeferredToolContext, per-iteration tool_specs recomputation, deferred-not-activated error pathsrc/agent/loop_/execution.rs—extra_toolsparameter onfind_tool/execute_tools_*functionssrc/agent/prompt.rs—build_deferred_tools_section()renderingsrc/channels/mod.rs— conditional deferred vs eager MCP wiring, system prompt injection,ChannelRuntimeContextfield, task-local scoping
Risk and rollback
No response
Breaking change?
No
Data hygiene checks
- I removed personal/sensitive data from examples, payloads, and logs.
- I used neutral, project-focused wording and placeholders.
Activity
mykolademyanov commentedon Mar 10, 2026
Once agents start interacting with tools or external systems, debugging usually becomes harder than expected.
One pattern that helps is separating reasoning from execution.
The model proposes actions, while another layer actually executes them, handles retries and records state.
Otherwise failures often get hidden inside the agent loop.
Short overview of this runtime pattern:
https://agentpatterns.tech/en/architecture/agent-runtime
Dave-London commentedon Mar 10, 2026
Deferred loading solves the schema side of the token problem nicely. The output side matters too — a single verbose tool call can burn thousands of tokens of formatting. We address this by returning structured JSON instead of raw CLI text, typically 70-95% fewer output tokens. https://github.com/Dave-London/Pare
feat(tool): on-demand MCP tool loading via tool_search
feat(tool): on-demand MCP tool loading via tool_search (#3446)
feat(tool): on-demand MCP tool loading via tool_search (zeroclaw-labs…
feat(tool): on-demand MCP tool loading via tool_search (zeroclaw-labs…
feat(tool): on-demand MCP tool loading via tool_search (zeroclaw-labs…