feat(mcp): per-message caching for prune_tools, integration tests#2333
Merged
feat(mcp): per-message caching for prune_tools, integration tests#2333
Conversation
28448ca to
988b6ee
Compare
, #2300) Add PruningCache to agent loop state, keyed on (message_content_hash, tool_list_hash), to avoid redundant LLM calls when the tool list and user message have not changed between agent loop iterations. - PruningCache with CachedResult::Ok/Failed enum; single-slot, ~48 bytes on stack plus at most max_tools McpTool clones (~3 KB) on hit - prune_tools_cached() wrapper; negative caching prevents LLM retry storms on persistent pruning failures - tool_list_hash hashes full metadata (name, description, input_schema) sorted by qualified name using BTreeMap-backed serde_json::to_vec; sentinel byte on schema serialization failure - Cache wired into rebuild_system_prompt (once per turn); apply_pruned_ mcp_tools updates shared_tools RwLock without calling sync_mcp_executor_ tools; both write sites carry cross-referencing ordering contract comments - Cache reset at: process_user_message_inner start, check_tool_refresh, handle_mcp_add, handle_mcp_remove - Err branch of pruning match in rebuild_system_prompt calls sync_mcp_executor_tools to self-heal shared_tools on pruning failure Add 13 integration tests covering: always_include partition (including cross-server bare-name matching), max_tools cap, min_tools_to_prune early-return (replaces tautology), LLM failure propagation, parse error, positive cache hit, cache miss on message/tool-list change, negative cache hit (skips LLM on second call), and reset-clears-negative. Closes #2298, #2300
988b6ee to
65c7364
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PruningCacheto agent loop state inzeph-core, keyed on(message_content_hash, tool_list_hash), eliminating redundant LLM calls when tool list and user message are unchanged between agent loop iterationsCachedResult::Failednegative caching to prevent LLM retry storms on persistent pruning failuresrebuild_system_prompt(once per turn);apply_pruned_mcp_toolsupdatesshared_toolsRwLock independently ofsync_mcp_executor_toolssync_mcp_executor_toolsto self-healshared_toolson failurealways_includepartition,max_toolscap,min_tools_to_pruneearly-return (replaces tautology), LLM failure, parse error, and all cache invalidation pathsTest plan
cargo nextest run -p zeph-mcp -p zeph-core --lib— 1455/1455 passedcargo +nightly fmt --check— cleancargo clippy -p zeph-mcp -p zeph-core -- -D warnings— clean (workspace-level pre-existing errors are not introduced by this PR, verified by checking main without changes)Closes #2298, #2300