Repository navigation
Commit ca3b490
test: verify MCP stdio tool execution and cleanup during evaluation
`test_mcp_stdio_agent_no_runtime_error` in `test_local_eval_service.py`
spawned an unpinned `npx -y @modelcontextprotocol/server-filesystem` process
and caught every exception inside `perform_inference()`, allowing a failed MCP
server startup, failed inference, or unfinished cleanup to pass as a successful
test.
Replace it in `test_local_eval_service.py` with tests against a local finite
Python stdio MCP server that verify `LocalEvalService.perform_inference()`
executes the MCP tool call, receives the native tool result, completes model
inference, and closes the MCP server cleanly without cleanup warnings or
errors. Additional tests verify that toolset load failures log an error while
inference succeeds without the toolset's tools, MCP tool errors (`isError=True`)
are forwarded to the model while inference succeeds, model errors return
`InferenceStatus.FAILURE` and close the toolset, and toolset close errors are
logged while inference succeeds.
Pull Request: #7474
PiperOrigin-RevId: 9977039431 parent bca4a42 commit ca3b490
1 file changed
Lines changed: 312 additions & 140 deletions
0 commit comments