Skip to content

Commit ca3b490

Browse files
astrogildacopybara-github
authored andcommitted
test: verify MCP stdio tool execution and cleanup during evaluation
`test_mcp_stdio_agent_no_runtime_error` in `test_local_eval_service.py` spawned an unpinned `npx -y @modelcontextprotocol/server-filesystem` process and caught every exception inside `perform_inference()`, allowing a failed MCP server startup, failed inference, or unfinished cleanup to pass as a successful test. Replace it in `test_local_eval_service.py` with tests against a local finite Python stdio MCP server that verify `LocalEvalService.perform_inference()` executes the MCP tool call, receives the native tool result, completes model inference, and closes the MCP server cleanly without cleanup warnings or errors. Additional tests verify that toolset load failures log an error while inference succeeds without the toolset's tools, MCP tool errors (`isError=True`) are forwarded to the model while inference succeeds, model errors return `InferenceStatus.FAILURE` and close the toolset, and toolset close errors are logged while inference succeeds. Pull Request: #7474 PiperOrigin-RevId: 997703943
1 parent bca4a42 commit ca3b490

1 file changed

Lines changed: 312 additions & 140 deletions

File tree

0 commit comments

Comments
 (0)