Summary
mcp_mocks are converted to Copilot SDK stdio server configurations without the Copilot CLI tools allowlist. The bundled CLI rejects the mock server with No tools specified for server "github", so MCP tools never reach the evaluated agent.
Reproduction
- Create an eval with
config.executor: copilot-sdk and an inline or fixture-backed mcp_mocks entry.
- Run the evaluation with the bundled Copilot CLI.
- Observe the CLI log:
No tools specified for server "github" followed by the server being skipped.
Expected behavior
Every server generated from mcp_mocks should include the Copilot CLI allowlist tools: ["*"], making its declared deterministic tools available to the agent.
Actual behavior
The mock server process is healthy, but its SDK configuration omits the allowlist; the CLI skips it without Waza surfacing a warning.
Environment
Reported with Waza 0.38.2 and bundled Copilot CLI 1.0.64-0 on Windows 10 and WSL2 Ubuntu.
Workaround
Use config.mcp_servers directly and add tools: ["*"].
Proposed fix
Add the allowlist to Waza's mock-to-SDK conversion, cover it with a regression test, and document the behavior.
Summary
mcp_mocksare converted to Copilot SDK stdio server configurations without the Copilot CLItoolsallowlist. The bundled CLI rejects the mock server withNo tools specified for server "github", so MCP tools never reach the evaluated agent.Reproduction
config.executor: copilot-sdkand an inline or fixture-backedmcp_mocksentry.No tools specified for server "github"followed by the server being skipped.Expected behavior
Every server generated from
mcp_mocksshould include the Copilot CLI allowlisttools: ["*"], making its declared deterministic tools available to the agent.Actual behavior
The mock server process is healthy, but its SDK configuration omits the allowlist; the CLI skips it without Waza surfacing a warning.
Environment
Reported with Waza 0.38.2 and bundled Copilot CLI 1.0.64-0 on Windows 10 and WSL2 Ubuntu.
Workaround
Use
config.mcp_serversdirectly and addtools: ["*"].Proposed fix
Add the allowlist to Waza's mock-to-SDK conversion, cover it with a regression test, and document the behavior.