You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1cdd4a7
Browse filesBrowse the repository at this point in the historyBrowse files
chore(python): per-file coverage gate in CI and local:gate
Add `npm run coverage:py` (scripts/coverage-py.mjs): for each Python
server, a locked sync, then `uv run --frozen pytest --cov
--cov-report=term-missing --cov-report=json`, then a per-file check of
coverage.json (scripts/lib/py-coverage.mjs) that fails any file below 90%
on lines or on branches. coverage.py's fail_under is global, hence the
script. Discovery, server selection and the carry-on-after-a-failure shape
are shared with validate-py.mjs.
Wire it in as a parallel `coverage` matrix job in python.yml and as a
stage of local:gate:stages after validate:py, and document it in
docs/quality-gate.md, AGENTS.md (the Python half of the coverage rule),
CONTRIBUTING.md and the pre-push-gate and testing skills.
Signed-off-by: cliffhall <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Copy file name to clipboardExpand all lines: .claude/skills/pre-push-gate/SKILL.md
+34-1Lines changed: 34 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -75,7 +75,8 @@ unquoted `#` or `:` in a `description`.
75
75
`verify:skills:cli` is the authoritative validator. It fetches the pinned
76
76
Claude Code CLI with `npx` when the installed one is a different version, so it
77
77
**fails offline**. (`validate:py`'s `sync` step is the other stage that can:
78
-
`uv sync --locked` downloads when a server's environment is missing or stale.)
78
+
`uv sync --locked` downloads when a server's environment is missing or stale.
79
+
`coverage:py` opens with the same sync, a no-op once `validate:py` has run.)
79
80
80
81
### `verify:typecheck-coverage`
81
82
@@ -205,6 +206,38 @@ the last `[validate:py] <server>: <step>` line above it. Re-run one server with
205
206
206
207
`uv: command not found` means `uv` is not installed. Never substitute `pip`.
207
208
209
+
### `coverage:py`
210
+
211
+
The Python per-file coverage gate: every file each server measures must reach
212
+
**90% on lines and 90% on branches**. The summary names the server, the step,
213
+
and for a shortfall each file and dimension below 90:
214
+
215
+
```
216
+
FAIL time — per-file check (2 below 90%)
217
+
src/time/src/mcp_server_time/server.py: lines 84.2% (75/89) is below 90%
218
+
src/time/src/mcp_server_time/server.py: branches 78.5% (11/14) is below 90%
219
+
```
220
+
221
+
Re-run one server with `npm run coverage:py -- <server>`. The per-file table
222
+
above the summary gives both figures for every file, and coverage.py's own
223
+
report above that (`term-missing`) lists the uncovered lines and partial
224
+
branches (`120->124` is the jump from line 120 to 124 never taken).
225
+
226
+
| Step | Fix |
227
+
| --- | --- |
228
+
|`sync`| As for `validate:py`: run `uv lock` in that server and commit the lockfile |
229
+
|`pytest --cov`| A test failed, so no verdict was reached; fix it as for `validate:py`'s `pytest`. `unrecognized arguments: --cov` means `pytest-cov` is missing from the server's dev dependencies |
230
+
|`per-file check`, "below 90%" | Write tests for the lines and branches the report lists. Drive the server in-process through a `ClientSession` where you can (`/testing`) |
231
+
|`per-file check`, "was not written" or "measured no files" | The run produced no usable `coverage.json`: check the server's `[tool.coverage.run]` (`branch = true`, `source` naming its package) |
232
+
|`per-file check`, "no covered_branches/num_branches" |`branch = true` is missing from `[tool.coverage.run]`|
233
+
234
+
**Never lower the gate**, and never drop a file from `source` to clear it.
235
+
Code that genuinely cannot run (a guard the types already rule out, a branch
236
+
the SDK's input validation makes unreachable) is marked at the source with
237
+
`# pragma: no cover # <reason>`; every pragma carries its reason
238
+
(`AGENTS.md`**Always test new or modified code**). A pragma on a line that is
239
+
merely awkward to test is not justified; write the test.
240
+
208
241
### `smoke`
209
242
210
243
`smoke — FAIL <server> over <transport>` followed by the reason and the tail
|`fetch`, `git`, `time`|`src/<server>/tests/`|`test_<subject>.py`| pytest, with `pytest-asyncio` (`asyncio_mode = "auto"`)|
31
31
| Root tooling |`scripts/`, beside the script |`<name>.test.mjs`|`node --test`|
32
32
33
33
Each vitest config includes only `**/__tests__/**/*.test.ts`, so a test file
@@ -41,10 +41,9 @@ connection harness does (`__tests__/harness.ts` in `everything`,
41
41
| Server | How its tests reach the code |
42
42
| --- | --- |
43
43
|`everything`, `filesystem`, `memory`, `sequentialthinking`| An SDK `Client` connected to the server's `createServer()` over `InMemoryTransport`, in the vitest process, through the server's own harness, plus unit tests of helper modules such as `lib.ts`. `everything` drives its stdio, SSE and Streamable HTTP transports in-process too; the other three have one thin spawn smoke of the built `dist/index.js`|
44
-
|`fetch`, `git`| Direct calls on the functions in `server.py`, with `unittest.mock`|
45
-
|`time`| A `ClientSession` against `serve()` in-process, over in-memory streams (`tests/test_protocol.py`); direct calls on the helpers in `server.py` (`tests/test_server.py`) |
44
+
|`fetch`, `git`, `time`| A `ClientSession` linked in-process to the real `serve()` over in-memory streams (below), plus direct calls on the pure functions in `server.py`|
46
45
47
-
Of the Python servers, only `time` opens a `ClientSession` in its tests.
46
+
No TypeScript test uses an in-memory transport yet.
48
47
49
48
## Choosing a harness for a new test
50
49
@@ -163,47 +162,71 @@ in the transport's `env` option (`MEMORY_FILE_PATH`, say), since the child
163
162
inherits only the SDK's default safelist (`PATH`, `HOME`, `USER` and the
164
163
like).
165
164
166
-
### Python: `ClientSession` over stdio
165
+
### Python: `ClientSession` over in-memory streams, in-process
167
166
168
-
Each Python server constructs its `Server` inside `serve()`, so it cannot be
169
-
handed to an in-memory session directly. `time` drives it in-process anyway, by
170
-
patching `stdio_server` to yield the server side of
171
-
`create_client_server_memory_streams()`; see the `serving()`helper in
172
-
`src/time/tests/test_protocol.py`. The other route spawns the module and talks
173
-
to it through the SDK's stdio client:
167
+
Each Python server builds its `Server` inside `serve()` and serves it over the
168
+
`stdio_server()` it imports into `server.py`. A test replaces that one name
169
+
with a context manager that yields the server side of a pair of in-memory
170
+
streams, runs the real `serve()`in a task group, and opens a `ClientSession`
171
+
on the client side. No process, no build, and the SDK's input validation and
172
+
error mapping run exactly as they do for a real client:
174
173
175
174
```python
176
-
import sys
175
+
from contextlib import asynccontextmanager
176
+
from typing import Any, AsyncIterator
177
177
178
178
import anyio
179
-
from mcp import ClientSession, StdioServerParameters
result =await session.call_tool("get_current_time", {"timezone": "UTC"})
207
+
assert result.isError isFalse
208
+
```
201
209
202
-
How an async test is written differs by server, because the dev dependencies
203
-
do: `fetch` and `time` have `pytest-asyncio` with `asyncio_mode = "auto"`,
204
-
so an `async def test_…` just works. `git` does not have it; wrap the
205
-
coroutine in `anyio.run(...)` as above (`git`'s `serve()` test does), rather
206
-
than adding a dependency for one test.
210
+
-**Patch the name `server.py` looks up** (`mcp_server_<name>.server.stdio_server`),
211
+
not `mcp.server.stdio.stdio_server`: `server.py` imported the function, so
212
+
the module attribute is what `serve()` calls.
213
+
-**Cancel the task group after the session closes.**`serve()` runs until its
214
+
streams end, so without the cancel the test never returns.
215
+
-**Assert on the wire shape where you can**, for example
216
+
`result.model_dump(by_alias=True, mode="json")`, rather than on Python
217
+
attribute names: the SDK v2 port (#4851) renames them, and a wire-level
218
+
assertion survives it.
219
+
-**No network in a test.**`fetch` replaces `httpx.AsyncClient` with one on an
220
+
`httpx.MockTransport`; `git` works against a temporary repository its
221
+
fixture creates and closes before removing.
222
+
223
+
`pytest-asyncio` with `asyncio_mode = "auto"` is a dev dependency of all three,
224
+
so an `async def test_…` runs as it stands.
225
+
226
+
A subprocess test over stdio (`StdioServerParameters(command=sys.executable,
227
+
args=["-m", "mcp_server_<name>"])` with `mcp.client.stdio.stdio_client`) is
228
+
kept to a thin smoke of the entry point. coverage.py does not measure a child
229
+
process, so code reached only that way reads as uncovered.
207
230
208
231
## Tests that pin a known bug
209
232
@@ -242,6 +265,8 @@ In Python the same text follows `#`. A bug with no issue yet reads
242
265
| One Python server |`uv run pytest` in `src/<server>`|
243
266
| One Python test |`uv run pytest <path to the test file>::<name>` in `src/<server>` (for example `tests/test_server.py::test_git_checkout_existing_branch`) |
244
267
| One Python server's whole chain |`npm run validate:py -- <server>`|
268
+
| One Python server's coverage gate |`npm run coverage:py -- <server>` at the root |
269
+
| Every Python server's coverage gate |`npm run coverage:py` at the root |
245
270
| Root tooling |`npm run test:scripts`|
246
271
247
272
`npm run typecheck -w src/<server>` is the only step that typechecks the test
@@ -281,13 +306,30 @@ Clearing a red file:
281
306
- Never lower a threshold, drop `perFile`, or add a file to the coverage
282
307
`exclude` to get green.
283
308
309
+
**The Python servers' coverage is a gate.**`npm run coverage:py -- <server>`
310
+
runs, in `src/<server>`:
311
+
312
+
```sh
313
+
uv run --frozen pytest --cov --cov-report=term-missing --cov-report=json
314
+
```
315
+
316
+
which prints coverage.py's report (the missing lines and partial branches)
317
+
and writes `coverage.json` (ignored by git). `scripts/lib/py-coverage.mjs` then
318
+
checks **every file** in it: at least 90% of its lines and 90% of its branches,
319
+
each file on its own. What is measured is the server's `[tool.coverage.run]`
320
+
(`branch = true`, `source` set to its package). `local:gate` and CI's
321
+
**Coverage \<server\>** job both run it. It is kept out of the plain
322
+
`uv run pytest` loop, which stays fast and uninstrumented.
323
+
324
+
A line or branch that genuinely cannot run is marked
325
+
`# pragma: no cover # <reason>`, with the reason on the same line; never
326
+
lower the gate. Diagnosing a red run is `/pre-push-gate`.
327
+
284
328
⚠️ **A spawned server is invisible to the report.** v8 coverage measures the
285
329
vitest process, not a child it starts, so code reached only through a spawn
286
330
smoke reads as uncovered. That is why behavior is tested in-process: a file
287
331
whose only test spawns it cannot clear the gate.
288
332
289
-
The Python servers have no coverage tooling yet (#4855).
290
-
291
333
## Error output
292
334
293
335
A test that expects an error keeps it off the console. Spy on the logger for
0 commit comments