Skip to content

Commit 4d0137d

Browse files
cliffhallclaude
andcommitted
docs(testing): match the harness text to the four server PRs as merged
Copilot round 2: `everything`'s shared helper is `__tests__/harness.ts` (#4978 removed `helpers.ts`), and the in-process example connected by hand, skipping the per-session id and `cleanup(sessionId)` the harness provides. The example now uses the harness, and a table names each server's factory, harness file and `connect(...)` arguments as #4978, #4977, #4973 and #4970 contain them. The spawn-smoke section names each server's smoke test and how each one behaves without a build (`memory`'s fails, the others skip), and says `everything` has none. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: cliffhall <[email protected]>
1 parent dd0d053 commit 4d0137d

1 file changed

Lines changed: 63 additions & 51 deletions

File tree

‎.claude/skills/testing/SKILL.md‎

Lines changed: 63 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -32,14 +32,15 @@ and those servers have no coverage gate yet (#4855).
3232

3333
Each vitest config includes only `**/__tests__/**/*.test.ts`, so a test file
3434
anywhere else in a workspace is never run and reports nothing. Shared test
35-
helpers go in `__tests__/` without the `.test` suffix, as
36-
`src/everything/__tests__/helpers.ts` does.
35+
helpers go in `__tests__/` without the `.test` suffix, as each server's
36+
connection harness does (`__tests__/harness.ts` in `everything`,
37+
`__tests__/helpers.ts` in the other three).
3738

3839
## What each server's tests do
3940

4041
| Server | How its tests reach the code |
4142
| --- | --- |
42-
| `everything`, `filesystem`, `memory`, `sequentialthinking` | An SDK `Client` connected to the server's `createServer()` over `InMemoryTransport`, in the vitest process; unit tests of the helper modules beside them; at most a thin spawn-based smoke of the built `dist/index.js` |
43+
| `everything`, `filesystem`, `memory`, `sequentialthinking` | An SDK `Client` connected to the server's `createServer()` over `InMemoryTransport`, in the vitest process, through the server's own harness, plus unit tests of helper modules such as `lib.ts`. `everything` drives its stdio, SSE and Streamable HTTP transports in-process too; the other three have one thin spawn smoke of the built `dist/index.js` |
4344
| `fetch`, `git`, `time` | Direct calls on the functions in `server.py`, with `unittest.mock` |
4445

4546
No Python test opens a `ClientSession` yet.
@@ -63,53 +64,47 @@ Which client harness is available depends on the language.
6364

6465
Each TypeScript server exports a `createServer(…)` factory and starts its
6566
transport only from a guarded `main()`, so importing it starts nothing and a
66-
test can link a real `Client` to a real server with no process and no build:
67+
test can link a real `Client` to a real server with no process and no build.
68+
**Each server's `__tests__/` has a harness that does the linking. Connect
69+
through it**, never by calling `createServer()` and
70+
`InMemoryTransport.createLinkedPair()` by hand: the harness also does the
71+
per-session setup and teardown the server needs.
6772

68-
| Server | Factory | Takes |
69-
| --- | --- | --- |
70-
| `everything` | `server/index.ts` | nothing; returns `{ server, cleanup }` |
71-
| `filesystem` | `server.ts` | the allowed directories |
72-
| `memory` | `index.ts` | the graph file's path |
73-
| `sequentialthinking` | `index.ts` | nothing |
73+
| Server | Factory | Harness | `connect(…)` takes |
74+
| --- | --- | --- | --- |
75+
| `everything` | `createServer()` in `server/index.ts`, returns `{ server, cleanup }` | `__tests__/harness.ts` | `{ capabilities, taskStore, setup, sessionId }`, all optional |
76+
| `filesystem` | `createServer(allowedDirectories)` in `server.ts` | `__tests__/helpers.ts` | the allowed directories, then `{ capabilities, listRoots }` |
77+
| `memory` | `createServer(memoryFilePath)` in `index.ts` | `__tests__/helpers.ts` | the graph file's path (`makeTempGraph()` makes one) |
78+
| `sequentialthinking` | `createServer()` in `index.ts` | `__tests__/helpers.ts` | `{ disableThoughtLogging }`, defaulting to `"true"` |
7479

75-
Each server's `__tests__/` already has a shared helper that does the
76-
connecting (and, where the server needs one, the temporary directory or file).
77-
**Use it** rather than writing the linking by hand again. What it does, shown
78-
for `everything`:
80+
Each `connect` returns the `client` and a `close()`; call `close()` in
81+
`afterEach`. The harnesses also export small result helpers (`call`,
82+
`textOf` and the like); read the one you are using before adding another.
7983

8084
```ts
81-
import { describe, it, expect, beforeEach, afterEach } from "vitest";
82-
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
83-
import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
84-
import { createServer } from "../server/index.js";
85+
import { afterEach, beforeEach, describe, expect, it } from "vitest";
86+
import type { Client } from "@modelcontextprotocol/sdk/client/index.js";
87+
import { connect, contentOf, textOf, type Session } from "./harness.js";
8588

8689
describe("echo, over the protocol", () => {
90+
let session: Session;
8791
let client: Client;
88-
let cleanup: () => void;
8992

9093
beforeEach(async () => {
91-
const created = createServer();
92-
cleanup = created.cleanup;
93-
const [clientTransport, serverTransport] =
94-
InMemoryTransport.createLinkedPair();
95-
client = new Client({ name: "test-client", version: "0.0.0" });
96-
await Promise.all([
97-
created.server.connect(serverTransport),
98-
client.connect(clientTransport),
99-
]);
94+
session = await connect();
95+
client = session.client;
10096
});
10197

10298
afterEach(async () => {
103-
await client.close();
104-
cleanup();
99+
await session.close();
105100
});
106101

107102
it("returns the message", async () => {
108103
const result = await client.callTool({
109104
name: "echo",
110105
arguments: { message: "hi" },
111106
});
112-
expect(result.content).toEqual([{ type: "text", text: "Echo: hi" }]);
107+
expect(textOf(contentOf(result)[0])).toBe("Echo: hi");
113108
});
114109

115110
it("reports bad input as a tool error", async () => {
@@ -119,12 +114,23 @@ describe("echo, over the protocol", () => {
119114
});
120115
```
121116

122-
Close the client (and, for `everything`, call `cleanup()`) in `afterEach`:
123-
`everything`'s `createServer()` starts timers (the roots sync after
124-
`initialize`, the task store) that otherwise outlive the test. A tool
125-
`everything` registers in `registerConditionalTools` appears only when the
126-
`Client` declares the matching capability (`roots`, `sampling`,
127-
`elicitation`) in its constructor options.
117+
What `everything`'s harness does that a hand-rolled connection would miss:
118+
119+
- **A session id per connection.** Logging, subscriptions, roots and the
120+
toggle tools keep per-session state in module-level maps keyed by the
121+
transport's session id, and `InMemoryTransport` has none, so every
122+
hand-rolled session would share the `undefined` key and leak state into the
123+
next test. `connect()` sets a fresh `sessionId` on the server transport
124+
(`sessionId: null` keeps it undefined, as stdio does), and `close()` runs the
125+
server's `cleanup(sessionId)`, which also stops the timers `createServer()`
126+
starts (the roots sync, the task store).
127+
- **Capabilities.** A tool `everything` registers in `registerConditionalTools`
128+
appears only when the client declares the matching capability: pass
129+
`capabilities` (`ALL_CAPABILITIES` declares every one), with `setup` to
130+
install the client's handlers for the requests the server sends back
131+
(sampling, elicitation, roots) and `taskStore` for the task-augmented ones.
132+
- **Notifications.** `session.notifications` collects every notification no
133+
specific handler consumed; `ofMethod(notifications, method)` filters it.
128134

129135
Assert on what the client receives (tool, resource and prompt lists, call
130136
results, errors, notifications), never on SDK internals or a handler's context
@@ -133,22 +139,28 @@ import and type changes alone.
133139

134140
### TypeScript: the spawn smoke of the built binary
135141

136-
What only the published entry point can show (that `dist/index.js` starts,
137-
reads its arguments and serves over stdio) is checked by a thin test that
138-
spawns it with `StdioClientTransport`. Everything else is tested in-process.
139-
Two hazards apply to that kind of test, and only to it:
140-
141-
- ⚠️ **It runs the last build, not your edit.** `npm test` does not build. Build
142-
first, or run `npm run validate -w src/<server>`, which builds before it
143-
tests.
144-
- ⚠️ **A spawn test guarded with `skipIf(!existsSync(distIndexPath))` is
145-
reported as skipped with no build**, and the run stays green having tested
146-
nothing over stdio. A skipped test in this repo usually means "not built".
142+
What only the published entry point can show (that `dist/index.js` starts as a
143+
bin and serves over stdio; in `memory` and `sequentialthinking`, also through
144+
a symlink, as npm's `.bin` entry runs it) is checked by one thin test per server that spawns it with
145+
`StdioClientTransport`: `filesystem`'s `bin-smoke.test.ts`, and
146+
`stdio-smoke.test.ts` in `memory` and `sequentialthinking`. `everything` has
147+
none; its transports are driven in-process. Everything else is tested
148+
in-process. Two hazards apply to these tests, and only to them:
149+
150+
- ⚠️ **They run the last build, not your edit.** `npm test` does not build.
151+
Build first, or run `npm run validate -w src/<server>`, which builds before
152+
it tests.
153+
- ⚠️ **A missing `dist/` is handled two ways.** `filesystem` and
154+
`sequentialthinking` guard the smoke with `skipIf(!existsSync(distIndexPath))`,
155+
so with no build it is reported as **skipped** and the run stays green having
156+
tested nothing over stdio. `memory`'s has no guard and **fails**. A skipped
157+
test in this repo usually means "not built".
147158

148159
Close the client in `finally` or `afterEach`: that is what stops the child
149160
process. Pipe its stderr (`stderr: "pipe"`), and pass environment variables
150-
in the transport's `env` option, since the child inherits only the SDK's
151-
default safelist (`PATH`, `HOME`, `USER` and the like).
161+
in the transport's `env` option (`MEMORY_FILE_PATH`, say), since the child
162+
inherits only the SDK's default safelist (`PATH`, `HOME`, `USER` and the
163+
like).
152164

153165
### Python: `ClientSession` over stdio
154166

@@ -241,7 +253,7 @@ Clearing a red file:
241253
`exclude` to get green.
242254

243255
⚠️ **A spawned server is invisible to the report.** v8 coverage measures the
244-
vitest process, not a child it starts, so code reached only through the spawn
256+
vitest process, not a child it starts, so code reached only through a spawn
245257
smoke reads as uncovered. That is why behavior is tested in-process: a file
246258
whose only test spawns it cannot clear the gate.
247259

0 commit comments

Comments
 (0)