Skip to content

Commit bbedff7

Browse files
cliffhallcliffhall
authored
Merge pull request #5288 from modelcontextprotocol/v2/docs/5170-client-smoke-playwright
docs(client-smoke): drive the Inspector web UI with Playwright
2 parents 846cc6c + d2108e9 commit bbedff7

2 files changed

Lines changed: 126 additions & 32 deletions

File tree

‎.claude/skills/client-smoke/SKILL.md‎

Lines changed: 125 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: client-smoke
3-
description: "Drive a locally built server with a real client: the Inspector CLI (scripted), the Inspector web UI (by hand) and an LLM client. Use when smoke-testing a server change; when calling a tool, reading a resource or getting a prompt from the command line; when choosing a protocol era; or when the Inspector CLI will not connect to, or misreads, a server command."
3+
description: "Drive a locally built server with a real client: the Inspector CLI (scripted), the Inspector web UI (through Playwright) and an LLM client. Use when smoke-testing a server change; when calling a tool, reading a resource or getting a prompt from the command line; when choosing a protocol era; or when the Inspector CLI will not connect to, or misreads, a server command."
44
disable-model-invocation: false
55
---
66

@@ -14,11 +14,11 @@ record is in `/pr-flow` (client evidence); building and starting the server is
1414

1515
Three clients, in the order to reach for them:
1616

17-
| Client | Driven by | Use it for |
18-
| --- | --- | --- |
19-
| Inspector **CLI** | A command. This is the **scripted path**, and the one an agent runs | One request, one JSON result: the evidence that goes in a PR |
20-
| Inspector **web** UI | A person, by hand. Nothing in this repo automates a browser | What only a UI shows: notifications arriving, elicitation and sampling prompts, a long-running tool's progress |
21-
| An **LLM client** | A prompt | Whether a model picks the tool and uses its result, which the PR checklist asks for |
17+
| Client | Driven by | Use it for |
18+
| -------------------- | ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |
19+
| Inspector **CLI** | A command. This is the **scripted path**, and the one an agent runs | One request, one JSON result: the evidence that goes in a PR |
20+
| Inspector **web** UI | A Playwright script in headless Chromium, which an agent writes and runs | What the CLI cannot do: elicitation and sampling round-trips, URL elicitation, tasks, notifications arriving, a long-running tool's progress |
21+
| An **LLM client** | A prompt | Whether a model picks the tool and uses its result, which the PR checklist asks for |
2222

2323
Build first (`npm run build -w src/<server>`). Every command below runs the
2424
last build of a TypeScript server, not your edit.
@@ -85,7 +85,7 @@ That rule produces the three failures people actually hit:
8585
`--cli -e KEY=VALUE node …` leaves no server command at all, and the CLI
8686
falls back to your personal catalog (`~/.mcp-inspector/mcp.json`). With
8787
several servers in it that fails with `Multiple servers found in config
88-
file. Please specify one with --server`. Either way it is no longer talking
88+
file. Please specify one with --server`. Either way it is no longer talking
8989
to your build, so move the option after the command.
9090
- **`Method is required`** means the options ended up on the wrong side of a
9191
`--`.
@@ -106,13 +106,13 @@ $INSPECT node src/memory/dist/index.js \
106106

107107
The exit code is the verdict, so check it rather than reading the text:
108108

109-
| Outcome | stdout | stderr | Exit |
110-
| --- | --- | --- | --- |
111-
| The request succeeded | `{"result": …}` | | 0 |
112-
| The tool ran and returned `isError: true` | `{"result":{"content": …, "isError": true}}` | `{"error":{"code":"tool_is_error", …}}` | 5 |
113-
| No such tool | | `{"error":{"code":"tool_not_found", …}}` | 5 |
114-
| An HTTP server could not be reached (refused, DNS, timeout) | | `{"error":{"code":"unreachable", …}}` | 4 |
115-
| Bad arguments, a failed version negotiation, or a stdio server that exited | | `{"error":{"code":"error", …}}` | 1 |
109+
| Outcome | stdout | stderr | Exit |
110+
| -------------------------------------------------------------------------- | -------------------------------------------- | ---------------------------------------- | ---- |
111+
| The request succeeded | `{"result": …}` | | 0 |
112+
| The tool ran and returned `isError: true` | `{"result":{"content": …, "isError": true}}` | `{"error":{"code":"tool_is_error", …}}` | 5 |
113+
| No such tool | | `{"error":{"code":"tool_not_found", …}}` | 5 |
114+
| An HTTP server could not be reached (refused, DNS, timeout) | | `{"error":{"code":"unreachable", …}}` | 4 |
115+
| Bad arguments, a failed version negotiation, or a stdio server that exited | | `{"error":{"code":"error", …}}` | 1 |
116116

117117
These are the codes observed on 2.9.0 for the cases this repo's servers
118118
produce; the CLI defines others (schema and skill checks, authentication)
@@ -166,11 +166,11 @@ or session code; it is deprecated but must keep working.
166166
The Inspector connects in one of three eras, chosen with `--protocol-era` (the
167167
CLI) or **Server Settings → Protocol Era** (the web UI):
168168

169-
| Era | What the Inspector does |
170-
| --- | --- |
171-
| `legacy` (the default) | A plain `initialize`. Negotiates 2025-11-25 or older |
172-
| `modern` | Pins 2026-07-28 through `server/discover`, with no fallback |
173-
| `auto` | Probes `server/discover`, and falls back to `initialize` |
169+
| Era | What the Inspector does |
170+
| ---------------------- | ----------------------------------------------------------- |
171+
| `legacy` (the default) | A plain `initialize`. Negotiates 2025-11-25 or older |
172+
| `modern` | Pins 2026-07-28 through `server/discover`, with no fallback |
173+
| `auto` | Probes `server/discover`, and falls back to `initialize` |
174174

175175
**Every server serves both eras**: the Python servers since #4853, the
176176
TypeScript servers since #4852 (stdio through the SDK's `serveStdio`, and
@@ -196,14 +196,19 @@ from the web UI. On 2026-07-28 the CLI does follow a task handle: a call to
196196

197197
## The Inspector web UI
198198

199-
The web client is the hand-driven path. It needs a person at a browser; this
200-
repo configures no browser automation, so an agent prepares the command and
201-
the steps and asks for what was observed.
199+
The web client is driven with **Playwright** in headless Chromium: an agent
200+
writes the script, runs it, and records what came back. It is not a step to
201+
hand to a maintainer. Everything below was run on **Inspector 2.12.0** and
202+
**Playwright 1.64.1**, against `everything` in both eras; pin both in any
203+
script you keep, since the selectors are the UI's, not an API.
202204

203205
```sh
204-
npx -y @modelcontextprotocol/inspector node src/everything/dist/index.js stdio
205-
npx -y @modelcontextprotocol/inspector -- uv --directory src/time run mcp-server-time
206-
npx -y @modelcontextprotocol/inspector --transport http --server-url http://localhost:3001/mcp
206+
npx -y @modelcontextprotocol/[email protected] --web --protocol-era modern -- \
207+
node src/everything/dist/index.js stdio
208+
npx -y @modelcontextprotocol/[email protected] --web --protocol-era legacy -- \
209+
uv --directory src/time run mcp-server-time
210+
npx -y @modelcontextprotocol/[email protected] --web \
211+
--transport http --server-url http://localhost:3001/mcp
207212
```
208213

209214
⚠️ **The web launcher reads `--` the other way round from the CLI.** Here the
@@ -212,14 +217,103 @@ it; in `--cli` mode the server command is before `--` and the options after.
212217
A line copied from one mode to the other starts the wrong thing. Pass a
213218
server's environment with `-e KEY=VALUE` before the `--`.
214219

215-
Use the web UI for what a single request cannot show: `toggle-simulated-logging` and
220+
Install Playwright in a scratch directory **outside the worktree**, never as a
221+
dependency of this repo (`npm i [email protected]`, then
222+
`npx playwright install chromium`), and run the script from there. The
223+
launcher inherits the script's working directory, so spawn it with `cwd` set
224+
to the worktree root: the server commands above use paths relative to it, and
225+
from the scratch directory no server would start.
226+
227+
### The recipe
228+
229+
- **Launch.** Spawn the launcher above with `MCP_AUTO_OPEN_ENABLED=false` in
230+
its environment, or it opens a tab in your own browser as well. It prints a
231+
tokenized URL (`http://127.0.0.1:6274?MCP_INSPECTOR_API_TOKEN=…`) on stdout;
232+
read it from there and navigate to it. Start the launcher in its own process
233+
group (`detached: true`) and kill the group when the script ends, so the
234+
server it spawned goes with it.
235+
- **Connect.** The connect control is a Mantine switch whose input is hidden:
236+
`getByRole("switch", { name: /Connect or disconnect/ }).check({ force: true })`.
237+
- **Pick a screen.** The Servers, Tools, Prompts and Resources tabs are radios
238+
that cannot be clicked; click their label, `locator('label[for$="-Tools"]')`.
239+
A tool is then a button named for its title and name:
240+
`getByRole("button", { name: / echo$/ })`.
241+
- **Fill and run.** A field is a textbox named for its property
242+
(`getByRole("textbox", { name: /message/ })`). Run with the **Execute Tool**
243+
button; it stays disabled until every required field is filled.
244+
- **Selects.** A Mantine Select (an `enum` field, in a tool form or in an
245+
elicitation) is a **read-only** textbox: `fill` does nothing. Click it, then
246+
press `ArrowDown` and `Enter`, which picks the first option.
247+
- **Read the result.** The panel's heading is **Results**, or **Tool Call
248+
Failed**. A result with `isError: true` keeps the **Results** heading and
249+
adds an alert named **Tool Error** holding the message, so check for that
250+
alert rather than the heading. Which heading a failure gets varies by era:
251+
rejecting a sampling request shows **Tool Call Failed** on 2026-07-28 and
252+
**Results** plus **Tool Error** on 2025-11-25.
253+
- **Elicitation.** A dialog named **Elicitation Request**, with **Cancel**,
254+
**Decline** and **Submit**. Its question is the paragraph that opens with
255+
`“`; the dialog's first `<p>` is its title.
256+
- **Multi-step elicitation.** After **Submit**, wait for the question to change
257+
or for the result heading, **never for the dialog to close**. Both eras
258+
reuse the open dialog for the next step, so it never closes between steps,
259+
and on the 2025 era it can read as hidden for a moment while the next
260+
question renders, so a wait for it to close either hangs or returns early.
261+
The URL error path's retry reuses the dialog the same way: tell its two
262+
prompts apart by their **Request ID** line.
263+
- **URL elicitation.** The dialog shows **Open in Browser**, and **I've
264+
completed it** only after that has been clicked. The click opens a popup
265+
page: catch it with `page.waitForEvent("popup")` around the click, close it,
266+
then click **I've completed it**.
267+
- **Sampling.** A dialog named **Sampling Request**: fill the **Response**
268+
textbox and click **Send Response**, or click **Reject**.
269+
- **Tasks.** On 2026-07-28 the Inspector follows the task handle itself: run
270+
`simulate-research-query` and wait for **Results** like any other call.
271+
- **Isolation.** Open a fresh page, and connect again, for each scenario. One
272+
scenario's open dialog or dropped connection then cannot fail the next.
273+
274+
Era differences that change what to expect, beyond the tool list (see
275+
[Protocol eras](#protocol-eras)):
276+
277+
| Scenario | 2025-11-25 (`legacy`) | 2026-07-28 (`modern`) |
278+
| --------------------------------------------- | -------------------------------------------------------- | ---------------------------------------------------------------------- |
279+
| `trigger-url-elicitation` with `errorPath` on | **Two** dialogs: the -32042 error path, then the retry's | **One**: there is no -32042 on 2026-07-28 |
280+
| `trigger-sampling-request`, rejected | **Results** with a **Tool Error** alert | **Tool Call Failed** |
281+
| Each prompt for input | Plain dialog | The dialog also shows `input_required` (the answer is sent as a retry) |
282+
283+
A sketch of one scenario, with the launch and teardown above:
284+
285+
```js
286+
const page = await browser.newPage();
287+
await page.goto(url);
288+
await page
289+
.getByRole("switch", { name: /Connect or disconnect/ })
290+
.check({ force: true });
291+
await page.locator('label[for$="-Tools"]').click();
292+
await page
293+
.getByRole("button", { name: / trigger-elicitation-request$/ })
294+
.click();
295+
await page.getByRole("button", { name: "Execute Tool" }).click();
296+
const dialog = page.getByRole("dialog", { name: "Elicitation Request" });
297+
await dialog.getByRole("textbox", { name: "String", exact: true }).fill("Ada");
298+
await dialog.getByRole("button", { name: "Submit" }).click();
299+
await page
300+
.getByRole("heading", { name: /^(Results|Tool Call Failed)$/ })
301+
.waitFor();
302+
// → Results: "✅ User provided the requested information! … Name: Ada"
303+
await page.close();
304+
```
305+
306+
Use the web UI for what a single CLI request cannot show:
307+
`trigger-elicitation-request`, `trigger-multi-step-elicitation`,
308+
`trigger-url-elicitation` and `trigger-sampling-request` (the client is asked
309+
something), the task tools on 2026-07-28, `toggle-simulated-logging` and
216310
`toggle-subscriber-updates` (notifications over time),
217-
`trigger-elicitation-request` and `trigger-sampling-request` (the client is
218-
asked something), `trigger-long-running-operation` (progress), and a
219-
filesystem server's response to a Roots change.
311+
`trigger-long-running-operation` (progress), and a filesystem server's
312+
response to a Roots change.
220313

221-
Evidence from the web UI is what was clicked and what appeared, in words. This
222-
repo does not attach screenshots.
314+
Evidence from the web UI is the scenario, era and what came back, in words: the
315+
result heading and its text, or the dialogs that appeared. This repo does not
316+
attach screenshots.
223317

224318
## An LLM client
225319

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ not in advance.
3131
| [`local-dev`](.claude/skills/local-dev/SKILL.md) | Install, build and run each server from the checkout over the transports it implements; local `npx`/`uvx` and client-config runs; stale builds and fresh worktrees; the `overrides`, lockstep and `uv.lock` procedures | Model-invoked, or `/local-dev` |
3232
| [`testing`](.claude/skills/testing/SKILL.md) | The in-process protocol-level harness, test placement, the commands per suite, `test` versus `coverage`, and clearing the per-file coverage gate | Model-invoked, or `/testing` |
3333
| [`release`](.claude/skills/release/SKILL.md) | A milestone release end to end: the release issue, the preparation PRs on `v2/main` (audit, Version Packages, Python CalVer), the pure `v2/main` → `main` merge PR, the release ledger, and what a maintainer publishes | **Name-only**: `/release` |
34-
| [`client-smoke`](.claude/skills/client-smoke/SKILL.md) | Driving a built server with the Inspector CLI (the scripted path), the Inspector web UI (by hand) and an LLM client; the CLI's argument split and exit codes; protocol eras and what can be exercised today | Model-invoked, or `/client-smoke` |
34+
| [`client-smoke`](.claude/skills/client-smoke/SKILL.md) | Driving a built server with the Inspector CLI (the scripted path), the Inspector web UI (through Playwright) and an LLM client; the CLI's argument split and exit codes; protocol eras and what can be exercised today | Model-invoked, or `/client-smoke` |
3535

3636
A PR that adds a skill under `.claude/skills/<name>/SKILL.md` adds its row to
3737
this table in the same change, and the table never lists a skill that does not

0 commit comments

Comments
 (0)