Skip to content

Commit 2e1c4b9

Browse files
ondraulehlaclaude
andcommitted
Probe: a server that rejects the protocol version the client asks for is up, not down
In modelcontextprotocol/registry#1715 two posts point out that a liveness result depends on the protocol version the client asks for. A server that answers "Unsupported protocol version" is up. It must not count as dead. On the board 15 registry entries did exactly that: they rejected 2025-11-25, the version that the SDK asks for, with HTTP 400 or a JSON-RPC error (-32022, -32602, -32004). They were "error" or "protocol_error", and 14 of them were down since early October. - New status unsupported_version. It is gated (reachable, not usable with this client), so it is never down. Its evidence state is "unknown". - Each evidence row now keeps the step that failed (method_phase: initialize or tools/list) and the JSON-RPC error code of the answer. The file says which client, SDK and requested version ran the checks (checked_with), and each row already had the version the server answered. - The run summary keeps the requested version. The server page uses it. - A package that answers with a newer version or rejects ours is not reclassified as crashed. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 0626d65 commit 2e1c4b9

17 files changed

Lines changed: 64 additions & 11 deletions

File tree

‎packages/mcpcost/src/cli.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,7 @@ function statusText(r: Row): string {
280280
}
281281
if (s === 'payment') return 'payment required';
282282
if (s === 'newer_protocol') return 'newer protocol than this SDK';
283+
if (s === 'unsupported_version') return 'rejects the protocol version this SDK asks for';
283284
if (s === 'not_found') return r.kind === 'stdio' ? 'command not found' : 'not found';
284285
return s.replace('_', ' ');
285286
}

‎packages/mcpcost/src/core.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,8 @@ export function classify(message: string | undefined, first: Observed | undefine
246246
const text = message ?? '';
247247
// The server speaks a protocol version that this SDK does not support yet.
248248
if (/protocol version is not supported/i.test(text)) return 'newer_protocol';
249+
// The server rejects the version this SDK asks for, with HTTP 400 or a JSON-RPC error: it is up, just not for this version.
250+
if (/unsupported (?:or missing )?(?:mcp )?protocol version/i.test(text)) return 'unsupported_version';
249251
if (first) {
250252
if (first.status === 401 || first.status === 403) return 'auth';
251253
if (first.status === 402) return 'payment';

‎packages/mcpcost/src/mcp.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ const text = (value: unknown) => ({ content: [{ type: 'text' as const, text: JSO
4242
const READ_ONLY = { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true } as const;
4343

4444
const TOKEN_FIELDS = {
45-
status: { type: 'string', description: 'ok (answered initialize and tools/list), auth (needs credentials), payment (402), not_found, protocol_error (not MCP), timeout, dns, tls, refused, server_error, rate_limited, newer_protocol, or for a local command: install_failed, no_bin, needs_env, needs_args, needs_network, crashed' },
45+
status: { type: 'string', description: 'ok (answered initialize and tools/list), auth (needs credentials), payment (402), not_found, protocol_error (not MCP), timeout, dns, tls, refused, server_error, rate_limited, newer_protocol, unsupported_version (rejects the protocol version the client asks for), or for a local command: install_failed, no_bin, needs_env, needs_args, needs_network, crashed' },
4646
error: { type: 'string', description: 'The first line of the failure, when status is not ok' },
4747
protocolVersion: { type: 'string', description: 'The MCP protocol version the server negotiated' },
4848
serverInfo: { type: 'object', description: 'name, version and title the server reports in initialize' },

‎packages/mcpcost/src/types.ts‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,9 @@ export type ProbeStatus =
7575
| 'server_error'
7676
| 'protocol_error'
7777
| 'newer_protocol'
78+
// The server rejects the protocol version that the client asks for in initialize (it may speak an older or a newer one).
79+
// It is up and speaks MCP; a client with another SDK version may connect.
80+
| 'unsupported_version'
7881
| 'timeout'
7982
| 'dns'
8083
| 'tls'
@@ -92,8 +95,11 @@ export type ProbeStatus =
9295
| 'needs_network'
9396
| 'crashed';
9497

95-
/** Reachable but not usable without something from the user: credentials, arguments, a network, a URL of their own, or a newer SDK. */
96-
export const GATED_STATUSES: ReadonlySet<string> = new Set(['auth', 'payment', 'newer_protocol', 'needs_env', 'needs_args', 'needs_network', 'needs_url']);
98+
/**
99+
* Reachable but not usable without something from the user: credentials, arguments, a network, a URL of their own, or
100+
* another SDK version (the server speaks a newer protocol, or rejects the version the client asks for).
101+
*/
102+
export const GATED_STATUSES: ReadonlySet<string> = new Set(['auth', 'payment', 'newer_protocol', 'unsupported_version', 'needs_env', 'needs_args', 'needs_network', 'needs_url']);
97103

98104
/** A result that is neither alive nor gated: the server did not give a valid answer in this probe. An entry the probe did not run has no result. */
99105
export function isFailing(status: string): boolean {

‎packages/mcpcost/tests/probe.test.ts‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,15 @@ describe('probeRemote classification with a fake fetch', () => {
193193
expect(r.error).toContain('Missing required _meta');
194194
});
195195

196+
it('maps an initialize that the server rejects for its protocol version to unsupported_version, not to a failure', async () => {
197+
const rejected = JSON.stringify({ jsonrpc: '2.0', id: 0, error: { code: -32022, message: 'Unsupported protocol version: 2025-11-25', data: { supported: ['2026-07-28'] } } });
198+
const r400 = await probeRemote(target, { fetch: respond(400, { 'content-type': 'application/json' }, rejected) });
199+
expect(r400.status).toBe('unsupported_version');
200+
expect(r400.httpStatus).toBe(400);
201+
const mcp = JSON.stringify({ jsonrpc: '2.0', id: 0, error: { code: -32602, message: 'Unsupported MCP protocol version.' } });
202+
expect((await probeRemote(target, { fetch: respond(200, { 'content-type': 'application/json' }, mcp) })).status).toBe('unsupported_version');
203+
});
204+
196205
it('maps DNS, TLS, refused and timeout failures', async () => {
197206
const withCause = (code: string) => Object.assign(new TypeError('fetch failed'), { cause: Object.assign(new Error(code), { code }) });
198207
expect((await probeRemote(target, { fetch: fail(withCause('ENOTFOUND')) })).status).toBe('dns');

‎probe/build-data.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import { mkdir, readFile, writeFile } from 'node:fs/promises';
1212
import { existsSync } from 'node:fs';
1313
import { parseArgs } from 'node:util';
1414
import { GATED_STATUSES, hostOf, isConfirmedDown, isFailing, isSchemaRefusal } from '../packages/mcpcost/src/index.js';
15+
import { LATEST_PROTOCOL_VERSION } from '@modelcontextprotocol/sdk/types.js';
1516
import { downChange, failingSinceOf, NOT_AN_ENTRY } from './rows.js';
1617
import { LISTED_META, type Listing } from './sources.js';
1718
import { BUNDLE_META, type Bundle } from './bundles.js';
@@ -500,6 +501,8 @@ const topHosts = hosts.slice(0, 500).map((h) => ({
500501
const weight = (r: ServerRow) => r.claudeTokens ?? (counted.length ? -1 : (r.toolsTokens ?? 0));
501502
const summary = {
502503
probedAt: run.probedAt,
504+
/** The protocol version the probe asks for in initialize (the newest one of its SDK). */
505+
requestedProtocolVersion: LATEST_PROTOCOL_VERSION,
503506
registryEntries: registryOnly.length,
504507
tracked: registryRaw.filter((r) => registryMeta.get(r.server.name)?.status === 'tracked').length,
505508
/** Servers outside the registry from public catalogs (probe/sources.ts). */

‎probe/dataset.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ const evidence = {
8888
auth: 'reachable, needs credentials or payment: 401, 403, 402',
8989
transient: 'no valid answer in this run, and not dead: a timeout, a 429, a 5xx or a refused connection; or a dead cause seen for less than two days',
9090
dead: 'no DNS, a TLS failure, 404/405/410, or an answer that is not MCP, in every check for two days or more (evidence.failing_since); a package that does not install or has no executable at two checks a week apart',
91-
unknown: 'a package that needs values, arguments or a network to start, or crashed on start; a URL with placeholders that each user fills in, so it was not called; or an error the probe could not classify'
91+
unknown: 'a package that needs values, arguments or a network to start, or crashed on start; a URL with placeholders that each user fills in, so it was not called; a server that rejects the protocol version the probe asks for (status unsupported_version: it is up); or an error the probe could not classify'
9292
},
9393
probed_at: summary.probedAt,
9494
checked_with: checkedWith,

‎probe/rows.test.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,16 @@ test('evidence carries the entry version and the first failed check', () => {
5050
assert.equal(row.evidence.failing_since, DAY1);
5151
assert.equal('version' in evidenceRow(server({ status: 'ok' })), false);
5252
});
53+
54+
test('a server that rejects the requested protocol version is unknown, never dead, and the row keeps the step and the error code', () => {
55+
const rejected = server({ status: 'unsupported_version', httpStatus: 400, failingSince: DAY1, error: 'Streamable HTTP error: Error POSTing to endpoint: {"jsonrpc":"2.0","id":0,"error":{"code":-32022,"message":"Unsupported protocol version: 2025-11-25"}}' });
56+
const row = evidenceRow(rejected);
57+
assert.equal(row.state, 'unknown');
58+
assert.equal(row.evidence.method_phase, 'initialize');
59+
assert.equal(row.evidence.jsonrpc_error_code, -32022);
60+
// A server that answered initialize and failed on tools/list says so; a package check has no phase.
61+
assert.equal(evidenceRow(server({ status: 'protocol_error', initMs: 120, error: 'MCP error -32601: Method not found' })).evidence.method_phase, 'tools/list');
62+
assert.equal(evidenceRow(server({ status: 'protocol_error', initMs: 120, error: 'MCP error -32601: Method not found' })).evidence.jsonrpc_error_code, -32601);
63+
assert.equal(evidenceRow(server({ status: 'crashed', transport: 'stdio' })).evidence.method_phase, undefined);
64+
assert.equal(evidenceRow(server({ status: 'ok' })).evidence.method_phase, undefined);
65+
});

‎probe/rows.ts‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ export const DATASET_FIELDS: Record<string, string> = {
1818
version: 'the version of the registry entry that was probed (the latest one)',
1919
host: 'the host of the remote endpoint; npmjs.com or pypi.org for a package server',
2020
transport: 'streamable-http, sse or stdio',
21-
status: 'the probe result: ok, auth, payment, newer_protocol, needs_url, not_found, protocol_error, dns, tls, timeout, refused, server_error, rate_limited, error; for a package: crashed, install_failed, no_bin, needs_env, needs_args, needs_network. needs_url: every URL of the entry has a placeholder that each user fills in, so the probe did not call it',
21+
status: 'the probe result: ok, auth, payment, newer_protocol, unsupported_version (the server rejects the protocol version the probe asks for), needs_url, not_found, protocol_error, dns, tls, timeout, refused, server_error, rate_limited, error; for a package: crashed, install_failed, no_bin, needs_env, needs_args, needs_network. needs_url: every URL of the entry has a placeholder that each user fills in, so the probe did not call it',
2222
failing_since: 'for a server with no valid answer: the first probe of the current run of failed probes (ISO 8601). The board calls a server down after two days of them; one failed probe is "no answer"',
2323
http_status: 'the HTTP status of the first response, when there was one',
2424
auth_scheme: 'the WWW-Authenticate scheme of a server that needs auth',
@@ -80,6 +80,8 @@ export type EvidenceState = 'alive' | 'auth' | 'transient' | 'dead' | 'unknown';
8080
export const EVIDENCE_STATE: Record<string, EvidenceState> = {
8181
ok: 'alive',
8282
newer_protocol: 'alive',
83+
// Up and speaking MCP, but not the version this client asks for: not dead, and not alive for this client either.
84+
unsupported_version: 'unknown',
8385
auth: 'auth',
8486
payment: 'auth',
8587
timeout: 'transient',
@@ -121,6 +123,10 @@ export interface EvidenceRow {
121123
tool_count?: number;
122124
/** For a failed check: the first check of the current run of failed checks, ISO 8601. */
123125
failing_since?: string;
126+
/** For an endpoint check that did not pass: the step it stopped at. */
127+
method_phase?: 'initialize' | 'tools/list';
128+
/** The JSON-RPC error code in the server's answer, when there is one. */
129+
jsonrpc_error_code?: number;
124130
error?: string;
125131
};
126132
}
@@ -133,6 +139,11 @@ export function evidenceRow(s: ServerRow): EvidenceRow {
133139
if (s.protocolVersion) evidence.protocol_version = s.protocolVersion;
134140
if (s.toolCount != null) evidence.tool_count = s.toolCount;
135141
if (s.failingSince) evidence.failing_since = s.failingSince;
142+
// What modelcontextprotocol/registry#1715 asks a liveness row to keep: the step that failed and the server's error code.
143+
// initMs is set only when initialize succeeded. The requested protocol version and the SDK are in checked_with.
144+
if (s.status !== 'ok' && s.transport !== 'stdio' && s.status !== 'needs_url') evidence.method_phase = s.initMs != null ? 'tools/list' : 'initialize';
145+
const code = /"code"\s*:\s*(-?\d+)|MCP error (-?\d+)/.exec(s.error ?? '');
146+
if (code) evidence.jsonrpc_error_code = Number(code[1] ?? code[2]);
136147
if (s.error) evidence.error = s.error;
137148
let state = EVIDENCE_STATE[s.status] ?? 'unknown';
138149
// Our own failed check is not proof that the entry is dead: that takes two days of failed checks.

‎probe/stdio-classify.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ const FOREIGN = /ENOEXEC|exec format error|cannot execute binary file|executable
2727
export function classify(result: Pick<ProbeResult, 'status' | 'error'>): { status: StdioStatus; text: string } {
2828
const text = `${result.error ?? ''}`;
2929
if (result.status === 'ok') return { status: 'ok', text };
30+
// The server answered initialize, with a protocol version this client does not take or a refusal of the one it asked for.
31+
if (result.status === 'newer_protocol' || result.status === 'unsupported_version') return { status: result.status, text };
3032
const key = text.split('\n').map((l) => l.trim()).find((l) => KEY_LINE.test(l)) ?? '';
3133
let status: StdioStatus = result.status;
3234
if (FOREIGN.test(text)) status = 'no_bin';

0 commit comments

Comments
 (0)