Skip to content

Commit 4a86760

Browse files
cdeustclaude
andcommitted
fix(ci): derive the host-contract tool bounds from the registries (#597)
The verifier carried MIN_FULL_TOOL_COUNT = 52 with a ceiling three above, so it stayed right when the surface moved to 54 (ADR-1066) and only broke when the prediction tools took it to 57: CI failed on "discovered 57 full tools; expected 52..55", taking down Test (SQLite backend) and Release dependency set. mcp_server/tool_surface.py reads the surface from the registry modules themselves and the verifier derives both bounds from it (ADR-1077). Three tests pin the derivation against what register_all actually registers, including that the verifier's bounds follow it. A literal that is right for a while, then wrong, and only caught once it drifts far enough is a second source of truth, not a check. Full suite: 8770 passed, 8 skipped. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01Cz7A6goxVzfKL9NT7k4zDH
1 parent 53ec2bb commit 4a86760

7 files changed

Lines changed: 195 additions & 2 deletions

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,15 @@ adheres to [Semantic Versioning](https://semver.org/).
88

99
### Added
1010

11+
- **The host-contract verifier counts the tools the server registers
12+
(ADR-1077).** `scripts/verify_mcp_hosts.py` held its own floor of 52 with a
13+
ceiling three above, so it stayed right through the move to 54 (ADR-1066)
14+
and only failed when the prediction tools took the count to 57, blocking two
15+
CI jobs on a number nothing had updated. `mcp_server/tool_surface.py` now
16+
reads the surface from the registry modules and the verifier derives its
17+
bounds from it, with a test pinning both against what `register_all`
18+
actually registers.
19+
1120
- **Prediction records, and a score for the confidences they carry (#597).**
1221
Cortex stored what happened and never what was expected, so nothing it held
1322
could turn out wrong in a way it noticed: 796 procedural skills mined from
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
<!-- Generated by Cortex wiki_reindex from wiki/adr/cortex/1077-the-advertised-tool-count-is-read-from-the-registries.md; ADR-1077. Read-only mirror: edit the canonical wiki page. -->
2+
3+
---
4+
created: 2026-09-16T18:27:56Z
5+
kind: adr
6+
number: 1077
7+
status: accepted
8+
tags: [tool-surface, ci, drift, issue-597]
9+
title: The advertised tool count is read from the registries
10+
---
11+
# ADR-1077: The advertised tool count is read from the registries
12+
13+
## Status
14+
15+
accepted
16+
17+
## Context
18+
19+
The number of tools this server advertises appears in a pinned test, in nine documents the doc-claim gate scans, in the docker-smoke floor and in `scripts/verify_mcp_hosts.py`, the host-contract verifier that drives a real stdio round trip against three clients. Every one of those carried its own copy of the number.
20+
21+
One copy went stale silently. `MIN_FULL_TOOL_COUNT = 52` with a ceiling of `+ 3` stayed at 52 when the surface moved to 54 (ADR-1066), because 54 still fell inside 52 to 55. The drift only surfaced three tools later, when the prediction tools took the count to 57 and the verifier failed in CI on `discovered 57 full tools; expected 52..55`, blocking `Test (SQLite backend)` and `Release dependency set` on PR #599.
22+
23+
A literal that is right for a while, then wrong, and only caught when it drifts far enough is not a check: it is a second source of truth waiting to disagree with the first.
24+
25+
## Decision
26+
27+
`mcp_server/tool_surface.py` reads the surface from the registry modules themselves: `standalone_tool_names()` is the union of their `SCHEMAS` maps minus the three gated names, and `UPSTREAM_TOOL_NAMES` holds those three. It imports registries only, never the server, so a caller pays no MCP startup for a count.
28+
29+
`scripts/verify_mcp_hosts.py` derives its bounds from that module rather than stating them. The remaining copies are prose for humans, and the doc-claim gate already cross-checks them against the pinned test name.
30+
31+
## Consequences
32+
33+
Easier: adding a tool moves the verifier's bounds with it, and a future ADR-1066 cannot pass CI while one check still counts the old surface.
34+
35+
`tests_py/test_tool_surface.py` pins the derivation against reality: the derived set equals what `register_all` actually registers, the gated names are exactly the difference between the two registrations, and the verifier's bounds equal the derived count and that count plus the gated ones.
36+
37+
Harder: `tool_surface.py` lists the standalone registries, so a new registry module must be added there as well as in `__main__`. The first test above fails the moment it is not.
38+
39+
Unchanged: the pinned baseline test and the documents keep their literals. They are read by people, and the gate that compares them to the code is already in place.

‎mcp_server/tool_surface.py‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
"""What this tree registers: the standalone tools and the gated ones.
2+
3+
The number appears in a test, in nine documents, in the docker-smoke floor
4+
and in the host-contract verifier, and every one of them used to carry its
5+
own copy. One copy went stale: `scripts/verify_mcp_hosts.py` held 52 while
6+
the surface moved to 54 (ADR-1066), which nothing noticed until the count
7+
reached 57 and broke its ceiling (issue #597).
8+
9+
This module is the one place that counts, read from the registries rather
10+
than stated. It imports the registry modules only, never the server, so a
11+
caller pays no MCP startup for a count.
12+
13+
source: ADR-1077"""
14+
15+
from __future__ import annotations
16+
17+
from mcp_server import (
18+
tool_registry_advanced,
19+
tool_registry_core,
20+
tool_registry_ingest,
21+
tool_registry_manage,
22+
tool_registry_memory,
23+
tool_registry_memory_maintenance,
24+
tool_registry_nav,
25+
tool_registry_predictions,
26+
tool_registry_wiki,
27+
tool_registry_wiki_drafts,
28+
)
29+
30+
# The three tools that register only when their upstream MCP server is
31+
# configured; every advertised tool works without one (ADR-0093).
32+
UPSTREAM_TOOL_NAMES: frozenset[str] = frozenset(
33+
{"ingest_codebase", "change_impact", "ingest_prd"}
34+
)
35+
36+
_STANDALONE_REGISTRIES = (
37+
tool_registry_core,
38+
tool_registry_memory,
39+
tool_registry_memory_maintenance,
40+
tool_registry_manage,
41+
tool_registry_nav,
42+
tool_registry_predictions,
43+
tool_registry_advanced,
44+
tool_registry_wiki,
45+
tool_registry_wiki_drafts,
46+
)
47+
48+
49+
def standalone_tool_names() -> frozenset[str]:
50+
"""Every tool name that registers without an upstream MCP server."""
51+
names: set[str] = set()
52+
for registry in _STANDALONE_REGISTRIES:
53+
names.update(registry.SCHEMAS)
54+
names.update(set(tool_registry_ingest.SCHEMAS) - UPSTREAM_TOOL_NAMES)
55+
return frozenset(names)

‎scripts/verify_mcp_hosts.py‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,14 +41,23 @@
4141
)
4242

4343
from mcp_server.tool_profiles import LEAN_TOOL_NAMES # noqa: E402
44+
from mcp_server.tool_surface import ( # noqa: E402
45+
UPSTREAM_TOOL_NAMES,
46+
standalone_tool_names,
47+
)
4448

4549

4650
CLIENTS = ("claude-code", "gemini-cli", "codex-cli")
4751
PROFILES: tuple[Literal["full", "lean"], ...] = ("full", "lean")
4852
STORAGE_SELECTIONS: tuple[Literal["sqlite", "auto"], ...] = ("sqlite", "auto")
53+
# The floor is what this tree registers without an upstream, read from the
54+
# registries themselves rather than copied: a literal here went stale when
55+
# the count moved 52 -> 54 (ADR-1066) and only failed the build three tools
56+
# later, when it moved to 57 (issue #597).
4957
# source: ADR-0788
50-
MIN_FULL_TOOL_COUNT = 52
51-
MAX_FULL_TOOL_COUNT = MIN_FULL_TOOL_COUNT + 3
58+
# The correction is recorded as ADR number 1077.
59+
MIN_FULL_TOOL_COUNT = len(standalone_tool_names())
60+
MAX_FULL_TOOL_COUNT = MIN_FULL_TOOL_COUNT + len(UPSTREAM_TOOL_NAMES)
5261

5362

5463
def _result(

‎tests_py/test_tool_surface.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
"""The advertised tool surface is read from the registries, not restated.
2+
3+
`scripts/verify_mcp_hosts.py` carried its own literal floor of 52 while the
4+
surface moved to 54 (ADR-1066), and nothing noticed until the count reached
5+
57 and broke that script's ceiling in CI (issue #597).
6+
7+
source: ADR-1077
8+
"""
9+
10+
from __future__ import annotations
11+
12+
import asyncio
13+
14+
from mcp.server.mcpserver import MCPServer
15+
16+
from mcp_server.__main__ import register_all
17+
from mcp_server.tool_surface import UPSTREAM_TOOL_NAMES, standalone_tool_names
18+
19+
20+
def _live(*, codebase: bool, prd: bool) -> set[str]:
21+
server = MCPServer(name="surface-test", version="0.0.0")
22+
register_all(server, codebase=codebase, prd=prd)
23+
return {tool.name for tool in asyncio.run(server.list_tools())}
24+
25+
26+
def test_the_derived_surface_is_what_registers() -> None:
27+
assert standalone_tool_names() == _live(codebase=False, prd=False)
28+
29+
30+
def test_the_upstream_names_are_exactly_the_gated_ones() -> None:
31+
gated = _live(codebase=True, prd=True) - _live(codebase=False, prd=False)
32+
33+
assert gated == set(UPSTREAM_TOOL_NAMES)
34+
35+
36+
def test_the_host_contract_bounds_follow_the_surface() -> None:
37+
from scripts.verify_mcp_hosts import MAX_FULL_TOOL_COUNT, MIN_FULL_TOOL_COUNT
38+
39+
assert MIN_FULL_TOOL_COUNT == len(standalone_tool_names())
40+
assert MAX_FULL_TOOL_COUNT == MIN_FULL_TOOL_COUNT + len(UPSTREAM_TOOL_NAMES)
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
created: 2026-09-16T18:27:56Z
3+
kind: adr
4+
number: 1077
5+
status: accepted
6+
tags: [tool-surface, ci, drift, issue-597]
7+
title: The advertised tool count is read from the registries
8+
---
9+
# ADR-1077: The advertised tool count is read from the registries
10+
11+
## Status
12+
13+
accepted
14+
15+
## Context
16+
17+
The number of tools this server advertises appears in a pinned test, in nine documents the doc-claim gate scans, in the docker-smoke floor and in `scripts/verify_mcp_hosts.py`, the host-contract verifier that drives a real stdio round trip against three clients. Every one of those carried its own copy of the number.
18+
19+
One copy went stale silently. `MIN_FULL_TOOL_COUNT = 52` with a ceiling of `+ 3` stayed at 52 when the surface moved to 54 (ADR-1066), because 54 still fell inside 52 to 55. The drift only surfaced three tools later, when the prediction tools took the count to 57 and the verifier failed in CI on `discovered 57 full tools; expected 52..55`, blocking `Test (SQLite backend)` and `Release dependency set` on PR #599.
20+
21+
A literal that is right for a while, then wrong, and only caught when it drifts far enough is not a check: it is a second source of truth waiting to disagree with the first.
22+
23+
## Decision
24+
25+
`mcp_server/tool_surface.py` reads the surface from the registry modules themselves: `standalone_tool_names()` is the union of their `SCHEMAS` maps minus the three gated names, and `UPSTREAM_TOOL_NAMES` holds those three. It imports registries only, never the server, so a caller pays no MCP startup for a count.
26+
27+
`scripts/verify_mcp_hosts.py` derives its bounds from that module rather than stating them. The remaining copies are prose for humans, and the doc-claim gate already cross-checks them against the pinned test name.
28+
29+
## Consequences
30+
31+
Easier: adding a tool moves the verifier's bounds with it, and a future ADR-1066 cannot pass CI while one check still counts the old surface.
32+
33+
`tests_py/test_tool_surface.py` pins the derivation against reality: the derived set equals what `register_all` actually registers, the gated names are exactly the difference between the two registrations, and the verifier's bounds equal the derived count and that count plus the gated ones.
34+
35+
Harder: `tool_surface.py` lists the standalone registries, so a new registry module must be added there as well as in `__main__`. The first test above fails the moment it is not.
36+
37+
Unchanged: the pinned baseline test and the documents keep their literals. They are read by people, and the gate that compares them to the code is already in place.

‎wiki/manifest.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3475,6 +3475,10 @@
34753475
"ADR-1076": {
34763476
"mirror": "ADR-1076-a-prediction-is-written-open-and-resolved-against-evidence-the-caller-supplies.md",
34773477
"path": "adr/cortex/1076-a-prediction-is-written-open-and-resolved-against-evidence-the-caller-supplies.md"
3478+
},
3479+
"ADR-1077": {
3480+
"mirror": "ADR-1077-the-advertised-tool-count-is-read-from-the-registries.md",
3481+
"path": "adr/cortex/1077-the-advertised-tool-count-is-read-from-the-registries.md"
34783482
}
34793483
},
34803484
"project": "cortex",

0 commit comments

Comments
 (0)