Repository navigation
MCP servers whose tool surface exists only as code have no route: detect reports the official MongoDB and Grafana servers as not agent projects #431
Description
Summary
Shipgate reports that the official MongoDB MCP server is not an agent project:
$ agents-shipgate detect --workspace mongodb-mcp-server --json
is_agent_project : false
frameworks : []
suggested_sources: []
next_action : "Stop: Workspace has no framework imports, no tool artifacts,
and no prompt directory."
That repository publishes 49 MCP tools, including drop-database, drop-collection,
delete-many, update-many, atlas-create-cluster and atlas-create-db-user. It is
precisely the kind of surface this product exists to gate, and there is no route to it today.
The important finding is what the repository actually contains, because it changes what the
ask should be. This is not a Zod problem.
The gap, measured
Reproduced on 9233c6b8, against
mongodb-js/mongodb-mcp-server#1417
("request elicitation for aggregations that include write stages") — a PR that adds a human
confirmation control to write-stage aggregations.
verify --preview on that diff:
diff completeness: complete files: 17
should_run : None evaluation_status: unclassified matched_rules: [] diff_tokens: []
The abstention itself is correct — #403's "skip requires positive evidence" is doing its job,
and this reports unclassified rather than claiming nothing changed. But no rule classified a
diff that adds a confirmation control to MCP write tools, and detect cannot see the project
at all, so there is nothing downstream to abstain into.
The only route the suggested-source probe found was filename-shaped, and it correctly rejected
what it found:
"excluded_sources": [{"type": "mcp", "path": "packages/mongodb-atlas-mcp-remote/package.json",
"reason": "MCP tools file must contain a tools array"}]What the repository already declares statically
Every tool is a class with two literal static fields:
export class AggregateTool extends MongoDBToolBase {
static toolName = "aggregate";
public description = "Run an aggregation against a MongoDB collection";
public argsShape = { ...ConnectionIdArgs, ...CollOperationArgs, ...AggregateArgs };
static operationType: OperationType = "read";
}Counted across the tool tree:
unique static toolName = "…" literals |
49 |
static operationType literals |
54 |
| distribution | read 19 · create 11 · metadata 9 · delete 6 · update 5 · connect 4 |
So the repository states, in static string literals, both the tool name and its operation
class — the two things the declaration questionnaire (#410) asks a human for. Thirty-five of
the fifty-four are already classified as something other than read.
Reading those two fields requires no Zod evaluation, no type inference, and no TypeScript
compilation. The Zod argsShape matters only for the SHIP-SCHEMA-* family, which is
secondary to the gate.
Surface discipline
Answering CONTRIBUTING.md § Surface discipline
directly, because this request is squarely in its scope.
1. Which headline metric does this move? Activation rate and time-to-first-verdict. For this
class of repository both are currently undefined rather than poor: detect says "Stop", so
there is no adoption path to be slow at. Every MCP server implemented in TypeScript — which is
the reference SDK's first-class language — is in this state.
2. Can an existing surface carry it? Partly, and this deserves an honest answer. An adopter
can export tools/list to JSON and point tool_sources[].type: mcp at it, which the existing
adapter reads at high confidence. That works and needs no new surface. Its limits: the export
is a build-and-run artifact this repository does not produce, it is not in the PR diff so it
cannot gate a PR without a CI step that builds and starts the server, and generating it is
exactly the runtime dependency the static default path exists to avoid. It is a good answer for
a team adopting deliberately; it is not an answer for detect finding the repository at all.
3. Does it respect the roadmap non-goals? The
non-goal is conditional, and worth quoting rather than paraphrasing:
New adapters (AutoGen, Semantic Kernel, LlamaIndex, additional language surfaces) are accepted
only when a real workflow needs one — they do not advance the core thesis and are not a
priority.
This is filed as evidence for that clause, not as an exception to it. The judgement of whether
"a real workflow needs one" is the maintainer's; what this issue supplies is the workflow: a
first-party vendor MCP server with 49 tools including database drops, which the tool cannot
currently see.
Options, smallest first
A. Trigger rule + documented export recipe. No new adapter. One content-shaped trigger rule
that recognises MCP tool registration in TypeScript, so the gate engages and routes the adopter
to produce an export, plus a documented CI recipe for generating it. Moves nothing on its own —
detect still says Stop — but stops the silent miss.
B. Name-and-operation extraction (recommended). A narrow static reader for the two literal
fields above: toolName and an operation-class field, at medium confidence, no Zod
evaluation. This produces a catalog of 49 tools that flows into the existing questionnaire,
where effects and authority are declared by a human anyway. It is the minimum that makes the
repository gateable, and it reuses the entire decision engine rather than adding a parallel one.
C. Full TypeScript/Zod adapter. Parse argsShape into tool schemas for the SHIP-SCHEMA-*
checks. Largest surface, hits the non-goal hardest, and buys the least per unit of work given
that effects and authority are declared regardless.
Recommendation: B, with A folded into it. C only if schema findings turn out to be what
adopters ask for, which no evidence currently suggests.
Worth noting that B generalises past this repository. toolName-style literal registration is
the common shape across TypeScript MCP servers; the specific field names differ, which is the
real design question — whether this is one reader with a configurable field mapping, or a
per-server pattern list, and which of those stays honest about what it actually proved.
Acceptance
-
detectonmongodb-js/mongodb-mcp-serverreturnsis_agent_project: truewith a named source. -
scanenumerates the 49 tools, withdrop-databaseanddelete-manypresent in the catalog. - Extraction confidence is measured, not assumed (
insufficient_evidenceis structural for every code-derived framework: two independent ceilings deny surface completeness by constant #393): a dynamically registered tool list does not report the same confidence as literal registration. - Adversarial: a tool whose name is built at runtime is reported as unenumerated rather than silently dropped, and the exclusion ledger (Epic: account for every narrowing decision — exclusion ledger, conservation invariant, and skip-requires-evidence #403) records it.
Found 2026-08-24 while walking mongodb-mcp-server#1417, chosen as a test case because it adds
a safeguard rather than a capability — a shape the gate should recognise as strengthening.
Activity
pengfei-threemoonslab commentedon Aug 27, 2026
This is scoped too narrowly — the gap is not TypeScript, it is any MCP server that does not emit an export. New evidence from walking grafana/mcp-grafana#1080, which is Go.
Same outcome as MongoDB, different language. Three vendor MCP servers now measured:
static toolName = "aggregate"mcpgrafana.MustTool("update_incident", …).snapJSONThe discriminator is not the language — two of these are Go. It is whether the repository happens to emit its tool list as a checked-in artifact, and only one of the three does.
The extraction remains trivial. One grep over the Go source produced the complete surface and the exact delta:
Which is the same finding as MongoDB's 49
static toolNameliterals: tool names are static string literals at a registration site. The spelling of the registration differs per server (static toolName =,MustTool("…"); the literal does not.Suggested changes to this issue:
Happy to retitle and rewrite the body if you want it restructured rather than amended by comment.
pengfei-threemoonslab commentedon Aug 29, 2026
Settling the scope and the open design question, and retitling per the 2026-08-27 comment.
Decision: Option B, generalized — a built-in registry of named idiom recognizers
The central question the Grafana evidence raised — one reader with a configurable pattern, or a per-server list — resolves to neither: a built-in, versioned registry of named registration idioms, shipped with the product.
detecttime there is no manifest, so patterns must be built-in for the activation case regardless. Post-adoption, a configurable regex would hand the definition of evidence to configuration that ships in the same PR as the tool it could hide — the P0: prove actual agent-to-tool bindings #268 class. If a configured source ever needs to select behavior, it selects a named idiom from the built-in registry (idiom: go_musttool), never supplies a pattern.insufficient_evidenceis structural for every code-derived framework: two independent ceilings deny surface completeness by constant #393), never per language: literal-at-registration-site =medium(the questionnaire supplies effects and authority anyway, per Option B); a checked-in export/snapshot stayshigh; a name built at runtime is unenumerated and lands in the exclusion ledger (Epic: account for every narrowing decision — exclusion ledger, conservation invariant, and skip-requires-evidence #403) — the adversarial acceptance bullet already in the body.Scope by measurement, not by ambition
Before implementing, a half-day survey: count registration idioms across the top ~30 registry servers, then ship the smallest set that covers the beachhead and the measured majority. Expected from the three walks:
server.tool(/server.registerTool(with a string-literal first argument;static toolName = "…"(+ an operation-class field where present, as in MongoDB);MustTool("…"/AddTool(shapes.Anything below the survey's coverage line waits for evidence, same rule as Option C.
Acceptance additions
mcp-grafanajoins MongoDB:detectreturnsis_agent_project: true, and the docs: STABILITY exit-code text says 20 is strict-only; advisory exits 20 on a fail_on match (and the base floor after #1074) #1080 delta enumerates 99 → 100, added:update_incident.github-mcp-serverregresses nothing: the snapshot route keepshighconfidence and remains preferred where both routes see the same server.Priority
Raising P2 → P1. Three first-party vendor MCP servers measured, two invisible; this issue is the front door for the whole class. Filed alongside it as one increment: #462 (the annotation-vs-evidence contradiction finding — the check that runs once these servers are visible) and #463 (surface-first rendering for unadopted repositories — the artifact a maintainer reads when it does). The three together are what turns "detect says Stop" into a report worth showing the people who maintain these servers.
[-]TypeScript MCP servers have no route: `detect` reports the official MongoDB MCP server (49 tools, incl. `drop-database`) as not an agent project[/-][+]MCP servers whose tool surface exists only as code have no route: `detect` reports the official MongoDB and Grafana servers as not agent projects[/+]pengfei-threemoonslab commentedon Aug 31, 2026
Milestone: v0.18.0 — the four-flags cycle. This route is the front door for cold evaluation of MCP servers; it feeds the incident fixtures (#471) and the coverage report (#473).
docs: link the two follow-ups the #431 survey and review produced
Read an MCP server's tool surface out of its own source (#431) (#483)
Fix four lexer defects the review found, in both readers (#485)
Read MCP registration sites in the zero-install detector (#485) (#526)
Register the ten distribution surfaces and test that they agree (#497) (
Address review: dead Poetry gate, class-scope lookup, duplicated rout…
Read the Python `@mcp.tool` decorator, in both detectors (#484) (#535)