Skill catalog (dynamic, via tool description) is invisible under tool search / deferred tools

Status update (2026-07-05): the core problem is fixed by #80 — delivering the catalog via MCP server instructions (--catalog=instructions, now the default) survives tool search: 3/3 previously-failing eval tasks pass with tool search ON. See the comments below for the verified evidence and remaining open questions (stdio instructions-freeze, caching/cost). This issue stays open to track those follow-ups.

Problem

Skilljack surfaces the model-facing skill catalog (<available_skills> manifest) inside the load-skill tool description — the dynamic channel, refreshed via tools/listChanged (getToolDescription in src/skill-tool.ts).

Claude Code / the Agent SDK now default to MCP Tool Search / deferred tool loading (launched 2026-01-14; CC ~2.1.x; built-in tools also deferred as of v2.1.69). Deferred MCP tools are listed by name but their descriptions are kept out of the model's context until ToolSearch fetches them on demand. Result: the model never sees skilljack's catalog, doesn't know a skill is relevant, and won't reliably auto-activate skilljack skills.

Evidence (controlled)

Skilljack served over HTTP (so its tools are present on the model's first turn, isolating this from the separate stdio connect race, #49753-style), model claude-sonnet-4-6, only ENABLE_TOOL_SEARCH toggled:

Constraint (superseded 2026-07-05)

Do not "fix" this by also placing the catalog into MCP server instructions. Server instructions (static) and the load-skill tool description (dynamic) are deliberately independent delivery strategies, and instructions is expected to become more optional in the next MCP spec release — so it is not a durable carrier for the dynamic catalog.

Reversed after controlled evals showed the instructions channel is the only one that survives tool search on default modern Claude Code. #80 made the catalog single-channel (--catalog=<instructions|tool-description>, never both) with instructions as the default. The staleness concern is real and tracked as a follow-up: instructions are frozen at startup on stdio (the SDK can't update them after construction); on HTTP they're rebuilt per request, so new connections always get a fresh catalog.

Directions to investigate

• Opt out of deferral for skilljack's tools / Make the catalog ToolSearch-discoverable / Alternative dynamic-catalog delivery — resolved by the instructions channel (Single-channel skill catalog with instructions default + HTTP discovery-on-change #80).
• Environment awareness — no longer needed for the default; only relevant for users who opt into --catalog=tool-description.

Remaining follow-ups (why this is still open)

1. stdio instructions-freeze: in instructions mode on stdio, skills added/removed at runtime aren't reflected in the catalog until restart. HTTP is mostly closed via per-request rebuild (Single-channel skill catalog with instructions default + HTTP discovery-on-change #80); already-connected HTTP clients still only see changes after reconnecting (MCP delivers instructions only at initialize).
2. Caching/cost: measure the prompt-caching impact of catalog-in-instructions (system-prompt cache prefix) vs catalog-in-tool-description (tools block) vs deferred (out of context). Results to be posted here.

Workaround (historical — no longer needed on the default)

Auto-activation now works out of the box with tool search enabled on the default --catalog=instructions (skilljack ≥ 0.12.0). ENABLE_TOOL_SEARCH=false is only needed if you opt into --catalog=tool-description, whose catalog is deferred out of context by tool search.

Related

• Fix: Single-channel skill catalog with instructions default + HTTP discovery-on-change #80 (single-channel catalog, instructions default, HTTP discovery-on-change); groundwork: Stateless HTTP transport + fix MCP-mode evals for agent-sdk 0.3.x tool search #79 (HTTP transport + eval fixes)
• stdio connect race (separate issue): Race between initialize control_request and MCP refresh causes tools to drop from first request anthropics/claude-code#49753
• Tool search: Built-in system tools now deferred behind ToolSearch (undocumented change) anthropics/claude-code#31002, #40314; Advanced tool use; Tool search — Agent SDK docs