feat!: port to MCP Python SDK v2 (2026-07-28 protocol revision) - #362

The 2026-07-28 revision lets a result tell the client how long it may be
reused. Without hints every field defaults to zero, so a client re-asks for
the tool list on every turn and re-reads a resource it already has — the
freshness the fields exist to buy is left on the table.

The list endpoints and server/discover describe registration tables built once
at startup, which nothing short of a restart can invalidate, so they get an
hour. That is also what makes them worth caching at all: a stable tool block
is what earns a client's prompt-cache hits.

resources/read is bounded by the watcher's poll interval instead. Most of what
it serves is immutable — a ZIM archive is a sealed file, so an entry body for a
given (archive, path) can never change — and would justify hours. But the hint
is per method, not per URI, and the same method also serves zim://files, a live
directory scan; the floor belongs to the mutable member. Deriving it from the
poll interval means a cached read is never staler than the server's own
detection latency. Per-URI hints would need the handler to construct the
result, which the resource layer does not currently do.

Scope is private throughout: these payloads embed server-local absolute paths
and configuration, so a shared intermediary must not serve one tenant's
response to another.

Verified on the wire rather than at the constructor, which is how the era
gating surfaced: the SDK stamps these fields only for clients that opened the
2026 way, and a legacy session correctly sees none of them. Tests pin both,
since a hint that silently fails to apply looks identical to one that works —
the server keeps answering correctly, it just never gets to skip a request.

Refs #360 (Phase 4).

The cache hints shipped so far are keyed by method, so one TTL had to cover
every resources/read. That method serves two very different things: a ZIM
archive, which is a sealed file whose content cannot change without the file
being replaced, and zim://files, a live scan of the allowed directories. The
honest method-wide value is the short one, so archive reads were capped at the
watcher's poll interval — five seconds by default — and the caching that
mattered most never happened. An overview read costs three blocking archive
opens; a client re-paid that every five seconds.

The SDK fills only fields a handler left unset, so a handler that stamps ttl_ms
itself wins over the server-wide hint for exactly the URIs it chooses.
_handle_read_resource does that for archive-backed URIs and leaves zim://files
alone, which lands it on the method-wide floor as before.

How long is a policy question, not a technical one, so it is configurable via
OPENZIM_MCP_RESOURCE_CACHE_TTL_SECONDS (default one hour). The trade-off is
staleness after an archive is replaced in place. A client on an open
subscriptions/listen stream is told immediately and re-reads, so the TTL costs
it nothing — the long TTL and the watcher are designed to work together. A
client that is not subscribed, which is anything on stdio, can serve the old
content until the TTL expires. Operators who hot-swap archives and cannot rely
on subscribers lower it, or set 0 to turn the override off entirely.

Zero disables rather than forbidding caching. The watcher's own detection
latency is already the tightest freshness the server can honestly promise, so
falling back to it is what "off" should mean; being stricter than the floor
would promise something the server cannot keep.

Era gating is pinned on this path specifically. The method-wide hints are
applied by the SDK, which decides whether the client's protocol revision has
any meaning for ttlMs; a handler-set value bypasses that decision point, so a
legacy session reading an archive URI is now asserted to still see nothing.

Also documents the cache hints for the first time — the per-method work landed
without any user-facing description of what clients may cache or for how long.

Refs #360 (Phase 4).

The TTL policy is stated as an exception — everything except zim://files is
backed by a sealed archive and may be cached for an hour. That is true of the
three resources registered today and silently wrong for any future one that is
computed rather than read out of a ZIM: it would inherit the long TTL without
anyone deciding it should, and nothing would fail. Pinning the registration set
turns that into a build failure, which is the prompt to classify the new
resource. Verified by registering a fourth resource and watching the assertion
trip.

Also records why the override sits on a private SDK method when the port spent
its effort moving off private reach-ins: a per-result hint is set by returning
a result with an explicit ttl_ms, and on this layer the only code that returns
a ReadResourceResult is that handler, which MCPServer binds as on_read_resource
at construction. The per-URI tests are the canary if a future SDK stops routing
through it.

Refs #360 (Phase 4).

The previous pages documented around two defects the port review just
closed: they told readers that stdio/SSE (and subscriptions_enabled=
false) servers acknowledge `subscriptions/listen` and advertise
subscribe/listChanged anyway, and that per-entry reads carry the
hour-long TTL. Both are now the honest versions — the capability and
handler are withheld together, and only the `zim://{name}` overview
(whose replacement actually publishes `resources/updated`) takes the
long TTL.

A follow-up pass over the two previous commits found the fixes sound
but their edges thin:

- The SSE transport also withholds the subscription capability (the
  gate is http-only, not merely non-stdio), and the docs promise it —
  but no test pinned it, so a gate rewritten as `transport != "stdio"`
  would have resurrected the false advertisement on SSE with the whole
  suite green. Pinned, and mutation-checked against exactly that edit.

- The `*_error` partial-failure overview shape was excluded from the
  long TTL but only the total-failure `"error"` shape was tested;
  deleting the clause left 3702 tests green. Pinned with a bogus-file
  overview that produces the shape naturally, mutation-checked.

- Four pages and one docstring still described the pre-fix behavior:
  troubleshooting and architecture-overview said stdio/SSE/disabled
  servers ack listen and stay silent (now method-not-found — an ack now
  *proves* the gate is open), the subscriptions page told clients to
  subscribe with the raw stem (now the template expansion, which is
  what delivery matches), the api-reference TTL row still covered
  "archive URIs" generally, and _cache_hints still claimed every
  archive-backed URI takes the long stamp.

The test still drove the pre-consolidation tool names — every call
answered "Unknown tool: get_server_health", so the health parse blew up
before any assertion ran, and the cache-populating calls were silently
populating nothing. It now speaks zim_health/zim_browse/zim_search/
zim_metadata and reads the legacy health payload from the combined
response's `health` key, and passes end-to-end: run #1 populates, the
graceful shutdown persists, run #2 reloads.

A second verified sweep over ground the port review didn't touch (the
Phase F wrappers, zim/ core, search pipeline, cache/security/limiter,
HTTP entry) confirmed ten distinct bugs:

- Title-mode promotion was discarded exactly when it mattered most: a
  filler-prose query whose full-phrase lookup returns 0 hits (the class
  promotion exists for) got `results: []` back while the resolved
  canonical was thrown away — yet one irrelevant raw hit DID get the
  canonical hoisted. An empty raw page now serves the promoted row.
- The promotion probe train (dozens of blocking libzim opens) ran
  inline on the event loop; now offloaded like every sibling call.
- Inbound-links cursors never verified archive identity, so a cursor
  from archive A silently paginated archive B at A's offset. The same
  guard every other paginated surface applies is now applied.
- zim_search validated `limit` against 1000 while cross_file caps at 50
  and filtered fulltext at 100 in the data layer, turning the promised
  invalid_limit envelope into a generic error naming an internal
  parameter. The wrapper now mirrors the per-dispatch caps.
- An oversize zim_get batch paid the full clamped batch price (the
  whole bucket) before the data layer rejected it having read nothing;
  now rejected up front at the flat price with a structured envelope.
- W-namespace browse resumed by surviving rows: a row dropped during
  materialisation re-served the survivor (duplicates) or re-issued the
  same cursor forever. The payload now reports candidate rows scanned.
- The M/<key> text presenter dropped all content_offset accounting: a
  tail slice read as the complete value and a past-end offset as an
  empty one. It now mirrors the regular-entry renderer.
- Telemetry's first-ever increment of a key raced across to_thread
  workers (Counter.__missing__ is a preemption point); now locked.
- A dotted persistence_path was rewritten by with_suffix, colliding
  e.g. openzim.prod and openzim.dev onto one snapshot file that
  last-writer-wins and peer-exit unlinks; the extension is appended.
- A non-ASCII auth token could never match for UTF-8-encoding clients
  (Starlette decodes headers as latin-1); rejected at config load like
  the blank-token footgun.

Also ports the live cache-persistence test off the pre-Phase-F tool
names it was still speaking. The zim_search description trim keeps the
advanced surface at 25,498 of the 25,500-byte schema budget.

…on surfaces

A third review pass over the port, aimed at the paths tests don't reach,
found six defects and three pieces of drift:

- A per-entry resource path that `sanitize_input` rejects reached 2026-era
  clients as an opaque "Internal server error". `ResourceManager.get_resource`
  calls `create_resource` outside the `try` that wraps `resource.read()`, so
  nothing converted the `OpenZimMcpValidationError` and the modern HTTP entry
  replaced it wholesale — discarding the only actionable part of the message
  and making a malformed URI look like a retryable server fault. Now
  INVALID_PARAMS with the message intact, converted at the only layer that
  can.

- Replacement notifications never reached clients that subscribed with the
  spelling they had successfully read from. `UriTemplate.match` percent-decodes
  before matching, so `zim://my archive` is a working `resources/read`, but the
  watcher published only the encoded form and SDK delivery is exact-string.
  Such a client sat on a URI nothing ever fires on — a stale overview for the
  life of the stream, extended by the hour-long resource TTL. Both spellings
  are published when they differ; unreserved stems still publish once.

- `publish_change` treated "not list_changed" as "replaced", so a future third
  change kind would go out as a `resources/updated` for whatever URI came with
  it — the list-membership/content conflation the 2026-07-28 split exists to
  end, reintroduced silently. Both kinds are now matched explicitly.

- The per-stream URI-set bound went missing with the 1.x registry, and the
  module docstring claimed the SDK's `ListenHandler` was the equivalent
  admission control. It is not: it caps stream count and event backlog and
  never inspects the URI set, which it holds for the stream's lifetime — a
  ceiling of 1024 streams x 4 MiB on a method the rate limiter does not cover.
  Restored as `install_bounded_listen_handler`, and the docstring now says what
  is and isn't bounded.

- Title-mode promotion carried the zero-result alt-spelling suggestions onto
  the confident hit it produced, against `_assemble_find_response`'s stated
  rule that a non-fuzzy hit carries none.

- An empty `M/<key>` metadata value rendered as a bare `## Content` heading:
  the presenter was a copy of `_render_entry_payload_text` that had dropped its
  `(No content)` fallback (and its content-type casing). Both now share one
  builder, so the copy cannot drift again.

- `serve_streamable_http` re-registered /healthz and /readyz on every call,
  stacking unreachable duplicate routes; the TTL decision re-parsed every
  overview body to answer one boolean; `subscriptions_enabled` still documented
  the removed subscribe/unsubscribe handlers; three website pages still named
  `EnvelopeAwareFastMCP`; and `TIMEOUTS.SUBSCRIPTION_SEND_SECONDS` documented a
  fan-out that no longer exists.

Every fix is pinned by a test that was mutation-checked against reverting it.
3733 pass.

Not fixed, deliberately: `completion/complete` recursively globs every allowed
directory per keystroke and is outside the rate limiter. Memoizing it trades
away the freshness `test_completion_reflects_archives_added_after_startup`
deliberately pins, which is a product call rather than a bug fix.

…yers

A fourth verified sweep, aimed at the subsystems the first three passes
never opened — the search pipeline core, query understanding, synthesis,
content rendering, and the archive/state layer — confirmed twenty-two
defects. One reported finding was refuted and is not addressed here.

Query understanding, the layer every `zim_query` call goes through:

- A 4 KB query cost 2.1 s of CPU and then returned a regex-timeout error
  instead of results. `_TRAILING_POLITENESS_RE` led with `\s+`, and a
  greedy repeat whose tail is a failing alternation backtracks O(n^2)
  over a whitespace run — a cost `safe_regex_sub` cannot bound, since
  CPython's `re` holds the GIL for the whole match. The anchor is now the
  fixed-width `\s` that `_PARAM_LEAK_RE` already uses; behaviour is
  unchanged because the match simply starts at the last space.

- The same politeness pass ate the entire topic when the topic *was* the
  polite word: `tell me about cheers` (the TV series), `tack`, `ta`, `tx`
  all returned "Topic Required" without touching the archive. The strip
  now refuses a pass that would consume the query's last content token.

- `get entries A/Mercury_(planet)` shipped `A/Mercury_` to the data layer.
  The batch extractor's character class dropped `(`, `)` and `@`, so it
  truncated exactly the disambiguated titles the tool itself emits. It now
  carries the single-entry sibling's tolerance, with paren-aware trimming
  so a parenthesised aside is still peeled.

- `search across all files for X` and `search all archives for X`
  classified as `search_all` but failed the extractor's narrower prefix,
  so the cross-archive search ran on the whole command sentence. The
  extractor now accepts the shape its own intent pattern admits.

- `query_rewrite.misspelling_map_path` / `misspelling_exclusion_path` were
  validated and stored and then read by nothing: an operator's curated map
  was silently ignored. Bound at handler construction, and the paths are
  now checked for readability at startup rather than failing mid-request.

Search and ranking:

- Zero-hit alt-spelling suggestions were derived from the path basename,
  which is only title-shaped on mwoffliner archives. On zimit/warc2zim
  archives the recovery hint was the empty string (paths ending in `/`)
  or an opaque `a682878.html` — and the empty value poisoned the dedup
  set, collapsing the whole suggestion pool to one blank entry. The
  entry's own title is authoritative and the archive is still open.

- The canonical-splice namespace gate parsed `iep.utm.edu/plato/` as
  namespace `iep.utm.edu`, so on every new-scheme archive the splice was
  unreachable. The gate now asks the archive for its scheme, matching the
  verdict `_matches_cheap_namespace` gives the same path. Unblocking it
  exposed a second defect it had been hiding: on archives that suffix
  every title with the site name, the one entry whose title matches a
  query *exactly* is whichever page carries a stripped title, so the
  splice promoted `iep.utm.edu/meno-2/` (titled "Plato") above the Plato
  article. The prepend now yields when a hit already carries the queried
  title, and the relevance order breaks the tie.

- The canonical-prefix suggestion probe excluded already-present rows
  from its scan instead of from its result, so it returned the *second*
  shortest title — promoting a non-canonical entry above the canonical
  and evicting a genuine suggestion from the page.

- Tail-hijack promotion guards tokenized the raw path, so on old-scheme
  archives the namespace letter counted as a token and every
  `len(cand_tokens) == 1` rule was unreachable: `antarctica coal aerosol`
  promoted `A/Aerosol`, the exact wrong-answer class those guards exist
  to block. Both schemes now produce the same promotion.

- `_is_list_article` matched anchored rules against the namespace-prefixed
  path, so catalog demotion silently no-opped on old-scheme archives. Its
  suffix table also treated a bare `_awards`/`_honors` tail as catalog
  shape, demoting `Academy_Awards` out of the answer lead; award catalogs
  are matched in their multi-word form now, and an explicit title-index
  promotion is exempt from the heuristic.

Rendering and bundling:

- A heading containing a run of backslashes hung the bundle build: the
  relaxed matcher emitted a per-character optional escape, letting the
  engine split n backslashes 2**n ways (16 s at n=26) before conceding.

- The compact search-snippet cap truncated inside `**bold**`, and the
  unpaired marker turned the rest of the response bold. It now runs the
  same repairs `create_snippet` applies at its own cut sites.

- `_heading_visible_text` folded HTML comment text into the heading, so a
  heading with a `<!-- -->` never matched the rendered markdown and the
  whole section vanished from the bundle.

- The `include_subsections=False` empty-body test added the heading line's
  length to its threshold, so a long-titled section swallowed a genuine
  lead paragraph. The measured slice starts at the body offset.

Envelopes, state, and the tool surface:

- Any response or query containing `<|endoftext|>` hard-failed the tool:
  tiktoken rejects special-token literals by default. A best-effort budget
  estimate must never fail the call.

- Combined `zim_health` / `zim_metadata` responses shipped `_meta: {}`,
  so a client budgeting context from `tokens_est` got nothing.

- `zim_health` leaked the absolute cache-persistence path over HTTP/SSE
  while redacting every sibling field — the docs already promised it was
  redacted.

- "File does not exist" reached the entry-level "Resource Not Found"
  template, whose recovery steps hunt inside an archive that was never
  opened; the caller was never told to list the real archives.

- Replaying a `next_cursor` without repeating `limit` silently changed
  page size: the cursor records `l` but no consumer read it, so zim_links
  jumped 2 -> 100 and zim_browse 5 -> 50 and 3 -> 200.

- A decodable cursor carrying no resume offset was served as a fresh
  page 1 — the exact failure simple mode returns a structured error for.

- `openzim-mcp build --help` exited 2, because argparse's SystemExit(0)
  was coerced by `or 2`.

…nd paging layers

Six defects in paths no test reaches, all found by aiming at the seams the
previous four passes did not cover. Every fix is mutation-checked: each was
reverted individually and its test confirmed to fail.

**Two ways a real heading became unlocatable, both silently losing a whole
section.** ``_compute_section_offsets`` logs one WARNING and ``continue``s
when no matcher finds a heading, so the section vanishes from the bundle and
``zim_get_section`` answers ``section_not_found`` for an id no listing ever
offered. First, ``_strip_md_inline_decorations`` dropped ``*`` and backticks
but never ``_`` — exactly what html2text emits for ``<i>``/``<em>``. Wikipedia
italicises the work title in the H1 of every article about a film, book,
album, or newspaper, so ``<h1><i>2040</i> (film)</h1>`` rendered as
``#  _2040_ (film)`` and the last-resort matcher compared that against
``2040 (film)``. On the shipped corpus ``A/2040_(film)`` reported
``heading_count: 0`` with an empty TOC. The strip is now emphasis-aware and
applied to *both* sides of the comparison, so a heading whose text really
carries an underscore still matches; an interior ``snake_case`` underscore is
preserved. Second, a ``<br>`` inside a heading becomes a real newline in the
rendered markdown and pushes the tail onto the following line, while
``_heading_visible_text`` concatenates the whole subtree — and every matcher
is line-anchored, so none could bridge it. ``_build_headings`` now records the
pre-break spelling under an opt-in ``line_text`` (opt-in because that dict is
also the wire payload of ``zim_get(view="structure")``, which must not
change), and the locator retries with it. A wikibooks article went from 1
heading to 10.

**A subscription on the ``.zim`` spelling never fired.** ``_resolve_zim_name``
deliberately accepts both the bare stem and the full filename, and the
``zim://files`` listing advertises ``name`` *with* the extension — so a client
that builds its URI from that listing reads ``zim://wikipedia.zim``
successfully and is handed the hour-long ``archive_read_ttl_ms``, then
subscribes with the same string. The watcher only published the stem, and SDK
delivery is exact-string, so that stream sat on a URI nothing ever fired on:
a stale overview for its whole life. This is the percent-encoding gap fixed in
bb5e559, along the extension axis instead; both axes are now published and
deduplicated, so an unreserved name still emits each URI exactly once.

**One open listen stream made the process unkillable.** ``uvicorn.Config``
left ``timeout_graceful_shutdown`` at ``None`` — wait forever — and a
``subscriptions/listen`` SSE loop only ends when the *client* disconnects. On
SIGTERM uvicorn blocked in "Waiting for connections to close" and never
reached the ASGI lifespan, which is where the MtimeWatcher is stopped.
Verified against a real server: still running 30s after SIGTERM before, exits
in 5.2s after. ``docker stop`` therefore burned its full grace period and
SIGKILLed, and SIGKILL skips ``atexit`` — discarding the cache persistence
save on every restart.

**The canonical-title splice made one search hit unreachable.** It prepended
a synthetic row then trimmed back to ``limit`` (a cosmetic fix for the
"showing 1-4 with limit=3" header), dropping the last BM25 hit — but told
neither resume mechanism, so the renderer still advertised ``offset + limit``
and ``next_cursor`` still encoded the pre-splice count. The displaced row fell
between the end of page 1 and the start of page 2. On the corpus, ``search for
biomass fuel`` with ``limit=3`` lost ``A/Aviation_biofuel`` entirely. The
splice no longer trims — which is what the filtered sibling
``search_with_filters_with_canonical_splice`` already does for the same query
— and records ``source_consumed`` so the cursor-less render path advances by
ranked rows rather than by rendered ones.

**Promotion reported a budget for a payload it had already replaced.**
``_merge_promotion_into_title_results`` rewrote ``results`` and carried the
old ``_meta`` forward verbatim, so ``chars``/``tokens_est`` described bytes
that never shipped — 47% of actual on the empty-page branch this PR added,
where a zero-row page with suggestions becomes a one-row hit. The envelope is
re-measured after the merge, carrying the source annotations through and
keeping bb5e559's rule that a promoted canonical sheds its zero-result hints.

Verification: 3943 passed / 0 failed / 213 skipped against the real corpus (22
new tests, from 3921). Live suite 25 passed / 0 failed, unchanged. black /
isort / flake8 / mypy / bandit clean, and the pre-commit hook set passes on
every changed file.

…pass exposed

Three defects, one of them introduced by the previous commit. Every fix is
mutation-checked: each was reverted individually and its test confirmed to fail.

**The reranker silently undid the pass-5 splice fix — and inverted it.** 1e1bd8e
made `_splice_title_match_into_search` prepend the canonical row as an EXTRA
rather than a replacement, so a spliced page carries `limit + 1` rows. Sixteen
lines later `_handle_search` hands that page to `_maybe_rerank_compact`, which
set `effective_top_k = limit` and cut it straight back down — re-applying the
very trim the fix removed, and violating the invariant its own comment states
("the caller's page size wins; rerank only REORDERS it"). That invariant held
only while `len(candidates) == limit`, which the splice broke.

The row that died was near-deterministically the canonical one. `BGEReranker`
scores `snippet or path` as the passage, and the synthetic row's snippet is the
sentinel `(canonical title match)` — a placeholder, not content. Measured on the
shipped corpus with the real BAAI/bge-reranker-base, it scored -5.5 against +4.6
for a genuine hit. So with the [reranker] extra installed, `search for effects of
climate change on humans` at limit=3 returned three rows and no canonical at all,
while the control run with the reranker disabled put `A/Effects_of_climate_change`
at rank 1. The splice's entire purpose was reversed by a placeholder string.

Fixed on both axes: the canonical row is held out of the comparison and pinned at
the head where the splice put it, and the cut is sized from the list actually
handed over rather than from the caller's page size — so the next thing that
grows a page cannot reintroduce the same defect. The sentinel is now a shared
constant rather than a literal repeated across three modules, since that
duplication is precisely what let the reranker diverge from the renderers that
already special-case it.

**Simple mode ignored the page size its own cursor advertised.** Every cursor
encodes `s.l`, and the advanced arm honours it through `effective_limit` — but
`decode_offset_cursor` never projected it, so `handle_zim_query` fell back to
each intent handler's hardcoded default. Replaying the `next_cursor` the footer
literally instructs the caller to pass resized a 3-row `walk namespace` page to
200 rows. Same precedence as the advanced arm now: explicit limit > cursor's `l`
> handler default, with the same positive-int validation so a hand-built cursor
degrades instead of erroring. This is the sibling-divergence shape again — one
arm got the fix, the other was missed.

**Percent-encoded link targets were never decoded.** ZIM stores entry paths as
raw UTF-8 (`A/El_Niño`) while the `<a href>` in archived HTML is RFC 3986-encoded
(`El_Ni%C3%B1o`), and `_resolve_link_to_entry_path` normalises without decoding.
So `zim_links(direction="related")` returned `{"path": "A/El_Ni%C3%B1o", "title":
"A/El_Ni%C3%B1o"}` — the title lookup failed too, leaving the raw path as its own
placeholder — and feeding that path to `zim_get`, the obvious next call, answered
"Entry not found". The inbound link graph indexed the same unfetchable keys.
Both consumers already open the archive, so they now pick whichever spelling it
can serve, raw first: some archives genuinely store a literal `%` in a path
(warc2zim assets like `I/Al_Gore%2C_2007.webp`), and decoding unconditionally
would break the paths that already work.

Verification: 3961 passed / 0 failed / 213 skipped against the real corpus (18
new tests, from 3943). Live suite 25 passed / 0 failed, unchanged. The reranker
fix is confirmed against the real cross-encoder end to end, not only the stub.
black / isort / flake8 / mypy / bandit clean.

…lly pay

The advanced surface measured 25,499 against a 25,500 cap — one byte of
headroom, so the next word added to any tool description would have turned
CI red on a release that adds descriptions.

The surface was not the problem. `_measure_tools` serialized with
`json.dumps`, which defaults to `", "` and `": "` separators, while the wire
is compact: pydantic emits no padding. Counting a space after every comma
and colon across eight tools inflated the figure by ~620 bytes. Measured
against a running server, the whole `tools/list` frame — JSON-RPC envelope
and all — is 24,881 bytes, comfortably inside the 25KiB MCP Tax pain band
the cap exists to defend.

This is the same failure the function's own docstring already warns about
from the other direction. Omitting `outputSchema` once let it measure a
number smaller than clients pay; counting padding made it measure a number
larger. Either way the cap is not measuring the thing it names.

Separators are pinned, `TOTAL_CAP` becomes the pain band itself (25KiB), and
a test asserts the padded serializer is not in use, so this cannot drift
back silently. Per-tool allocations are untouched: every ceiling simply
gained the padding back as real headroom, leaving 618 bytes on the total.

Suite 3972 passed, 0 failed.

…r X"

The `search_all` cue-strip treated `zim` as a head noun only, so `zims?`
claimed the qualifier in `search all zim files for photosynthesis` and the
alternation stopped there. What survived as the cross-archive search terms
was `files for photosynthesis` — the head noun and the preposition both
polluting BM25, on what is probably the most natural way a user has of
phrasing this. `zim archives`, `zim file` and `.zim files` failed the same
way; `search all files for X` and `search all zims for X` were fine, which
is why the variant list never caught it.

`zim` is now matched as a qualifier ahead of the head noun as well as a head
noun in its own right, in the extractor and in the `INTENT_PATTERN` that
gates it. Both, because the extractor's prefix has to stay at least as wide
as the gate — a prefix that fails to match returns the whole command
sentence as the search terms — and because the gate rejected `.zim` outright,
which would have left that half of the extractor unreachable.

Five phrasings added to `_SEARCH_ALL_VARIANTS`, each confirmed to fail
first, against the class that already pins exactly this property. Suite
3982 passed, 0 failed.

…ages over

`truncate_content` emitted `content[:max_length].strip()` but advertised
`content_offset = current_offset + max_length` — the *unstripped* page size.
Any whitespace sitting on a page boundary was therefore stripped out of the
page and then skipped over by the next offset, so it left the document
entirely. A caller doing exactly what the footer tells it to do — "Pass
`content_offset=N` to read the next page" — and concatenating the result got
the words on either side of every boundary fused: `word00001word00002`, one
character lost per page, silently. Paragraph breaks went the same way, so
paged article text lost its structure as well as its spacing.

The page still ends stripped; the difference is that the stripped whitespace
is now handed to the next page instead of deleted, because `next_offset`
advances by what was actually emitted. Leading whitespace is still trimmed
at offset 0, where there is no preceding page for it to belong to — on any
later page it is precisely the boundary whitespace its predecessor deferred.

Verified by paging a body end to end at several page sizes and asserting the
concatenation equals the original, including one case whose boundaries land
on paragraph breaks. All four assertions fail against the unfixed code.
Suite 3987 passed, 0 failed.

…e it

html2text puts a space after emphasis it closes even when the source had
none, so `<h2><b>Foo</b>bar</h2>` renders as `## **Foo** bar` while
`_heading_visible_text` reads `Foobar` off the soup. Stripping decorations
left `'Foo bar'` against `'Foobar'`, the last-resort matcher missed, and
`_compute_section_offsets` logged one warning and dropped the section — so
`view="toc"` never listed it and `zim_get_section` answered
`section_not_found` for a heading the article plainly has. Worse, the
preceding section's slice then ran on through the missing heading and
swallowed its body.

This is the third instance of the family the fifth sweep documented, and it
survived that pass because the natural fixture does not reproduce it: code
spans gain no space, so `<code>` abutting text matches either way. Only
`<b>`/`<i>`/`<em>`/`<strong>` do.

The cascade gains one final comparison with the spaces removed. It runs only
where the section would otherwise be lost, and both sides must still agree
on every non-space character, so it cannot promote a match the exact pass
rejected on content — asserted by a test that a genuinely different heading
still fails to match.

6 new tests, 4 confirmed to fail against the unfixed code. Suite 3993
passed, 0 failed.

… failure modes

Two problems the reappearance handling introduced, both found by reviewing
that change rather than the code it fixed.

`_scan` swallows an `OSError` per directory and returns a snapshot missing
every archive underneath it, which reads exactly like a mass deletion.
Recording those as removals meant the next healthy pass saw all of them
"reappear" and published a replacement for each — one transient stat failure
on a network mount turning into a notification storm and a re-read of every
archive on the server. `_scan` now reports whether it walked everything, and
the replacement inference is withheld when it did not. The listing
notification keeps its long-standing behaviour; only the new inference is
gated.

The removed-path memory was also consumed before the publishes rather than
after. `_snapshot` is deliberately assigned last so that a pass dying
mid-publish is retried wholesale on the next interval — but the memory did
not keep that discipline, so the retry no longer recognised the reappearance
and the `resources/updated` the client was waiting for was lost for good.
Both are now updated together, after every publish has succeeded.

2 new tests, both confirmed to fail against the previous commit: the first
saw six fabricated replacements from a single failed scan, the second saw
the notification vanish. Suite 3995 passed, 0 failed.

`SHUTDOWN_GRACE_SECONDS` was added so termination would be "clean rather
than killed", the stated benefit being that the cache-persistence save
registered in `OpenZimMcpCache.__init__` survives. It does not. Python's
default SIGTERM disposition terminates without unwinding, so `atexit` never
runs, and measuring it shows the gap plainly: closing stdin persists 5.3KB
and exits 0, while SIGTERM writes nothing and exits -15. Every `docker
stop`, pod eviction and systemd restart threw the cache away and forced the
next start to be cold, on both transports.

The HTTP transport looked safe and is not. uvicorn does run its own graceful
ASGI shutdown, but then restores the previous signal handler and re-raises
the signal to report the right exit status — and with the default
disposition restored, that re-raise killed the process just the same.

`main` now installs a SIGTERM handler that raises `SystemExit`, which
unwinds the server for both transports, uvicorn included: its re-raise lands
on this handler instead, after its own shutdown has completed.

Unwinding alone is not enough to reach `atexit`, though, and relying on it
would have replaced a fast wrong stop with a hang. The stdio transport parks
a worker thread on a blocking read of stdin, and nothing closes stdin when
the stop came from a signal, so interpreter finalization waits on that
thread until the supervisor's SIGKILL — measured at over 8 seconds before
being killed. So the flush is explicit via `cache.shutdown()` (which also
deregisters the now-redundant atexit handlers) and the process leaves
through `os._exit`, by which point `server.run()` has returned and the ASGI
lifespan that stops the watcher has completed with it.

Both transports now exit 143 promptly, and stdio persists 5.3KB where it
previously persisted nothing. 5 unit tests plus a live test that drives a
real subprocess through populate → SIGTERM → assert file. Suite 4000 passed,
0 failed; bandit -ll, mypy, flake8, black, isort clean.

Reading `zim://{name}/entry/{path}` for an entry the archive doesn't have
answered `-32603` with "Error reading resource <uri>" — an internal-error
code, which tells a client the server broke and the request is worth
retrying, for a request that can never succeed, and without naming the entry.

libzim signals a missing entry with a bare `KeyError('Cannot find entry')`.
That is neither `MCPError` nor `OpenZimMcpArchiveError`, so it escaped both
converters at this boundary and the SDK replaced it wholesale. It now
converts to `-32602` naming the entry and the archive, matching the sibling
unknown-archive case one level up, which has answered `-32602` with a
descriptive message since the third pass.

Verified on the wire against a real archive: a missing entry now reads
`Entry not found: 'A/...' is not in '<archive>'.` where it previously read
the generic internal error, and a valid entry still returns its 12,637-char
body. The test is confirmed to fail against the unfixed module. Suite 4013
passed, 0 failed.

3.0.0 bumps the link-graph `SCHEMA_VERSION` from 2 to 3, so every sidecar
built by 2.x is rejected on load and every operator using
`zim_links(direction="inbound")` meets this error path on upgrade. It told
them to run `openzim-mcp build link-graph <archive>`, which exits 1 with
"sidecar already exists; pass --force to overwrite" whenever a sidecar file
is present next to the archive — which, in the situation that produces this
message, it always is. The one instruction shipped for the release's only
breaking on-disk change could not be followed.

The message now names `--force` and says why it is needed. The invalidation
is also documented in the API reference, where the sidecar was described as
going stale only when the archive's UUID changes — a schema bump was not
listed as a staleness cause at all, so nothing told a reader upgrading to
3.0.0 that their sidecars were about to stop working.

Also corrects the deployment guide, which claimed
`OPENZIM_MCP_SUBSCRIPTIONS_ENABLED=false` leaves `subscriptions/listen`
succeeding but silent. It does the opposite: the handler is dropped, so the
call fails with method-not-found and the capability reads `false`. Verified
against a constructed server; three other pages, including a later section
of that same page, already described it correctly.

Suite 4014 passed, 0 failed.

`test_removed_path_memory_is_bounded` asserted that the newest removal
survives a full memory, but a single tick diffs a *set* of removed paths
whose iteration order is arbitrary — so which entry survives eviction is not
a property of one pass at all. It passed locally and failed on CI's
different hash seed. The bound assertion stays; the recency guarantee moves
to its own test that spans two ticks, which is where it actually holds.
Verified across four `PYTHONHASHSEED` values.

`_uri_spellings` now publishes only spellings that can reach the wire.
Percent-encoding the surrogates fixed the raise inside the watcher, but the
loop also published the unencoded spelling, and a URI carrying a lone
surrogate cannot be JSON-encoded (`json.dumps(..., ensure_ascii=False)`
raises on it). That would have moved the same crash off the poller and onto
the subscriber's notification stream — a narrower blast radius than the
original bug, but the same bug.

Three CodeQL alerts raised by this pass's test code, one of them an error:
`returncode` was only assigned on the success path of the SIGTERM live test,
and the two discovery helpers in the phase-C module mixed explicit returns
with a terminal `pytest.skip` that CodeQL reads as falling through.

Suite 4015 passed, 0 failed; live suite 38 passed, 0 failed.

514f83e corrected the human-readable `content_offset=N` hint in the
truncation note but not `_meta.more_at_offset`, which is the field a
programmatic client actually pages by. That is computed from
`_content_chars` in two payload builders in `zim/content.py`, each
independently repeating `max_content_length if was_truncated else
len(content)` — so the structured path went on skipping the whitespace at
every page boundary and gluing the words together, and the two fields now
disagreed with each other about where the next page starts.

Both derive from one `paged_slice_length` now, alongside the body hint, so a
third copy cannot drift the way the second one did. That the arithmetic was
written out three times is why fixing one site looked complete.

The earlier tests only exercised `truncate_content` directly and passed with
`zim/content.py` reverted, so they were not pinning this at all. The new one
drives a real payload builder and fails against the unfixed arithmetic
(20 vs 19 on a boundary that lands mid-gap).

Suite 4019 passed, 0 failed; mypy, flake8, black, isort clean.

… changed

The reappearance inference trusted absence alone, and the completeness guard
added with it only covers a scan that *raises*. The ordinary way a watched
directory loses its listing raises nothing: an NFS or CIFS mount dropping, a
Docker volume being remounted, a CSI volume re-attaching. The directory stays
readable and simply lists empty. Every archive was then recorded as removed
and, when the mount came back, announced as replaced — measured here as ten
`resources/updated` notifications for five untouched files. Every subscribed
client would evict a correct cached overview and re-read it, three blocking
archive opens apiece, because a mount flapped.

The memory now carries the `(mtime, size)` each path had when it vanished,
and a reappearance publishes only when that differs from what came back.
This settles the flap without having to detect one: a mount that returns
intact brings every file back byte-identical and says nothing, while the
straddling `rm` + `cp` this inference exists for changes mtime, size or both
and still publishes. It also closes the case where a path is deleted and much
later re-created — the stat is compared, not merely the name.

`_last_scan_complete` stays as the guard for a scan that genuinely failed
part way; it is now the second line rather than the only one.

2 new tests, the first confirmed to produce ten false replacements against
the previous commit. Suite 4021 passed, 0 failed, stable across three
`PYTHONHASHSEED` values.

Cursors are unsigned base64 JSON, so `s.l` is entirely client-controlled.
Every tool checks an explicit `limit` against its cap, but that check only
runs when the caller passes one — a request carrying nothing but a cursor
skipped it, and both resolvers accepted any positive int. Measured against a
real server: a forged cursor naming `l = 1_000_000_000` reached
`search_zim_file_data` with exactly that, versus a documented cap of 1000.

`search_zim_file_data` has no range check of its own, so
`result_count = min(limit, total_results - offset)` becomes the whole match
set. On a full English Wikipedia archive a common term matches on the order
of a million entries, each row costing a libzim lookup plus an HTML snippet
render, and the finished payload is then serialized and written to the
response cache. One unauthenticated request, no explicit `limit` for anything
to validate.

Both resolvers now bound `s.l` by `MAX_SEARCH_RESULT_LIMIT`, which is exactly
the rule already documented for the lower bound: a hand-built cursor can
carry anything, and anything outside the acceptable range falls back to the
handler default rather than erroring. The server never mints a page above
that cap, so no legitimate cursor is turned away — asserted by a test that a
cursor at the cap is still honoured.

The tighter per-tool caps keep working as before: `walk_namespace` already
rejects a forged 100000 with "limit must be between 1 and 500", which is why
this only ever bit the search backend.

8 tests, 3 confirmed to fail against the unfixed resolvers. Suite 4029
passed, 0 failed.

The SDK's DNS-rebinding Host/Origin check runs inside the transport, and on
the legacy branch that is too late to matter. Era routing sends any request
without an `MCP-Protocol-Version` header down that branch, which creates the
transport, registers it in the session manager's table and starts its task
*before* reaching the handler that validates the headers. Measured: ten
requests carrying `Origin: http://evil.example`, every one correctly answered
`403 Invalid Origin header`, and the session table grew from 0 to 10.

Nothing reaps them. The app builder exposes no `session_idle_timeout`, so the
manager takes the `None` default and its idle-expiry branch never runs; the
table is emptied only by an explicit DELETE or process exit.

The exposure is the default deployment rather than an exotic one:
`--transport http` binds 127.0.0.1 with no token — which `check_safe_startup`
permits precisely because it is localhost — and no CORS origins, so the auth
middleware short-circuits and no CORS middleware is installed. Any page the
user visits can `fetch()` the endpoint in a loop and leak a session and a
live task per request, with the tool surface never invoked and nothing in the
logs naming a leak.

The same check now runs as the innermost of the three middlewares, so a
request that fails it is answered without the SDK ever seeing it. Auth and
CORS still answer first, and the health probes are exempt for the same reason
they are exempt from auth — asserted, so the gate cannot shadow them.

2 tests; the first is confirmed to fail against the ungated app (0 -> 10).
Suite 4031 passed, live suite 38 passed, 0 failed. bandit -ll clean.

…elease rewrites

`_MD_LINK_RE` stopped at the first `)`, but html2text backslash-escapes the
parens inside a link target — so a heading linking to a parenthetically
disambiguated article, which is how Wikipedia distinguishes most ambiguous
titles, reduced to `Mercury (mythology) "Mercury (mythology)")` instead of
`Naming after Mercury (mythology)`. That never matches the soup-side text, so
the heading was dropped, its section vanished from the bundle, and the
preceding section's slice ran on through and swallowed its body. It now uses
the same escape-aware alternation `compact_format._MARKDOWN_LINK_RE` adopted
for the CodeQL `py/redos` finding, where each character belongs to exactly
one branch.

Cache keys follow the rule the bundle prefix already states — a change to
what a value CONTAINS needs a new prefix:

- `bundle:v2e` -> `v2f`, because this pass makes two more classes of heading
  locatable, so affected articles gain sections they previously lacked.
- `entry:` -> `entry:v3:`, which was never versioned at all. Its stat token
  covers the archive changing, not this server changing what it renders from
  an unchanged archive — and 3.0.0 does that three times over (the empty
  `M/<key>` fallback, the page-boundary whitespace fix, `more_at_offset`). An
  operator upgrading with `persistence_enabled` would have been served the
  2.7.0 rendering until the TTL expired, every fix inert on exactly the
  entries it was written for.
- `browse_ns_data:v2d` -> `v2e`, since the new-scheme W payload gained
  `scanned_count` this release.

The transport-security gate's test no longer needs the ZIM corpus; hard-coding
that path failed the four CI jobs that run without it.

Suite 4039 passed with the corpus and 3978 without, 0 failed either way.

…rtesy word

`_TOPIC_LIKE_POLITENESS` exists because `Cheers` and `tack` are ordinary
article titles as well as courtesy words, and the strip refuses a pass that
would consume the query's last topic-bearing token. But that guard asks
`_carries_topic` what is left, and its scaffolding set covered only half the
intents: it held `structure`, `section` and `toc` but not `outline`, `table`
or `contents`. So `outline of tack` left `outline of` behind, which read as
topic-bearing, the guard stayed silent, the operand was peeled, and the
extractor anchored on the last keyword — producing a confident structure dump
for an entry named `of`, with nothing surfaced to say the topic had been lost.
`table of contents for cheers` gave `entry_path='for'` the same way.

The set now covers the scaffolding of the phrases that reach a greedy-tail
extractor. Adding a word here can only make the guard fire more often, and
firing means keeping the query the user actually typed, so the direction is
the safe one — `outline of Berlin` and friends are asserted unchanged, as is
a genuine trailing `please`.

9 tests, the first five confirmed to return `of` / `for` against the previous
commit. Suite 4048 passed, 0 failed.

The reassembly tests built their `ContentProcessor` against a hard-coded
`/tmp`, which Windows resolves to `D:\tmp` — not present, so config
validation rejected it and both Windows jobs failed while every other
platform passed. `tmp_path` gives a real directory on all three. The repo
already carries this note in `test_availability_fixes` and `test_config`;
this file missed it.

SDK 2.0.0 ships its per-version method tables without a ping row for the
modern revision (python-sdk#3273), so every keepalive from a
2026-07-28-negotiated client dies as -32601 before the SDK's own ping
handler is consulted - and fixing only the request gate would turn the
miss into an unhandled KeyError in the outbound sieve. install a startup
shim that copies the 2025-11-25 ping rows into both tables, through the
one seam every consumer shares: the dict under the MappingProxyType.

A canary test pins the upstream gap and fails the day the locked SDK
defines the rows itself, which is the signal to delete the shim. The
wire tests assert the behavior in-memory and, via the live suite, over
real HTTP against a spawned server process.

This was the concrete blocker holding the v2 port (#360) out of a
release; with modern keepalives answered, merging can cut 3.0.0.

paged_slice_length returned 0 whenever a mid-article slice was entirely
whitespace: lead is 0 past the top of the article and rstrip empties the
rest. The footer hint and _meta.more_at_offset both derive from it, so
each named the offset the caller was already at - an infinite loop for
any client following either, on any preformatted body with a whitespace
run at least one page wide.

Deferral now has a floor: the all-whitespace page consumes its full
slice, and truncate_content emits the run verbatim. Deferring it to the
next page defers it to the same page forever, and dropping it would fuse
the words on either side of the run - the exact defect the reassembly
suite exists to prevent. The sweep-8 harness's own nxt > offset assert
catches regressions.

This release changes what _generate_search_suggestions produces - the
canonical probe's Strategy B now takes the minimum over all prefix
matches and declines when the shortest title is already on the page -
but the suggestions_data key kept its v2b prefix. The key embeds an
archive stat token and the archive did not change, so an operator
upgrading with persistence_enabled restored the 2.7.0 snapshot and kept
serving the wrongly promoted payloads until TTL expiry, the fix inert on
exactly the cached queries it was written for.

Bumped to v2c under the release's own rule, stated at the entry:v3 and
browse_ns_data:v2e bumps: a change to what a payload contains needs a
new prefix. A sweep-9 guard test pins the retired spelling out of the
source the same way the sweep-8 one does for entry keys.

The range said <3.0 but the code performs 2.0.0-specific surgery:
sdk_compat.py patches the dicts under the SDK's method tables, and
server.py/subscriptions.py install handlers through the private
_request_handlers registry. CI resolves the lockfile and stays green
either way; fresh installs (pip/uvx, Smithery, Docker) resolve the
newest wheel the range admits, so a 2.1 that moves any of those seams
would break at runtime with the suite still green - the
lockfile-masks-range-defects failure mode again.

Raising the ceiling is a deliberate bump PR that re-audits the seams;
the ping-shim canary says when the biggest one can go.

Three claims the branch made false, and one name that no longer exists.

The subscriptions page said delivery "no longer touches SDK internals"
- the bounded listen handler and sdk_compat.py are exactly that - and
told clients a raw non-ASCII stem "would never match", which the
four-spelling publish exists precisely to make untrue. The server's own
pre-ack listen bound (256 URIs per stream, 2048 chars per URI,
INVALID_PARAMS) was documented nowhere while the pages presented the
SDK's two stream caps as the only limits; it now sits in the delivery
list, the security page's defense-in-depth table, and the deployment
bullet, with the rejection codes distinguished (fix the request vs try
again later).

FastMCP does not exist in SDK v2. Every remaining reference on the
website and in code comments now names what ships - the SDK, MCPServer,
or the MCP SDK default, whichever keeps the sentence's claim precise;
each behavior claim (base64 wrapping, non-dict wrapping, func_metadata,
the host allow-list default) was re-verified against the installed 2.0.0
before renaming. The one survivor is sdk_compat.py's "FastMCP 4",
which names the third-party client framework.