docs(readme): predictions and calibration, one store for Claude Code and Codex - #603

ZETETIC-REVIEW: REQUEST_CHANGES

Docs-only diff, 34 added and 3 removed across 2 files, reviewed at 40263d6 against main 9656fc4 in a read-only worktree. Every return shape and key in the new README block was checked by running the three handlers, not by reading them. Two claims in the new text do not survive that check.

Blocking

1. README.md:176, band: [0.7, 0.8] is not a band calibration can return

BAND_EDGES in mcp_server/core/calibration.py:43 is (0.0, 0.2, 0.4, 0.6, 0.8, 1.0), and _band_of buckets a confidence into one of those five intervals. A mean_confidence of 0.78 falls in [0.6, 0.8]. Band edges are structure, not an illustrative number: a reader who writes code against band: [0.7, 0.8] writes a matcher that never fires.

Ran the README's own example through the handlers on a temporary SQLite store, plus three more resolved rows to fill a band:

$ PYTHONPATH=. .venv/bin/python run_readme.py
predict -> {"prediction_id": 1, "status": "open"}
resolve_prediction -> {"prediction_id": 1, "verdict": "confirmed", "resolved": true}
calibration ->
{
 "scored": 4,
 "abandoned": 0,
 "brier": 0.16022499999999998,
 "uninformative_brier": 0.25,
 "confirmed": 3,
 "refuted": 1,
 "reliability": [
  {"band": [0.6, 0.8], "resolved": 3, "mean_confidence": 0.7433333333333333,
   "observed_frequency": 0.6666666666666666},
  {"band": [0.8, 1.0], "resolved": 1, "mean_confidence": 0.8,
   "observed_frequency": 1.0}
 ],
 "open": 0,
 "domain": "all"
}
bands produced -> [[0.6, 0.8], [0.8, 1.0]]
UNINFORMATIVE_BRIER = 0.25 BAND_EDGES = (0.0, 0.2, 0.4, 0.6, 0.8, 1.0)

Fix: write band: [0.6, 0.8].

2. README.md:290 marks the Codex plugin ✅ for tools its profile does not expose

The middle column is headed "Local stdio hosts (Gemini CLI, Codex CLI, ChatGPT desktop, Cursor, Windsurf, VS Code, Agents SDK)". For Codex CLI and ChatGPT desktop the documented path is the native plugin, which this same commit describes twenty lines later at README.md:310 as a 10-tool lean surface. docs/codex-plugin.md:11 and plugins/hypermnesia-mcp-codex/README.md:25 both pin it to hypermnesia-mcp --profile lean. LEAN_TOOL_NAMES in mcp_server/tool_profiles.py:26 holds exactly query_methodology, remember, recall, unified_search, recall_hierarchical, consolidate, memory_stats, check_setup, wiki_read, wiki_list, and none of the prediction tools is among them:

lean allows predict -> False
lean allows resolve_prediction -> False
lean allows calibration -> False
len(LEAN_TOOL_NAMES) = 10

LEAN_INSTRUCTIONS states the hidden surface is also rejected on call, so a Codex plugin user who follows the table gets a refusal, not a tool. The three tools are indeed registered in mcp_server/tool_registry_predictions.py and belong to the standalone surface (mcp_server/tool_surface.py:36-42), so the ✅ is right for a stdio host that launches the full profile and wrong for the plugin the README recommends to those two hosts.

Fix: qualify the cell the way the query_methodology row already qualifies its own, for instance ⚠️ full profile only; the Codex plugin's lean surface does not register them, or add the exclusion to the Codex paragraph at README.md:310.

Non-blocking

1. Both new claims describe unreleased code. The prediction tools landed at dd29c89 on 2026-09-16, the backend alignment at 9656fc4, and the newest tag is v4.22.0 at f4f382e on 2026-09-15. docs/codex-plugin.md:34 hedges with "In this source checkout (pending release), direct MCP startup reads the same ~/.claude/methodology/backend.json selection"; README.md:310 drops the hedge while the plugin's .mcp.json still starts the published package through uvx.
2. README.md:312 says explicit CORTEX_MEMORY_STORE_BACKEND or a database URL still wins. resolve_backend_env in mcp_server/infrastructure/backend_marker.py:71-79 also honours CORTEX_BACKEND between those two. Partial rather than wrong.

Checked and correct

predict requires claim, prediction, test, confidence (mcp_server/handlers/predict.py:36) and returns {prediction_id, status: "open"}, matching README.md:169.
resolve_prediction requires prediction_id, verdict, observed, source_kind, source_ref and returns {prediction_id, verdict, resolved: true}, matching README.md:173. VERDICTS is ("confirmed", "refuted", "abandoned") and SOURCE_KINDS is ("review", "ci", "test", "manual"), so the example's "confirmed" and "ci" are both legal.
calibration_report returns every key the README shows, plus abandoned, open and domain that it does not claim to list; reliability rows carry exactly band, resolved, mean_confidence, observed_frequency.
UNINFORMATIVE_BRIER is 0.25 at mcp_server/core/calibration.py:38, so the prose about a constant 0.5 forecast holds.
The sentence that the verdict, the observation and the source reference come from the caller matches resolve_prediction's refusal path, which rejects a resolution carrying no reference.
Prose in the new subsection carries no em dash, no bold-label bullet, no bullet list at all and not the word the redaction rules forbid.
The CHANGELOG edit removes a second ### Added inside the Unreleased block; the entries below it now sit under the heading at CHANGELOG.md:9.

Gates

$ python scripts/check_doc_claims.py
doc claims OK (1 declared not-a-claim exemption(s))
  CONTRIBUTING.md:36: exempt from the tests claim
$ python scripts/check_project_wiki.py
Project wiki mirrors match their canonical sources.
$ UV_NO_SYNC=1 uv run ruff check
All checks passed!
$ UV_NO_SYNC=1 uv run ruff format --check
1483 files already formatted

gh pr checks 603 settled green at 40263d6: 14 SUCCESS, 11 SKIPPED, nothing failing or pending. Test (SQLite backend) 5m54s, Fuzz (PR batch) address 5m5s and undefined 5m4s, Craftsmanship Gate 18s, Lint 15s, CI Green 7s. No Python changed, so Type Check and the Build/Docker jobs were skipped by the changed-paths filter.

Green CI does not reach either finding: no gate reads the README's illustrative output against BAND_EDGES, and scripts/verify_mcp_hosts.py counts the tools the server registers rather than the ones a profile keeps.

Two edits clear this: [0.6, 0.8] at README.md:176, and a qualified cell at README.md:290.

ZETETIC-REVIEW: REQUEST_CHANGES

Verdict on 73c4277, the branch tip. It supersedes my comment on 40263d6, which the head has moved past twice since. Reviewed in a read-only worktree at that sha.

Closed since 40263d6

README.md:176 reads band: [0.6, 0.8], a band _band_of can produce for a mean_confidence of 0.78 under BAND_EDGES = (0.0, 0.2, 0.4, 0.6, 0.8, 1.0).
The host table is one column per host, and the Codex plugin column carries ❌ for predictions with the reason and the alternative.
README.md:319 names CORTEX_BACKEND between the other two, matching resolve_backend_env at backend_marker.py:71-79.
README.md:288 and README.md:318 hedge the shared backend selection to the release after 4.22.0, which is right while .mcp.json launches the published package through uvx.
The worktree row no longer asserts a doctor check. doctor.py:295 lists eight checks on PostgreSQL and five on SQLite with no worktree among them, grep -c worktree mcp_server/doctor_mcp.py is 0, and the check that would report one lives in #602, still open. The row is now correct about what enforces the convention.

Blocking

1. README.md:279 attributes the lean profile to the startup budget; the cited document does not say that, and ADR-0693 says otherwise

The sentence is "The Codex plugin deliberately starts the server with --profile lean so its first uvx handshake stays inside the host's startup budget (measured in docs/codex-plugin.md)".

docs/codex-plugin.md mentions lean at lines 11, 61 and 70 and the startup budget at lines 55 to 63, and never joins the two. Line 59 measures 103.84 seconds from clean UV_CACHE_DIR and UV_TOOL_DIR against the manifest's startup_timeout_sec: 180. That is the two-driver package download, which the flag cannot shorten: tool_profiles.resolve parses --profile after uvx has installed and imported the package, and it gates registration rather than installation. The citation points at a number that does not measure the stated cause.

ADR-0693 records the real reason, quoting the module docstring: "Cortex registers its whole tool surface to every client on every session (issue #177): the largest fixed token cost in the ecosystem, paid before the user types anything. A profile narrows that surface for sessions that only need the common recall/onboarding loop". Fixed token cost per session, not seconds of startup.

Fix: attribute the choice to the token cost of the full surface and cite ADR-0693 or issue #177, or keep the fact and drop the causal clause.

2. README.md:297 sources both worktree values to docs/agent-guidance.md, which names one of them and excludes the other

The row now reads "Worktree directory (repository convention, docs/agent-guidance.md)" with .claude/worktrees/<name>/ for Claude Code and .Codex/worktrees/<name>/ for the Codex plugin. docs/agent-guidance.md:160-165 says:

- Do NOT create git worktrees outside this repository — no `/private/tmp/cortex-*`,
  no `../Cortex-wt-*` sibling, no `~/.claude/worktrees/`. The only location is
  `.claude/worktrees/<name>/` (already gitignored via `.claude/*`)

"The only location" backs the Claude Code cell and rules out the Codex one. The merged #601 frames the Codex directory the same way, in its own .gitignore:62-65 comment: "Codex puts its own worktrees under .Codex/worktrees/ rather than the .claude/worktrees/ this project's rule names, and the directory was one git add -A away from being committed". So .Codex/worktrees/ is where Codex in fact puts them and what #601 stopped from being committed, not what the cited convention prescribes.

Fix: scope the citation to the Claude Code cell and describe the Codex cell as observed behaviour gitignored since #601, or amend docs/agent-guidance.md first so the citation becomes true.

Every other cell, measured at 73c4277

$ PYTHONPATH=. python -c "from mcp_server.tool_profiles import LEAN_TOOL_NAMES, resolve; ..."
lean==README: True n= 10 diff= frozenset()
resolve default= ToolProfile.FULL
standalone= 57 bounds= (57, 60)

The ten names in the Tool surface cell are exactly LEAN_TOOL_NAMES at tool_profiles.py:26, symmetric difference empty, so the cell cannot drift from the code unnoticed.
"all 57 tools" matches tool_surface.standalone_tool_names(); full_tool_bounds() is (57, 60) with both upstream integrations.
"full is the default profile" matches resolve([], {}), and ADR-0693 records that keeping full as the default was deliberate rather than accidental.
The Codex plugin column rests on plugins/hypermnesia-mcp-codex/.mcp.json, whose args are ["--from", "hypermnesia-mcp[postgresql,sqlite]", "hypermnesia-mcp", "--profile", "lean"] with startup_timeout_sec: 180.
The wiki-writes row holds in both directions: wiki_write, curate_wiki, add_rule, codebase_analyze and ingest_findings are each in the standalone surface and none is in lean.
The autonomous-wiki row draws the split correctly: consolidate is in lean, curate_wiki is not.
The profiling row marks lean ⚠️ rather than ❌, and query_methodology is one of the ten.
The backend row matches backend_marker.py: the installer writes ~/.claude/methodology/backend.json, and apply_backend_resolution runs at mcp_server/__main__.py:22 before the registry imports build settings.
The third column's codex mcp add is the command the README itself gives at line 333.
The prediction subsection still checks out against the handlers: predict requires claim, prediction, test, confidence and returns {prediction_id, status}; resolve_prediction requires prediction_id, verdict, observed, source_kind, source_ref and returns {prediction_id, verdict, resolved}; calibration_report returns every key the example shows; UNINFORMATIVE_BRIER is 0.25.

Gates at 73c4277

$ python scripts/check_doc_claims.py
doc claims OK (1 declared not-a-claim exemption(s))
  CONTRIBUTING.md:36: exempt from the tests claim
$ python scripts/check_project_wiki.py
Project wiki mirrors match their canonical sources.

gh pr checks 603: 14 SUCCESS, 11 SKIPPED, nothing failing or pending. Test (SQLite backend) 5m41s, Fuzz (PR batch) address 5m13s and undefined 4m44s, HOL plugin scan 1m6s, Analyze (python) 1m2s, Lint 18s, Craftsmanship Gate 17s, CI Green 8s. No Python changed, so the type and build jobs were filtered out.

Neither finding is reachable by a gate: check_doc_claims.py does not compare a README rationale against the ADR that records the real one, and nothing checks that a cited convention document contains the value cited to it.

Two edits clear this: a cause at README.md:279 that matches ADR-0693, and a citation at README.md:297 scoped to the cell docs/agent-guidance.md actually names.

ZETETIC-REVIEW: APPROVE

Verdict on 22f766d, the branch tip. Both blocking findings from 73c4277 are closed against their sources, and re-running every table measurement at this sha leaves nothing blocking. This supersedes my earlier comments on 40263d6 and 73c4277.

The two findings, closed

README.md:280 now states the reason ADR-0693 records

New text: "the full tool surface is the largest fixed token cost a session pays before the user types anything (ADR-0693, issue #177), and the plugin keeps that cost to ten tools".

docs/adr/ADR-0693-mcp-server-tool-profiles.md:17 quotes the module docstring: "Cortex registers its whole tool surface to every client on every session (issue #177): the largest fixed token cost in the ecosystem, paid before the user types anything." The README's paraphrase narrows the ADR's superlative from the ecosystem to a session rather than inflating it, keeps "paid before the user types anything" intact, and cites both the ADR and the issue that the ADR itself names five times. "keeps that cost to ten tools" is the checkable half, and it checks out: plugins/hypermnesia-mcp-codex/.mcp.json passes --profile lean, and len(LEAN_TOOL_NAMES) is 10. The startup-budget clause and the citation to a measurement that did not measure it are both gone.

README.md:298 sources each worktree cell to what actually names it

New row: ".claude/worktrees/<name>/, the location docs/agent-guidance.md names" and ".Codex/worktrees/<name>/, where Codex puts its own; ignored at the repository root since #601".

docs/agent-guidance.md:161-162 reads "The only location is .claude/worktrees/<name>/", which is exactly what the first cell now claims for it, and nothing more.
.gitignore:62-65 reads "Codex puts its own worktrees under .Codex/worktrees/ rather than the .claude/worktrees/ this project's rule names", with .Codex/ on line 66. The second cell now says "where Codex puts its own", which is the gitignore comment's own framing, and attributes the ignoring to #601 rather than the convention. Neither cell borrows the other's authority.

Re-measured at 22f766d

$ PYTHONPATH=. python -c "from mcp_server.tool_profiles import LEAN_TOOL_NAMES, resolve; ..."
lean==README: True n= 10 diff= frozenset()
resolve default= ToolProfile.FULL
standalone= 57 bounds= (57, 60)
 lean predict : False
 lean resolve_prediction : False
 lean calibration : False
 lean wiki_write : False
 lean curate_wiki : False
 lean add_rule : False
 lean codebase_analyze : False
 lean ingest_findings : False
 lean consolidate : True
 lean query_methodology : True

Tool surface cell: the ten names are exactly LEAN_TOOL_NAMES at tool_profiles.py:26, symmetric difference empty. "all 57 tools" matches standalone_tool_names(), and full_tool_bounds() is (57, 60) with both upstream integrations. "full is the default profile" matches resolve([], {}).
Predictions row and wiki-writes row: every named tool is in the standalone surface and absent from lean, so both ❌ cells and both ✅ cells hold.
Autonomous-wiki row: consolidate is in lean and curate_wiki is not, which is the split the cell draws.
Profiling row: query_methodology is one of the ten, so ⚠️ rather than ❌ is right.
Backend row: the installer writes ~/.claude/methodology/backend.json, apply_backend_resolution runs at mcp_server/__main__.py:22 before the registry imports build settings, and the precedence in backend_marker.py:71-79 is the one README.md:319 states, CORTEX_BACKEND included.
Prediction subsection: predict requires claim, prediction, test, confidence and returns {prediction_id, status}; resolve_prediction requires prediction_id, verdict, observed, source_kind, source_ref and returns {prediction_id, verdict, resolved}; the reliability row keys are band, resolved, mean_confidence, observed_frequency; band: [0.6, 0.8] is producible under BAND_EDGES; UNINFORMATIVE_BRIER is 0.25.
CHANGELOG: the second ### Added inside the Unreleased block is gone and its entries sit under the heading at line 9.

Non-blocking, for a later change rather than this one

1. Nothing pins the README's ten lean names to LEAN_TOOL_NAMES. scripts/verify_mcp_hosts.py and tests_py/test_tool_profiles.py both import the frozenset, but neither reads the README, and check_doc_claims.py treats README.md as a claims surface without comparing that list. The same holds for "all 57 tools". A future profile change would leave the table stale with every gate green, which is how the ✅ I flagged at 40263d6 got written in the first place.
2. .gitignore:66 is the bare pattern .Codex/, which git matches at any depth, not only at the repository root. The cell's "ignored at the repository root" is true and is what fix(gitignore): anchor .Codex/ to repo root, report outside worktrees (ADR-1078) #602 is about to make exact; it is worth rereading that cell when fix(gitignore): anchor .Codex/ to repo root, report outside worktrees (ADR-1078) #602 merges.

Gates at 22f766d

$ python scripts/check_doc_claims.py
doc claims OK (1 declared not-a-claim exemption(s))
  CONTRIBUTING.md:36: exempt from the tests claim
$ python scripts/check_project_wiki.py
Project wiki mirrors match their canonical sources.

gh pr checks 603: 14 SUCCESS, 11 SKIPPED, nothing failing or pending. Test (SQLite backend) 8m27s, Fuzz (PR batch) address 4m49s and undefined 4m44s, Analyze (python) 1m11s, HOL plugin scan 1m3s, Analyze (javascript-typescript) 58s, Analyze (actions) 36s, Lint 22s, Craftsmanship Gate 15s, CI Green 7s, upstream identity contract 9s, Changed paths 5s, plugin-scanner 3s, CodeQL 3s. No Python changed, so the type and build jobs were filtered out.

The diff is 2 files, 54 added and 15 removed, and every factual claim in it now resolves to a file in this repository that states it.