Skip to content

fix(core): remove os/pathlib from wiki_coverage cluster (#560, 3/3) - #569

Merged
cdeust merged 5 commits into
mainfrom
fix/560-core-no-os-wiki
Sep 15, 2026
Merged

cdeust merged 5 commits into
mainfrom
fix/560-core-no-os-wiki

Conversation

@cdeust

@cdeust cdeust commented Sep 15, 2026 •

Copy link
Copy Markdown
Owner

Summary

Third and final PR of the multi-PR #560 fix: the wiki filesystem cluster (6 of the 17 mcp_server/core/ modules that imported os/pathlib). Together with #564 (2 modules) and #568 (9 modules) together they complete #560.

  • wiki_groomer.py, wiki_file_doc_skeleton.py: pathlib/os.path usage was pure string manipulation (no I/O) - replaced with local helpers pinned equal to PurePosixPath/os.path.splitext by new tests.
  • wiki_drift.py + new mcp_server/infrastructure/wiki_drift_fs.py: real filesystem I/O (read a page, mtime, walk the wiki tree, walk a source tree for file-exists checks) moved to infrastructure; core goes through configure_wiki_drift_filesystem(...). Public function signatures are unchanged, so existing handler call sites and all 11 existing tests needed zero changes.
  • wiki_coverage_dashboard.py + new mcp_server/infrastructure/wiki_dashboard_fs.py: same treatment for gap counting, kind-page counts, domain-dir listing, and the actual dashboard file writes.
  • wiki_axis_registry.py + new mcp_server/infrastructure/wiki_axis_fs.py: the _schema/ file reads moved to infrastructure via configure_schema_file_reader(...).
  • wiki_coverage.py (1330 lines, the largest module) + new mcp_server/infrastructure/wiki_coverage_fs.py: 5 functions doing real I/O (stat, listdir, walk, open+read) moved behind one configure_wiki_coverage_filesystem(...) seam. _SKIP_DIRECTORIES now imports the shared frozenset already extracted for wiki_drift_fs.py instead of duplicating it (infrastructure may not import core, so it lives in mcp_server/shared/wiki_skip_directories.py).

Seam-fragility audit (per the CI failure caused by PR #568's reranker seam being called from a bare CI script that bypassed every composition root): grepped scripts/, .github/, benchmarks/, and mcp_server/hooks/ for every function and configure_* name this PR introduces - zero hits outside mcp_server/__main__.py, tests_py/_composition_root_wiring.py, and handlers. Unlike the reranker seam, none of these four wiki seams have bare-script exposure today, so their fail-fast "raise if unconfigured" design is safe (a wiki audit silently returning "no drift found" on misconfiguration would be a worse failure than a loud crash). Also re-audited PR #564's seams: both already safe (graceful degrade or no I/O at all).

Test plan

  • python scripts/check_craftsmanship.py --base origin/main returns OK
  • grep for os/pathlib in these 6 files returns empty
  • ruff check . and ruff format --check . are clean
  • pyright mcp_server/ reports 0 errors
  • python scripts/verify_mcp_hosts.py -- hypermnesia-mcp passes on all 6 client/profile combinations
  • Full pytest -q: 8447 passed, 165 skipped, 0 failed

Refs #560 (the last of the three PRs to merge, #568, closes the issue)

🤖 Generated with Claude Code

cdeust added a commit that referenced this pull request Sep 15, 2026
Addresses the reviewer's explicit "apply the same no-silent-default rule
to #569, for example a wiki-root seam that falls back to a default
path" note on PR #568's review.

get_registry() used to collapse "configure_default_wiki_root was never
called" and "the configured provider legitimately returned None" into
the same wiki_root=None -> defaults-only behavior. The first case is a
missing composition root, not a valid answer, and silently degrading to
built-in defaults would hide any wiki-relative custom axis taxonomy a
user configured via _schema/ files (CLAUDE.md: no silent fallbacks).
Now: an explicitly-configured provider that returns None still yields
defaults-only (unchanged, legitimate); a provider that was never
configured at all raises RuntimeError.

Wired tests_py/_composition_root_wiring.py's
configure_default_wiki_root(lambda: None) explicitly, matching the
test session's pre-existing implicit behavior (conftest.py isolates
tests from any real wiki data) but now stated rather than silently
defaulted. All 78 tests across test_wiki_axis_registry.py,
test_wiki_classifier.py, test_ble001_sweep_core.py, and
test_craftsmanship_imports.py pass unchanged.

Also fixed a malformed `source: ADR-0297, issue #560` citation in
wiki_coverage_fs.py's docstring (decision-integrity gate parses the
first token after "source:" as a single decision ID; a trailing comma
made it unparseable) -- same class of typo already fixed elsewhere in
this PR, missed on this file.

Verified: scripts/check_craftsmanship.py --base origin/main returns OK;
pyright mcp_server/ reports 0 errors; full pytest reports 8447 passed,
165 skipped, 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
cdeust and others added 3 commits September 15, 2026 05:58
…t/axis_registry/coverage_dashboard (#560)

Checkpoint commit, not yet pushed: 5 of the wiki-cluster's 6 modules done
(wiki_coverage.py remains). Same injection-seam idiom as the ablation/
env-flag PR (#568), but this checkpoint has NOT yet been audited for the
"unconfigured raises" ordering-contract problem the orchestrator flagged
on that PR — every provider here needs the same audit before this is
pushed for review.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
… modules

Third and final PR of the multi-PR #560 fix: wiki_coverage.py (1330
lines, the largest of the 6 wiki-filesystem-cluster modules) had 5
functions doing real filesystem I/O (os.stat, os.listdir, os.walk,
open+read). New mcp_server/infrastructure/wiki_coverage_fs.py holds all
of it; core now goes through one configure_wiki_coverage_filesystem(
stat_page=, markdown_file_sizes=, domain_kind_membership=,
walk_source_files=, walk_markdown_contents=) injection point, wired by
mcp_server/__main__.py and tests_py/_composition_root_wiring.py (the
same idiom every other module in this multi-PR fix uses).

- _has_substantive_anchor: os.stat -> injected stat_page(wiki_root,
  rel_path) -> (size, mtime) | None.
- _count_substantive_pages: isdir+listdir+getsize -> injected
  markdown_file_sizes(dom_path) -> {filename: size}.
- list_domains: isdir+listdir membership counting across known-kind
  directories -> injected domain_kind_membership(wiki_root, known_kinds)
  -> {candidate_domain: [kind, ...]}; the plausibility filter and
  >=2-kinds threshold stay in core as pure logic over the returned dict.
- list_source_files: os.walk+splitext+relpath -> injected
  walk_source_files(root, skip_directories, source_extensions), core's
  own frozensets passed in as plain data (no I/O in the constants
  themselves).
- _index_wiki_file_references: os.walk+open+read every .md file ->
  injected walk_markdown_contents(wiki_root) returning the raw text of
  every page; the regex token-extraction stays in core (pure).
- Two bare os.path.basename() calls (pure string ops, not part of the
  walk) replaced with token.rsplit("/", 1)[-1].
- _SKIP_DIRECTORIES stopped duplicating the frozenset already extracted
  to mcp_server/shared/wiki_skip_directories.py for
  infrastructure/wiki_drift_fs.py's identical need (DRY boy-scout fix;
  that shared module exists specifically because infrastructure/ may not
  import core/, so the pure data had to move to shared/ for both infra
  modules to reuse).

Also fixes a pyright regression introduced by wiki_axis_registry.py's
fix in this same branch: configure_default_wiki_root's provider type
changed from Path | str | None to str | None (core cannot reference
Path even in a type hint without importing pathlib), so __main__.py's
own configure_default_wiki_root(lambda: WIKI_ROOT) needed to become
lambda: str(WIKI_ROOT) (WIKI_ROOT is a Path from
infrastructure/config.py) -- caught by pyright, not by any test.

Seam-fragility audit (per the #568/PR2 CI failure where a bare CI
preload script bypassed every composition root and hit an unconfigured
seam): grepped scripts/, .github/, benchmarks/, and mcp_server/hooks/
for every wiki_drift/wiki_axis_registry/wiki_coverage_dashboard/
wiki_coverage function and configure_* name introduced in this PR and
the prior wiki-cluster commit -- zero hits outside mcp_server/__main__.py,
tests_py/_composition_root_wiring.py, and handlers/. Unlike the reranker
seam, these four wiki seams have no bare-script exposure today, so their
"raise RuntimeError if unconfigured" design (correct here: a wiki audit
silently returning "no drift found" when actually misconfigured is a
worse failure mode than a loud crash, unlike an env-var read where
"unconfigured = false" is a safe default) is not a live landmine. Also
re-audited PR #564's abstention_gate.py (classifier_factory already
degrades gracefully to no-filtering when unconfigured, confirmed safe)
and codebase_graph.py (no I/O seam at all, pure string ops only).

Fixes #560 (all 17 of the audited core/ modules that imported os or
pathlib now have zero such imports; part 1/3 was PR #564, part 2/3 was
PR #568).

Verified: scripts/check_craftsmanship.py --base origin/main returns OK;
grep for os/pathlib in this branch's 6 wiki modules returns empty; ruff
check/format are clean; pyright mcp_server/ reports 0 errors; `python
scripts/verify_mcp_hosts.py -- hypermnesia-mcp` passes on all 6
client/profile combinations; full pytest in progress at commit time (see
hand-back notes for the confirmed count).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Addresses the reviewer's explicit "apply the same no-silent-default rule
to #569, for example a wiki-root seam that falls back to a default
path" note on PR #568's review.

get_registry() used to collapse "configure_default_wiki_root was never
called" and "the configured provider legitimately returned None" into
the same wiki_root=None -> defaults-only behavior. The first case is a
missing composition root, not a valid answer, and silently degrading to
built-in defaults would hide any wiki-relative custom axis taxonomy a
user configured via _schema/ files (CLAUDE.md: no silent fallbacks).
Now: an explicitly-configured provider that returns None still yields
defaults-only (unchanged, legitimate); a provider that was never
configured at all raises RuntimeError.

Wired tests_py/_composition_root_wiring.py's
configure_default_wiki_root(lambda: None) explicitly, matching the
test session's pre-existing implicit behavior (conftest.py isolates
tests from any real wiki data) but now stated rather than silently
defaulted. All 78 tests across test_wiki_axis_registry.py,
test_wiki_classifier.py, test_ble001_sweep_core.py, and
test_craftsmanship_imports.py pass unchanged.

Also fixed a malformed `source: ADR-0297, issue #560` citation in
wiki_coverage_fs.py's docstring (decision-integrity gate parses the
first token after "source:" as a single decision ID; a trailing comma
made it unparseable) -- same class of typo already fixed elsewhere in
this PR, missed on this file.

Verified: scripts/check_craftsmanship.py --base origin/main returns OK;
pyright mcp_server/ reports 0 errors; full pytest reports 8447 passed,
165 skipped, 0 failed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@cdeust
cdeust force-pushed the fix/560-core-no-os-wiki branch from c6236e9 to 88b7a9c Compare September 15, 2026 04:01
@cdeust

cdeust commented Sep 15, 2026

Copy link
Copy Markdown
Owner Author

ZETETIC-REVIEW: REQUEST_CHANGES

Reviewed at 88b7a9c, rebased on 427b923.

  1. Production regression in the background consolidation. The six wiki filesystem seams (configure_wiki_coverage_filesystem, configure_wiki_drift_filesystem, configure_dashboard_filesystem, configure_default_wiki_root, configure_schema_file_reader) are wired only in mcp_server/__main__.py and tests_py/_composition_root_wiring.py, and they raise when unwired. mcp_server/hooks/consolidate_background.py, spawned by SessionStart, imports handlers.consolidate directly and runs it with "wiki": True (line 79). That path goes through handlers/consolidation/wiki_maintenance.py, which reaches core/wiki_coverage and core/wiki_coverage_dashboard, so on this branch every autonomous consolidation fails in the hook process. scripts/regenerate_file_docs.py reaches core/wiki_coverage the same way. CI cannot see this because the suite always wires the seams in conftest.
  2. The design multiplies wiring sites. Together with fix(core): remove os/pathlib from ablation.py and 8 retrieval env-flag modules (#560, 2/3) #568, every new seam must be added by hand to __main__, 14 hook entry points, the benchmark bootstrap and the scripts. One composition-root function should wire every seam (environment reader, reranker filesystem, wiki filesystem), with every entry point calling it. A test must enumerate the entry points mechanically (hook commands in .claude-plugin/plugin.json, benchmarks/**/run_*.py, scripts/*.py reaching core, __main__) and assert that each one calls it. A subprocess test must run consolidate_background with wiki: True on a throwaway store without the test conftest, and assert that it completes.

cdeust and others added 2 commits September 15, 2026 07:11
…point

Review of #569 found the wiki filesystem seams wired only in __main__ and
the test conftest while they raise when unwired: the SessionStart-spawned
hooks/consolidate_background.py runs consolidate with wiki=True in its own
process, so every autonomous consolidation lost its dashboards (the
sub-task error was recorded nested in stats and the cycle still reported
status=ok). scripts/regenerate_file_docs.py had the same gap.

- mcp_server/composition_root.py: wire_composition_root(), idempotent,
  wires the wiki root, the schema reader and the coverage, drift and
  dashboard filesystems in one place.
- Called by __main__, scripts/launcher.py, every hooks/*.py __main__
  block, every script and benchmark runner that reaches core.
- tests_py/scripts/test_composition_root_wiring_coverage.py enumerates
  the entry points mechanically (plugin.json hook commands, hooks with a
  __main__ block, scripts and benchmarks importing core/handlers) and
  requires an actual wire_composition_root() call in each.
- tests_py/hooks/test_consolidate_background_wiki_composition_root.py
  runs the hook as a real subprocess on a throwaway SQLite store without
  the test conftest. Measured: unwired, the hook logs
  "dashboards=error: RuntimeError: ... not configured"; wired, it logs
  "dashboards=written=0". The hook's wiki summary line now reports the
  dashboard outcome instead of defaulting to "ok".

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…y import

CI's craftsmanship gate refused hooks/* importing the top-level
mcp_server.composition_root: the layer table lets hooks/ import core,
infrastructure and shared only. hooks/ is also the only layer that may
itself import both core and infrastructure and be imported by every
entry point (the top-level __main__, scripts and benchmarks are not
layer-checked), so the single wiring function moves to
mcp_server/hooks/wiring.py.

- post_tool_capture wires in _dispatch_with_store_cleanup, after the
  capture-mode gate, instead of inside process_event: excluded events
  still import nothing from core or infrastructure (W3-1b), and
  process_event stays within the 40-line cap.
- Single-line imports keep post_commit_reindex.py under 300 lines.
- The coverage test matches a real top-level main block, not the phrase
  quoted in a docstring.
- Measured cost of the wiring import in a hook process: 24-35 ms
  cumulative (python -X importtime, 3 runs, this machine).

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@cdeust

cdeust commented Sep 15, 2026

Copy link
Copy Markdown
Owner Author

ZETETIC-REVIEW: APPROVE

Re-reviewed at f77040a, base 427b923. Both points of the earlier REQUEST_CHANGES are addressed.

  • The six wiki seams still raise when unwired. They are now wired by a single idempotent wire_composition_root() in mcp_server/hooks/wiring.py. Every entry point calls it: __main__, scripts/launcher.py, every hook's main block, every script and benchmark runner that reaches core, and the test session. It lives in hooks/ because that is the only layer that may import both core and infrastructure and also be imported by hook processes. The CI craftsmanship gate refused the earlier top-level placement.
  • Measured on the hook path: python -m mcp_server.hooks.consolidate_background on a throwaway SQLite store logs dashboards=written=0. The same hook body run without the wiring logs dashboards=error: RuntimeError: ... not configured. The hook's summary line no longer defaults the dashboard status to ok.
  • test_composition_root_wiring_coverage.py enumerates entry points mechanically and requires an actual call, not a mention. The subprocess test has no wall-clock verdict.
  • post_tool_capture wires after the capture-mode gate, so excluded events still import nothing from core or infrastructure (W3-1b).
  • Cost of the wiring import per hook process: 24-35 ms cumulative (-X importtime, 3 runs).
  • Gates on the final state: full suite 8617 passed (one coverage-test false positive fixed, the hook tests re-run 472 passed). pyright 0 diagnostics. The craftsmanship gate passes with --base origin/main, as CI runs it. CI is green on all 24 required checks.

@cdeust
cdeust merged commit 872855f into main Sep 15, 2026
28 checks passed
@cdeust
cdeust deleted the fix/560-core-no-os-wiki branch September 15, 2026 05:42
cdeust added a commit that referenced this pull request Sep 15, 2026
…g modules (#560, 3/3)

Last of the three #560 PRs (after #564 and #569): ablation.py,
conflict_monitor.py, pg_recall_weights.py, recall_pipeline.py,
reconsolidation.py, reranker_model.py, retrieval_dispatch.py,
telemetry.py and thermodynamics.py no longer import os or pathlib.

- core/environment.py: read_environment_variable/read_environment_float
  read through an injected reader, evaluated on every call, so runtime
  toggles (CORTEX_ABLATE_*, CORTEX_DECAY_DISABLED, CORTEX_*_BETA) still
  take effect after import. An unconfigured reader raises: returning
  "unset" silently would make every ablation run measure the un-ablated
  system (benchmarks set CORTEX_ABLATE_* without importing __main__).
- core/reranker_model.py: the FlashRank cache lookup, existence check and
  sha256 move to infrastructure/reranker_cache.py behind a seam that
  raises when unwired.
- Both seams join the wiki seams in mcp_server.hooks.wiring
  .wire_composition_root(), the single function every entry point calls.
  It now re-applies its bindings on every call instead of short-circuiting
  after the first, so a seam reset by a test is restored. The per-hook
  copies, benchmarks/lib/_environment_wiring.py, _reranker_wiring.py and
  the import-time wiring in hooks/_telemetry.py are gone;
  benchmarks/lib/_composition_root_wiring.py calls the same function.
- doctor.ensure_reranker_ready() wires through that function; CI's
  reranker preload step calls it instead of reaching into core directly.
- tests_py/benchmarks/test_ablation_runner_wiring.py drives the
  ablation-runner path with a CORTEX_ABLATE_* flag and proves the
  mechanism is disabled.

Fixes #560

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
cdeust added a commit that referenced this pull request Sep 15, 2026
…g modules (#560, 2/3) (#568)

* fix(core): remove os/pathlib from ablation.py and 8 retrieval env-flag modules (#560, 3/3)

Last of the three #560 PRs (after #564 and #569): ablation.py,
conflict_monitor.py, pg_recall_weights.py, recall_pipeline.py,
reconsolidation.py, reranker_model.py, retrieval_dispatch.py,
telemetry.py and thermodynamics.py no longer import os or pathlib.

- core/environment.py: read_environment_variable/read_environment_float
  read through an injected reader, evaluated on every call, so runtime
  toggles (CORTEX_ABLATE_*, CORTEX_DECAY_DISABLED, CORTEX_*_BETA) still
  take effect after import. An unconfigured reader raises: returning
  "unset" silently would make every ablation run measure the un-ablated
  system (benchmarks set CORTEX_ABLATE_* without importing __main__).
- core/reranker_model.py: the FlashRank cache lookup, existence check and
  sha256 move to infrastructure/reranker_cache.py behind a seam that
  raises when unwired.
- Both seams join the wiki seams in mcp_server.hooks.wiring
  .wire_composition_root(), the single function every entry point calls.
  It now re-applies its bindings on every call instead of short-circuiting
  after the first, so a seam reset by a test is restored. The per-hook
  copies, benchmarks/lib/_environment_wiring.py, _reranker_wiring.py and
  the import-time wiring in hooks/_telemetry.py are gone;
  benchmarks/lib/_composition_root_wiring.py calls the same function.
- doctor.ensure_reranker_ready() wires through that function; CI's
  reranker preload step calls it instead of reaching into core directly.
- tests_py/benchmarks/test_ablation_runner_wiring.py drives the
  ablation-runner path with a CORTEX_ABLATE_* flag and proves the
  mechanism is disabled.

Fixes #560

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

* test(ablation): the runner wiring test needs psycopg, like the runner (#560)

benchmarks/lib/ablation_runner.py imports db_snapshot, which imports
psycopg; the SQLite-only CI job has no postgresql extra and failed at
import. The PostgreSQL test jobs keep running it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
@cdeust cdeust mentioned this pull request Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant