Skip to content

Commit 9656fc4

Browse files
authored
fix: align Claude and Codex backend selection and verify shared handoff (#600)
* fix: share backend selection across Claude and Codex entry points * docs: record shared backend startup decision and review follow-up * docs: renumber shared startup decision to ADR-1078
1 parent dd29c89 commit 9656fc4

11 files changed

Lines changed: 639 additions & 6 deletions

‎CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,13 @@ adheres to [Semantic Versioning](https://semver.org/).
5555

5656
### Fixed
5757

58+
- **Claude and direct MCP startup honour the same saved backend (#600).** A
59+
saved SQLite selection was honoured by the Claude launcher while direct
60+
console/module startup could select a reachable PostgreSQL server, sending
61+
the two hosts to different stores. Direct startup now resolves the saved
62+
selection before loading memory settings, preserving explicit overrides.
63+
Existing databases are not migrated or merged.
64+
5865
- **Procedural skills are stored on the default backend too (#596).** The
5966
`procedural_skills` table existed only on PostgreSQL, so on SQLite the
6067
session-end writer mined its skills and lost every one to an
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
<!-- Generated by Cortex wiki_reindex from wiki/adr/cortex/1078-resolve-saved-backend-before-direct-mcp-startup-imports.md; ADR-1078. Read-only mirror: edit the canonical wiki page. -->
2+
3+
---
4+
created: 2026-09-16T19:44:08Z
5+
kind: adr
6+
number: 1078
7+
status: proposed
8+
tags: [architecture, storage, codex, startup]
9+
title: Resolve saved backend before direct MCP startup imports
10+
---
11+
# ADR-1078: Resolve saved backend before direct MCP startup imports
12+
13+
## Status
14+
15+
proposed
16+
17+
## Context
18+
19+
ADR-0505 resolves the saved backend in scripts/launcher.py. Direct console and python -m mcp_server startup skipped that resolution: Claude could honour a saved SQLite selection while Codex selected a reachable PostgreSQL server. PR #600 reproduces four failing bootstrap cases before the fix. Settings-dependent registry imports bind configuration early, so resolution inside main() would happen too late.
20+
21+
## Decision
22+
23+
Call the existing apply_backend_resolution(os.environ) at module startup before settings-dependent imports. Preserve precedence: explicit CORTEX_MEMORY_STORE_BACKEND, then CORTEX_BACKEND alias, then non-empty DATABASE_URL or CORTEX_MEMORY_DATABASE_URL, then saved methodology/backend.json, otherwise existing automatic selection. Retain the existing blank DATABASE_URL normalization. Importing this entry point intentionally mutates the process environment when resolution requires it. Keep the file-wide Ruff suppression scoped to E402 only: the intervening resolver call makes fourteen pre-existing downstream imports trigger E402, as independently measured in the PR review; one file-level explanation records this shared ordering constraint instead of fourteen repeated annotations.
24+
25+
## Consequences
26+
27+
Claude launcher and direct MCP startup apply the same saved backend policy without duplicating the resolver or merging existing stores. Explicit overrides remain authoritative. Tests isolate configuration roots and use subprocesses to avoid ambient environment leakage. Nine bootstrap cases pass after the fix; the reviewer reproduced four failures on the baseline and reported 8670 full-suite passes at db22b167. This establishes startup selection and tested process handoff, not installed-host hook parity, semantic retrieval quality, or PostgreSQL handoff. Project ADR sharing additionally requires a common physical project_root; lean authoring permissions remain unchanged. Evidence: https://github.com/cdeust/Cortex/pull/600#issuecomment-5703423724 and tests_py/test_shared_backend_bootstrap.py.

‎docs/codex-plugin.md‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,20 @@ codex plugin add hypermnesia-mcp-codex@cortex-codex-plugins
3131

3232
Restart the ChatGPT desktop app and start a new task so Codex loads the new
3333
plugin components. The plugin uses `uvx`, so `uv` must be available on `PATH`.
34-
The first launch installs both storage drivers. Cortex tries PostgreSQL first
35-
at the configured `DATABASE_URL` (or its local `cortex` default), then falls
36-
back to SQLite only when no explicit PostgreSQL target was supplied and the
37-
default server is unavailable. An explicitly configured but unreachable
38-
`DATABASE_URL` remains an error rather than silently redirecting writes.
34+
The first launch installs both storage drivers. In this source checkout (pending
35+
release), direct MCP startup reads the
36+
same `~/.claude/methodology/backend.json` selection as the Claude launcher
37+
before loading memory settings. `CORTEX_CLAUDE_DIR` relocates that shared
38+
configuration root. Explicit `CORTEX_MEMORY_STORE_BACKEND`, then
39+
`CORTEX_BACKEND`, then a non-empty `DATABASE_URL` or
40+
`CORTEX_MEMORY_DATABASE_URL` take precedence over the saved selection.
41+
42+
Without a saved selection or explicit backend, Cortex keeps its existing
43+
`auto` behavior: PostgreSQL first, then SQLite only when no explicit PostgreSQL
44+
target was supplied and the default server is unavailable. An explicitly
45+
configured but unreachable database URL remains an error rather than silently
46+
redirecting writes. Both hosts must use the same configuration root and storage
47+
settings to share memories; this does not merge previously separate stores.
3948

4049
An optional prewarm can download the package before restarting Codex; it is a
4150
startup optimization, not an installation prerequisite:
@@ -86,3 +95,6 @@ authentication and per-user or per-organization isolation, reviewable tool
8695
metadata, domain verification, operational monitoring, and the applicable
8796
privacy and legal material. None of those remote-deployment claims are made by
8897
this local package.
98+
99+
See [shared memory and decisions](shared-host-memory.md) for the common storage
100+
and ADR-root contract, opt-in authoring, and the limits of the handoff tests.

‎docs/shared-host-memory.md‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Shared memory and decisions across Claude and Codex
2+
3+
## What this change establishes
4+
5+
Claude and Codex can connect independent Cortex MCP processes to the same
6+
storage and decision files. The direct console entry point now applies the
7+
saved backend selection before importing memory settings, matching the Claude
8+
launcher (ADR-0505). Previously, a saved SQLite selection could be ignored by
9+
the direct entry point: its automatic selection could choose a reachable local
10+
PostgreSQL server instead.
11+
12+
This is a source-checkout change. Installing the currently published package
13+
does not establish that this fix is present; it must be released first.
14+
15+
## Shared configuration contract
16+
17+
Both processes need the same backend and physical storage location. For an
18+
isolated SQLite evaluation, set these variables identically in both launches:
19+
20+
```sh
21+
export CORTEX_CLAUDE_DIR=/absolute/path/to/disposable-cortex
22+
export CORTEX_MEMORY_STORE_BACKEND=sqlite
23+
export CORTEX_MEMORY_DB_PATH="$CORTEX_CLAUDE_DIR/memory.db"
24+
export CORTEX_MEMORY_SQLITE_FALLBACK_PATH="$CORTEX_CLAUDE_DIR/memory.db"
25+
```
26+
27+
These paths are examples for disposable data. This workflow neither migrates
28+
nor merges existing databases. PostgreSQL deployments must likewise use the
29+
same explicit database target; cross-process PostgreSQL handoff is not covered
30+
by the new SQLite tests.
31+
32+
Backend precedence remains: explicit `CORTEX_MEMORY_STORE_BACKEND`, then
33+
`CORTEX_BACKEND`, then a non-empty database URL, then the saved
34+
`methodology/backend.json` selection. Without these, existing automatic
35+
selection remains unchanged. An explicit unreachable PostgreSQL target must
36+
not silently redirect writes to SQLite.
37+
38+
## ADRs also need shared files
39+
40+
A shared database alone does not share project decision records. Project-mode
41+
`wiki_adr` stores files under the supplied absolute `project_root`, updates the
42+
project decision index, and returns `memory_sync: project-files-only`
43+
(ADR-0056). Both clients must pass the same physical project root with a valid
44+
`wiki/manifest.json`. Matching project labels in separate worktrees do not
45+
make their files shared.
46+
47+
Use exact ADR identifiers for handoff. A missing or stale index fails closed;
48+
initialize or repair it with `wiki_reindex` before relying on exact lookup.
49+
For global wiki operations, the shared `CORTEX_CLAUDE_DIR` also determines the
50+
wiki root.
51+
52+
## Tool permissions
53+
54+
The bundled Codex plugin keeps its ten-tool `lean` profile (ADR-0693).
55+
It can recall a project ADR but cannot create one. An explicitly configured
56+
`--profile full` MCP process can author decisions. This exposes the full tool
57+
surface; it is not a narrowly scoped decision-writing profile.
58+
59+
A minimal evaluation is:
60+
61+
1. A full-profile client creates a project ADR with `wiki_adr`.
62+
2. A second client recalls its exact identifier using the same `project_root`.
63+
3. Another full-profile client writes a follow-up ADR that cites the first and
64+
describes the observed outcome.
65+
4. Both clients retrieve the follow-up and its reference.
66+
67+
The follow-up reference is explicit text. These tests do not establish
68+
automatic outcome attribution, automatic conflict resolution, or equivalent
69+
Claude and Codex lifecycle hooks.
70+
71+
## Reproducible checks
72+
73+
The added tests use disposable roots and independent processes:
74+
75+
- `tests_py/test_shared_backend_bootstrap.py`: saved selection and explicit
76+
override precedence at direct entry-point startup. Before the fix, four of
77+
nine cases failed; after the fix all nine pass.
78+
- `tests_py/test_shared_host_decisions.py`: actual MCP calls for full-to-lean
79+
exact ADR handoff, denied lean authoring, linked follow-up, simultaneous ADR
80+
allocation, and separation between two explicit project roots.
81+
- `tests_py/infrastructure/test_shared_host_store.py`: independent SQLite
82+
processes observe acknowledged writes, retain distinct concurrent records,
83+
and read persisted content after reopening.
84+
85+
Run these with the repository Python environment and
86+
`CORTEX_MEMORY_STORE_BACKEND=sqlite`. The storage test bypasses embedding and
87+
semantic retrieval; its assertions concern committed rows. The MCP clients
88+
are test clients, not installed Claude or Codex applications. Project-root
89+
separation is a routing check, not an authorization boundary.
90+
91+
### Test liveness policy
92+
93+
The test harness uses 30-second bootstrap and protocol deadlines, a 90-second
94+
MCP scenario deadline, and 5 seconds per shutdown stage. These are engineering
95+
time budgets to fail stalled tests and clean up children before the existing
96+
300-second pytest ceiling, not algorithm constants or latency benchmarks.
97+
98+
Validation on the local macOS checkout: all 12 new tests and 47 related
99+
backend, launcher, profile, and ADR tests passed. Ruff lint, formatting,
100+
and the craftsmanship gate passed. The store protocol test uses POSIX pipe
101+
selection; these results do not establish Windows coverage. Independent review
102+
approved the change after subprocess deadlines and cleanup were added.
103+
104+
## Next acceptance step
105+
106+
Run the released or explicitly checkout-backed server through both installed
107+
hosts against a disposable store. Verify the configured roots, available
108+
tools, decision creation, exact retrieval, and outcome handoff. Only then
109+
assess packaging changes and a public directory submission. No live host
110+
configuration, personal memory data, or marketplace listing is changed here.

‎mcp_server/__main__.py‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,20 @@
66
python -m mcp_server
77
"""
88

9+
# ruff: noqa: E402 — backend selection must precede settings-dependent imports.
10+
# File-wide: the resolver call requires the same exemption for 14 existing imports.
11+
912
from __future__ import annotations
1013

14+
import os
1115
import signal
1216
import sys
1317

14-
# source: ADR-0093
18+
from mcp_server.infrastructure.backend_marker import apply_backend_resolution
19+
20+
# source: ADR-0505 — direct console/module launches must resolve the saved
21+
# backend before registry imports instantiate MemorySettings, just as launcher.py.
22+
apply_backend_resolution(os.environ)
1523

1624

1725
try: # pragma: no cover — defensive; sentence-transformers is mandatory
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
"""Disposable process for shared-store tests; no embeddings or model calls.
2+
3+
source: docs/shared-host-memory.md, exact storage visibility contract.
4+
"""
5+
6+
import json
7+
import os
8+
import sys
9+
10+
from mcp_server.infrastructure.memory_config import get_memory_settings
11+
from mcp_server.infrastructure.memory_store import get_shared_store, reset_shared_store
12+
13+
14+
def main():
15+
settings = get_memory_settings()
16+
store = get_shared_store(settings.DB_PATH, settings.EMBEDDING_DIM)
17+
print(json.dumps({"pid": os.getpid(), "backend": type(store).__name__}), flush=True)
18+
try:
19+
for line in sys.stdin:
20+
request = json.loads(line)
21+
if request["operation"] == "write":
22+
result = store.insert_memory(
23+
{
24+
"content": request["content"],
25+
"domain": "shared-host-test",
26+
"source": request["source"],
27+
}
28+
)
29+
else:
30+
result = store.get_memory(request["id"])
31+
if result:
32+
result = {k: result.get(k) for k in ("id", "content", "source")}
33+
print(json.dumps(result), flush=True)
34+
finally:
35+
reset_shared_store()
36+
37+
38+
if __name__ == "__main__":
39+
main()
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
"""Independent host processes observe the same committed SQLite memories.
2+
3+
source: docs/shared-host-memory.md. This tests persistence, not semantic recall
4+
or automatic host capture; project ADR isolation is tested through real MCP.
5+
"""
6+
7+
from concurrent.futures import ThreadPoolExecutor
8+
from contextlib import contextmanager
9+
import json
10+
import os
11+
from pathlib import Path
12+
import select
13+
import subprocess
14+
import sys
15+
import tempfile
16+
import time
17+
18+
19+
REPO = Path(__file__).resolve().parents[2]
20+
21+
22+
# Test liveness/cleanup budgets, not memory-store performance requirements.
23+
# source: docs/shared-host-memory.md — test liveness budget policy.
24+
PROTOCOL_TIMEOUT = 30
25+
# source: docs/shared-host-memory.md — test liveness budget policy.
26+
CLEANUP_TIMEOUT = 5
27+
28+
29+
def _readline(process):
30+
deadline = time.monotonic() + PROTOCOL_TIMEOUT
31+
line = bytearray()
32+
while not line.endswith(b"\n"):
33+
remaining = deadline - time.monotonic()
34+
ready, _, _ = select.select([process.stdout], [], [], max(0, remaining))
35+
assert ready and remaining > 0, "shared-store worker response timed out"
36+
chunk = os.read(process.stdout.fileno(), 1)
37+
assert chunk, "shared-store worker exited without acknowledgement"
38+
line.extend(chunk)
39+
return json.loads(line)
40+
41+
42+
def _exchange(process, payload):
43+
data = (json.dumps(payload) + "\n").encode()
44+
deadline = time.monotonic() + PROTOCOL_TIMEOUT
45+
while data:
46+
remaining = deadline - time.monotonic()
47+
_, ready, _ = select.select([], [process.stdin], [], max(0, remaining))
48+
assert ready and remaining > 0, "shared-store worker request timed out"
49+
data = data[os.write(process.stdin.fileno(), data) :]
50+
return _readline(process)
51+
52+
53+
def _stop(process):
54+
process.stdin.close()
55+
try:
56+
process.wait(timeout=CLEANUP_TIMEOUT)
57+
except subprocess.TimeoutExpired:
58+
process.terminate()
59+
try:
60+
process.wait(timeout=CLEANUP_TIMEOUT)
61+
except subprocess.TimeoutExpired:
62+
process.kill()
63+
process.wait(timeout=CLEANUP_TIMEOUT)
64+
finally:
65+
process.stdout.close()
66+
67+
68+
def _environment(root):
69+
env = {k: v for k, v in os.environ.items() if not k.startswith("CORTEX_")}
70+
env.pop("DATABASE_URL", None)
71+
database = str(root / "memory.db")
72+
env.update(
73+
CORTEX_CLAUDE_DIR=str(root),
74+
CORTEX_MEMORY_STORE_BACKEND="sqlite",
75+
CORTEX_MEMORY_DB_PATH=database,
76+
CORTEX_MEMORY_SQLITE_FALLBACK_PATH=database,
77+
)
78+
return env
79+
80+
81+
@contextmanager
82+
def _worker(root):
83+
with tempfile.TemporaryFile(mode="w+") as errors:
84+
process = subprocess.Popen(
85+
[sys.executable, "-m", "tests_py.fixtures.shared_host_store_worker"],
86+
cwd=REPO,
87+
env=_environment(root),
88+
stdin=subprocess.PIPE,
89+
stdout=subprocess.PIPE,
90+
stderr=errors,
91+
bufsize=0,
92+
)
93+
try:
94+
yield process, _readline(process)
95+
finally:
96+
failed = sys.exc_info()[0] is not None
97+
_stop(process)
98+
if not failed:
99+
errors.seek(0)
100+
assert process.returncode == 0, errors.read()
101+
102+
103+
def _concurrent_writes(claude, codex):
104+
with ThreadPoolExecutor(max_workers=2) as pool:
105+
writes = [
106+
pool.submit(
107+
_exchange,
108+
process,
109+
{
110+
"operation": "write",
111+
"content": f"ADR-0056 outcome from {host}",
112+
"source": host,
113+
},
114+
)
115+
for process, host in [
116+
(claude, "claude-test"),
117+
(codex, "codex-test"),
118+
]
119+
]
120+
return [future.result(timeout=PROTOCOL_TIMEOUT) for future in writes]
121+
122+
123+
def test_cross_process_handoff_and_concurrent_acknowledged_writes(tmp_path):
124+
with _worker(tmp_path) as (claude, first):
125+
with _worker(tmp_path) as (codex, second):
126+
assert first["pid"] != second["pid"]
127+
assert first["backend"] == second["backend"] == "SqliteMemoryStore"
128+
original = _exchange(
129+
claude,
130+
{
131+
"operation": "write",
132+
"content": "ADR-0056: adopt shared memory",
133+
"source": "claude-test",
134+
},
135+
)
136+
seen = _exchange(codex, {"operation": "read", "id": original})
137+
assert seen["content"] == "ADR-0056: adopt shared memory"
138+
assert seen["source"] == "claude-test"
139+
identifiers = _concurrent_writes(claude, codex)
140+
assert len({original, *identifiers}) == 3
141+
for identifier, host in zip(
142+
identifiers, ["claude-test", "codex-test"], strict=True
143+
):
144+
for process in (claude, codex):
145+
record = _exchange(process, {"operation": "read", "id": identifier})
146+
assert record["content"] == f"ADR-0056 outcome from {host}"
147+
assert record["source"] == host
148+
with _worker(tmp_path) as (reopened, _):
149+
assert _exchange(reopened, {"operation": "read", "id": original}) == seen

0 commit comments

Comments
 (0)