Skip to content

Commit db22b16

Browse files
committed
fix: share backend selection across Claude and Codex entry points
1 parent 0fc6bd1 commit db22b16

7 files changed

Lines changed: 575 additions & 6 deletions

File tree

‎docs/codex-plugin.md‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,20 @@ codex plugin add hypermnesia-mcp-codex@cortex-codex-plugins
3131

3232
Restart the ChatGPT desktop app and start a new task so Codex loads the new
3333
plugin components. The plugin uses `uvx`, so `uv` must be available on `PATH`.
34-
The first launch installs both storage drivers. Cortex tries PostgreSQL first
35-
at the configured `DATABASE_URL` (or its local `cortex` default), then falls
36-
back to SQLite only when no explicit PostgreSQL target was supplied and the
37-
default server is unavailable. An explicitly configured but unreachable
38-
`DATABASE_URL` remains an error rather than silently redirecting writes.
34+
The first launch installs both storage drivers. In this source checkout (pending
35+
release), direct MCP startup reads the
36+
same `~/.claude/methodology/backend.json` selection as the Claude launcher
37+
before loading memory settings. `CORTEX_CLAUDE_DIR` relocates that shared
38+
configuration root. Explicit `CORTEX_MEMORY_STORE_BACKEND`, then
39+
`CORTEX_BACKEND`, then a non-empty `DATABASE_URL` or
40+
`CORTEX_MEMORY_DATABASE_URL` take precedence over the saved selection.
41+
42+
Without a saved selection or explicit backend, Cortex keeps its existing
43+
`auto` behavior: PostgreSQL first, then SQLite only when no explicit PostgreSQL
44+
target was supplied and the default server is unavailable. An explicitly
45+
configured but unreachable database URL remains an error rather than silently
46+
redirecting writes. Both hosts must use the same configuration root and storage
47+
settings to share memories; this does not merge previously separate stores.
3948

4049
An optional prewarm can download the package before restarting Codex; it is a
4150
startup optimization, not an installation prerequisite:
@@ -86,3 +95,6 @@ authentication and per-user or per-organization isolation, reviewable tool
8695
metadata, domain verification, operational monitoring, and the applicable
8796
privacy and legal material. None of those remote-deployment claims are made by
8897
this local package.
98+
99+
See [shared memory and decisions](shared-host-memory.md) for the common storage
100+
and ADR-root contract, opt-in authoring, and the limits of the handoff tests.

‎docs/shared-host-memory.md‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Shared memory and decisions across Claude and Codex
2+
3+
## What this change establishes
4+
5+
Claude and Codex can connect independent Cortex MCP processes to the same
6+
storage and decision files. The direct console entry point now applies the
7+
saved backend selection before importing memory settings, matching the Claude
8+
launcher (ADR-0505). Previously, a saved SQLite selection could be ignored by
9+
the direct entry point: its automatic selection could choose a reachable local
10+
PostgreSQL server instead.
11+
12+
This is a source-checkout change. Installing the currently published package
13+
does not establish that this fix is present; it must be released first.
14+
15+
## Shared configuration contract
16+
17+
Both processes need the same backend and physical storage location. For an
18+
isolated SQLite evaluation, set these variables identically in both launches:
19+
20+
```sh
21+
export CORTEX_CLAUDE_DIR=/absolute/path/to/disposable-cortex
22+
export CORTEX_MEMORY_STORE_BACKEND=sqlite
23+
export CORTEX_MEMORY_DB_PATH="$CORTEX_CLAUDE_DIR/memory.db"
24+
export CORTEX_MEMORY_SQLITE_FALLBACK_PATH="$CORTEX_CLAUDE_DIR/memory.db"
25+
```
26+
27+
These paths are examples for disposable data. This workflow neither migrates
28+
nor merges existing databases. PostgreSQL deployments must likewise use the
29+
same explicit database target; cross-process PostgreSQL handoff is not covered
30+
by the new SQLite tests.
31+
32+
Backend precedence remains: explicit `CORTEX_MEMORY_STORE_BACKEND`, then
33+
`CORTEX_BACKEND`, then a non-empty database URL, then the saved
34+
`methodology/backend.json` selection. Without these, existing automatic
35+
selection remains unchanged. An explicit unreachable PostgreSQL target must
36+
not silently redirect writes to SQLite.
37+
38+
## ADRs also need shared files
39+
40+
A shared database alone does not share project decision records. Project-mode
41+
`wiki_adr` stores files under the supplied absolute `project_root`, updates the
42+
project decision index, and returns `memory_sync: project-files-only`
43+
(ADR-0056). Both clients must pass the same physical project root with a valid
44+
`wiki/manifest.json`. Matching project labels in separate worktrees do not
45+
make their files shared.
46+
47+
Use exact ADR identifiers for handoff. A missing or stale index fails closed;
48+
initialize or repair it with `wiki_reindex` before relying on exact lookup.
49+
For global wiki operations, the shared `CORTEX_CLAUDE_DIR` also determines the
50+
wiki root.
51+
52+
## Tool permissions
53+
54+
The bundled Codex plugin keeps its ten-tool `lean` profile (ADR-0693).
55+
It can recall a project ADR but cannot create one. An explicitly configured
56+
`--profile full` MCP process can author decisions. This exposes the full tool
57+
surface; it is not a narrowly scoped decision-writing profile.
58+
59+
A minimal evaluation is:
60+
61+
1. A full-profile client creates a project ADR with `wiki_adr`.
62+
2. A second client recalls its exact identifier using the same `project_root`.
63+
3. Another full-profile client writes a follow-up ADR that cites the first and
64+
describes the observed outcome.
65+
4. Both clients retrieve the follow-up and its reference.
66+
67+
The follow-up reference is explicit text. These tests do not establish
68+
automatic outcome attribution, automatic conflict resolution, or equivalent
69+
Claude and Codex lifecycle hooks.
70+
71+
## Reproducible checks
72+
73+
The added tests use disposable roots and independent processes:
74+
75+
- `tests_py/test_shared_backend_bootstrap.py`: saved selection and explicit
76+
override precedence at direct entry-point startup. Before the fix, four of
77+
nine cases failed; after the fix all nine pass.
78+
- `tests_py/test_shared_host_decisions.py`: actual MCP calls for full-to-lean
79+
exact ADR handoff, denied lean authoring, linked follow-up, simultaneous ADR
80+
allocation, and separation between two explicit project roots.
81+
- `tests_py/infrastructure/test_shared_host_store.py`: independent SQLite
82+
processes observe acknowledged writes, retain distinct concurrent records,
83+
and read persisted content after reopening.
84+
85+
Run these with the repository Python environment and
86+
`CORTEX_MEMORY_STORE_BACKEND=sqlite`. The storage test bypasses embedding and
87+
semantic retrieval; its assertions concern committed rows. The MCP clients
88+
are test clients, not installed Claude or Codex applications. Project-root
89+
separation is a routing check, not an authorization boundary.
90+
91+
### Test liveness policy
92+
93+
The test harness uses 30-second bootstrap and protocol deadlines, a 90-second
94+
MCP scenario deadline, and 5 seconds per shutdown stage. These are engineering
95+
time budgets to fail stalled tests and clean up children before the existing
96+
300-second pytest ceiling, not algorithm constants or latency benchmarks.
97+
98+
Validation on the local macOS checkout: all 12 new tests and 47 related
99+
backend, launcher, profile, and ADR tests passed. Ruff lint, formatting,
100+
and the craftsmanship gate passed. The store protocol test uses POSIX pipe
101+
selection; these results do not establish Windows coverage. Independent review
102+
approved the change after subprocess deadlines and cleanup were added.
103+
104+
## Next acceptance step
105+
106+
Run the released or explicitly checkout-backed server through both installed
107+
hosts against a disposable store. Verify the configured roots, available
108+
tools, decision creation, exact retrieval, and outcome handoff. Only then
109+
assess packaging changes and a public directory submission. No live host
110+
configuration, personal memory data, or marketplace listing is changed here.

‎mcp_server/__main__.py‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,19 @@
66
python -m mcp_server
77
"""
88

9+
# ruff: noqa: E402 — backend selection must precede settings-dependent imports.
10+
911
from __future__ import annotations
1012

13+
import os
1114
import signal
1215
import sys
1316

14-
# source: ADR-0093
17+
from mcp_server.infrastructure.backend_marker import apply_backend_resolution
18+
19+
# source: ADR-0505 — direct console/module launches must resolve the saved
20+
# backend before registry imports instantiate MemorySettings, just as launcher.py.
21+
apply_backend_resolution(os.environ)
1522

1623

1724
try: # pragma: no cover — defensive; sentence-transformers is mandatory
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
"""Disposable process for shared-store tests; no embeddings or model calls.
2+
3+
source: docs/shared-host-memory.md, exact storage visibility contract.
4+
"""
5+
6+
import json
7+
import os
8+
import sys
9+
10+
from mcp_server.infrastructure.memory_config import get_memory_settings
11+
from mcp_server.infrastructure.memory_store import get_shared_store, reset_shared_store
12+
13+
14+
def main():
15+
settings = get_memory_settings()
16+
store = get_shared_store(settings.DB_PATH, settings.EMBEDDING_DIM)
17+
print(json.dumps({"pid": os.getpid(), "backend": type(store).__name__}), flush=True)
18+
try:
19+
for line in sys.stdin:
20+
request = json.loads(line)
21+
if request["operation"] == "write":
22+
result = store.insert_memory(
23+
{
24+
"content": request["content"],
25+
"domain": "shared-host-test",
26+
"source": request["source"],
27+
}
28+
)
29+
else:
30+
result = store.get_memory(request["id"])
31+
if result:
32+
result = {k: result.get(k) for k in ("id", "content", "source")}
33+
print(json.dumps(result), flush=True)
34+
finally:
35+
reset_shared_store()
36+
37+
38+
if __name__ == "__main__":
39+
main()
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
"""Independent host processes observe the same committed SQLite memories.
2+
3+
source: docs/shared-host-memory.md. This tests persistence, not semantic recall
4+
or automatic host capture; project ADR isolation is tested through real MCP.
5+
"""
6+
7+
from concurrent.futures import ThreadPoolExecutor
8+
from contextlib import contextmanager
9+
import json
10+
import os
11+
from pathlib import Path
12+
import select
13+
import subprocess
14+
import sys
15+
import tempfile
16+
import time
17+
18+
19+
REPO = Path(__file__).resolve().parents[2]
20+
21+
22+
# Test liveness/cleanup budgets, not memory-store performance requirements.
23+
# source: docs/shared-host-memory.md — test liveness budget policy.
24+
PROTOCOL_TIMEOUT = 30
25+
# source: docs/shared-host-memory.md — test liveness budget policy.
26+
CLEANUP_TIMEOUT = 5
27+
28+
29+
def _readline(process):
30+
deadline = time.monotonic() + PROTOCOL_TIMEOUT
31+
line = bytearray()
32+
while not line.endswith(b"\n"):
33+
remaining = deadline - time.monotonic()
34+
ready, _, _ = select.select([process.stdout], [], [], max(0, remaining))
35+
assert ready and remaining > 0, "shared-store worker response timed out"
36+
chunk = os.read(process.stdout.fileno(), 1)
37+
assert chunk, "shared-store worker exited without acknowledgement"
38+
line.extend(chunk)
39+
return json.loads(line)
40+
41+
42+
def _exchange(process, payload):
43+
data = (json.dumps(payload) + "\n").encode()
44+
deadline = time.monotonic() + PROTOCOL_TIMEOUT
45+
while data:
46+
remaining = deadline - time.monotonic()
47+
_, ready, _ = select.select([], [process.stdin], [], max(0, remaining))
48+
assert ready and remaining > 0, "shared-store worker request timed out"
49+
data = data[os.write(process.stdin.fileno(), data) :]
50+
return _readline(process)
51+
52+
53+
def _stop(process):
54+
process.stdin.close()
55+
try:
56+
process.wait(timeout=CLEANUP_TIMEOUT)
57+
except subprocess.TimeoutExpired:
58+
process.terminate()
59+
try:
60+
process.wait(timeout=CLEANUP_TIMEOUT)
61+
except subprocess.TimeoutExpired:
62+
process.kill()
63+
process.wait(timeout=CLEANUP_TIMEOUT)
64+
finally:
65+
process.stdout.close()
66+
67+
68+
def _environment(root):
69+
env = {k: v for k, v in os.environ.items() if not k.startswith("CORTEX_")}
70+
env.pop("DATABASE_URL", None)
71+
database = str(root / "memory.db")
72+
env.update(
73+
CORTEX_CLAUDE_DIR=str(root),
74+
CORTEX_MEMORY_STORE_BACKEND="sqlite",
75+
CORTEX_MEMORY_DB_PATH=database,
76+
CORTEX_MEMORY_SQLITE_FALLBACK_PATH=database,
77+
)
78+
return env
79+
80+
81+
@contextmanager
82+
def _worker(root):
83+
with tempfile.TemporaryFile(mode="w+") as errors:
84+
process = subprocess.Popen(
85+
[sys.executable, "-m", "tests_py.fixtures.shared_host_store_worker"],
86+
cwd=REPO,
87+
env=_environment(root),
88+
stdin=subprocess.PIPE,
89+
stdout=subprocess.PIPE,
90+
stderr=errors,
91+
bufsize=0,
92+
)
93+
try:
94+
yield process, _readline(process)
95+
finally:
96+
failed = sys.exc_info()[0] is not None
97+
_stop(process)
98+
if not failed:
99+
errors.seek(0)
100+
assert process.returncode == 0, errors.read()
101+
102+
103+
def _concurrent_writes(claude, codex):
104+
with ThreadPoolExecutor(max_workers=2) as pool:
105+
writes = [
106+
pool.submit(
107+
_exchange,
108+
process,
109+
{
110+
"operation": "write",
111+
"content": f"ADR-0056 outcome from {host}",
112+
"source": host,
113+
},
114+
)
115+
for process, host in [
116+
(claude, "claude-test"),
117+
(codex, "codex-test"),
118+
]
119+
]
120+
return [future.result(timeout=PROTOCOL_TIMEOUT) for future in writes]
121+
122+
123+
def test_cross_process_handoff_and_concurrent_acknowledged_writes(tmp_path):
124+
with _worker(tmp_path) as (claude, first):
125+
with _worker(tmp_path) as (codex, second):
126+
assert first["pid"] != second["pid"]
127+
assert first["backend"] == second["backend"] == "SqliteMemoryStore"
128+
original = _exchange(
129+
claude,
130+
{
131+
"operation": "write",
132+
"content": "ADR-0056: adopt shared memory",
133+
"source": "claude-test",
134+
},
135+
)
136+
seen = _exchange(codex, {"operation": "read", "id": original})
137+
assert seen["content"] == "ADR-0056: adopt shared memory"
138+
assert seen["source"] == "claude-test"
139+
identifiers = _concurrent_writes(claude, codex)
140+
assert len({original, *identifiers}) == 3
141+
for identifier, host in zip(
142+
identifiers, ["claude-test", "codex-test"], strict=True
143+
):
144+
for process in (claude, codex):
145+
record = _exchange(process, {"operation": "read", "id": identifier})
146+
assert record["content"] == f"ADR-0056 outcome from {host}"
147+
assert record["source"] == host
148+
with _worker(tmp_path) as (reopened, _):
149+
assert _exchange(reopened, {"operation": "read", "id": original}) == seen

0 commit comments

Comments
 (0)