Skip to content

Commit 6fb0800

Browse files
committed
docs(audit): schema-version fence test + v26 doc sync + accepted-findings dispositions
#3 (Med) Docs claimed schema v23 while code is at v26. Updated ARCHITECTURE.md (listing v24 historian_runs, v25 pi_stable_id_scheme, v26 memory_mutation_log + cached_m1_bytes) and STRUCTURE.md to v26. Added the safeguard the audit asked for: export LATEST_MIGRATION_VERSION (derived from the MIGRATIONS array) and a schema-version-fence.test.ts asserting LATEST_SUPPORTED_VERSION === LATEST_MIGRATION_VERSION, so forgetting to bump the fence ceiling (a bug the project already hit in v2 work) now fails CI. Documented the remaining accepted/deferred findings in AUDIT-KNOWN-ISSUES.md so future audits don't re-flag them: - A14 #2 vestigial session_facts/plugin_messages (inert; drop gated on min TUI) + code pointer at getSessionFactsVersion - A15 #18 in-session memory mutation eventual-consistency (by design) - A16 #24 session queries not harness-scoped (latent; IDs don't collide) - A17 #26 materializeM0 double budget-guard (observation, no risk) - A18 #16 legacy V5 key-files path (deferred; V6 trust enforcement added earlier) - A19 #13 Pi inject-compartments byte-parity test (guarded by PARITY.md + audits)
1 parent 3a355c2 commit 6fb0800

6 files changed

Lines changed: 91 additions & 2 deletions

File tree

‎ARCHITECTURE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -383,7 +383,7 @@ Magic Context runs in three effective modes depending on `ctx_reduce_enabled` an
383383

384384
**Storage:** Use the SQLite database created by `src/features/magic-context/storage-db.ts` under the cortexkit data directory resolved by `src/shared/data-path.ts` (`~/.local/share/cortexkit/magic-context/context.db` on Linux/macOS, XDG-equivalent on Windows). Legacy OpenCode-plugin-folder DBs are migrated forward on first boot. The same DB is shared cross-harness between OpenCode and Pi; session-scoped tables include a `harness` discriminator (`'opencode'` / `'pi'`) while project-scoped tables (memories, git commits) are shared.
385385

386-
**Schema migrations:** `src/features/magic-context/migrations.ts` declares versioned migrations v1–v23 (`LATEST_SUPPORTED_VERSION = 23` in `storage-db.ts` is the schema-fence ceiling and MUST be bumped with every new migration). Notable: v10 `tool_owner_message_id` (composite tool-tag identity); v11 `todo_synthetic_*` (synthetic-todowrite); v12 orphan `memory_embeddings` cleanup; v13 `pending_compaction_marker_state` (deferred-marker drain); v14 project-scoped key files + version counter; v15 `deferred_execute_state` (boundary execution); v16 context-limit cache sentinels; v17 multi-anchor note-nudge/auto-search JSON storage; v18 `pending_pi_compaction_marker_state`; v19 compartment-state lease table; v20 subagent invocation token accounting; v21 session lifetime work metrics; **v22 the v2.0 cache-architecture foundation (m[0]/m[1] split tables, `project_state` epoch counter, plus per-compartment `p1`–`p4` tier columns, `importance`, `episode_type`, `p1_embedding`, and `legacy` flag); v23 `compartment_events` (historian-extracted causal_incident / trajectory_correction, stored-not-rendered in v2.0).** Migration runner uses `schema_migrations` table with version-ordered execution and sibling-startup race protection (duplicate-insert is tolerated).
386+
**Schema migrations:** `src/features/magic-context/migrations.ts` declares versioned migrations v1–v26 (`LATEST_SUPPORTED_VERSION = 26` in `storage-db.ts` is the schema-fence ceiling and MUST be bumped with every new migration; a unit test — `schema-version-fence.test.ts` — asserts `LATEST_SUPPORTED_VERSION === LATEST_MIGRATION_VERSION` so the two can't drift). Notable: v10 `tool_owner_message_id` (composite tool-tag identity); v11 `todo_synthetic_*` (synthetic-todowrite); v12 orphan `memory_embeddings` cleanup; v13 `pending_compaction_marker_state` (deferred-marker drain); v14 project-scoped key files + version counter; v15 `deferred_execute_state` (boundary execution); v16 context-limit cache sentinels; v17 multi-anchor note-nudge/auto-search JSON storage; v18 `pending_pi_compaction_marker_state`; v19 compartment-state lease table; v20 subagent invocation token accounting; v21 session lifetime work metrics; **v22 the v2.0 cache-architecture foundation (m[0]/m[1] split tables, `project_state` epoch counter, plus per-compartment `p1`–`p4` tier columns, `importance`, `episode_type`, `p1_embedding`, and `legacy` flag); v23 `compartment_events` (historian-extracted causal_incident / trajectory_correction, stored-not-rendered in v2.0); v24 `historian_runs` telemetry (per-run chunk range, compartment/fact/event counts, importance min/max/avg, status + failure reason, FK to `subagent_invocations`); v25 `pi_stable_id_scheme` (Pi stable-id cutover watermark); v26 `memory_mutation_log` + `cached_m1_bytes` (memory supersede-delta — non-additive in-session memory mutations render as an m[1] `<memory-updates>` delta instead of bumping the project epoch, plus the frozen-m[1]-bytes cache column).** Migration runner uses `schema_migrations` table with version-ordered execution and sibling-startup race protection (duplicate-insert is tolerated).
387387

388388
**Harness-aware behavior:** `src/shared/harness.ts` exposes `setHarness()`/`getHarness()` for the runtime to identify itself; production INSERTs into session-scoped tables tag rows with the current harness. Pi-specific session-resolution paths are skipped on OpenCode and vice versa.
389389

‎STRUCTURE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,7 @@
119119
- `src/features/magic-context/memory/memory-migration.ts`: `/ctx-session-upgrade` 9-cat→5-cat memory re-eval (active-only, permanent-safe, epoch-bumping).
120120
- `src/features/magic-context/storage-db.ts`: Create durable storage; run versioned migrations; resolve runtime SQLite backend.
121121
- `src/features/magic-context/storage-meta-persisted.ts`: Read and write per-session persisted scalars and JSON blobs.
122-
- `src/features/magic-context/migrations.ts`: Versioned schema migrations v1–v23 (`LATEST_SUPPORTED_VERSION` in `storage-db.ts` must track the highest).
122+
- `src/features/magic-context/migrations.ts`: Versioned schema migrations v1–v26 (`LATEST_SUPPORTED_VERSION` in `storage-db.ts` must track the highest; `schema-version-fence.test.ts` asserts they stay in lockstep).
123123
- `src/features/magic-context/message-index.ts`: FTS-backed raw-message index for `ctx_search`.
124124
- `src/features/magic-context/search.ts`: Unified retrieval over memories, raw messages, and git commits.
125125

‎docs/AUDIT-KNOWN-ISSUES.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -269,3 +269,62 @@ auditor re-finding one adds no new information.
269269
- **Dashboard token breakdown** is still v1-shaped (no Docs/User-Profile buckets;
270270
reads the legacy `memory_block_cache`) — a display divergence from the TUI
271271
sidebar's v2 breakdown.
272+
273+
### A14. Vestigial `session_facts` / `plugin_messages` tables
274+
275+
Both are documented as retired (facts are promoted memories; the plugin-message
276+
bus is superseded by RPC) but are still created on every DB, migrated, and
277+
deleted in `clearSession`. `getSessionFactsVersion` returns a hard-coded `0` and
278+
the `sessionFactsVersion` branch in `mustMaterialize` is kept inert-safe but
279+
never fires; no module imports `plugin-messages.ts` at runtime. Dropping them is
280+
a schema migration that should be gated on the minimum supported TUI version no
281+
longer polling the old bus — deferred to avoid a migration during active
282+
dogfooding. No correctness impact; the inert branch is a no-op.
283+
284+
### A15. In-session memory mutations have no dedicated cache-bust signal
285+
286+
A non-additive `ctx_memory` mutation (delete/update/archive/merge) queues a
287+
`memory_mutation_log` row but does not itself schedule a cache-busting pass — the
288+
`<memory-updates>` delta recomputes on the next cache-bust triggered by other
289+
work. This is the intended supersede-delta eventual-consistency tradeoff (the
290+
whole point is to NOT hard-bust m[0] on every memory edit). Delete visibility is
291+
therefore delayed until the next natural bust. Accepted by design; see the m[1]
292+
memory-mutation note in ARCHITECTURE.md.
293+
294+
### A16. Session-scoped queries don't include the `harness` discriminator
295+
296+
`harness` columns exist on session-scoped tables, but session-scoped reads key on
297+
`session_id` alone — the code relies on OpenCode and Pi session IDs never
298+
colliding in practice (different id formats). If they ever did collide,
299+
session-scoped state could alias across harnesses. Accepted as latent; revisit
300+
only if a shared ID scheme is introduced.
301+
302+
### A17. `materializeM0` runs two over-budget guards for the same budget
303+
304+
`renderM0` escalates `decayPressureMultiplier` (up to 3×, re-rendering m[0] each
305+
iteration) while `renderDecayedCompartments` independently demotes oldest-first.
306+
Both use the same unified estimator and converge deterministically, so there is
307+
no cache-stability risk — only redundant compute on very large histories. An
308+
observation, not a defect; consolidate only if materialize latency becomes a
309+
concern.
310+
311+
### A18. Legacy V5 key-files path ships alongside active V6
312+
313+
The old per-session `session_meta.key_files` storage + candidate/heuristic
314+
selection remains while the live Dreamer path uses V6 `runKeyFilesTask` +
315+
project-scoped `project_key_files`. Drift-prone duplication in a path-containment
316+
subsystem. Removing/hard-isolating V5 is worthwhile but risky to do casually in a
317+
security-sensitive area; deferred to a focused change. (The V6 validator's
318+
candidate-set + doc-exclusion enforcement was added this round — see the
319+
key-files trust fix.)
320+
321+
### A19. No byte-equivalence parity test for Pi's `inject-compartments-pi`
322+
323+
Pi re-implements parts of m[0]/m[1] rendering (`resolvePiStableId`,
324+
`SYNTH_USER_ID_PREFIX`) while sharing hash/mutation/trim helpers from
325+
`@magic-context/core`. `mustMaterialize` must produce byte-identical m[0]
326+
decisions across harnesses or caches diverge. Parity is currently enforced by
327+
PARITY.md discipline + nine rounds of cross-harness audits rather than an
328+
automated equivalence test. A test feeding identical DB state through both paths
329+
and asserting byte-identical m[0] is a worthwhile future addition; the divergence
330+
risk is documented, not unguarded.

‎packages/plugin/src/features/magic-context/migrations.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -999,6 +999,17 @@ const MIGRATIONS: Migration[] = [
999999
},
10001000
];
10011001

1002+
/**
1003+
* Highest version in the MIGRATIONS array. `LATEST_SUPPORTED_VERSION` in
1004+
* storage-db.ts (the schema-fence ceiling) MUST equal this — a stale ceiling
1005+
* makes the DB refuse to open after the new migration applies (a real bug the
1006+
* project hit during v2 work). A unit test asserts the two stay in lockstep.
1007+
*/
1008+
export const LATEST_MIGRATION_VERSION: number = MIGRATIONS.reduce(
1009+
(max, m) => Math.max(max, m.version),
1010+
0,
1011+
);
1012+
10021013
function ensureMigrationsTable(db: Database): void {
10031014
db.exec(`
10041015
CREATE TABLE IF NOT EXISTS schema_migrations (
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
/// <reference types="bun-types" />
2+
3+
import { describe, expect, it } from "bun:test";
4+
import { LATEST_MIGRATION_VERSION } from "./migrations";
5+
import { LATEST_SUPPORTED_VERSION } from "./storage-db";
6+
7+
// Guards the #1 bug class the project already hit during v2 work: adding a
8+
// migration but forgetting to bump LATEST_SUPPORTED_VERSION (the schema-fence
9+
// ceiling). A stale ceiling makes the DB refuse to open after the new migration
10+
// applies, silently disabling Magic Context. Keep the fence and the migration
11+
// list in lockstep.
12+
describe("schema version fence", () => {
13+
it("LATEST_SUPPORTED_VERSION equals the highest migration version", () => {
14+
expect(LATEST_SUPPORTED_VERSION).toBe(LATEST_MIGRATION_VERSION);
15+
});
16+
});

‎packages/plugin/src/hooks/magic-context/inject-compartments.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -679,6 +679,9 @@ function getMaxMemoryId(db: Database, projectPath: string | undefined): number {
679679
// The m[0] snapshot keeps a sessionFactsVersion field for shape stability, but
680680
// it is pinned to 0 so fact changes never drive m[0] re-materialization —
681681
// rendered bytes no longer depend on session_facts. (Avoids wasted rebuilds.)
682+
// session_facts is a retired table (facts are promoted memories now); this
683+
// branch is kept inert-safe but never fires. Do NOT rewire facts through here.
684+
// See docs/AUDIT-KNOWN-ISSUES.md A14 (vestigial table, drop gated on min TUI).
682685
function getSessionFactsVersion(_db: Database, _sessionId: string): number {
683686
return 0;
684687
}

0 commit comments

Comments
 (0)