Skip to content

Commit f5cfa31

Browse files
committed
fix: round-3 council findings (seq=0 m0 gap, dreamer archived undercount, dashboard path-norm)
Third blind speed bug-hunt (2/member, HEAD a579235). 3 real fixes, 3 verified accepted-by-design (documented), 2 dismissed. HIGH #3 - seq=0 first-compartment invisible (OpenCode parity gap): getMaxCompartmentSeq returned 0 for an empty session, same as the first real compartment (sequence 0). So mustMaterialize saw 0===0 (no refold into m[0]) and m[1]'s filter excluded it - the first compartment was missing from history until a SECOND one published. The incremental publish path does not clear the m[0] cache, so it was live. Ported Pi's proven fix: empty session now reports EMPTY_MAX_COMPARTMENT_SEQ (-1); normalizeCachedMaxCompartmentSeq reinterprets a legacy stored 0 as empty only when the live session has zero compartments. +1 regression test; 2 empty-session assertions updated 0 -> -1. MED #8 - dreamer archived count under-reported (runner.ts): archivedIds and mergedIds are disjoint in getMemoryCountsByStatus (a superseded memory goes to mergedIds, never archivedIds), so subtracting merged again double-counted and drove archived toward zero. Dropped the term. Feeds the dreamer memory-changes drill-down. MED #2 - dashboard key-files & smart-notes path mismatch (db.rs): both used literal WHERE project_path = ? while the frontend passes resolved identity, so symlinked/legacy/non-canonical paths returned 0 rows (same class as the #87 memories fix). Generalized resolve_paths_for_memory_filter -> resolve_paths_for_table_filter and routed get_project_key_files + get_smart_notes through it (IN-query over all paths normalizing to the identity). Verified accepted-by-design -> AUDIT-KNOWN-ISSUES A9/A10 + A8 addendum: #1 stale key-files render one pass (cache-stability design), #4 m[0] no reselect/reorder after non-additive mutation (supersede-delta), #6 materialize boundary expiry cutoff (A8), #7 merge status-promotion lag (supersede-delta). Dismissed: session_facts_version missing-column (column exists), Pi deferred-materialization (re-flag of A6). plugin 1712/0 (+1), Pi 403/0, dashboard cargo + 106 Rust tests, tsc + biome clean, dist rebuilt.
1 parent bc58cd5 commit f5cfa31

5 files changed

Lines changed: 200 additions & 33 deletions

File tree

‎docs/AUDIT-KNOWN-ISSUES.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,50 @@ the prompt cache (the exact bug fixed by freezing the cutoff). TTL precision is
117117
sacrificed for byte-stable replay; the memory drops on the next cache-busting
118118
pass. Do not add an expiry trigger to `mustMaterialize`.
119119

120+
### A9. Key-files render last-known-good content for one pass when disk drifts
121+
122+
`key-files-block.ts` — the render loop emits every stored `project_key_files`
123+
row regardless of its `staleReason` (or a freshness check that just detected
124+
drift); the freshness check only skips re-*checking* an already-stale row, never
125+
re-*rendering* it, and the stale flag is flushed AFTER the render. So a file that
126+
was deleted or content-drifted on disk still renders its last-known-good stored
127+
content for one pass, then corrects on the next key-file refresh. This is the
128+
documented design (ARCHITECTURE.md: "Injection renders only stored DB content.
129+
Disk drift queues a CAS-protected stale update and never changes the bytes
130+
already being rendered; stale writes do not bump the version because they do not
131+
affect `<key-files>` output."). Skipping stale rows at render time would change
132+
the rendered bytes every time a pinned file changed mid-session — the exact m[0]
133+
cache bust the design avoids. Accepted.
134+
135+
### A10. ctx_memory non-additive mutations don't reselect or reorder m[0]
136+
137+
`tools.ts` — `delete`/`update`/`archive`/`merge` route through
138+
`queueMemoryMutation` (supersede-delta m[1] rows), NOT a `project_memory_epoch`
139+
bump, so they deliberately do not re-materialize m[0]. Two consequences are
140+
within this accepted tradeoff:
141+
- **Freed budget isn't reclaimed immediately.** After an archive/delete frees
142+
injection budget, a memory that was previously trimmed out of m[0] is not
143+
pulled back in until the next natural m[0] materialization. Reclaiming it
144+
would require re-materializing m[0] = a cache bust on a routine mutation.
145+
- **Status-promotion reordering lags.** `merge` can promote the canonical to
146+
`permanent` (if any source was permanent); since selection is permanent-first
147+
only under budget pressure, the reorder doesn't take effect in m[0] until the
148+
next materialization. The canonical stays present throughout; only its
149+
trim-priority is briefly stale, and only under budget pressure.
150+
Both self-heal on the next hard bust. This is the supersede-delta design: m[1]
151+
deltas express add/remove/update, not reselection/reordering, precisely so
152+
routine memory edits don't bust the m[0] prefix. (The dashboard differs — it
153+
bumps the epoch on status changes — because an external editor cannot otherwise
154+
signal a running session; see the dashboard's own path.) Accepted.
155+
156+
> **A8 addendum (materialize-boundary cutoff):** within a single `materializeM0`,
157+
> the m[0] baseline memories are read with a `Date.now()` expiry cutoff a sub-ms
158+
> before `materializedAt` is stamped, so a memory expiring in that window can
159+
> render in m[0] one cycle past its TTL. This is the same TTL-precision-at-the-
160+
> boundary tradeoff A8 already accepts (m[0] stays internally consistent and the
161+
> memory drops on the next materialization); m[1]'s `id > maxMemoryId` filter
162+
> prevents any double-render. Not a separate bug.
163+
120164
## Growing-data factors (bounded or slow; future cleanup)
121165

122166
These are intentionally listed so a future cleanup/maintenance task can address

‎packages/dashboard/src-tauri/src/db.rs‎

Lines changed: 58 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1676,27 +1676,38 @@ fn resolve_paths_for_memory_filter(
16761676
conn: &Connection,
16771677
project_filter: &str,
16781678
) -> Result<Vec<String>, rusqlite::Error> {
1679-
// Pull every distinct stored memory project_path. The set is small (one
1680-
// per project that ever wrote memories), so the cost is bounded by the
1681-
// number of distinct projects, not memories.
1682-
let mut stmt = conn.prepare("SELECT DISTINCT project_path FROM memories")?;
1679+
resolve_paths_for_table_filter(conn, "memories", project_filter)
1680+
}
1681+
1682+
/// Resolve a project-identity filter (`git:<sha>` / `dir:<hash>`) to the set of
1683+
/// raw `project_path` values stored in `table` that normalize to the same
1684+
/// identity. The plugin writes rows under the resolved identity, but legacy
1685+
/// rows, symlinked, or non-canonical paths may be stored differently — so a
1686+
/// literal `WHERE project_path = ?` misses them. This mirrors the memories
1687+
/// path-resolution so key-files and smart-notes group the same way the Memories
1688+
/// tab does. Falls back to the raw filter when nothing matches (legacy behavior).
1689+
fn resolve_paths_for_table_filter(
1690+
conn: &Connection,
1691+
table: &str,
1692+
project_filter: &str,
1693+
) -> Result<Vec<String>, rusqlite::Error> {
1694+
// `table` is a fixed internal literal (never user input), so interpolating
1695+
// it into the DISTINCT query is safe. The set is small (one row per project
1696+
// that ever wrote to the table), bounded by project count, not row count.
1697+
let mut stmt = conn.prepare(&format!("SELECT DISTINCT project_path FROM {table}"))?;
16831698
let all_paths: Vec<String> = stmt
16841699
.query_map([], |row| row.get::<_, String>(0))?
16851700
.collect::<Result<Vec<_>, _>>()?;
16861701

1687-
// Resolve each stored path to an identity and keep the ones that match
1688-
// the requested filter. `resolve_project_identity` caches results so
1689-
// repeated calls are cheap.
16901702
let normalized_filter = normalize_stored_project_path(project_filter);
16911703
let mut matched: Vec<String> = all_paths
16921704
.into_iter()
16931705
.filter(|path| normalize_stored_project_path(path) == normalized_filter)
16941706
.collect();
16951707

16961708
if matched.is_empty() {
1697-
// No stored path resolves to this identity. Treat the incoming value
1698-
// as a legacy raw path filter for backward compatibility — same as
1699-
// pre-fix behavior, which only worked when frontend sent raw paths.
1709+
// No stored path resolves to this identity. Treat the incoming value as
1710+
// a legacy raw path filter for backward compatibility.
17001711
matched.push(project_filter.to_string());
17011712
}
17021713
Ok(matched)
@@ -2866,21 +2877,35 @@ pub fn get_project_key_files(
28662877
conn: &Connection,
28672878
project_path: &str,
28682879
) -> Result<Vec<KeyFileRow>, rusqlite::Error> {
2869-
let version: i64 = conn
2870-
.query_row(
2871-
"SELECT version FROM project_key_files_version WHERE project_path = ?1",
2872-
[project_path],
2880+
// Resolve the identity filter to the set of stored project_path values so
2881+
// symlinked / non-canonical / legacy raw paths still match (mirrors the
2882+
// memories path resolution).
2883+
let paths = resolve_paths_for_table_filter(conn, "project_key_files", project_path)?;
2884+
if paths.is_empty() {
2885+
return Ok(Vec::new());
2886+
}
2887+
let version: i64 = {
2888+
let placeholders = build_in_placeholders(paths.len(), 1);
2889+
let params = rusqlite::params_from_iter(paths.iter());
2890+
conn.query_row(
2891+
&format!(
2892+
"SELECT COALESCE(MAX(version), 0) FROM project_key_files_version \
2893+
WHERE project_path IN ({placeholders})"
2894+
),
2895+
params,
28732896
|row| row.get(0),
28742897
)
2875-
.unwrap_or(0);
2876-
let mut stmt = conn.prepare(
2898+
.unwrap_or(0)
2899+
};
2900+
let placeholders = build_in_placeholders(paths.len(), 1);
2901+
let mut stmt = conn.prepare(&format!(
28772902
"SELECT project_path, path, content, content_hash, local_token_estimate,
28782903
generated_at, generated_by_model, generation_config_hash, stale_reason
28792904
FROM project_key_files
2880-
WHERE project_path = ?1
2881-
ORDER BY generated_at DESC, path ASC",
2882-
)?;
2883-
let rows = stmt.query_map([project_path], |row| {
2905+
WHERE project_path IN ({placeholders})
2906+
ORDER BY generated_at DESC, path ASC"
2907+
))?;
2908+
let rows = stmt.query_map(rusqlite::params_from_iter(paths.iter()), |row| {
28842909
Ok(KeyFileRow {
28852910
project_path: row.get(0)?,
28862911
path: row.get(1)?,
@@ -3401,14 +3426,22 @@ pub fn get_smart_notes(
34013426
conn: &Connection,
34023427
project_path: &str,
34033428
) -> Result<Vec<Note>, rusqlite::Error> {
3404-
let mut stmt = conn.prepare(
3429+
// Smart notes are stored under the resolved project identity; resolve the
3430+
// filter to all stored paths that normalize to it so symlinked / legacy raw
3431+
// paths still surface (mirrors the memories + key-files path resolution).
3432+
let paths = resolve_paths_for_table_filter(conn, "notes", project_path)?;
3433+
if paths.is_empty() {
3434+
return Ok(Vec::new());
3435+
}
3436+
let placeholders = build_in_placeholders(paths.len(), 1);
3437+
let mut stmt = conn.prepare(&format!(
34053438
"SELECT id, type, status, content, session_id, project_path, surface_condition,
34063439
created_at, updated_at, last_checked_at, ready_at, ready_reason
34073440
FROM notes
3408-
WHERE project_path = ?1 AND type = 'smart' AND status != 'dismissed'
3409-
ORDER BY created_at ASC",
3410-
)?;
3411-
let rows = stmt.query_map(rusqlite::params![project_path], |row| {
3441+
WHERE project_path IN ({placeholders}) AND type = 'smart' AND status != 'dismissed'
3442+
ORDER BY created_at ASC"
3443+
))?;
3444+
let rows = stmt.query_map(rusqlite::params_from_iter(paths.iter()), |row| {
34123445
Ok(Note {
34133446
id: row.get(0)?,
34143447
note_type: row.get(1)?,

‎packages/plugin/src/features/magic-context/dreamer/runner.ts‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1015,10 +1015,12 @@ export async function runDream(args: {
10151015
const memoryChanges = {
10161016
written: countNewIds(memoryCountsBefore.ids, memoryCountsAfter.ids),
10171017
deleted: countNewIds(memoryCountsAfter.ids, memoryCountsBefore.ids),
1018-
archived: Math.max(
1019-
0,
1020-
countNewIds(memoryCountsBefore.archivedIds, memoryCountsAfter.archivedIds) - merged,
1021-
),
1018+
// archivedIds already EXCLUDES merged/superseded rows — getMemoryCountsByStatus
1019+
// routes a memory with superseded_by_memory_id into mergedIds and never into
1020+
// archivedIds (the two sets are disjoint). So the archived delta is already
1021+
// merge-free; subtracting `merged` again double-counted and under-reported
1022+
// archived (often to zero).
1023+
archived: countNewIds(memoryCountsBefore.archivedIds, memoryCountsAfter.archivedIds),
10221024
merged,
10231025
};
10241026
const persistedMemoryChanges = Object.values(memoryChanges).some((value) => value > 0)

‎packages/plugin/src/hooks/magic-context/inject-compartments.test.ts‎

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -483,6 +483,51 @@ describe("m[0]/m[1] materialization", () => {
483483
).toBe("max_compartment_seq");
484484
});
485485

486+
it("mustMaterialize detects the FIRST compartment (sequence 0) published after an empty materialize", () => {
487+
// Regression for the seq=0 collision: an empty session materialized m[0]
488+
// with maxCompartmentSeq sentinel; publishing the first compartment at
489+
// sequence 0 must be seen as a CHANGE so it folds into m[0]. The old code
490+
// stored 0 for "empty" too, so 0 === 0 left the first compartment invisible
491+
// (never refolded into m[0], and excluded from m[1] by `sequence > 0`).
492+
db = makeDb();
493+
const projectDirectory = makeProjectDir();
494+
materializeM0({
495+
db,
496+
sessionId: SESSION_ID,
497+
state: readStateFromMeta(),
498+
projectPath: PROJECT_PATH,
499+
projectDirectory,
500+
});
501+
const state = readStateFromMeta();
502+
// First compartment for this session — sequence 0.
503+
replaceAllCompartmentState(
504+
db,
505+
SESSION_ID,
506+
[
507+
{
508+
sequence: 0,
509+
startMessage: 1,
510+
endMessage: 1,
511+
startMessageId: "m1",
512+
endMessageId: "m1",
513+
title: "First",
514+
content: "First summary",
515+
},
516+
],
517+
[],
518+
);
519+
520+
expect(
521+
mustMaterialize({
522+
db,
523+
sessionId: SESSION_ID,
524+
state,
525+
projectPath: PROJECT_PATH,
526+
projectDirectory,
527+
}).reason,
528+
).toBe("max_compartment_seq");
529+
});
530+
486531
it("mustMaterialize detects a new m0_mutation_log entry by monotonic id", () => {
487532
db = makeDb();
488533
const projectDirectory = makeProjectDir();
@@ -593,7 +638,9 @@ describe("m[0]/m[1] materialization", () => {
593638
expect(Buffer.from(row.cached_m1_bytes as Buffer).toString("utf8")).toBe(result.m1Text);
594639
expect(row.cached_m0_project_memory_epoch).toBe(0);
595640
expect(row.cached_m0_project_user_profile_version).toBe(0);
596-
expect(row.cached_m0_max_compartment_seq).toBe(0);
641+
// Empty session: maxCompartmentSeq is the empty sentinel (-1), not 0, so
642+
// the first compartment (sequence 0) is correctly detected as a change.
643+
expect(row.cached_m0_max_compartment_seq).toBe(-1);
597644
expect(row.cached_m0_max_memory_id).toBe(0);
598645
expect(row.cached_m0_max_mutation_id).toBe(0);
599646
expect(row.cached_m0_max_memory_mutation_id).toBe(0);
@@ -774,7 +821,8 @@ describe("m[0]/m[1] materialization", () => {
774821
expect(state.cachedM0Bytes).toBeInstanceOf(Buffer);
775822
expect(state.cachedM1Bytes).toBeInstanceOf(Buffer);
776823
expect(state.cachedM0ProjectMemoryEpoch).toBe(0);
777-
expect(state.cachedM0MaxCompartmentSeq).toBe(0);
824+
// Empty session: maxCompartmentSeq is the empty sentinel (-1), not 0.
825+
expect(state.cachedM0MaxCompartmentSeq).toBe(-1);
778826
expect(state.cachedM0MaxMutationId).toBe(0);
779827
expect(state.cachedM0MaxMemoryMutationId).toBe(0);
780828
expect(state.cachedM0ProjectDocsHash).toBe("");

‎packages/plugin/src/hooks/magic-context/inject-compartments.ts‎

Lines changed: 42 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -629,15 +629,42 @@ function numberFromRow(row: unknown, key: string): number {
629629
return typeof value === "number" && Number.isFinite(value) ? value : 0;
630630
}
631631

632+
// Sentinel for "no compartments yet". Must be < 0 so it is distinct from the
633+
// first real compartment, which is sequence 0. readNewCompartments filters
634+
// `sequence > maxSeq`, so an empty session must report -1 for the first
635+
// compartment (seq 0) to be included in m[1]; and `mustMaterialize` must see
636+
// -1 (empty) -> 0 (first publish) as a CHANGE so the first compartment is
637+
// folded into m[0]. Using 0 for "empty" made the first compartment invisible
638+
// (0 === 0 = no refold, and `sequence > 0` excluded it). Mirrors Pi's
639+
// EMPTY_MAX_COMPARTMENT_SEQ.
640+
const EMPTY_MAX_COMPARTMENT_SEQ = -1;
641+
632642
function getMaxCompartmentSeq(db: Database, sessionId: string): number {
633643
const row = cachedStatement(
634644
maxCompartmentSeqStatements,
635645
db,
636-
"SELECT COALESCE(MAX(sequence), 0) AS s FROM compartments WHERE session_id = ?",
646+
"SELECT COALESCE(MAX(sequence), -1) AS s FROM compartments WHERE session_id = ?",
637647
).get(sessionId);
648+
// The query returns -1 (EMPTY_MAX_COMPARTMENT_SEQ) for an empty session and
649+
// the real max sequence (>= 0) otherwise.
638650
return numberFromRow(row, "s");
639651
}
640652

653+
/**
654+
* Reinterpret a legacy persisted `0` watermark as "empty" when — and only
655+
* when — the session genuinely has no compartments. Builds before the seq=0
656+
* fix persisted `0` for empty sessions; comparing that against the new -1
657+
* sentinel would force one spurious materialize. Guarded against the live
658+
* compartment count so a real seq-0 compartment keeps `0` as its watermark.
659+
* Mirrors Pi's normalizeCachedMaxCompartmentSeq.
660+
*/
661+
function normalizeCachedMaxCompartmentSeq(db: Database, sessionId: string, stored: number): number {
662+
if (stored === 0 && getMaxCompartmentSeq(db, sessionId) === EMPTY_MAX_COMPARTMENT_SEQ) {
663+
return EMPTY_MAX_COMPARTMENT_SEQ;
664+
}
665+
return stored;
666+
}
667+
641668
function getMaxMemoryId(db: Database, projectPath: string | undefined): number {
642669
if (!projectPath) return 0;
643670
const row = cachedStatement(
@@ -736,7 +763,20 @@ export function mustMaterialize(args: {
736763
if (args.state.cachedM0ProjectUserProfileVersion !== current.projectUserProfileVersion) {
737764
return { value: true, reason: "project_user_profile_version" };
738765
}
739-
if (args.state.cachedM0MaxCompartmentSeq !== current.maxCompartmentSeq) {
766+
// Normalize a legacy stored 0 to the empty sentinel so an empty session whose
767+
// cache predates the seq=0 fix doesn't compare 0 (legacy empty) against the
768+
// first real compartment (seq 0) incorrectly — and so an empty→first-publish
769+
// transition (-1 → 0) is correctly seen as a CHANGE that folds the first
770+
// compartment into m[0].
771+
const cachedMaxSeq =
772+
args.state.cachedM0MaxCompartmentSeq === null
773+
? null
774+
: normalizeCachedMaxCompartmentSeq(
775+
args.db,
776+
args.sessionId,
777+
args.state.cachedM0MaxCompartmentSeq,
778+
);
779+
if (cachedMaxSeq !== current.maxCompartmentSeq) {
740780
return { value: true, reason: "max_compartment_seq" };
741781
}
742782
// NOTE: maxMemoryId is deliberately NOT a materialization trigger. New

0 commit comments

Comments
 (0)