Skip to content

Commit c9a06ea

Browse files
committed
fix(pi): round-3 audit — post-commit entry-id ref-map, cutover stamp gating, placeholder GC, schema-fence null-guard
Round-3 blind audit (1 council + 4 Oracles) findings, verified against source: - #2 (HIGH, regression in round-2 Cluster B): sticky reminder / note nudges / auto-search received the PRE-pipeline entryIdByRef, which keys pre-commit objects and misses every commit-cloned (dirty) message. Build a REAL-only post-commit entryIdByRef (no pi-msg-* fallback, so branch-fail correctly degrades) and a separate full-fallback map for placeholder strip. - #6 (MED): cutover scheme stamp was gated on executedWorkThisPass, which is set true even when heuristic cleanup THREW (empty-pending-ops path) — would stamp while the re-drop never ran, leaving resurrected tools in context. Gate on heuristicsExecuted (re-drop ran) when heuristics enabled; executedWorkThisPass when disabled (no drops to redo). Retries cutover on failure, no loop. - #7 (MED): stripped_placeholder_ids grew unbounded (OpenCode prunes on replay). Prune below-boundary ids on cache-busting passes only (storage-only GC), using the carried ref-map; defer passes never mutate persisted replay state. - #9 (MED): buildPiCompactionSummary joined ALL compartment titles (hundreds after recomp/upgrade) — cap to 5 titles + remainder count. - Oracle MED (made live by v25 fence): Pi bootstrap did not null-guard openDatabase(), which FAILS CLOSED by returning null (not throwing) on the schema fence — Pi would crash at runDeferredV22Backfill instead of degrading. Worker commits integrated + audited separately: - 2f94291: Pi m0 fresh-render frozen snapshot (compartments+profile) + mustMaterialize guard - 38e7e7a: tag-transcript setToolOutput throw-fallback + multi-part folded tool_result pairing Verified: plugin 1667, Pi 367, 0 fail; tsc + biome clean both packages. Confirm-pass dismissed 2 council findings as false positives (synthetic-untaggable is tail-only; discovery-gate fires on execute via cacheBustingPass).
1 parent 38e7e7a commit c9a06ea

6 files changed

Lines changed: 261 additions & 25 deletions

File tree

‎packages/pi-plugin/src/context-handler.ts‎

Lines changed: 71 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1785,12 +1785,27 @@ export function registerPiContextHandler(
17851785

17861786
// Stamp the new stable-id scheme ONLY after a successful cutover pass
17871787
// that actually executed (re-tagged + re-dropped under the new scheme).
1788-
// runPipeline returned without throwing = the controlled bust completed;
1789-
// stamping here (not before) means a mid-pass failure leaves the old
1790-
// scheme so the next pass retries the cutover instead of skipping it.
1791-
// Guard on executedWorkThisPass: if the forced execute somehow degraded
1792-
// to no-op, don't stamp (retry next pass).
1793-
if (stableIdSchemeCutover && result.executedWorkThisPass) {
1788+
// Stamp the new scheme only when the cutover's ESSENTIAL work completed.
1789+
// Re-keying tag identity (pi-msg-* → entry-id) orphans the old drop
1790+
// state, so previously-dropped tools resurface as fresh active tags;
1791+
// the cutover is not "done" until heuristic cleanup actually re-dropped
1792+
// them. `executedWorkThisPass` is too loose — it's set true even when
1793+
// the heuristics try-block THREW (the empty-pending-ops path also sets
1794+
// it), which would stamp the scheme while the re-drop never happened,
1795+
// leaving resurrected tools in context permanently. Gate on
1796+
// `heuristicsExecuted` (the re-drop ran without throwing) when heuristics
1797+
// are enabled; when disabled there are no drops to redo, so a completed
1798+
// pipeline (executedWorkThisPass) is sufficient. The forced pass reliably
1799+
// runs heuristics (shouldRunHeuristics is true under the forced
1800+
// execute+materialize), so a genuine throw defers the stamp and the next
1801+
// pass retries the cutover rather than skipping it — no infinite loop
1802+
// unless heuristics throws every pass, which is a separate bug that this
1803+
// gate correctly refuses to paper over.
1804+
const cutoverWorkComplete =
1805+
options.heuristics === undefined
1806+
? result.executedWorkThisPass
1807+
: result.heuristicsExecuted;
1808+
if (stableIdSchemeCutover && cutoverWorkComplete) {
17941809
try {
17951810
updateSessionMeta(options.db, sessionId, {
17961811
piStableIdScheme: PI_STABLE_ID_SCHEME,
@@ -1822,7 +1837,10 @@ export function registerPiContextHandler(
18221837
db: options.db,
18231838
messages: outputMessages,
18241839
entryIds: entryIds ?? null,
1825-
entryIdByRef,
1840+
// Post-commit/post-splice ref-map from runPipeline — the
1841+
// pass-start `entryIdByRef` keys pre-commit objects and misses
1842+
// cloned (dirty) messages, resolving anchors to the wrong id.
1843+
entryIdByRef: result.postCommitEntryIdByRef,
18261844
hasRecentReduceCall: hasRecentPiCtxReduceExecution(sessionId),
18271845
isCacheBustingPass: cacheBustingPass,
18281846
});
@@ -1859,7 +1877,8 @@ export function registerPiContextHandler(
18591877
messages: outputMessages,
18601878
projectIdentity,
18611879
entryIds: strictEntryIds,
1862-
entryIdByRef,
1880+
// Post-commit/post-splice ref-map (see sticky reminder above).
1881+
entryIdByRef: result.postCommitEntryIdByRef,
18631882
// Same signal OpenCode uses to gate sticky-anchor GC
18641883
// (isCacheBustingPass = history-refresh OR work executed).
18651884
isCacheBusting: isCacheBusting || result.executedWorkThisPass,
@@ -1882,7 +1901,8 @@ export function registerPiContextHandler(
18821901
db: options.db,
18831902
messages: outputMessages,
18841903
entryIds: strictEntryIds,
1885-
entryIdByRef,
1904+
// Post-commit/post-splice ref-map (see sticky reminder above).
1905+
entryIdByRef: result.postCommitEntryIdByRef,
18861906
options: {
18871907
enabled: true,
18881908
scoreThreshold: options.autoSearch.scoreThreshold,
@@ -2645,6 +2665,16 @@ interface RunPipelineResult {
26452665
targetCount: number;
26462666
reasoningWatermark: number;
26472667
activeTags: ReturnType<typeof getActiveTagsBySession>;
2668+
/**
2669+
* REAL-SessionEntry-id map keyed by AgentMessage object identity, built AFTER
2670+
* transcript.commit() and BEFORE injection splices. The ONLY correct map for
2671+
* consumers running after runPipeline mutated the array (sticky reminder,
2672+
* note nudges, auto-search): the pass-start `entryIdByRef` keys pre-commit
2673+
* objects and misses every cloned (dirty) message. Contains NO pi-msg-*
2674+
* fallback ids, so a branch-resolution failure leaves a message unmapped and
2675+
* the consumer correctly falls to its degraded (entryIds === null) path.
2676+
*/
2677+
postCommitEntryIdByRef: ReadonlyMap<object, string>;
26482678
}
26492679

26502680
async function runPipeline(args: RunPipelineArgs): Promise<RunPipelineResult> {
@@ -3141,15 +3171,29 @@ async function runPipeline(args: RunPipelineArgs): Promise<RunPipelineResult> {
31413171
// commit() just locks the result in.
31423172
transcript.commit();
31433173

3144-
// Carried stable-id map for placeholder stripping (3b-A). Built AFTER commit()
3145-
// — commit reassigns args.messages[idx] = working[idx] (cloned objects for
3146-
// dirty messages), so building before would key stale refs the post-commit
3147-
// array no longer holds. Built BEFORE injectM0M1Pi — injection splices/unshifts
3148-
// the array, after which positional entryIds[index] is stale; object identity
3149-
// survives the splice, so a ref-keyed map resolves correctly post-injection.
3150-
// stripPiDroppedPlaceholderMessages reads carried ids by object-ref (skip-on-
3151-
// miss: the only legitimate misses are injection's synthetic m[0]/m[1] prepends).
3152-
const placeholderStableIdByRef = new Map<object, string>();
3174+
// Two post-commit stable-id maps, both keyed by AgentMessage object identity.
3175+
// Built AFTER commit() — commit reassigns args.messages[idx] = working[idx]
3176+
// (cloned objects for dirty messages), so a map built BEFORE the pipeline (the
3177+
// pass-start `entryIdByRef`) keys the PRE-commit objects and misses every
3178+
// cloned message. Built BEFORE injectM0M1Pi — injection splices/unshifts the
3179+
// array, after which positional entryIds[index] is stale; object identity
3180+
// survives both the clone and the splice, so a ref-keyed map resolves correctly
3181+
// for consumers that run AFTER the pipeline mutates the array. There is NO
3182+
// structural splice between pass start and here (only commit's in-place
3183+
// reassignment), so positional entryIds[i] is still authoritative at build time.
3184+
//
3185+
// Two maps because the consumers have DIFFERENT identity contracts:
3186+
// 1. postCommitStableIdByRef (FULL fallback via resolvePiStableId, incl.
3187+
// pi-msg-* index ids) — for stripPiDroppedPlaceholderMessages, which needs
3188+
// a stable id for EVERY message (skip-on-miss + legacy path).
3189+
// 2. postCommitEntryIdByRef (REAL SessionEntry ids ONLY, no pi-msg-* fallback)
3190+
// — for sticky reminder / note nudges / auto-search. These must fall to
3191+
// their degraded path (entryIds === null) when branch resolution failed;
3192+
// a pi-msg-* fallback id would defeat that and anchor to an unstable id.
3193+
// The only legitimate misses are injection's synthetic m[0]/m[1] prepends,
3194+
// which carry no SessionEntry id and must not be anchored.
3195+
const postCommitStableIdByRef = new Map<object, string>();
3196+
const postCommitEntryIdByRef = new Map<object, string>();
31533197
for (let i = 0; i < args.messages.length; i++) {
31543198
const m = args.messages[i];
31553199
if (!m || typeof m !== "object") continue;
@@ -3159,7 +3203,13 @@ async function runPipeline(args: RunPipelineArgs): Promise<RunPipelineResult> {
31593203
args.entryIds,
31603204
args.entryIdByRef ?? undefined,
31613205
);
3162-
if (id) placeholderStableIdByRef.set(m as object, id);
3206+
if (id) postCommitStableIdByRef.set(m as object, id);
3207+
// Real-only: positional entryIds[i] is a real SessionEntry id or undefined
3208+
// (never pi-msg-*). Authoritative here because nothing has spliced yet.
3209+
const realId = args.entryIds?.[i];
3210+
if (typeof realId === "string" && realId.length > 0) {
3211+
postCommitEntryIdByRef.set(m as object, realId);
3212+
}
31633213
}
31643214

31653215
// 6. <session-history> injection — writes compartments, facts, and
@@ -3226,7 +3276,7 @@ async function runPipeline(args: RunPipelineArgs): Promise<RunPipelineResult> {
32263276
sessionId: args.sessionId,
32273277
messages: args.messages,
32283278
isCacheBusting: args.isCacheBusting,
3229-
stableIdByRef: placeholderStableIdByRef,
3279+
stableIdByRef: postCommitStableIdByRef,
32303280
// F4 cutover: when the stable-id scheme just changed, force rediscovery so
32313281
// previously-stripped placeholders get re-keyed under the new scheme this
32323282
// pass (discovery is otherwise gated on isCacheBusting = history-refresh).
@@ -3360,6 +3410,7 @@ async function runPipeline(args: RunPipelineArgs): Promise<RunPipelineResult> {
33603410
getOrCreateSessionMeta(args.db, args.sessionId)
33613411
.clearedReasoningThroughTag ?? 0,
33623412
activeTags,
3413+
postCommitEntryIdByRef,
33633414
};
33643415
}
33653416

‎packages/pi-plugin/src/index.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -454,6 +454,22 @@ export default async function (pi: ExtensionAPI): Promise<void> {
454454
return;
455455
}
456456

457+
// openDatabase() FAILS CLOSED by returning null (not throwing) on the schema
458+
// fence — when the shared cross-harness DB has been migrated to a schema newer
459+
// than this binary supports (e.g. a newer OpenCode/Pi build ran a migration).
460+
// The try/catch above only catches genuine open exceptions, so without this
461+
// guard `db` is null and the very next call (runDeferredV22Backfill) would
462+
// crash the whole Pi process. Degrade gracefully instead: skip hook
463+
// registration until the binary is upgraded. The detailed reason is already
464+
// logged by openDatabase ("storage fatal: … newer than this binary supports").
465+
if (!db) {
466+
warn(
467+
`Magic Context (pi) storage unavailable at ${dbPath} (cache schema is newer than this binary supports). ` +
468+
"Plugin will not register hooks; upgrade/restart Pi/OpenCode/Magic Context to recover.",
469+
);
470+
return;
471+
}
472+
457473
// v22 deferred legacy-memory identity backfill. openDatabase() has already
458474
// run migrations; the runner is fire-and-forget and logs failures without
459475
// blocking Pi startup.

‎packages/pi-plugin/src/pi-historian-runner.test.ts‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,43 @@ import {
1717
import { getUserMemoryCandidates } from "@magic-context/core/features/magic-context/user-memory/storage-user-memory";
1818
import { closeQuietly } from "@magic-context/core/shared/sqlite-helpers";
1919
import type { SubagentRunner } from "@magic-context/core/shared/subagent-runner";
20-
import { runPiHistorian } from "./pi-historian-runner";
20+
import {
21+
buildPiCompactionSummary,
22+
runPiHistorian,
23+
} from "./pi-historian-runner";
2124
import { createTestDb } from "./test-utils.test";
2225

26+
describe("buildPiCompactionSummary", () => {
27+
const mk = (title: string) => ({ title, startMessage: 1, endMessage: 2 });
28+
29+
it("joins all titles when at or below the cap", () => {
30+
const summary = buildPiCompactionSummary(["a", "b", "c", "d", "e"].map(mk));
31+
expect(summary).toBe("Magic Context compacted: a; b; c; d; e");
32+
expect(summary).not.toContain("more");
33+
});
34+
35+
it("caps the title list and stays bounded for large compartment counts", () => {
36+
const many = Array.from({ length: 545 }, (_, i) => mk(`segment-${i}`));
37+
const summary = buildPiCompactionSummary(many);
38+
// Bounded: only the first 5 titles appear, plus a remainder count.
39+
expect(summary).toContain("Magic Context compacted 545 segments:");
40+
expect(summary).toContain(
41+
"segment-0; segment-1; segment-2; segment-3; segment-4",
42+
);
43+
expect(summary).toContain("…and 540 more");
44+
expect(summary).not.toContain("segment-5;");
45+
// Length must not scale with compartment count.
46+
expect(summary.length).toBeLessThan(200);
47+
});
48+
49+
it("falls back to message range when titles are empty", () => {
50+
const summary = buildPiCompactionSummary([
51+
{ title: " ", startMessage: 3, endMessage: 9 },
52+
]);
53+
expect(summary).toBe("Magic Context compacted messages 3-9.");
54+
});
55+
});
56+
2357
function rawMessages(count = 12) {
2458
return Array.from({ length: count }, (_, index) => {
2559
const ordinal = index + 1;

‎packages/pi-plugin/src/pi-historian-runner.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1000,7 +1000,20 @@ export function buildPiCompactionSummary(
10001000
const last = compartments[compartments.length - 1];
10011001
return `Magic Context compacted messages ${first?.startMessage ?? "?"}-${last?.endMessage ?? "?"}.`;
10021002
}
1003-
return `Magic Context compacted: ${titles.join("; ")}`;
1003+
// Cap the title list so the marker summary stays bounded. This summary lives
1004+
// behind the compaction boundary (not model-visible — filterCompacted stops
1005+
// here and <session-history> replaces it), but it IS written to the JSONL and
1006+
// can surface in Pi's resume picker. Joining ALL titles grows unbounded with
1007+
// compartment count (a recomp/upgrade on a long session has hundreds), so cap
1008+
// to the first N and summarize the remainder.
1009+
const MAX_SUMMARY_TITLES = 5;
1010+
if (titles.length <= MAX_SUMMARY_TITLES) {
1011+
return `Magic Context compacted: ${titles.join("; ")}`;
1012+
}
1013+
const shown = titles.slice(0, MAX_SUMMARY_TITLES).join("; ");
1014+
return `Magic Context compacted ${titles.length} segments: ${shown}; …and ${
1015+
titles.length - MAX_SUMMARY_TITLES
1016+
} more`;
10041017
}
10051018

10061019
/**

‎packages/pi-plugin/src/strip-placeholders-pi.test.ts‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,93 @@ describe("stripPiDroppedPlaceholderMessages", () => {
6666
}
6767
});
6868

69+
it("prunes below-boundary ids from the persisted set on cache-busting passes", () => {
70+
const db = createTestDb();
71+
try {
72+
// Pass 1: discover two placeholders under real carried ids.
73+
const phA = assistantMessage("[dropped §2§]", 2);
74+
const phB = assistantMessage("[dropped §3§]", 3);
75+
const pass1 = [userMessage("keep", 1), phA, phB];
76+
const map1 = new Map<object, string>([
77+
[pass1[0] as object, "entry-keep"],
78+
[phA as object, "entry-A"],
79+
[phB as object, "entry-B"],
80+
]);
81+
stripPiDroppedPlaceholderMessages({
82+
db,
83+
sessionId: "ses-prune",
84+
messages: pass1,
85+
isCacheBusting: true,
86+
stableIdByRef: map1,
87+
});
88+
expect(getStrippedPlaceholderIds(db, "ses-prune").size).toBe(2);
89+
90+
// Pass 2 (cache-busting): entry-A has fallen below the compaction
91+
// boundary (no longer in the window); only entry-B remains present.
92+
// The persisted set must prune entry-A.
93+
const phB2 = assistantMessage("[dropped §3§]", 3);
94+
const pass2 = [userMessage("keep", 1), phB2];
95+
const map2 = new Map<object, string>([
96+
[pass2[0] as object, "entry-keep"],
97+
[phB2 as object, "entry-B"],
98+
]);
99+
stripPiDroppedPlaceholderMessages({
100+
db,
101+
sessionId: "ses-prune",
102+
messages: pass2,
103+
isCacheBusting: true,
104+
stableIdByRef: map2,
105+
});
106+
const remaining = getStrippedPlaceholderIds(db, "ses-prune");
107+
expect(remaining.has("entry-B")).toBe(true);
108+
expect(remaining.has("entry-A")).toBe(false); // pruned below-boundary
109+
expect(remaining.size).toBe(1);
110+
} finally {
111+
closeQuietly(db);
112+
}
113+
});
114+
115+
it("does NOT prune on defer passes (storage-only GC gated to cache-busting)", () => {
116+
const db = createTestDb();
117+
try {
118+
const phA = assistantMessage("[dropped §2§]", 2);
119+
const pass1 = [userMessage("keep", 1), phA];
120+
const map1 = new Map<object, string>([
121+
[pass1[0] as object, "entry-keep"],
122+
[phA as object, "entry-A"],
123+
]);
124+
stripPiDroppedPlaceholderMessages({
125+
db,
126+
sessionId: "ses-defer-noprune",
127+
messages: pass1,
128+
isCacheBusting: true,
129+
stableIdByRef: map1,
130+
});
131+
expect(
132+
getStrippedPlaceholderIds(db, "ses-defer-noprune").has("entry-A"),
133+
).toBe(true);
134+
135+
// Defer pass where entry-A is absent — must NOT prune (defer passes
136+
// never mutate persisted replay state).
137+
const pass2 = [userMessage("keep", 1)];
138+
const map2 = new Map<object, string>([
139+
[pass2[0] as object, "entry-keep"],
140+
]);
141+
stripPiDroppedPlaceholderMessages({
142+
db,
143+
sessionId: "ses-defer-noprune",
144+
messages: pass2,
145+
isCacheBusting: false,
146+
stableIdByRef: map2,
147+
});
148+
expect(
149+
getStrippedPlaceholderIds(db, "ses-defer-noprune").has("entry-A"),
150+
).toBe(true);
151+
} finally {
152+
closeQuietly(db);
153+
}
154+
});
155+
69156
it("uses the carried-id map by object-ref and survives an index shift", () => {
70157
const db = createTestDb();
71158
try {

0 commit comments

Comments
 (0)