Skip to content

Commit 382f394

Browse files
cdeustclaude
andcommitted
fix(memory): derive consolidation stage lazily on the read path
A3 made HEAT lazy (effective_heat decays from heat_base_set_at on every read) but left STAGE eager — consolidation_stage advances only in the consolidation handler, never on the recall/read path. Between consolidation passes a trace stayed frozen at its insert stage 'labile' (alpha=2.0, floor=0.0) while its heat decayed under the labile law, collapsing to ~0. Benchmark and import runs trigger no consolidation pass, so the forgetting law could never shape ranking: everything old read as ~0 uniformly. Fix: make STAGE as lazy as HEAT. New PL/pgSQL effective_stage() re-derives the stage at read time from elapsed dwell + stored signal columns, mirroring cascade_advancement.compute_advancement_readiness applied cumulatively (DA gate disabled — encoding-time, unavailable on read; monotonic forward-only; stage_hours consumed as a dwell budget stage-by-stage; reconsolidating/unknown pass through unchanged). effective_heat() now keys BOTH alpha (Kandel 2001) AND the permastore floor (Bahrick 1984, Benna & Fusi 2016) on the derived stage instead of the static column. Also includes the step-1 decay-clock anchor: insert_memory sets heat_base_set_at = COALESCE(provided, created_at, now()) in both pg_store and sqlite_store, so imported/backfilled rows decay from their true age rather than from insert time. Verified on cortex_test (rolled-back probe): fresh/imp0.5 -> 1.00 (early_ltp) 365d/imp0.5/0acc -> 0.05 (late_ltp floor) 365d/imp0.5/5acc -> 0.10 (consolidated floor) 365d/imp0.2/0acc -> ~0 (labile, forgetting preserved) test_pg_effective_stage_parity.py pins the SQL ladder against the Python oracle (grid + monotonicity + off-chain passthrough). I2 allow-list line numbers updated for the heat_base_set_at shift (fix confined to pg_schema, no writer added). Suites: infrastructure+invariants 415 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_019o58McF4LRfvGNNXaqG2Au
1 parent 295587f commit 382f394

7 files changed

Lines changed: 429 additions & 17 deletions

File tree

‎mcp_server/infrastructure/pg_schema.py‎

Lines changed: 116 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -605,6 +605,96 @@
605605
# - Stage floors (Bahrick 1984 permastore, Benna & Fusi 2016)
606606
# - p_factor (global decay rate per hour) = 0.95 default
607607

608+
EFFECTIVE_STAGE_FN = """
609+
-- effective_stage — LAZY read-time derivation of the consolidation stage.
610+
--
611+
-- Root cause (memory 4202985): A3 made HEAT lazy (decayed from stage_hours
612+
-- on the read path) but left STAGE eager — advanced only by the
613+
-- consolidation handler (handlers/consolidation/cascade.py), never on
614+
-- recall/read. Benchmark and import runs trigger no consolidation pass, so
615+
-- a row stays frozen at its insert stage 'labile' (α=2.0, floor=0.0) while
616+
-- its heat decays under the labile law → collapses to ~0. This function
617+
-- re-derives the stage lazily from elapsed dwell + the row's stored signal
618+
-- columns so α and the floor track the trace's true maturity.
619+
--
620+
-- Mirrors cascade_advancement.compute_advancement_readiness applied
621+
-- cumulatively (advance as far as time + signals permit, one shot),
622+
-- forward-chain only (labile→early_ltp→late_ltp→consolidated), monotonic
623+
-- (never demote below the stored stage). The DA gate of the LABILE→EARLY_LTP
624+
-- transition is encoding-time and unavailable on the read path, so only the
625+
-- importance gate is used (dopamine treated as absent).
626+
--
627+
-- stage_hours is treated as a dwell BUDGET consumed stage-by-stage: leaving
628+
-- a stage requires budget ≥ that stage's effective min_dwell, and consumes it.
629+
--
630+
-- min_dwell hours: labile=0, early_ltp=1, late_ltp=6, consolidated=24.
631+
-- source: cascade_stages._STAGE_PROPERTIES (Kandel 2001)
632+
-- schema acceleration on min_dwell (cascade_advancement._effective_min_dwell):
633+
-- systems-consolidation stages (late_ltp, consolidated): × 15^(-schema)
634+
-- (Tse 2007 ~10-15× acceleration; 15.0 base is an engineering choice)
635+
-- synaptic-tag stages (labile, early_ltp): × (1 - schema·0.2)
636+
-- signal gates (cascade_advancement._check_*_advancement):
637+
-- labile→early_ltp: importance > 0.3 (DA path disabled at read)
638+
-- early_ltp→late_ltp: access_count ≥ 1 OR importance > 0.4
639+
-- late_ltp→consolidated: access_count ≥ (3 if schema < 0.5 else 1)
640+
-- 'reconsolidating' and any unknown stage are returned unchanged — they are
641+
-- access-triggered (Nader 2000), not time-derivable.
642+
CREATE OR REPLACE FUNCTION effective_stage(
643+
p_stage TEXT,
644+
p_hours DOUBLE PRECISION,
645+
p_importance REAL,
646+
p_access INTEGER,
647+
p_schema REAL
648+
) RETURNS TEXT AS $$
649+
DECLARE
650+
cur TEXT := p_stage;
651+
budget DOUBLE PRECISION := GREATEST(0.0, COALESCE(p_hours, 0.0));
652+
imp DOUBLE PRECISION := COALESCE(p_importance, 0.5);
653+
acc INTEGER := COALESCE(p_access, 0);
654+
sch DOUBLE PRECISION := COALESCE(p_schema, 0.0);
655+
dwell DOUBLE PRECISION;
656+
late_thr INTEGER;
657+
BEGIN
658+
-- Only the forward synaptic-tag chain is derived lazily.
659+
IF cur NOT IN ('labile', 'early_ltp', 'late_ltp', 'consolidated') THEN
660+
RETURN cur;
661+
END IF;
662+
663+
-- LABILE → EARLY_LTP. min_dwell(labile)=0 → no budget consumed.
664+
IF cur = 'labile' THEN
665+
IF imp > 0.3 THEN
666+
cur := 'early_ltp';
667+
ELSE
668+
RETURN cur;
669+
END IF;
670+
END IF;
671+
672+
-- EARLY_LTP → LATE_LTP. effective min_dwell = 1.0 · (1 - schema·0.2).
673+
IF cur = 'early_ltp' THEN
674+
dwell := 1.0 * (1.0 - sch * 0.2);
675+
IF budget >= dwell AND (acc >= 1 OR imp > 0.4) THEN
676+
budget := budget - dwell;
677+
cur := 'late_ltp';
678+
ELSE
679+
RETURN cur;
680+
END IF;
681+
END IF;
682+
683+
-- LATE_LTP → CONSOLIDATED. effective min_dwell = 6.0 · 15^(-schema).
684+
IF cur = 'late_ltp' THEN
685+
dwell := 6.0 * POWER(15.0, -sch);
686+
late_thr := CASE WHEN sch < 0.5 THEN 3 ELSE 1 END;
687+
IF budget >= dwell AND acc >= late_thr THEN
688+
cur := 'consolidated';
689+
END IF;
690+
END IF;
691+
692+
-- CONSOLIDATED is terminal on the forward chain.
693+
RETURN cur;
694+
END;
695+
$$ LANGUAGE plpgsql IMMUTABLE PARALLEL SAFE;
696+
"""
697+
608698
EFFECTIVE_HEAT_FN = """
609699
-- p_factor default: 0.95 per DAY (pre-A3 DECAY_MEMORIES_FN ran ~daily,
610700
-- each run applied factor 0.95 once). Converted to per-hour equivalent:
@@ -625,6 +715,7 @@
625715
stage_floor DOUBLE PRECISION;
626716
base_scaled DOUBLE PRECISION;
627717
decayed DOUBLE PRECISION;
718+
eff_stage TEXT;
628719
BEGIN
629720
-- Pinned: protected or explicit no_decay. heat_base is authoritative;
630721
-- factor still applies (homeostatic contraction affects even anchors
@@ -647,9 +738,26 @@
647738
stage_hours := GREATEST(0.0, EXTRACT(EPOCH FROM
648739
(t_now - COALESCE(m.stage_entered_at, m.created_at))) / 3600.0);
649740
741+
-- Lazily derive the effective consolidation stage. A3 made HEAT lazy but
742+
-- left STAGE eager (advanced only by the consolidation handler, never on
743+
-- the read path), so between consolidation passes a row's heat decayed
744+
-- under the labile α while the trace had already earned a later,
745+
-- slower-decaying stage. effective_stage() re-derives the stage from
746+
-- elapsed dwell (stage_hours) + the stored signal columns so α and the
747+
-- floor below match the trace's true maturity, monotonically (never
748+
-- below the stored stage). 'reconsolidating'/unknown stages pass through
749+
-- unchanged. source: effective_stage() + cascade_advancement gates.
750+
eff_stage := effective_stage(
751+
m.consolidation_stage,
752+
stage_hours,
753+
m.importance,
754+
m.access_count,
755+
m.schema_match_score
756+
);
757+
650758
-- α(stage) — Kandel 2001 stage-dependent decay exponent.
651759
-- source: pg_schema.py:748-756
652-
alpha := CASE m.consolidation_stage
760+
alpha := CASE eff_stage
653761
WHEN 'labile' THEN 2.0
654762
WHEN 'early_ltp' THEN 1.2
655763
WHEN 'late_ltp' THEN 0.8
@@ -669,8 +777,10 @@
669777
* (1.0 - EXP(-LEAST(stage_hours / 1.0, 80.0)));
670778
671779
-- Stage permastore floor — Bahrick 1984 + Benna & Fusi 2016.
672-
-- source: pg_schema.py:742-747
673-
stage_floor := CASE m.consolidation_stage
780+
-- Uses the lazily-derived eff_stage (see above) so a trace that has
781+
-- matured into late_ltp/consolidated gets its retention floor even
782+
-- between consolidation passes. source: pg_schema.py:742-747
783+
stage_floor := CASE eff_stage
674784
WHEN 'consolidated' THEN 0.10
675785
WHEN 'late_ltp' THEN 0.05
676786
WHEN 'reconsolidating' THEN 0.05
@@ -1606,6 +1716,9 @@ def get_all_ddl() -> list[str]:
16061716
# rename lands before indexes on heat_base are created.
16071717
MIGRATIONS_DDL,
16081718
INDEXES_DDL,
1719+
# effective_stage() must be created before effective_heat(), which
1720+
# calls it to derive α/floor lazily on the read path.
1721+
EFFECTIVE_STAGE_FN,
16091722
EFFECTIVE_HEAT_FN,
16101723
EFFECTIVE_HEAT_FROZEN_FN,
16111724
# A3 canonical read path: lazy effective_heat() computes decay at

‎mcp_server/infrastructure/pg_store.py‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -450,10 +450,19 @@ def insert_memory(self, data: dict[str, Any]) -> int:
450450
from mcp_server.core.temporal import normalize_date_to_iso
451451

452452
raw_created = normalize_date_to_iso(raw_created) or raw_created
453+
# A3 decay clock: anchor heat_base_set_at to the event date, not NOW().
454+
# effective_heat() decays from COALESCE(heat_base_set_at, last_accessed,
455+
# created_at); for a never-touched insert the faithful "last canonical
456+
# touch" IS the event (created_at), so a historical-dated memory
457+
# (import / benchmark loader) engages the SQL forgetting law instead of
458+
# reading hours_elapsed≈0. No-op for fresh writes where created_at≈now.
459+
# Source: docs/program/phase-3-a3-migration-design.md §3.1 (clock = last
460+
# touch); benchmark root-cause memory 4202968.
461+
heat_base_anchor = data.get("heat_base_set_at") or raw_created or now
453462
row = self._execute(
454463
"""INSERT INTO memories (
455464
content, embedding, tags, source, domain,
456-
directory_context, created_at, last_accessed,
465+
directory_context, created_at, last_accessed, heat_base_set_at,
457466
heat_base, surprise_score, importance,
458467
emotional_valence, confidence, store_type,
459468
is_protected, consolidation_stage,
@@ -465,7 +474,7 @@ def insert_memory(self, data: dict[str, Any]) -> int:
465474
arousal, dominant_emotion, supersedes_id
466475
) VALUES (
467476
%(content)s, %(embedding)s, %(tags)s::jsonb, %(source)s, %(domain)s,
468-
%(directory_context)s, %(created_at)s, %(last_accessed)s,
477+
%(directory_context)s, %(created_at)s, %(last_accessed)s, %(heat_base_set_at)s,
469478
%(heat)s, %(surprise_score)s, %(importance)s,
470479
%(emotional_valence)s, %(confidence)s, %(store_type)s,
471480
%(is_protected)s, %(consolidation_stage)s,
@@ -485,6 +494,7 @@ def insert_memory(self, data: dict[str, Any]) -> int:
485494
"directory_context": data.get("directory_context", ""),
486495
"created_at": raw_created or now,
487496
"last_accessed": now,
497+
"heat_base_set_at": heat_base_anchor,
488498
"heat": data.get("heat", 1.0),
489499
"surprise_score": data.get("surprise_score", 0.0),
490500
"importance": data.get("importance", 0.5),

‎mcp_server/infrastructure/sqlite_store.py‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -203,10 +203,15 @@ def insert_memory(self, data: dict[str, Any]) -> int:
203203
except ImportError:
204204
pass
205205
content = data["content"]
206+
# A3 decay clock parity with PgMemoryStore: anchor heat_base_set_at to
207+
# the event date (created_at), not NOW(), so historical-dated inserts
208+
# engage the SQL forgetting law. No-op for fresh writes. See
209+
# PgMemoryStore.insert_memory for the rationale.
210+
heat_base_anchor = data.get("heat_base_set_at") or raw_created or now
206211
cur = self._conn.execute(
207212
"""INSERT INTO memories (
208213
content, tags, source, domain,
209-
directory_context, created_at, last_accessed,
214+
directory_context, created_at, last_accessed, heat_base_set_at,
210215
heat_base, surprise_score, importance,
211216
emotional_valence, confidence, store_type,
212217
is_protected, consolidation_stage,
@@ -216,7 +221,7 @@ def insert_memory(self, data: dict[str, Any]) -> int:
216221
hippocampal_dependency, is_benchmark, agent_context,
217222
is_global, supersedes_id
218223
) VALUES (
219-
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
224+
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
220225
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?
221226
)""",
222227
(
@@ -227,6 +232,7 @@ def insert_memory(self, data: dict[str, Any]) -> int:
227232
data.get("directory_context", ""),
228233
raw_created or now,
229234
now,
235+
heat_base_anchor,
230236
data.get("heat", 1.0),
231237
data.get("surprise_score", 0.0),
232238
data.get("importance", 0.5),
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
"""Live-PG behavior tests for the A3 decay-clock anchor in insert_memory.
2+
3+
Root cause (benchmark module #6 campaign, 2026-06-30): ``insert_memory``
4+
let ``heat_base_set_at`` take its schema DEFAULT NOW(), so a
5+
historical-dated insert (an imported 2023 conversation, a benchmark
6+
haystack with multi-year timestamps) read ``hours_elapsed ≈ 0`` in
7+
``effective_heat()`` and the SQL forgetting law never engaged —
8+
``effective_heat ≡ heat_base`` regardless of decay law.
9+
10+
The fix anchors ``heat_base_set_at`` to the event date (``created_at``)
11+
on insert. ``effective_heat()`` decays from
12+
``COALESCE(heat_base_set_at, last_accessed, created_at)``; for a
13+
never-touched insert the faithful "last canonical touch" IS the event,
14+
so the clock now reads real elapsed time.
15+
16+
These tests pin the two invariants of that fix:
17+
- INV-FRESH: a fresh insert (created_at ≈ now) is a no-op —
18+
effective_heat == heat_base (hours_elapsed ≈ 0).
19+
- INV-AGED: a historical-dated insert engages the law —
20+
effective_heat < heat_base (hours_elapsed > 0).
21+
22+
Runs against cortex_test (conftest redirects DATABASE_URL).
23+
"""
24+
25+
from __future__ import annotations
26+
27+
from datetime import datetime, timedelta, timezone
28+
29+
import pytest
30+
31+
from mcp_server.infrastructure.pg_store import PgMemoryStore
32+
from tests_py.conftest import _USE_PG # type: ignore
33+
34+
pytestmark = pytest.mark.skipif(
35+
not _USE_PG, reason="PostgreSQL not available — decay clock needs live schema"
36+
)
37+
38+
_DOMAIN = "decay-clock-anchor-test"
39+
40+
41+
@pytest.fixture
42+
def store():
43+
s = PgMemoryStore()
44+
yield s
45+
try:
46+
s._execute("DELETE FROM memories WHERE domain = %s", (_DOMAIN,))
47+
s._conn.commit()
48+
finally:
49+
s.close()
50+
51+
52+
def _effective_heat(store: PgMemoryStore, memory_id: int) -> float:
53+
row = store._execute(
54+
"SELECT effective_heat(m, NOW(), 1.0) AS eff "
55+
"FROM memories m WHERE id = %s",
56+
(memory_id,),
57+
).fetchone()
58+
assert row is not None
59+
return float(row["eff"])
60+
61+
62+
def _hours_elapsed(store: PgMemoryStore, memory_id: int) -> float:
63+
row = store._execute(
64+
"SELECT EXTRACT(EPOCH FROM (NOW() - heat_base_set_at)) / 3600.0 AS hrs "
65+
"FROM memories WHERE id = %s",
66+
(memory_id,),
67+
).fetchone()
68+
assert row is not None
69+
return float(row["hrs"])
70+
71+
72+
def test_fresh_insert_is_decay_noop(store: PgMemoryStore) -> None:
73+
"""INV-FRESH: created_at ≈ now ⇒ clock ≈ now ⇒ effective_heat == heat_base."""
74+
mid = store.insert_memory(
75+
{
76+
"content": "fresh memory, no historical date",
77+
"heat": 1.0,
78+
"domain": _DOMAIN,
79+
"is_benchmark": True,
80+
}
81+
)
82+
assert _hours_elapsed(store, mid) < 1.0
83+
assert _effective_heat(store, mid) == pytest.approx(1.0, abs=1e-3)
84+
85+
86+
def test_historical_insert_engages_forgetting_law(store: PgMemoryStore) -> None:
87+
"""INV-AGED: a year-old created_at backdates the clock ⇒ law decays heat."""
88+
old = (datetime.now(timezone.utc) - timedelta(days=365)).isoformat()
89+
mid = store.insert_memory(
90+
{
91+
"content": "imported 2023 conversation",
92+
"heat": 1.0,
93+
"domain": _DOMAIN,
94+
"is_benchmark": True,
95+
"created_at": old,
96+
}
97+
)
98+
# ~8760h must have elapsed on the decay clock, not ≈ 0.
99+
assert _hours_elapsed(store, mid) > 8000.0
100+
# The SQL forgetting law must have pulled effective_heat strictly below
101+
# the stored heat_base. (Exact value depends on stage/law constants;
102+
# this test pins only that the clock engages, not the law family.)
103+
assert _effective_heat(store, mid) < 1.0
104+
105+
106+
def test_explicit_heat_base_set_at_overrides_created_at(store: PgMemoryStore) -> None:
107+
"""An explicit heat_base_set_at wins over created_at (e.g. re-anchoring)."""
108+
old_created = (datetime.now(timezone.utc) - timedelta(days=365)).isoformat()
109+
recent_anchor = (datetime.now(timezone.utc) - timedelta(hours=1)).isoformat()
110+
mid = store.insert_memory(
111+
{
112+
"content": "old event, recently re-anchored",
113+
"heat": 1.0,
114+
"domain": _DOMAIN,
115+
"is_benchmark": True,
116+
"created_at": old_created,
117+
"heat_base_set_at": recent_anchor,
118+
}
119+
)
120+
# Clock reads the explicit anchor (≈1h), not the 365-day created_at.
121+
assert _hours_elapsed(store, mid) < 24.0

0 commit comments

Comments
 (0)