Skip to content

Commit 6961659

Browse files
authored
Merge branch 'main' into feat/254-level1-deny
2 parents 52dee2f + 315092b commit 6961659

6 files changed

Lines changed: 183 additions & 7 deletions

File tree

‎dsh-mneme/CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,10 @@
3838

3939
## [Unreleased]
4040

41+
## 🐛 修复
42+
43+
- **saveWithDedupe 透传显式 epistemic_status(trustEpistemicWeighting 写入通路缺口)**:创建路径此前丢弃显式字段,store 回退内容标记推断(中文正则)——英文/无标记内容一律落 subjective,EPISTEMIC_WEIGHTS 对整池均匀 ×0.7,`trustEpistemicWeighting` 的重排对工具主路径写入的记忆**整体空转**(v0.4.5 起消费者在、写入通路断;patch 路径 store.update 本来就通)。E 系列四臂实验现场抓到(重排臂与对照臂注入序逐条相同)。修复后 undefined 保持推断行为(存量零变化),显式值优先(契约同 resolveEpistemicStatus);回归测试锁「observation 加权前后分数不变——字段丢失则全体缩放当场红」。
44+
4145
## [0.8.8] - 2026-09-27
4246

4347
## 🆕 新增
@@ -189,6 +193,7 @@
189193
- **能力说明第 5 条写全 `memory_forget` 的副作用面,`memory_save` 尾句收短(#249 文案精确性)**:原文只写「`memory_forget` 只停注入」,比工具实况窄——它实际让条目从注入、检索结果与列表三处消失,写窄会让模型低估其影响面;改为与 `memory_archive`(从列表 / 检索 / 注入 / 巩固四处隐藏)同口径,回归测试锁完整短语在场。
190194
- **运行时完整性判据从未生效:状态词对不上,而且结论根本没读(issue #268)**:`verifyPayload` 只认 `integrity.status === "sha512-matched"`,而下载通道在 `mneme-runtime.json` 里记的是 `{status: "verified", checked, detail}`(`src/runtime/download.js`,自 #133 起)——这份结论一喂进来就恒判 `ok:false`,接线即系统性判失败;更根本的是两个生产调用方(`memory_runtime` 的 verify 分支、`scripts/mneme-runtime.mjs` 的 `runVerify`)都只传 `cacheDir`,**这一层在生产路径上从未运行过**:清单里明写的 mismatch 也被静默放过(红测试证实)。附带第二处形状违约:`loader.js` 把清单里的**对象**直接填进 `describeLocalRuntime` 里声明为 `string|null` 的 `integrity` 字段,经免鉴权的 `/api/dsh-mneme/semantic` 外发,CLI `status` 还会把它打印成 `[object Object]`。修法收成一处:新增 `layout.js` 的 `recordedIntegrity()` 归一清单字段的两种形态(对象 / 缺失),判据与投影都只调它——`verifyPayload` 在调用方未显式传结论时读 `describePayload` 已解析的清单,判据同时认 `verified` 与 `sha512-matched`,并把显式 `unverified` 与「没传」同判(原先一个 `ok:true`、一个 `ok:false`);不一致的结论(含清单里记下的 mismatch)照样判失败。
191195

196+
192197
- **睡眠冲突/模式阶段的输出预算可配(issue #257)**:`src/dream/sleep.js` 冲突消解与模式发现两处 `maxTokens: 2048` 硬编码提为 `sleepMaxTokens`(默认 8192,schema + 整数白名单成对落位,面板可调)。实测依据(报告者 llama.cpp 环境):默认档 24 对裁决需 2097 token,恰好压在 2048 边界(53 次运行 48 败 5 胜的「间歇性失败」指纹);`sleepActionSet: full` 六分支实测需 6967(3.4 倍越界)——该档位自 #126 引入起从未跑通过。流式计费按实际用量,调大不增加成本。
193198

194199
- **审计记账改读 `chunk.usage`,token 不再恒为 0(issue #242)**:dsh-llm 的 StreamChunk 契约把用量嵌在 `{type:"usage", usage:TokenUsage}`(TokenUsage = inputTokens / outputTokens / …),chunk 顶层没有 token 字段——dream / summarize 的审计读取把整个 chunk 当用量对象,input/output 恒为 undefined,审计行落 0(实测 7 天 49 次 success 调用 token 全 0,面板「LLM 消耗」长期显示 0)。改读 `chunk.usage ?? chunk`,`?? chunk` 兜底兼容用量平铺在顶层的替身(嵌套 + 平铺双形状回归测试)。
@@ -200,6 +205,8 @@
200205

201206
- **全工具矩阵的「DTO 键集 ⊆ output schema」系统性断言(issue #195)**:#184(memory_get 内联 schema 漏声明 v0.8.1 的 scope 来源三键 → 任何被标注过的行都过不了 in-process 校验)此前只有单点回归护住 `memory_get` 一个工具,换一个工具、换一个键,同类事故可以原样重演。新增 `test/tools-dto-schema-matrix.test.js`,四层断言各管一段:① 9 个工具每个可安全触达分支的**真实 execute 返回值**过生产同款校验器 `validateJsonSchemaValue`(不写手抄期望值);② DTO 唯一产地 `toApiList` 在全形态(极简 / 敏感度 / 事件时间 / 单维与全量 scope 标注)下的输出 ⊆ `MEMORY_ITEM_SCHEMA`,并**反向**要求声明里的每个键都被至少一种形态真实产出(死声明会在下次增键时暴露);③ 全部工具 schema 的结构不变量(闭合、required ⊆ properties、每项带 type——否则前两层会因校验器形同虚设而静默失效);④ 负例锁:注入未声明键**必须**报错。护栏自证:两次变异测试(删共享 schema 一个键 / 给 memory_get 塞手抄小副本)分别让 2 条与 3 条断言转红。`memory_runtime` 的 provision(联网下载)与 verify 命中载荷(真实加载模型)不在单测内驱动,由 ③ 兜底声明合规。
202207

208+
- **supersede 旧值回归集(anti-update 探针,E6/#280 链)**:`scripts/benchmark-recall.js` 的 TEST_CASES 支持可选 `forbidden` 字段——forbidden id 出现在 top-K 即记 `forbiddenHit`,legacy/fused 与三融合配方全配置判定,CLI 报告加 LEAK 标记;seed 复刻 applySupersede 的落库状态(loser `archived=1` + 正文 superseded-by 注记,无专属取代列)。新增 `test/recall-anti-update.test.js` 6 例,把三件事钉成回归锁:四路默认检索对归档行是**排除**不是降权(谁把排除改成降权立刻红)、取代是归档不是删除(行还在、未遗忘、注记在)、`includeArchived` 显式口子仍在(排除发生在检索层而非行消失)。依据 StatemenBench anti-update probes。测试 1198 → **1204**。
209+
203210
## 🏗️ 工程
204211

205212
- 致谢:heptaspirit(#247 注入命中留痕 + #267 agent 主动整理接口 + #277 能力说明文案精确性)、davidekingsss(#248 审计记账修复 + #253 审计边界测试)。

‎dsh-mneme/lib/service.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1288,6 +1288,11 @@ export function createService({ store, mirror, config, onWrite, logger, document
12881288
tags: memory.tags ?? [],
12891289
importance: memory.importance ?? 3,
12901290
source: memory.source ?? "manual",
1291+
// epistemic_status 必须显式透传:省略时 store 会回退到内容标记推断
1292+
// (中文正则),英文/无标记内容一律落 subjective——E2 四臂实验当场抓到
1293+
// (trustEpistemicWeighting 的重排对 saveWithDedupe 写入的记忆整体空转,
1294+
// R+ 臂与 A 臂注入序逐条相同)。undefined 保持推断行为,显式值优先。
1295+
epistemic_status: memory.epistemic_status,
12911296
...(memory.evidence !== undefined ? { evidence: memory.evidence } : {}),
12921297
// v0.8.0 A1:scope 标注透传(store 端归一化,未标注落 NULL)。
12931298
// v0.8.1 底座:来源(auto/explicit)与决策时间随行透传。

‎dsh-mneme/scripts/benchmark-recall.js‎

Lines changed: 57 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,11 @@ const SEED = [
5252
// Standard query set: each case is a query plus the ids that MUST appear in
5353
// the top-K for the case to count as a hit. Covers the three recall paths —
5454
// multi-term lexical (BM25's home turf), identifier lookup, and semantic.
55+
// Optional per case: `forbidden` — ids that must NOT appear in the top-K.
56+
// Anti-update probes (E6, #280 链;StatemenBench anti-update probes):锁的
57+
// 回归类是「supersede 后旧值复现」——被取代旧条靠 archived=1 从四路检索里
58+
// **排除**(不是降权)。未来谁把任一默认检索路径改成 archived 宽容的降权、
59+
// 或把归档行重新放回候选池,forbiddenHit 就会 >0,这里先红。
5560
export const TEST_CASES = [
5661
{ query: "rust 异步", expected: ["mem_async_pattern", "mem_rust_switch"], note: "scattered terms — BM25 territory" },
5762
{ query: "ZFS-4421 checksum", expected: ["mem_zfs_bug"], note: "identifier + keyword" },
@@ -67,7 +72,12 @@ export const TEST_CASES = [
6772
// leading token with a cross-topic memory that must rank below it) and an
6873
// exact-token case that leans on the BM25 path.
6974
{ query: "zfs 磁盘 替换", expected: ["mem_zfs_bug"], note: "shared-token distractor" },
70-
{ query: "tokio spawn channel", expected: ["mem_async_pattern"], note: "exact async tokens" }
75+
{ query: "tokio spawn channel", expected: ["mem_async_pattern"], note: "exact async tokens" },
76+
// Anti-update probes(配 seedService 里的 supersede pair):第二条 query 的
77+
// 措辞只与被取代的旧值(Go 选型)重叠——归档旧条若经任一默认路径复现,
78+
// forbiddenHit > 0;同时新值必须照常命中(取代后话题有活答案)。
79+
{ query: "编译模块 语言 选型", expected: ["mem_rust_switch"], forbidden: ["mem_rust_switch_old"], note: "anti-update: winner must answer the topic" },
80+
{ query: "Go 部署 简单", expected: ["mem_rust_switch"], forbidden: ["mem_rust_switch_old"], note: "stale-value bait: wording matches only the superseded value" }
7181
];
7282

7383
export function seedService(overrides = {}) {
@@ -95,6 +105,15 @@ export function seedService(overrides = {}) {
95105
const row = store.save({ id: m.id, type: m.type, title: m.title, content: m.content, tags: m.tags, importance: m.importance, source: "seed" });
96106
store.setEmbedding(row.id, hashVec(`${m.title} ${m.content}`));
97107
}
108+
// Supersede pair(anti-update 探针的种子):复刻 dream/decisions.js
109+
// applySupersede 留下的确切落库状态——没有独立的取代列,取代关系完全由
110+
// loser 的 archived=1 + 正文 (superseded by: …) 注记表达。winner 就是
111+
// 上面的 mem_rust_switch 种子行,让「编译语言」话题始终有活答案,
112+
// 归档的 Go 旧决策必须保持不可见。
113+
const loserContent = "项目编译模块使用 Go 语言,选型理由是部署简单\n\n(superseded by: 语言迁移决策)";
114+
const loser = store.save({ id: "mem_rust_switch_old", type: "decision", title: "编译模块语言(旧)", content: loserContent, importance: 4, tags: ["rust"], source: "seed" });
115+
store.setEmbedding(loser.id, hashVec(`编译模块语言(旧) ${loserContent}`));
116+
store.setArchived(loser.id, true);
98117
return service;
99118
}
100119

@@ -109,18 +128,31 @@ export async function runBenchmark({ topK = 5, mode = "auto" } = {}) {
109128
const rows = [];
110129
let hits = 0;
111130
let mrrSum = 0;
131+
let leaks = 0;
112132
for (const tc of TEST_CASES) {
113133
const results = await service.searchMemories(tc.query, { mode, topK, useRerank: false });
114134
const ids = results.map((r) => r.id);
115135
const metrics = service.computeRetrievalMetrics(ids, tc.expected);
116136
if (metrics.recall === 1) hits++;
117137
mrrSum += metrics.mrr;
118-
rows.push({ query: tc.query, note: tc.note, expected: tc.expected, got: ids, ...metrics });
138+
// forbidden 命中数:anti-update 探针的判定(见 TEST_CASES 头注释)。
139+
const forbiddenHit = (tc.forbidden ?? []).filter((id) => ids.includes(id)).length;
140+
if (forbiddenHit > 0) leaks++;
141+
rows.push({
142+
query: tc.query,
143+
note: tc.note,
144+
expected: tc.expected,
145+
...(tc.forbidden ? { forbidden: tc.forbidden } : {}),
146+
forbiddenHit,
147+
got: ids,
148+
...metrics
149+
});
119150
}
120151
runs.push({
121152
config: cfg.name,
122153
recallAtK: +(hits / TEST_CASES.length).toFixed(3),
123154
avgMrr: +(mrrSum / TEST_CASES.length).toFixed(3),
155+
forbiddenHits: leaks,
124156
rows
125157
});
126158
}
@@ -141,18 +173,30 @@ export async function runFusionBenchmark({ topK = 5, mode = "auto" } = {}) {
141173
const rows = [];
142174
let hits = 0;
143175
let mrrSum = 0;
176+
let leaks = 0;
144177
for (const tc of TEST_CASES) {
145178
const results = await service.searchMemories(tc.query, { mode, topK, useRerank: false });
146179
const ids = results.map((r) => r.id);
147180
const metrics = service.computeRetrievalMetrics(ids, tc.expected);
148181
if (metrics.recall === 1) hits++;
149182
mrrSum += metrics.mrr;
150-
rows.push({ query: tc.query, note: tc.note, expected: tc.expected, got: ids, ...metrics });
183+
const forbiddenHit = (tc.forbidden ?? []).filter((id) => ids.includes(id)).length;
184+
if (forbiddenHit > 0) leaks++;
185+
rows.push({
186+
query: tc.query,
187+
note: tc.note,
188+
expected: tc.expected,
189+
...(tc.forbidden ? { forbidden: tc.forbidden } : {}),
190+
forbiddenHit,
191+
got: ids,
192+
...metrics
193+
});
151194
}
152195
runs.push({
153196
config: recipe,
154197
recallAtK: +(hits / TEST_CASES.length).toFixed(3),
155198
avgMrr: +(mrrSum / TEST_CASES.length).toFixed(3),
199+
forbiddenHits: leaks,
156200
rows
157201
});
158202
}
@@ -164,10 +208,13 @@ function printFusionReport(report) {
164208
console.log(`\n=== ${run.config} (topK=${report.topK}, mode=${report.mode}) ===`);
165209
for (const r of run.rows) {
166210
const ok = r.recall === 1 ? "PASS" : "MISS";
167-
console.log(` [${ok}] "${r.query}" (${r.note}) recall=${r.recall} mrr=${r.mrr}`);
211+
const leak = r.forbiddenHit > 0 ? " LEAK" : "";
212+
console.log(` [${ok}${leak}] "${r.query}" (${r.note}) recall=${r.recall} mrr=${r.mrr}`);
168213
if (r.recall < 1) console.log(` expected ⊇ ${r.expected.join(", ")} got: ${r.got.join(", ") || "—"}`);
214+
if (r.forbiddenHit > 0) console.log(` forbidden hit ×${r.forbiddenHit}: ${r.got.filter((id) => r.forbidden.includes(id)).join(", ")}`);
169215
}
170-
console.log(` → Recall@${report.topK}: ${(run.recallAtK * 100).toFixed(1)}% avg MRR: ${run.avgMrr}`);
216+
console.log(` → Recall@${report.topK}: ${(run.recallAtK * 100).toFixed(1)}% avg MRR: ${run.avgMrr}` +
217+
(run.forbiddenHits > 0 ? ` forbiddenHits: ${run.forbiddenHits}` : ""));
171218
}
172219
const summary = report.runs.map((r) => `${r.config}=${(r.recallAtK * 100).toFixed(1)}%`).join(" ");
173220
console.log(`\n融合配方 A/B (Recall@${report.topK}, ${report.mode}): ${summary}`);
@@ -178,10 +225,13 @@ function printReport(report) {
178225
console.log(`\n=== ${run.config} (topK=${report.topK}, mode=${report.mode}) ===`);
179226
for (const r of run.rows) {
180227
const ok = r.recall === 1 ? "PASS" : "MISS";
181-
console.log(` [${ok}] "${r.query}" (${r.note}) recall=${r.recall} mrr=${r.mrr}`);
228+
const leak = r.forbiddenHit > 0 ? " LEAK" : "";
229+
console.log(` [${ok}${leak}] "${r.query}" (${r.note}) recall=${r.recall} mrr=${r.mrr}`);
182230
if (r.recall < 1) console.log(` expected ⊇ ${r.expected.join(", ")} got: ${r.got.join(", ") || "—"}`);
231+
if (r.forbiddenHit > 0) console.log(` forbidden hit ×${r.forbiddenHit}: ${r.got.filter((id) => r.forbidden.includes(id)).join(", ")}`);
183232
}
184-
console.log(` → Recall@${report.topK}: ${(run.recallAtK * 100).toFixed(1)}% avg MRR: ${run.avgMrr}`);
233+
console.log(` → Recall@${report.topK}: ${(run.recallAtK * 100).toFixed(1)}% avg MRR: ${run.avgMrr}` +
234+
(run.forbiddenHits > 0 ? ` forbiddenHits: ${run.forbiddenHits}` : ""));
185235
}
186236
const [legacy, fused] = report.runs;
187237
const lift = ((fused.recallAtK - legacy.recallAtK) * 100).toFixed(1);

‎dsh-mneme/src/service.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1288,6 +1288,11 @@ export function createService({ store, mirror, config, onWrite, logger, document
12881288
tags: memory.tags ?? [],
12891289
importance: memory.importance ?? 3,
12901290
source: memory.source ?? "manual",
1291+
// epistemic_status 必须显式透传:省略时 store 会回退到内容标记推断
1292+
// (中文正则),英文/无标记内容一律落 subjective——E2 四臂实验当场抓到
1293+
// (trustEpistemicWeighting 的重排对 saveWithDedupe 写入的记忆整体空转,
1294+
// R+ 臂与 A 臂注入序逐条相同)。undefined 保持推断行为,显式值优先。
1295+
epistemic_status: memory.epistemic_status,
12911296
...(memory.evidence !== undefined ? { evidence: memory.evidence } : {}),
12921297
// v0.8.0 A1:scope 标注透传(store 端归一化,未标注落 NULL)。
12931298
// v0.8.1 底座:来源(auto/explicit)与决策时间随行透传。

‎dsh-mneme/test/epistemic.test.js‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,3 +296,30 @@ test("conflict resolution keeps the LLM winner when disabled", () => {
296296
assert.equal(store.getById(w.memory.id).archived, false);
297297
assert.equal(store.getById(l.memory.id).archived, true);
298298
});
299+
300+
// E2 四臂实验(Kaggle v11)当场抓到的回归:saveWithDedupe 此前丢弃显式
301+
// epistemic_status,store 回退内容推断,英文/无标记内容一律 subjective——
302+
// EPISTEMIC_WEIGHTS 对整池均匀 ×0.7,trustEpistemicWeighting 的重排整体空转
303+
// (R+ 臂与 A 臂注入序逐条相同)。显式值必须穿透创建路径;undefined 保持推断。
304+
test("saveWithDedupe forwards explicit epistemic_status (rerank needs non-uniform weights)", async () => {
305+
const store = createStore(":memory:");
306+
const builder = createService({ store, mirror: null, config: {} });
307+
const bait = builder.saveWithDedupe({ type: "preference", title: "m#0", content: "User feels that daily stand-up meetings are micromanagement theater", importance: 3, epistemic_status: "subjective" });
308+
const fact = builder.saveWithDedupe({ type: "preference", title: "m#1", content: "User is a Lead Software Engineer with 15 years of experience", importance: 3, epistemic_status: "observation" });
309+
assert.equal(store.getById(bait.memory.id).epistemic_status, "subjective", "explicit subjective stored");
310+
assert.equal(store.getById(fact.memory.id).epistemic_status, "observation", "explicit field must survive saveWithDedupe (English content, inference cannot classify)");
311+
312+
const plain = createService({ store, mirror: null, config: { signalTransparency: true } });
313+
const weighted = createService({ store, mirror: null, config: { signalTransparency: true, trustEpistemicWeighting: true } });
314+
for (const s of [plain, weighted]) s.setVectorIndex(createVectorIndex({ store }));
315+
const opts = { mode: "auto", topK: 5, useRerank: false };
316+
const pRows = await plain.searchMemories("software engineer", opts);
317+
const wRows = await weighted.searchMemories("software engineer", opts);
318+
const pScore = Object.fromEntries(pRows.map((r) => [r.id, r.signals?.final]));
319+
const wScore = Object.fromEntries(wRows.map((r) => [r.id, r.signals?.final]));
320+
assert.ok(pScore[fact.memory.id] > 0, "observation entry is a candidate");
321+
assert.ok(
322+
Math.abs(wScore[fact.memory.id] - pScore[fact.memory.id]) < 1e-9,
323+
"observation keeps weight 1.0 — if scaled, the explicit field was dropped (all rows uniform subjective)"
324+
);
325+
});

0 commit comments

Comments
 (0)