Skip to content

Commit d2df0b1

Browse files
committed
refactor(api): provenance 读出口改用 token 门控并抽共用模块(#509 评审)
维护者四条评审意见,逐条落实: 1. policy open → token。records 带蒸馏窗口的逐字会话原文,与开放的路由 串起来即可「先枚举 id、再按 id 捞回原文」,可达结果与 /export 同量级、 只是换了形状。SECURITY.md 中英两份同步从开放读移入受门控,并写明界线。 2. 两张皮的整段复制抽成 src/provenance-http.js(照 reclaim-http.js 的先例), 配静态锁:口径字面量与共用答复只许出现在该模块,两个入口都要真的走它。 3. `?? []` 把「装配没接线」与「这条记忆没有来源」压成同一读数——前者改回 503 provenance-unavailable(与 RECLAIM_UNAVAILABLE 同构),后者仍是空 windows 200。 4. 范围缩回「区间 + 原文」:#421 要的 distill_run_id / run_type / trigger_source 三个键在库里不存在(蒸馏归因没有 run 概念),补它们要改 schema + 写入路径,不属本 PR。 Refs #509
1 parent 7c072dc commit d2df0b1

9 files changed

Lines changed: 344 additions & 85 deletions

File tree

‎SECURITY.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ The plugin can optionally start its own HTTP server (`externalApiEnabled`, defau
5151

5252
- **Loopback by default**: the server binds `127.0.0.1:8790` (`externalApiHost` / `externalApiPort`, or persisted panel settings). Rebinding to a non-loopback address exposes the store to your network and is an explicit operator decision.
5353
- **Token authentication**: every route except `GET /health` requires `Authorization: Bearer <token>`; the token is auto-generated on first use, persisted in the plugin settings store and verified with `timingSafeEqual`. To rotate it, clear the persisted value (settings kv key `external_api`) and let the next boot mint a new one — there is no rotate endpoint yet.
54-
- **Surface**: memories read/write (`GET/POST /memories`, `GET/PUT/DELETE /memories/:id`, `GET /search`, `GET /context`, `GET /status`), plus profile/rules **reads** (`GET /profile`, `GET /rules`), `POST /bootstrap` and `POST /maintenance/reclaim` (the host-side twin `/api/dsh-mneme/maintenance/reclaim` also answers `GET` for dry-run previews). Writes go through the same dedupe + quality-filter path as in-conversation saves. The full route list lives in [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md).
54+
- **Surface**: memories read/write (`GET/POST /memories`, `GET/PUT/DELETE /memories/:id`, `GET /search`, `GET /context`, `GET /status`), profile/rules **reads** (`GET /profile`, `GET /rules`), `GET /memory/provenance`, `POST /bootstrap` and `POST /maintenance/reclaim` (the host-side twin `/api/dsh-mneme/maintenance/reclaim` also answers `GET` for dry-run previews). Writes go through the same dedupe + quality-filter path as in-conversation saves. The full route list lives in [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md).
5555
- **CLI**: `bin/cli.mjs` is a thin client over this API and stores its credentials in `~/.dsh-mneme/cli.json` - treat that file like any credential file.
5656

5757
---
@@ -153,8 +153,8 @@ The token is a lock on the **write** surface, not a wall around the whole API.
153153

154154
- **The real boundary is loopback + the DSH host listener.** The host-side routes (`/api/dsh-mneme/*`) are mounted on the DSH port, which binds `127.0.0.1` and is fenced by the host against DNS rebinding and cross-site browser requests. That fence does **not** establish identity: any local process, and any page running in the host's own origin, is inside the perimeter.
155155
- **Authentication is opt-in and fail-open.** `apiToken` defaults to empty; while it is empty, `isAuthorized()` returns true for every request — reads *and* writes. Setting a token gates the write routes and the sensitive reads; it does not gate plain reads.
156-
- **Open reads (no token even when one is configured)**: `GET /`, `/info`, `/version-check`, `/list`, `/recall-stats`, `/search`, `/semantic`, `/semantic/llm-audit`, `/semantic/llm-audit/stats`, `/semantic/graph/*`, `/entities`, `/memories/entities`, `/dream-status`, `/inject-status`, `/inject-preview`, `/pipeline`, `/candidates`, `/llm-providers`, `/memory/provenance`, plus the `GET`/`HEAD` branches of `/profile`, `/rules`, `/persona`, `/mode`, `/features`, `/commands`, `/nudge`. Treat them as readable by anything local.
157-
- **Gated routes (when a token is configured)**: every write — `/create`, `/update`, `/delete`, `/import`, `/vector-config`, `/vector-reindex`, `/runtime/provision`, `/maintenance/reclaim`, `/test-model`, `/conflicts/resolve`, `/profile/extract` (POST-only: one host-LLM call that returns a profile **draft**; other methods get 405 before any side effect), and the non-`GET` branches of `/profile`, `/rules`, `/persona`, `/mode`, `/features`, `/commands`, `/nudge` — plus four reads: `/export`, `/external-api`, `/conflicts`, `/health`.
156+
- **Open reads (no token even when one is configured)**: `GET /`, `/info`, `/version-check`, `/list`, `/recall-stats`, `/search`, `/semantic`, `/semantic/llm-audit`, `/semantic/llm-audit/stats`, `/semantic/graph/*`, `/entities`, `/memories/entities`, `/dream-status`, `/inject-status`, `/inject-preview`, `/pipeline`, `/candidates`, `/llm-providers`, plus the `GET`/`HEAD` branches of `/profile`, `/rules`, `/persona`, `/mode`, `/features`, `/commands`, `/nudge`. Treat them as readable by anything local.
157+
- **Gated routes (when a token is configured)**: every write — `/create`, `/update`, `/delete`, `/import`, `/vector-config`, `/vector-reindex`, `/runtime/provision`, `/maintenance/reclaim`, `/test-model`, `/conflicts/resolve`, `/profile/extract` (POST-only: one host-LLM call that returns a profile **draft**; other methods get 405 before any side effect), and the non-`GET` branches of `/profile`, `/rules`, `/persona`, `/mode`, `/features`, `/commands`, `/nudge` — plus five reads: `/export`, `/external-api`, `/conflicts`, `/health`, `/memory/provenance`. That last one is gated because it returns the **verbatim raw text** of distilled conversation windows (opt-in via `distillRecords.enabled`); chained with the open list route it would otherwise let any local process enumerate ids and then pull that text back, which reaches the same result as `/export` in a different shape.
158158
- **The standalone data plane** (`127.0.0.1:8790`) is the opposite shape: every route except `GET /health` requires its own Bearer token. Pointing `externalApiHost` at a non-loopback address therefore turns it into a plaintext, token-only HTTP surface — an explicit operator decision.
159159
- **Consequence**: if you expose the plugin beyond loopback (reverse proxy, non-loopback bind), set `apiToken` *and* check that the plain reads above are acceptable to expose — they are not covered by it.
160160
- **How this list stays true**: every route declares its own `policy` (`open` / `token` / `token-write`) at registration, and `createRouteRegistrar` throws when a route has none or reuses a path. A new route that forgets its classification **fails to boot** instead of silently opening. The two lists above are compared against the routes actually mounted by `dsh-mneme/test/security-doc.test.js` in both directions, so adding a route without updating this section fails CI rather than quietly drifting — the matrix is derived from code, not from anyone's memory.
@@ -432,7 +432,7 @@ This project is licensed under the **MIT License**. See [LICENSE](https://github
432432

433433
- **默认只绑回环**:服务绑 `127.0.0.1:8790`(`externalApiHost` / `externalApiPort`,或面板里持久化的设置)。改绑非回环地址等于把记忆库暴露给你的网络,这是运营者的显式决定。
434434
- **Token 鉴权**:除 `GET /health` 外每条路由都要 `Authorization: Bearer <token>`;token 首次使用时自动生成、持久化在插件设置库、用 `timingSafeEqual` 校验。要轮换就清掉持久化的值(设置 kv 键 `external_api`),下次启动会重新铸造——目前没有 rotate 端点。
435-
- **路由面**:记忆读写(`GET/POST /memories`、`GET/PUT/DELETE /memories/:id`、`GET /search`、`GET /context`、`GET /status`),profile/rules 的**读**(`GET /profile`、`GET /rules`),以及 `POST /bootstrap` 与 `POST /maintenance/reclaim`(宿主侧孪生路由 `/api/dsh-mneme/maintenance/reclaim` 另外还答 `GET`,用于 dry-run 预览)。写入走与对话内保存相同的去重 + 质量过滤路径。完整路由表见 [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md)。
435+
- **路由面**:记忆读写(`GET/POST /memories`、`GET/PUT/DELETE /memories/:id`、`GET /search`、`GET /context`、`GET /status`),profile/rules 的**读**(`GET /profile`、`GET /rules`),`GET /memory/provenance`,以及 `POST /bootstrap` 与 `POST /maintenance/reclaim`(宿主侧孪生路由 `/api/dsh-mneme/maintenance/reclaim` 另外还答 `GET`,用于 dry-run 预览)。写入走与对话内保存相同的去重 + 质量过滤路径。完整路由表见 [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md)。
436436
- **CLI**:`bin/cli.mjs` 是这个 API 上的薄客户端,凭据存在 `~/.dsh-mneme/cli.json`——请把该文件当凭据文件对待。
437437

438438
---
@@ -534,8 +534,8 @@ Token 是**写**面上的锁,不是把整个 API 围起来的墙。
534534

535535
- **真正的边界是回环 + 宿主监听器。** 宿主侧路由(`/api/dsh-mneme/*`)挂在 DSH 端口上,该端口绑 `127.0.0.1`,并由宿主挡掉 DNS rebinding 与跨站浏览器请求。这道围栏**不建立身份**:任何本地进程、以及任何跑在宿主同源页面里的脚本,都在这道围栏之内。
536536
- **鉴权是 opt-in 且 fail-open。** `apiToken` 默认为空;为空期间 `isAuthorized()` 对每个请求都返回 true——读和写都放行。配了 token 之后,它守的是写路由与敏感读,不守普通读。
537-
- **开放读(即使配了 token 也不需要)**:`GET /`、`/info`、`/version-check`、`/list`、`/recall-stats`、`/search`、`/semantic`、`/semantic/llm-audit`、`/semantic/llm-audit/stats`、`/semantic/graph/*`、`/entities`、`/memories/entities`、`/dream-status`、`/inject-status`、`/inject-preview`、`/pipeline`、`/candidates`、`/llm-providers`、`/memory/provenance`,以及 `/profile`、`/rules`、`/persona`、`/mode`、`/features`、`/commands`、`/nudge` 的 `GET`/`HEAD` 分支。把它们当作「本地任何东西都能读」。
538-
- **受门控的路由(配了 token 之后)**:全部写操作——`/create`、`/update`、`/delete`、`/import`、`/vector-config`、`/vector-reindex`、`/runtime/provision`、`/maintenance/reclaim`、`/test-model`、`/conflicts/resolve`、`/profile/extract`(只收 POST:花一次宿主 LLM 产一段画像**草稿**,其它方法在产生任何副作用之前就 405),以及 `/profile`、`/rules`、`/persona`、`/mode`、`/features`、`/commands`、`/nudge` 的非 `GET` 分支——外加四条读:`/export`、`/external-api`、`/conflicts`、`/health`。
537+
- **开放读(即使配了 token 也不需要)**:`GET /`、`/info`、`/version-check`、`/list`、`/recall-stats`、`/search`、`/semantic`、`/semantic/llm-audit`、`/semantic/llm-audit/stats`、`/semantic/graph/*`、`/entities`、`/memories/entities`、`/dream-status`、`/inject-status`、`/inject-preview`、`/pipeline`、`/candidates`、`/llm-providers`,以及 `/profile`、`/rules`、`/persona`、`/mode`、`/features`、`/commands`、`/nudge` 的 `GET`/`HEAD` 分支。把它们当作「本地任何东西都能读」。
538+
- **受门控的路由(配了 token 之后)**:全部写操作——`/create`、`/update`、`/delete`、`/import`、`/vector-config`、`/vector-reindex`、`/runtime/provision`、`/maintenance/reclaim`、`/test-model`、`/conflicts/resolve`、`/profile/extract`(只收 POST:花一次宿主 LLM 产一段画像**草稿**,其它方法在产生任何副作用之前就 405),以及 `/profile`、`/rules`、`/persona`、`/mode`、`/features`、`/commands`、`/nudge` 的非 `GET` 分支——外加五条读:`/export`、`/external-api`、`/conflicts`、`/health`、`/memory/provenance`。最后这条受门控是因为它回的是蒸馏窗口的**逐字会话原文**(`distillRecords.enabled` 打开才有):否则把它和开放的那个列表路由串起来,任何本地进程都能先枚举 id、再按 id 把那些原文捞回去——可达结果与 `/export`(一次拿全库)同量级,只是换了形状。
539539
- **独立数据面**(`127.0.0.1:8790`)是反过来的形状:除 `GET /health` 外每条路由都要自己的 Bearer token。因此把 `externalApiHost` 指向非回环地址,就等于把它变成一个「明文、只靠 token」的 HTTP 面——这是运营者的显式决定。
540540
- **推论**:如果你把插件暴露到回环之外(反代、绑非回环地址),请设 `apiToken`,**并**核对上面那串开放读是否也能被看到——它们不受 token 保护。
541541
- **这张表怎么保持为真**:每条路由在注册时自己声明 `policy`(`open` / `token` / `token-write`),`createRouteRegistrar` 在路由没声明或复用同一路径时直接抛错。新路由忘了归类会**启动失败**,而不是静默开门。上面两张清单还会被 `dsh-mneme/test/security-doc.test.js` 拿真实装配出来的路由做双向比对——加了路由却不更新这一节,CI 会红,而不是悄悄漂走。这张矩阵来自代码,不来自谁的记忆。

‎dsh-mneme/lib/api-standalone.js‎

Lines changed: 13 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import { TYPES } from "./store.js";
1919
import { bootstrapFromDirectory } from "./bootstrap.js";
2020
import { PACKAGE_VERSION } from "./version-check.js";
2121
import { parseReclaimJson, readReclaimFields, reclaimReply, RECLAIM_UNAVAILABLE } from "./reclaim-http.js";
22+
import { parseProvenanceQuery, provenanceReply } from "./provenance-http.js";
2223

2324
const DEFAULT_PORT = 8790;
2425
const DEFAULT_HOST = "127.0.0.1";
@@ -415,29 +416,21 @@ export function createStandaloneApi({ service, store, config = {}, logger, setti
415416
}
416417

417418
// --- GET /memory/provenance: 记忆级溯源读出口(#399 写、#421 读)--------
418-
// 与宿主内 /api/dsh-mneme/memory/provenance 同一份实现口径:一个链接一个
419-
// 窗口(区间 + 区间内原文行,按 seq 升序)。没有链接的记忆返回空 windows
420-
// ——「不是蒸馏来的」是有效答案,不是错误。daemon 与宿主共用 service,
421-
// 所以这里不需要另算一遍。
419+
// 与宿主内 /api/dsh-mneme/memory/provenance 同一份实现口径,就在
420+
// src/provenance-http.js 里写一份(#411 的教训:两张皮整段复制必然漂移)。
421+
// 一个链接一个窗口(区间 + 区间内原文行,按 seq 升序);没有链接的记忆返回
422+
// 空 windows——「不是蒸馏来的」是有效答案,不是错误。
423+
//
424+
// 这张面本来就要 Bearer token(除 /health 外无例外),所以不涉及宿主侧那次
425+
// open → token 的归类调整;daemon 与宿主共用 service,口径天然一致。
422426
if (req.method === "GET" && pathname === "/memory/provenance") {
423-
const id = (url.searchParams.get("id") ?? "").trim();
424-
if (!id) {
425-
sendJson(res, 400, { error: "missing-id" });
427+
const parsed = parseProvenanceQuery(url.searchParams.get("id"));
428+
if (!parsed.ok) {
429+
sendJson(res, parsed.status, { error: parsed.error });
426430
return;
427431
}
428-
try {
429-
const links = service.provenanceFor?.(id) ?? [];
430-
const windows = links.map((link) => ({
431-
session_id: link.session_id,
432-
seq_start: link.seq_start,
433-
seq_end: link.seq_end,
434-
created_at: link.created_at,
435-
records: service.distillRecordsFor?.(link.session_id, link.seq_start, link.seq_end) ?? []
436-
}));
437-
sendJson(res, 200, { memoryId: id, windows });
438-
} catch {
439-
sendJson(res, 500, { error: "internal" });
440-
}
432+
const { status, body } = provenanceReply({ service, id: parsed.id, warn: (m) => logger?.warn?.(m) });
433+
sendJson(res, status, body);
441434
return;
442435
}
443436

‎dsh-mneme/lib/api.js‎

Lines changed: 15 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import { computeHeat } from "./heat.js";
1010
import { EFFORT_REJECT_RE, describeStreamFailure, resolveRoute, withEffortFallback } from "./dream.js";
1111
import { getInjectionSnapshot } from "./inject.js";
1212
import { renderUserSettingsBlock } from "./user-settings.js";
13+
import { parseProvenanceQuery, provenanceReply } from "./provenance-http.js";
1314
import { describeLocalRuntime, publicRuntimeStatus } from "./runtime/loader.js";
1415
import { hostModulesDir, provisionRuntime } from "./runtime/provision.js";
1516
import { classify, fetchLatestVersion, PACKAGE_VERSION } from "./version-check.js";
@@ -1294,7 +1295,6 @@ export function createApi(ctx, service, settings, commands, embedder, semantic =
12941295
});
12951296

12961297
// --- 记忆级溯源读出口(#399 写、#421 读)-----------------------------------
1297-
// Read-only like list/search/semantic, so it stays open when apiToken is set.
12981298
// #399 把蒸馏窗口的原文落进 distill_records、把 memory → (session, seq 区间)
12991299
// 的归因写进 memory_provenance,但只做了写:库里有答案、service.provenanceFor
13001300
// 能查,外部(面板 / standalone API / 脚本)拿不到,「这条记忆是从哪几句蒸出来
@@ -1303,29 +1303,28 @@ export function createApi(ctx, service, settings, commands, embedder, semantic =
13031303
// 一个链接一个窗口:区间 + 区间里的原文行(按 seq 升序)。没有链接的记忆
13041304
// (memory_save / 工具写入 / 冷启动)返回空 windows——「不是蒸馏来的」本身是
13051305
// 有效答案,不是错误,所以不 404。
1306+
//
1307+
// policy 是 **token**,不是 open(#509 维护者评审定):records 带的是蒸馏窗口的
1308+
// 逐条会话原文,而 /list 是 open 且逐条带 id——两者串起来等于「先枚举 id、再按 id
1309+
// 捞回原文」,可达结果与 /export(一次拿全库,本仓 2026-10 审计后列 gated)同量级,
1310+
// 只是换了形状。抽稀不该靠「调用方不知道有这条路」。
1311+
//
1312+
// 口径只在 src/provenance-http.js 写一份,两个入口都调它(#411 的教训:两张皮
1313+
// 整段复制必然漂移)。
13061314
register({
13071315
kind: "exact",
13081316
path: "/api/dsh-mneme/memory/provenance",
1309-
policy: "open",
1317+
policy: "token",
13101318
handler(req, res) {
13111319
try {
13121320
const url = new URL(req.url, "http://localhost");
1313-
const id = (url.searchParams.get("id") ?? "").trim();
1314-
if (!id) {
1315-
sendJson(res, 400, { error: "missing-id" });
1321+
const parsed = parseProvenanceQuery(url.searchParams.get("id"));
1322+
if (!parsed.ok) {
1323+
sendJson(res, parsed.status, { error: parsed.error });
13161324
return;
13171325
}
1318-
const links = service.provenanceFor?.(id) ?? [];
1319-
const windows = links.map((link) => ({
1320-
session_id: link.session_id,
1321-
seq_start: link.seq_start,
1322-
seq_end: link.seq_end,
1323-
created_at: link.created_at,
1324-
// 到期裁剪后区间可能变空:链接刻意保留并指向空区间(clearDistillRecords
1325-
// 的语义),面板据此显示「原文已过期」而非「没有来源」。
1326-
records: service.distillRecordsFor?.(link.session_id, link.seq_start, link.seq_end) ?? []
1327-
}));
1328-
sendJson(res, 200, { memoryId: id, windows });
1326+
const { status, body } = provenanceReply({ service, id: parsed.id, warn: (m) => logger?.warn?.(m) });
1327+
sendJson(res, status, body);
13291328
} catch {
13301329
sendJson(res, 500, { error: "internal" });
13311330
}

0 commit comments

Comments
 (0)