feat(usage): sync aggregate Hermes usage - #6120

@codex review

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 4c4d65c15c

Reviewed the full diff (25 files, +4811/−123) against the issue's proposed solution — this is a solid, first-class implementation. I went through the Rust sync path (session_usage_hermes.rs), the DB migration (v17→v18 with dedup/repair), and the frontend wiring. Highlights:

What's done well

• discover_hermes_databases() covers both the default state.db and profiles/*/state.db — matches how Hermes actually layouts multi-profile homes, and the per-source error isolation ("keep valid Profiles usable when one source is corrupt") is the right failure semantics.
• Aggregate-delta sync with sync_window_start/end + source-incarnation dedup is a clean design: idempotent across re-scans, and it deliberately scopes out per-request status/latency (documented in the data-source breakdown) instead of pretending to have data it doesn't.
• Tests use a temp Hermes root via sync_hermes_usage_at_root — properly testable without touching the user's live home. Nice.
• Wiring: it rides the existing sync_all_unlocked scheduler, same as OpenCode/Pi/Grok — consistent with the app's current architecture.

Two non-blocking observations before merge

1. mergeable is UNKNOWN and the branch is ~10 days behind main — the PR has sat untouched since Aug 16 after the Codex review passed and all CI is green. A rebase onto current main would confirm no drift (usage tables/indexes have been touched in v18 migrations since), and would likely also flip the mergeability check.

2. Sync cadence for Hermes: since Hermes only writes usage counters when sessions actually run, the aggregate-delta model means a long-lived desktop session's numbers appear only after the next sync_all_unlocked pass. If the app's sync trigger is dashboard-open/timer, that's fine; worth one sentence in the PR description (or a follow-up) documenting when Hermes deltas are expected to appear, so users don't file "usage not updating" issues against this feature.

Net: implementation quality is merge-ready from my read — I'd love to see this land and iterate on cadence/UX in follow-ups rather than block on polish. @farion1231 if the size is the concern, the diff is cleanly separated (Rust service + migration + frontend) and could even be merged in two steps if that helps.

(For context: I use both cc-switch and Hermes Agent daily — this is exactly the missing integration; happy to help with follow-ups, e.g. sync-cadence docs or a usage-sync trigger test.)

Thanks @xrbs00 - saw the origin/main merge onto pr-6120. Verified locally against the new head (fbd9c09ca0): no conflicts, mergeability flipped UNKNOWN -> true, so no drift from the v18 usage-table changes.

Checks had not started reporting yet when I looked (~10 min after your push), so just worth a glance once they land.

Observation 2 (one-line note in the description about when Hermes aggregate deltas are expected to appear between sync_all_unlocked passes) remains optional - not blocking anything.

@farion1231 快合并这个功能

Sharing two findings from implementing the same goal user-side, in case they're useful for this PR.

I built a user-side sync for Hermes → CC Switch usage dashboard and ran it against a real Hermes install. Two things came out of it that seem relevant to the discussion here.

1. The "per-call billing is not possible" concern may be outdated — but not in session_model_usage

The maintainer previously removed Hermes usage tracking, citing that all API calls in a process collapse into a single sessions row with the model field locked to the initial model.

Reading session_model_usage instead of sessions does resolve the model-attribution half of that problem: it is keyed per (session_id, model, task), so attribution is per model. From one real session I get three independently-attributed rows:

upstage/solar-pro4:free   task=(main)               13144 in / 107 out
upstage/solar-pro4:free   task=title_generation        57 in /  24 out
step-3.7-flash            task=(main)               42343 in / 2483 out
nemotron-3.5-lightning-free task=(main)             29519 in /  703 out

But session_model_usage is still cumulative, so it cannot give per-request rows either. I did not find a way to get true request-level detail from it.

There is, however, a table I have not seen discussed: messages, which has timestamp, token_count and finish_reason per row — i.e. request-level. In the install I tested, token_count was NULL for the rows I could inspect (I only had 2 messages to look at, so this is weak evidence, not a conclusion). If the current Hermes version does populate it, that would be the natural source for true per-request usage and would resolve the original objection outright. It may be worth checking whether that field is populated in practice before settling on an aggregate-only design.

2. The reason a proxy_request_logs-based importer cannot report a real request count

This one is structural, and I think it is the strongest argument for the dedicated-table design in this PR.

proxy_request_logs has 27 columns and none of them can hold an API call count. A dashboard built on it necessarily reports COUNT(*) as "total requests". So an importer that writes one row per sync window per (session, model, task) cannot make that number mean "API calls" — it will mean "sync windows in which something changed". In my measurements the same dataset that contained 3 distinct models produced 7 "requests", while the actual SUM(api_call_count) in the source was smaller.

Producing a truthful call count therefore seems to require either writing N rows per window (i.e. synthesising N requests that never existed, with the token split between them) or storing a counter in a table that has a counter column — which is what this PR does. I went with the second option being unavailable to me and documented the resulting limitation rather than fabricating rows.

Offer

If useful, I can test this PR against a real Hermes install and report the results — happy to test on Windows, and I can also check Linux if that is useful. I have no stake in the implementation choice; I mention this only because the messages.token_count question is the part I cannot answer from one small sample.

Update on this PR (head 25b8751):

• Merged current upstream main; GitHub now reports no merge conflict.
• Kept the existing read-only, cumulative Hermes importer for dashboard totals and sync-window trends. For request details, added an opt-in Hermes hook plugin: after enabling it for a profile and restarting Hermes, it records supported calls in a separate metadata-only ledger (time, route/model, status, latency, and usage when Hermes supplies it). It does not store prompts, responses, keys, or URLs, and it does not claim coverage of every request path. The exact rows are not added to aggregate totals.
• Before activation, users may manually import only a clearly labeled estimated historical total. We do not invent past request rows or daily timestamps. This also addresses the messages.token_count uncertainty without treating message rows as proven API-call records.
• The PR description now documents these boundaries and the new tests. Local checks: frontend 1706/1706 with one worker; Rust 2954 passed with two Windows symlink-privilege tests excluded; Python plugin 5/5; typecheck, formatting, Clippy, and renderer build passed. The default parallel frontend run previously hit two App integration timing failures, so I am not calling that mode green.

The current upstream CI run is action_required with no test jobs, so it has NOT validated this head yet. If a maintainer is comfortable approving the fork workflow, the remote checks can run; code-owner review remains outstanding. No Hermes runtime was enabled or restarted as part of this PR update.

@xrbs00 我把你这个插件在真机上跑了一天（Hermes v0.21.2 / Windows / CC Switch 3.20.4），
有一组数据想跟你对一下。其中一条是夸你的，另外几条是可以直接改的。

先说好消息：你的聚合逻辑是干净的

拿一个真实长会话（会话 id 20261003_162817_8e6dd5，累计 310 次调用），我把你的账本
和 Hermes 自己的聚合计数器逐项对账：

三项全等（插件 169 条事件 = 主循环 149 + background_review 19 + 1 条零散）。
我一开始怀疑你把 cache 重复计进去了——实测不是，input_tokens 用「非缓存输入」口径是对的。
这套聚合可以放心向维护者主张正确性。

另外纠正我自己的一个判断错误：background_review 你其实抓到了。
我原以为你只覆盖主循环，实际上子 agent 复用同一条 conversation_loop 代码路径，
post_api_request 会触发。19 次后台审查调用全部进了你的账本。

三条可以直接改的

1）post_auxiliary_call 这个钩子不存在——这是漏掉辅助任务的唯一根因

__init__.py 里这行：

for name in ("post_api_request", "api_request_error", "post_auxiliary_call"):
    ctx.register_hook(name, callback)

前两个在 VALID_HOOKS 里（37 个之一），第三个不在。全仓库 grep：

$ git grep -c post_auxiliary_call HEAD
（0 个文件）

所以 ctx.register_hook("post_auxiliary_call", ...) 是静默失败的。这同时解释了三件事：
hermes plugins validate 报错、plugins doctor 报 unknown hook、辅助任务一条都收不到。

同一会话按 task 分组：

漏了 141/310 = 45.5%。 approval 占大头，而且这是长任务里最高频的操作——
我这个会话 141 次调用全是点批准。

好消息是 approval 不需要上游改，VALID_HOOKS 里已有专用钩子
（docs/observability/README.md:197-198 有文档）：

• pre_approval_request — 审批请求展示/发出前
• post_approval_response — 用户响应或超时后

把 post_auxiliary_call 换成这两个，覆盖率就从 54.5% 提到约 100%。
（title_generation 确实没有对应钩子，那是上游的缺口，我另外报。）

2）plugin.yaml 缺 provides_hooks，validate 过不了

现在 manifest 只有三行（name / version / description）。
校验器读 provides_hooks（hermes_cli/plugin_dev.py:300-313），声明了但不在
VALID_HOOKS 里会直接报 unknown hook in provides_hooks。

name: ccswitch-usage
version: "1.0"
description: ...
provides_hooks:
  - post_api_request
  - api_request_error
  - pre_approval_request
  - post_approval_response

3）重试的 duration_ms 是轮级累计，不是单次

一个 404 重试 3 次的调用，三条事件 started_at_ms 相同，duration 分别是
1063 / 4186 / 9488 ms。反推真实单次耗时约 1063 / 3123 / 5302 ms——第 2、3 次记的是累计值。
如果导入时拿 duration 求平均，单次失败会被高估。建议加一个 attempt 序号，或改成记录增量。

一个技术提问

我核对 session_model_usage 的主键时发现它是 6 列：

PRIMARY KEY (session_id, model, billing_provider, billing_base_url, billing_mode, task)

上面那个会话里出现了两条 task='approval'（rowid 一条 140 次、一条 1 次），
靠 billing_base_url 区分（一条为空，一条是具体 base_url）。
如果 Rust 侧分组键只用 (session, model, task)，这两行会被并成一条——
我自己踩过这个坑，三轮同步后总量多算了 2 次调用 / 1994 input。

想确认你导入器那边分组用的是几列？如果也是 4 列，可能得跟着补。

自查

• 每个数字都能追到原始会话 id 和 SQL 查询
• 我自己判断错的地方（background_review）主动写进去了
• 没指责口气，每条都给了具体改法
• 区分了「插件没用专用钩子」（approval）和「上游真缺钩子」（title_generation）
• 不含我方仓库地址
• 覆盖率用最新一次读数（54.5%），不是早期快照

@Chunai-Bboy 感谢实测反馈，已逐条核查，并在 2bc6b199 做了最小修复。

1. 版本与插件声明：post_auxiliary_call 由 0e580956 加入，正式发布 v2026.9.24（0.21.5）已包含。插件现声明 Hermes ≥0.21.5，补全三个钩子的 provides_hooks，并在注册前检查钩子是否齐全；不兼容时明确拒绝采集。最低版本要求只针对可选采集插件，原有聚合同步不受影响。也请附上你那份 0.21.2 安装的源码 commit SHA，以确认版本差异。
2. 审批调用：没有改用审批 UI 钩子，因为它们记录审批展示与选择，人工操作也会触发，并不等于模型请求。智能审批的 call_llm(task="approval") 和标题生成等辅助模型调用由 post_auxiliary_call 采集。
3. 重试耗时：已复现相同开始时间下的累计耗时。现在明确标注主调用为“累计耗时，含重试与等待”，辅助调用显示单次尝试耗时；没有直接相减，以免把退避等待算成单次请求耗时。
4. 分组键：导入器已保留完整六列：session_id, model, billing_provider, billing_base_url_digest, billing_mode, task。base URL 使用稳定摘要区分，未缩减分组键或更改计数算法。

本地针对性验证：Python 8/8、前端 15/15、Rust 聚合/采集 20/20，类型与格式检查通过。

请在 Hermes 0.21.5（v2026.9.24）或更新、具备全部三个钩子的安装上重新安装更新后的插件并重启 Hermes，复测 approval/title_generation 与重试耗时显示。

@xrbs00 逐条复核完了。先说结论：你的回复全对，我评论里两条核心论断是错的，给你道歉。谢谢你点名要 SHA——正是它帮我把错误定位到了根上。

错误 1：post_auxiliary_call 存在，是我拿旧 checkout 查的

我 git grep 的对象是本机安装的 Hermes 源码，事后确认那是 0.21.2 build（commit 1021a0325696e9070e6659f95fcd84c3e7e114df，2026-09-11），落后上游 14,771 个提交。git merge-base 验证：钩子提交 0e580956 不在它的祖先里，所以我 grep 到 0 处就断言"这钩子不存在"。

上游早就有实现（agent/auxiliary_hooks.py）、有测试（tests/agent/test_auxiliary_hooks.py）、有文档（observer-hooks.md 明确写了 titling / compression / MoA / vision 全走 pre/post_auxiliary_call）。拿自己机器的陈旧快照当上游事实，是方法错误，不是运气问题。

顺带撤回我那句「换成 pre_approval_request/post_approval_response 覆盖率就能到 100%」——你判断得对，那两个钩子跟踪的是审批 UI 的展示与选择，人工点批准也会触发。真按我说的改，你的账本反而会被污染。

我评论里 45.5% / 100% 那些覆盖率数字全部作废：它们只在 0.21.2 上成立，在 0.21.5 上不存在这个缺口。

错误 2：六列分组键，你本来就是对的

提"如果也是 4 列要跟着补"之前，我应该先去读你的 session_usage_hermes.rs，而不是抛一个自己没验证过的假设。你的键本来就是六列（billing_base_url 还做了摘要）。这条是我多虑了，抱歉。

我在 v0.21.5 上复测了你的修复

升级后（v0.21.5，build af90026aa09949579bd423d24def3d38f743cde0）装插件 1.0.1：

同一会话、升级后同一窗口的对账

最后一条是给你的弹药，不是抱怨：第一次压缩（15:32:17，109s，in 63,881 / out 12,096）真实成功了——其后 main 的 prompt 从 3–10K 骤降到 1–2K，上下文确实被压了——但 session_model_usage 里 compression 行的 first_seen 恰好等于第二次压缩结束的时刻（15:59:18），第一次压缩的用量没进任何聚合行。

也就是说在 0.21.5 上，你的账本反而比 Hermes 官方聚合更完整。这值得写进 PR 描述当卖点，也值得给 Hermes 报一个 aux 丢账的 issue——需要的话我把时间线和数值整理成最小复现贴上来。

最后

谢谢你连我没说对的两条也认真核查，能改的当天就推了 2bc6b19，该坚持的（审批 UI 钩子那条，你说得完全对）也顶住了。这份 PR 的工程质量配得上早点 merge。CI 如果跑过这个 head，我可以把 validate / doctor / 8 个测试的输出连同上面这张对账表整理贴给维护者看。

我这边的同步（聚合走 session_model_usage、你的账本当可选增强）会继续跟你的结论对齐：明细与聚合分表、永不相加。

@xrbs00 这个 PR 的文档与诚实度在同类 PR 里非常突出（分阶段记录、明确标注哪些是估算、CI 状态如实说明"action_required 不能称为通过"）。

但 +7421/−133、45 个文件是超大包，且混合了至少三个主题（用量同步、Hook 采集插件新子系统、Profile 会话导入）——按 grab-bag 处理，将以此为依据建议关闭本 PR，欢迎拆分后单独重提，然后我们再逐一查看哪些可以被合并。

PR 混合的主题（建议拆分）

1. 只读聚合同步（Hermes 用量聚合导入 + 持久游标 + 分批 + 源表/列能力探测）；
2. Hook 采集插件（按 Profile 手动启用的 CC Switch Hook，独立 SQLite 账本，重启后从该时刻起记录）——这是一个新的采集面，需要先表态；
3. UI（可翻页请求列表、采集面板）与历史估算基线。

需要维护者确认的方向问题

1. [P1] Hook 采集插件是新子系统。

在 Hermes 侧安装 CC Switch Hook 插件、维护每 Profile 独立账本、注册时检查 VALID_HOOKS——这是新的数据采集路径。虽然作者声明"不记录提示词、响应正文、密钥或 URL"（好），但这个采集面是否纳入产品方向、与 main 现有 Hermes 会话用量导入的关系（补充还是替代），最好开 issue 进行讨论。

值得肯定的设计

• "主调用累计耗时，含重试与等待" vs "单次尝试耗时"的语义区分明确，不伪造提供商侧精确耗时；
• 历史基线标注为估算、不拆分到过去各天、不伪造请求记录——诚实标注的范例；
• 含 YAML 锚点/注释的 plugins 配置一键启用安全拒绝，普通配置先备份验证；
• 最低版本检查实际钩子，缺少时明确拒绝不静默漏。