Skip to content

Commit c9670fd

Browse files
committed
feat(gui): add routing profiles and why-this-route views (RI-10)
1 parent 68d3aa0 commit c9670fd

18 files changed

Lines changed: 765 additions & 6 deletions

File tree

‎devlog/_plan/260804_router_intelligence/001_pr_stack_status.md‎

Lines changed: 36 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,8 @@ other; closing one is a maintainer decision and neither is stale.
4848
| RI-06 | `feat/ri-06-health-aware-routing` | `dev` (post-#1012 merge) | `af692bb7a` | #1013 | https://github.com/lidge-jun/opencodex/pull/1013 | MERGED |
4949
| RI-07 | `feat/ri-07-quota-aware-routing` | `dev` (post-#1013 merge) | `1f07c00b8` | #1014 | https://github.com/lidge-jun/opencodex/pull/1014 | MERGED |
5050
| RI-08 | `feat/ri-08-cost-aware-routing` | `dev` (post-#1014 merge) | `410db97e4` | #1015 | https://github.com/lidge-jun/opencodex/pull/1015 | MERGED |
51-
| RI-09 | `feat/ri-09-route-explainability-api` | `dev` (post-#1015 merge) | `d887c1202` | #1016 | https://github.com/lidge-jun/opencodex/pull/1016 | rebased / review in progress |
52-
| RI-10 | `feat/ri-10-routing-intelligence-ui` | `feat/ri-09` head | pending | pending | pending | queued |
51+
| RI-09 | `feat/ri-09-route-explainability-api` | `dev` (post-#1015 merge) | `68d3aa083` | #1016 | https://github.com/lidge-jun/opencodex/pull/1016 | MERGED |
52+
| RI-10 | `feat/ri-10-routing-intelligence-ui` | `dev` (post-#1016 merge; rebasing) | pending | #1018 | https://github.com/lidge-jun/opencodex/pull/1018 | OPEN |
5353

5454
## Per-PR acceptance log
5555

@@ -270,7 +270,7 @@ other; closing one is a maintainer decision and neither is stale.
270270
(5) assembleCandidateEvidence typed as `OcxConfig` after the RI-08 health/
271271
quota/cost merge.
272272
- Final commit: `d887c120282c8d381f92cd11c1eebe2373a27281`
273-
- PR: #1016 https://github.com/lidge-jun/opencodex/pull/1016
273+
- PR: #1016 https://github.com/lidge-jun/opencodex/pull/1016 (MERGED)
274274
- Verification:
275275
- `bun x tsc --noEmit`: PASSED (0 errors)
276276
- `bun run test tests/route-explainability.test.ts`: 10/10 pass -
@@ -282,6 +282,39 @@ other; closing one is a maintainer decision and neither is stale.
282282
- `bun run privacy:scan`: passed
283283
- Remaining Low findings: none
284284

285+
### RI-10 - feat/ri-10-routing-intelligence-ui
286+
287+
- Base SHA: `68d3aa0836648ae1d7b592fc5aa3b30146c0886d` (`dev` after #1016 merge)
288+
- Reviewed commit: same as final (author self-review before push)
289+
- Findings (self-review): 4 fixed pre-push -
290+
1. GUI lint: hardcoded "profiles" in an error literal (i18n rule) - now a
291+
key-less status code;
292+
2. GUI lint: setState-in-effect for the initial load - deferred via
293+
setTimeout(0);
294+
3. missing `.table`/`.checkbox` CSS classes - added to styles.css reusing
295+
the `.tbl` grammar;
296+
4. dev-mode GUI session bootstrap cannot authenticate through the Vite
297+
proxy - the screenshot is captured same-origin against the production
298+
GUI served by the backend instead.
299+
- Final commit: pending (recorded after commit)
300+
- PR: #1018 https://github.com/lidge-jun/opencodex/pull/1018
301+
- Verification:
302+
- `bun x tsc --noEmit`: PASSED (0 errors)
303+
- `bun run lint:gui`: PASSED (0 errors)
304+
- `bun run build:gui`: PASSED (production build + prepare:package)
305+
- `bun run test` (12 routing suites): 258/258 pass
306+
- `bun run privacy:scan`: passed
307+
- docs-site `bun run build`: 216 pages built, PASSED
308+
- Locale parity: compile-checked TKey set (all six locales updated)
309+
- Screenshot: live same-origin capture of `#routing` (profiles + dry-run +
310+
analytics) against a temporary config; uploaded to the PR via comment
311+
attachment
312+
- Environment note: the temporary screenshot backend briefly rewrote the
313+
Codex/Grok fence to port 10200; restored with `ocx ensure` to the live
314+
proxy (10100) and verified. Test processes and temp files cleaned up.
315+
- Remaining Low findings: none
316+
- Remaining Low findings: none
317+
285318
## Baseline note
286319

287320
The full-suite baseline on this Windows machine did not complete within the

‎docs-site/src/content/docs/ja/reference/configuration/routing.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,3 +84,24 @@ selector の後には bare native OpenAI-family id だけを指定できます
8484
- 空ではない `inputModalities` 交差。省略されたメンバー値を `["text"]` として扱います。
8585

8686
コンテキスト メタデータのない裸のリレー ID、または接続されていないモダリティを持つターゲットは、カタログからコンボを削除します。同期によって概要の警告が表示され、ダッシュボードで **注意が必要** とマークされます。コンテキスト メタデータを追加し、モダリティを調整したり、検出可能な互換性のある機能を備えたターゲット モデルを追加したりできます。
87+
88+
## ルーティングポリシープロファイル(`config.routingProfiles`)
89+
90+
明示的に要求された `policy/<id>`(または設定されたエイリアス)が、固定された候補許可リストの中から、ハードな能力要件と決定的で説明可能なスコアリングで選択します。既存のモデル ID が暗黙的にプロファイルを通ることはありません。`candidates`(明示的な許可リスト)、オプションの `alias`、`require`(`minContextWindow`、`minQuotaHeadroom`、`tools`、`imageInput`、`structuredOutput`、`localOnly`、`remoteAllowed`、`encryptedCodexTasks`、`reasoningEffort`、`serviceTier`)、`optimize`(latency/health/cost/quota の重み)、`limits.maxEstimatedCostUsd`、`unknownEvidence`(allow/penalize/exclude)をサポートします。未知はゼロや無料にはなりません。
91+
92+
CLI: `ocx route policy list`、`ocx route policy show <id>`、`ocx route policy dry-run <id> --model-context <tokens> --tools`、`ocx route policy evaluate <id>`。
93+
94+
コンボは明示的な順序・重み付きターゲットのルーティングとフェイルオーバーです。ポリシープロファイルは、候補間の証拠に基づく選択です。
95+
96+
## リクエスト履歴とルーティング分析
97+
98+
- `GET /api/request-history` - 派生インデックス(`routing-history.sqlite`)からのカーソルページング付き全履歴。フィルタ: `provider`、`model`、`requestedModel`、`status`、`conversationId`、`surface`、`inboundProtocol`、`apiKeyId`、`profileId`、`fallback`、`from`、`to`。
99+
- `GET /api/request-history/:requestId/route-decision` - このルートが選ばれた理由(トレース、候補、除外、スコア、プロファイル+リビジョン、実行試行、結果)。
100+
- `GET /api/routing-analytics` - 成功率・失敗率・フォールバック率、p50/p95/p99 の所要時間と TTFT、不完全ストリーム率、クールダウン失敗数、成功あたりの推定コスト、カバレッジ、信頼度、切り捨てフラグ。
101+
- `GET /api/routing-profiles`、`POST /api/routing-profiles/dry-run` - プロファイル参照とドライラン評価(上流への送信なし)。
102+
103+
CLI: `ocx logs explain <request-id>`、`ocx logs rebuild-index`、`ocx logs index-status`。
104+
105+
## 移行
106+
107+
`routingProfiles` は任意の追加設定です。既存の設定ファイルと古い `usage.jsonl` 行はそのまま読み込めます。インデックスは使い捨てで、削除すると次回クエリ時に `usage.jsonl` から自動再構築されます。自動チューニングは行われません。

‎docs-site/src/content/docs/ko/reference/configuration/routing.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,3 +82,24 @@ combo는 목록에 오를 수 없더라도 계속 직접 라우팅할 수 있습
8282
- 비어 있지 않은 `inputModalities` 교집합. 생략된 member value는 `["text"]`로 취급합니다.
8383

8484
context metadata가 없는 bare relay id이거나 modalities가 서로 겹치지 않는 target이 있으면 combo가 catalog에서 빠집니다. sync는 summary warning을 내고 dashboard는 이를 **Needs attention**으로 표시합니다. context metadata를 추가하거나, modalities를 맞추거나, 발견 가능한 호환 capability를 가진 model을 대상으로 삼으십시오.
85+
86+
## 라우팅 정책 프로필 (`config.routingProfiles`)
87+
88+
명시적으로 요청된 `policy/<id>`(또는 설정된 별칭)가 고정된 후보 허용 목록에서 하드 능력 요구사항과 결정적·설명 가능한 점수로 선택합니다. 기존 모델 ID가 암시적으로 프로필을 통과하지 않습니다. `candidates`(명시적 허용 목록), 선택적 `alias`, `require`(`minContextWindow`, `minQuotaHeadroom`, `tools`, `imageInput`, `structuredOutput`, `localOnly`, `remoteAllowed`, `encryptedCodexTasks`, `reasoningEffort`, `serviceTier`), `optimize`(latency/health/cost/quota 가중치), `limits.maxEstimatedCostUsd`, `unknownEvidence`(allow/penalize/exclude)를 지원합니다. 알 수 없음은 0이나 무료가 되지 않습니다.
89+
90+
CLI: `ocx route policy list`, `ocx route policy show <id>`, `ocx route policy dry-run <id> --model-context <tokens> --tools`, `ocx route policy evaluate <id>`.
91+
92+
콤보는 명시적인 순서·가중치 대상 라우팅 및 장애 조치입니다. 정책 프로필은 후보 간 증거 기반 선택입니다.
93+
94+
## 요청 기록 및 라우팅 분석
95+
96+
- `GET /api/request-history` - 파생 인덱스(`routing-history.sqlite`)에서 커서 페이지네이션으로 전체 기록을 조회. 필터: `provider`, `model`, `requestedModel`, `status`, `conversationId`, `surface`, `inboundProtocol`, `apiKeyId`, `profileId`, `fallback`, `from`, `to`.
97+
- `GET /api/request-history/:requestId/route-decision` - 이 경로가 선택된 이유(추적, 후보, 제외, 점수, 프로필+리비전, 실행 시도, 결과).
98+
- `GET /api/routing-analytics` - 성공/실패/취소/폴백 비율, p50/p95/p99 소요 시간 및 TTFT, 불완전 스트림 비율, 쿨다운 실패 수, 성공 요청당 추정 비용, 커버리지, 신뢰도, 잘림 플래그.
99+
- `GET /api/routing-profiles`, `POST /api/routing-profiles/dry-run` - 프로필 조회와 드라이런 평가(업스트림 전송 없음).
100+
101+
CLI: `ocx logs explain <request-id>`, `ocx logs rebuild-index`, `ocx logs index-status`.
102+
103+
## 마이그레이션
104+
105+
`routingProfiles`는 선택적 추가 설정입니다. 기존 설정 파일과 이전 `usage.jsonl` 행은 그대로 읽힙니다. 인덱스는 일회용이며 삭제 시 다음 쿼리에서 `usage.jsonl`로 자동 재구축됩니다. 자동 튜닝은 없습니다.

‎docs-site/src/content/docs/reference/configuration/routing.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,3 +189,32 @@ Codex picker list it only when every target exposes capabilities that can be int
189189
A bare relay id with no context metadata or targets with disjoint modalities removes the combo from
190190
the catalog. Sync emits a summary warning and the dashboard marks it **Needs attention**. Add context
191191
metadata, align modalities, or target models with discoverable compatible capabilities.
192+
193+
## Request history and routing analytics
194+
195+
- `GET /api/request-history` - cursor-paginated full history from the derived
196+
index (`routing-history.sqlite`), with filters (`provider`, `model`,
197+
`requestedModel`, `status`, `conversationId`, `surface`, `inboundProtocol`,
198+
`apiKeyId`, `profileId`, `fallback`, `from`, `to`) and opaque `cursor`
199+
pagination. `GET /api/request-history/:requestId` returns one canonical row.
200+
- `GET /api/request-history/:requestId/route-decision` - the why-this-route
201+
explanation: trace (candidates, exclusions, score components, profile +
202+
revision), execution attempt sequence, and final outcome.
203+
- `GET /api/routing-analytics` - success/failure/cancelled/fallback rates,
204+
p50/p95/p99 duration and TTFT, incomplete-stream rate, cooldown-triggering
205+
failures, cost per successful request, coverage, confidence, and an
206+
explicit truncation flag.
207+
- `GET /api/routing-profiles`, `POST /api/routing-profiles/dry-run` - profile
208+
inspection and dry-run evaluation (no upstream dispatch).
209+
210+
CLI: `ocx logs explain <request-id>`, `ocx logs rebuild-index`,
211+
`ocx logs index-status`, `ocx route policy list | show | dry-run | evaluate`.
212+
213+
## Migration
214+
215+
`routingProfiles` is optional and additive: existing config files load
216+
unchanged. Old `usage.jsonl` rows without `routeDecision` parse unchanged.
217+
The history index is disposable - deleting `routing-history.sqlite` triggers
218+
an automatic rebuild from `usage.jsonl` on the next query; `ocx logs
219+
rebuild-index` forces one. Nothing in this system auto-tunes weights,
220+
budgets, or candidate sets.

‎docs-site/src/content/docs/ru/reference/configuration/routing.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,3 +99,24 @@ Combo остаётся доступной для прямой маршрутиз
9999
из каталога. Sync выводит итоговое предупреждение, а дашборд помечает её как **Needs attention**.
100100
Добавьте метаданные контекста, согласуйте модальности или выберите модели с обнаруживаемыми
101101
совместимыми возможностями.
102+
103+
## Профили маршрутизации (`config.routingProfiles`)
104+
105+
Явно запрошенный `policy/<id>` (или настроенный псевдоним) выбирает среди фиксированного разрешённого списка кандидатов по жёстким требованиям к возможностям и детерминированной объяснимой оценке. Существующие идентификаторы моделей никогда не проходят через профиль неявно. Поддерживаются: `candidates` (явный список), необязательный `alias`, `require` (`minContextWindow`, `minQuotaHeadroom`, `tools`, `imageInput`, `structuredOutput`, `localOnly`, `remoteAllowed`, `encryptedCodexTasks`, `reasoningEffort`, `serviceTier`), `optimize` (веса latency/health/cost/quota), `limits.maxEstimatedCostUsd`, `unknownEvidence` (allow/penalize/exclude). Неизвестное не становится нулём или бесплатным.
106+
107+
CLI: `ocx route policy list`, `ocx route policy show <id>`, `ocx route policy dry-run <id> --model-context <tokens> --tools`, `ocx route policy evaluate <id>`.
108+
109+
Комбо — это явная маршрутизация с порядком/весами и отказоустойчивостью. Профиль — это выбор на основе доказательств среди кандидатов.
110+
111+
## История запросов и аналитика маршрутизации
112+
113+
- `GET /api/request-history` - полная история с курсорной пагинацией из производного индекса (`routing-history.sqlite`). Фильтры: `provider`, `model`, `requestedModel`, `status`, `conversationId`, `surface`, `inboundProtocol`, `apiKeyId`, `profileId`, `fallback`, `from`, `to`.
114+
- `GET /api/request-history/:requestId/route-decision` - объяснение выбора маршрута (трасса, кандидаты, исключения, оценки, профиль+ревизия, попытки, результат).
115+
- `GET /api/routing-analytics` - доли успеха/отказа/отмены/фолбэка, p50/p95/p99 длительность и TTFT, доля неполных потоков, сбои кулдауна, оценка стоимости за успешный запрос, покрытие, доверие, флаг усечения.
116+
- `GET /api/routing-profiles`, `POST /api/routing-profiles/dry-run` - просмотр профилей и пробная оценка (без отправки запросов).
117+
118+
CLI: `ocx logs explain <request-id>`, `ocx logs rebuild-index`, `ocx logs index-status`.
119+
120+
## Миграция
121+
122+
`routingProfiles` — необязательная аддитивная настройка. Существующие конфиги и старые строки `usage.jsonl` загружаются без изменений. Индекс одноразовый: при удалении он автоматически перестраивается из `usage.jsonl` при следующем запросе. Автонастройки нет.

‎docs-site/src/content/docs/zh-cn/reference/configuration/routing.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,3 +89,24 @@ selector 校验、冲突规则和隐私说明见[提供方配置](/reference/con
8989
如果是一个没有上下文元数据的裸 relay id,或者目标之间的模态互不相交,combo 就会从
9090
目录中移除。同步时会输出一条汇总警告,仪表板会将其标记为 **Needs attention**。
9191
补充上下文元数据、对齐模态,或者把目标模型切换为可发现且兼容的能力。
92+
93+
## 路由策略配置文件(`config.routingProfiles`)
94+
95+
显式请求的 `policy/<id>`(或配置的别名)会在固定的候选白名单中,根据硬性能力要求与确定性、可解释的评分进行选择。现有模型 ID 永远不会隐式经过配置文件。支持 `candidates`(显式白名单)、可选 `alias`、`require`(`minContextWindow`、`minQuotaHeadroom`、`tools`、`imageInput`、`structuredOutput`、`localOnly`、`remoteAllowed`、`encryptedCodexTasks`、`reasoningEffort`、`serviceTier`)、`optimize`(latency/health/cost/quota 权重)、`limits.maxEstimatedCostUsd`、`unknownEvidence`(allow/penalize/exclude)。未知不会被当作零或免费。
96+
97+
CLI:`ocx route policy list`、`ocx route policy show <id>`、`ocx route policy dry-run <id> --model-context <tokens> --tools`、`ocx route policy evaluate <id>`。
98+
99+
组合是显式的有序/加权目标路由与故障转移;策略配置文件是基于证据在候选之间进行选择。
100+
101+
## 请求历史与路由分析
102+
103+
- `GET /api/request-history` - 从派生索引(`routing-history.sqlite`)进行游标分页的全历史查询。过滤器:`provider`、`model`、`requestedModel`、`status`、`conversationId`、`surface`、`inboundProtocol`、`apiKeyId`、`profileId`、`fallback`、`from`、`to`。
104+
- `GET /api/request-history/:requestId/route-decision` - 为什么选择此路由(跟踪、候选、排除、分数、配置文件+版本、执行尝试、结果)。
105+
- `GET /api/routing-analytics` - 成功/失败/取消/回退率、p50/p95/p99 耗时与 TTFT、不完整流率、冷却失败数、每次成功请求的估算成本、覆盖率、置信度、截断标志。
106+
- `GET /api/routing-profiles`、`POST /api/routing-profiles/dry-run` - 配置文件查看与试运行评估(不发送上游请求)。
107+
108+
CLI:`ocx logs explain <request-id>`、`ocx logs rebuild-index`、`ocx logs index-status`。
109+
110+
## 迁移
111+
112+
`routingProfiles` 是可选的增量配置:现有配置文件与旧 `usage.jsonl` 行均可原样加载。索引是一次性的——删除后会在下次查询时从 `usage.jsonl` 自动重建。系统不会自动调优。

0 commit comments

Comments
 (0)