feat(security): RBAC M5 — audit query/export + per-user budget API (#3054)#3203
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
RBAC M5 introduces per-user spending caps so an operator can limit individual users on top of the existing global / per-agent / per-provider budgets. UserConfig gains an optional UserBudgetConfig with the same hourly / daily / monthly window structure as BudgetConfig plus an alert_threshold (0..=1) shared with the per-agent path. None means "no per-user cap" — the user is still bounded by the other layers. Refs #3054
…udit variants Append four RBAC-themed variants to AuditAction so the new admin-only endpoints in librefang-api can correlate denials and quota breaches with the user / channel that triggered them. The hash chain stays intact because the new variants only add fresh rows — pre-existing entries hash identically (their action string is unchanged). A regression test pins the Display output of every new variant so a casual rename surfaces as a test failure rather than a silent invalidation of every persisted hash. Refs #3054
RBAC M5 needs per-user budget detail and ranking endpoints. Adding user_id / channel columns to usage_events lets the metering pipeline attribute every LLM call to the caller (resolved from SenderContext via auth.identify) so the dashboard can roll spend up by user without scanning the whole table. - Schema v23 migration adds NULL-able user_id / channel columns and indexes both (idx_usage_user_time, idx_usage_channel_time). Pre-M5 rows return NULL — they fall outside any per-user filter, which is the right default since they pre-date attribution. - UsageRecord gains optional user_id / channel fields plus an anonymous() helper for kernel-internal call sites that don't carry a SenderContext (background_skill_review, ephemeral side-questions). - New UsageStore queries: query_user_hourly / _daily / _monthly and a query_user_ranking that aggregates all three windows in a single CASE-when round-trip. - Kernel cost-attribution wiring: execute_llm_agent and the streaming spawn now resolve UserId from sender_context (via auth.identify) and stamp every UsageRecord with it. When check_all_and_record returns QuotaExceeded the kernel now also records a BudgetExceeded audit entry so an operator can correlate denials with users. Refs #3054
Lands the four endpoints requested by RBAC M5:
- GET /api/audit/query — filter the hash-chained audit log by user,
action, agent, channel, and ISO-8601 time range. Newest-first, hard
cap 5000 rows. Filtering is in-memory (the AuditLog::recent path
already verifies the chain on read), so SQL injection surface is
zero.
- GET /api/audit/export?format=json|csv — same filters, streamed via
axum::body::Body::from_stream so the kernel never buffers a full
result set. CSV emits a fixed header row with RFC-4180 cell
escaping pinned by a regression test.
- GET /api/budget/users — per-user cost ranking (top N) joined with
the user's configured limits when present so the dashboard can
render % bars without a follow-up call.
- GET /api/budget/users/{user_id} — single-user detail with hourly /
daily / monthly spend, alert threshold, and an alert_breach flag.
Accepts both the canonical UUID and the raw configured name
(re-derived via UserId::from_name) so operators can paste either.
All four endpoints are admin-only and gate in-handler against
UserRole::Admin — they stay AUTHENTICATED (not on the is_public
allowlist). A 403 also fires a PermissionDenied audit event so an
operator can spot probes via /api/audit/query.
Middleware glue:
- AuthState gains an optional audit_log handle; when a user-scoped
role check returns FORBIDDEN we now record PermissionDenied with
the caller's UserId so the chain shows who tripped which path.
- The dirty middleware/config glue from PR #3196 (threading
authenticated UserId through to record_with_context) is folded in
unchanged.
OpenAPI registers all four new operations.
Refs #3054
The /api/shutdown, /api/config/reload, and /api/config/set handlers now accept the AuthenticatedApiUser extension and pass the caller's UserId + a 'api' channel through to record_with_context. Combined with the RBAC M5 audit chain, an operator can now see exactly which user triggered each ConfigChange entry instead of an anonymous 'system' record. This is the dirty middleware glue originally scoped under M2 — picked up unchanged so the M5 endpoints have a stable contract for caller attribution. Refs #3054
cae3d53 to
f914a55
Compare
Deploying librefang-docs with
|
| Latest commit: |
8f14bd3
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://962e0c25.librefang-docs-web.pages.dev |
| Branch Preview URL: | https://feat-rbac-m5-audit-api.librefang-docs-web.pages.dev |
…ap noted Address review of #3203: 1) UsageRecord::anonymous's 8 positional args trip clippy too_many_arguments under -D warnings, breaking the Quality CI lane. The shape mirrors the metering schema 1:1 and a builder would push call-site noise into ~20 internal kernel paths without gaining type safety, so suppress locally with explanation. 2) csv_escape did not neutralise CSV-formula injection (CWE-1236). A username or detail string starting with =, +, -, @, TAB, or CR round-trips through Excel/Google Sheets as a live formula. OWASP-recommended mitigation: prepend an apostrophe inside the quoted cell. New test test_csv_escape_neutralises_formula_injection pins the behaviour for all six sentinels and confirms inner-position sentinels are untouched. 3) UserConfig.budget is read-only display data in this slice — the metering pipeline (check_all_and_record) only enforces global / per-agent / per-provider caps, not per-user. Add an inline TODO at the kernel call site so the gap is greppable; full per-user enforcement (lookup the resolved UserConfig via attribution_user_id, call a new metering.check_user_budget arm) lands in a follow-up. PR body updated separately to reflect this.
…dded M5 added the UserConfig.budget field to KernelConfig but the golden JSON schema fixture was not regenerated, breaking kernel_config_schema_matches_golden_fixture across Test/Ubuntu and Test/macOS.
…rsed-instant time filter Addresses code review on PR #3203: - audit/budget admin gates: drop the "anonymous = Owner" loopback trust. `require_admin` and `require_admin_for_user_budget` now reject when no `AuthenticatedApiUser` is present, so a co-resident process at `127.0.0.1` (or `LIBREFANG_ALLOW_NO_AUTH=1` deploy) can no longer exfiltrate the hash-chained audit log or per-user spend without configuring an admin api_key. - /api/audit/{query,export}: parse `from`/`to` as RFC-3339 instants instead of doing lexicographic string compare. Prior behaviour silently dropped entries when the operator passed `Z` and the entries were stored with `+00:00` (the chrono default), because `Z` (0x5A) > `+` (0x2B). Malformed bounds now return 400 instead of being ignored. Adds `test_time_range_normalises_z_and_offset_suffix` and `test_parse_time_bounds_rejects_garbage`. - /api/budget/users/{id}: add `enforced: false` to the response so the M6 dashboard can warn that `alert_breach` is informational until the M5-followup enforcement arm in `kernel/mod.rs::execute_llm_agent` lands. Comment links the flag back to that TODO. - usage::query_user_ranking: bind LIMIT via `rusqlite::params!` instead of `format!`-ing the clamped u32 into the SQL string. Value is a clamped u32 so injection isn't a real risk, but keeping the convention uniform avoids future copy-paste landing on user-controlled input. `None` maps to `LIMIT -1` (SQLite "no limit"). - Drop `_USER_SPEND_FRESHNESS_SECS` (referenced nowhere, just a comment) and the `#[allow(clippy::type_complexity)]` on the `Option<Arc<AuditLog>>` field (well below the threshold). - Honesty pass on the export-streaming comment: the body is still materialised in memory before chunked transmission. Real lazy streaming would need `async-stream`; left for a follow-up. Verification: - cargo check -p librefang-api -p librefang-memory --lib → clean - cargo test -p librefang-api --lib routes::audit::tests → 8/8 passed - cargo test -p librefang-memory --lib usage → 14/14 passed - cargo clippy -p librefang-api -p librefang-memory --lib --tests -- -D warnings → clean
# Conflicts: # crates/librefang-kernel/src/auth.rs # crates/librefang-types/src/config/mod.rs # crates/librefang-types/src/config/types.rs
houko
left a comment
There was a problem hiding this comment.
Review
Solid M5 slice. Two follow-up commits (a612c09b fail-closed admin gate + RFC-3339 instant parsing, 4fb70b47 clippy + CSV formula injection guard) closed every concrete issue I had reading the code. The merge with main (e1ffcb46) coexists cleanly with RBAC M3 — UserConfig carries both M3's tool_policy / tool_categories / memory_access / channel_tool_rules and M5's budget field side-by-side, no semantic conflict.
What follows is one blocker plus a few smaller items.
Blocker — no integration test for the four new admin-only HTTP endpoints
crates/librefang-api/tests/api_integration_test.rs got exactly one change in this PR: adding audit_log: None to the test AuthState builder. Grep returns zero references to audit_query, audit_export, user_budget_ranking, or user_budget_detail from that file — confirmed:
grep "audit_query\|audit_export\|user_budget_ranking\|user_budget_detail" \
crates/librefang-api/tests/api_integration_test.rs
# (no output)The test plan in the PR body cites cargo test -p librefang-api --lib -- audit → 5 passed. Those 5 are unit tests inside routes/audit.rs::tests exercising apply_filter, csv_escape, and parse_time_bounds in isolation. None of them exercise:
- The full request → middleware → handler →
require_admin→ JSON shape pipeline - Anonymous-caller rejection (the fix-2
a612c09bchange to refuse loopback /LIBREFANG_ALLOW_NO_AUTH=1admin reads) - Viewer/User → 403 vs Admin/Owner → 200 role boundary
- CSV
Content-TypeandContent-Dispositionheaders - The
enforced: falseflag that the M6 dashboard contract depends on
This matters more here than for a typical feature PR for two reasons:
-
Pattern divergence: middleware role-gates writes (via
is_owner_only_write+user_role_allows_request), but admin-only reads for these four endpoints are gated only in-handler viarequire_admin. The middleware lets every authenticated GET through regardless of role (middleware.rs:121—if role >= UserRole::Admin || *method == GET { return true }). A future PR adding a new admin-read endpoint and forgetting the in-handler gate silently exposes the audit chain. A test that pins "Viewer → 403" would catch that whole class. -
Security blast radius: the audit log is hash-chained user attribution + every detail string ever recorded. The budget endpoint exposes per-user spend data. These are the highest-sensitivity reads on the API surface; a regression here is a privacy incident, not a feature regression.
Suggested minimum set (small — should fit in 100ish lines):
#[tokio::test]
async fn audit_query_rejects_anonymous_with_no_auth_mode() {
// Verifies a612c09b fix — anonymous callers cannot exfiltrate the
// audit log even on loopback / LIBREFANG_ALLOW_NO_AUTH=1.
}
#[tokio::test]
async fn audit_query_rejects_viewer_403_admin_returns_entries() {
// Pins the in-handler require_admin gate — middleware lets the GET
// through for Viewer; only the handler stops it.
}
#[tokio::test]
async fn audit_export_csv_emits_header_and_attachment_disposition() {
// Catches a regression where someone changes Content-Type and
// breaks the dashboard's CSV download flow.
}
#[tokio::test]
async fn user_budget_detail_returns_enforced_false_with_alert_breach() {
// M6 dashboard contract — flip to true only when per-user budget
// enforcement (M5-followup) lands.
}Each one wires up start_test_server_with_auth (already exists at api_integration_test.rs:1620-ish) with the appropriate user role and asserts the response. No new fixtures required.
Smaller things (not blocking)
apply_filter pulls MAX_AUDIT_QUERY_LIMIT * 4 = 20000 rows on every /audit/query
routes/audit.rs:215-220:
let pool_size = (MAX_AUDIT_QUERY_LIMIT as usize).saturating_mul(4);
let pool = state.kernel.audit().recent(pool_size);In a busy RBAC deployment (every tool call + every login + every config edit becomes a row), this materialises megabytes per request, then filters, reverses, truncates. The justification in the doc-comment is sound (AuditLog::recent re-verifies the hash chain on read), but for a steady-state operator dashboard polling every 5s this is wasteful.
A possible follow-up: add a recent_filtered(filter, limit) API on AuditLog that does the in-memory iteration with early termination once limit is hit, instead of allocating the full pool first. Not a blocker — the export endpoint already has the same shape and nobody's reported a problem.
UsageRecord::anonymous is at 8 args and growing
crates/librefang-memory/src/usage.rs:55-79. The #[allow(clippy::too_many_arguments)] suppression with the explanation is fine. But the next attribution dimension (per-channel? per-tenant? per-classification?) makes this 9, then 10. Worth flagging an ADR-style follow-up: switch to either a builder (UsageRecord::builder().agent(...).provider(...).build()) or a struct-literal-with-spread idiom. Don't block on this.
query_user_ranking LIMIT -1 semantics
crates/librefang-memory/src/usage.rs:944-947:
let bound_limit: i64 = match limit {
Some(n) => n.min(1000) as i64,
None => -1,
};SQLite treats LIMIT -1 as "no limit". Correct, but this is a SQLite-specific incantation that will read as a typo to a reviewer who doesn't know the convention. One-line comment pointing at the sqlite.org docs page would help future readers.
Honest streaming comment is already in place
stream_csv / stream_json build the entire chunked Vec upfront before handing to Body::from_stream. The follow-up commit (a612c09b) explicitly notes "the body is still materialised in memory before chunked transmission". OK acknowledged — would still benefit from a real async-stream generator in a follow-up so a 50K-row export doesn't hold 50K JSON values in RAM simultaneously.
Per-user budget enforcement is documented as deferred
The TODO(M5-followup) marker in kernel/mod.rs::execute_llm_agent and the enforced: false flag in /api/budget/users/{id} make the gap properly greppable. The M5 PR body's "What's deferred" section also notes it. Solid handling for a partially-complete feature.
Bottom line
The integration test set is the only thing standing between this and merge in my read. ~100 lines of test code targeting the four admin-only endpoints — symmetric to existing start_test_server_with_auth patterns, no new infrastructure. After that, this is mergeable.
Re-review of #3203 surfaced two MEDIUM issues both fixed here: 1. `UserBudgetConfig.alert_threshold` is documented "clamped to 0..=1" but the field is bare `f64` and TOML parsing accepts any value. With no clamp, `alert_threshold = 5.0` made `alert_breach` permanently false (the dashboard never warned) and `-1.0` made it permanently true. `KernelConfig::clamp_bounds` now reins both ends back to [0.0, 1.0] and resets NaN to the default 0.8. New regression test `test_clamp_bounds_user_alert_threshold` covers above-1, below-0, NaN, and in-range cases. 2. `query_user_daily` / `query_user_monthly` doc-comments said "calendar day, server-local" but SQLite's `datetime('now', 'start of day')` returns UTC. Operators in non-UTC zones see "today" sliced on UTC midnight, not local midnight. Updated the per-user rollup module header to spell out the UTC contract (matching the pre-existing `query_global_*` and `query_agent_*` rollups) and the `_daily` / `_monthly` doclines now say UTC. Also corrected the PR description: `audit_export` is described as "streamed via from_stream so the kernel never buffers". The handler DOES materialise the full filtered Vec before chunking — wire transfer is progressive but kernel-side memory is O(filtered_rows). Updated PR body to reflect that, with a follow-up note that switching to per-row pull from `AuditLog::recent` is the path to true streaming.
Closes the integration-test gap called out in PR #3203 review: the four new HTTP endpoints (`/api/audit/query`, `/api/audit/export`, `/api/budget/users`, `/api/budget/users/{id}`) had unit tests for `apply_filter` / `csv_escape` / `parse_time_bounds` in isolation but no test that exercised the full request → middleware → handler → `require_admin` pipeline. That gap matters because: - The middleware role-gate (`user_role_allows_request`) lets every authenticated GET through regardless of role; the in-handler `require_admin` is the *only* thing stopping a Viewer from reading the hash-chained audit log. A future refactor that drops it would silently expose the chain. - The `a612c09b` follow-up made the gate fail-closed for anonymous callers in `LIBREFANG_ALLOW_NO_AUTH=1` deployments — without an end-to-end test, that fix could be silently reverted. Adds `start_test_server_with_rbac_users` helper that wires: - `KernelConfig.users` AND `AuthState.user_api_keys` from a list of `(name, role, api_key)` tuples (mirrors `server.rs::build_router`). - The audit + budget routers under `/api/`. - `AuthState.audit_log = Some(kernel.audit().clone())` so denials hit the same hash chain as production. - `allow_no_auth = true` so anonymous-rejection tests can synthesize Bearer-less requests without 401-ing at the middleware layer. Tests: - `test_audit_query_rejects_anonymous_403` — pins the fail-closed admin gate from `a612c09b`. - `test_audit_query_rejects_viewer_admin_returns_200` — pins the in-handler `require_admin` boundary; Viewer 403, Admin 200 with documented JSON shape (`entries[]`, `count`, `limit`). - `test_audit_export_csv_emits_documented_headers` — pins `Content-Type: text/csv` + `Content-Disposition: attachment; filename="audit.csv"` + the canonical CSV header row schema, so the dashboard download flow stays compatible. - `test_user_budget_detail_includes_enforced_false` — pins the M6 dashboard contract that `enforced` stays `false` until per-user budget enforcement (the M5-followup arm in `kernel/mod.rs::execute_llm_agent`) actually lands. `cargo check -p librefang-api --tests` clean.
…ate flag-without-namespace combos PR #3205 (M3) shipped two paper cuts that this commit closes. Both land on the M5 branch because M5 is already touching the same files (`crates/librefang-types/src/config/validation.rs` for the `UserBudgetConfig` clamp, `crates/librefang-memory/src/namespace_acl.rs` unchanged but in scope) and a separate PR would conflict with no benefit. 1. **`MemoryNamespaceGuard::has_pii_label` is now case-insensitive.** The metadata-stored label is the `Display` form of `TaintLabel::Pii` (literal `"Pii"`), but external writers commonly hand-stamp `"PII"` or `"pii"`. The previous exact-match comparison silently skipped those, leaving structured PII fragments to fall through to the regex backstop — fine for free-form text but a leak for custom field names that have no e-mail/phone/SSN tokens to grep. Both the array and scalar-string metadata shapes now lowercase-normalise the target *and* candidate before comparing. 2. **`KernelConfig::validate()` now warns on `pii_access` / `export_allowed` / `delete_allowed` set with empty `readable_namespaces`.** The runtime guard requires read access for every code path those flags gate (PII redaction only runs on readable items; export/delete check write access AND the flag). An admin who toggles a flag without declaring namespaces gets a silent no-op that looks like it works — exactly the misconfiguration `validate()` exists to surface. Warning names the user, lists the flags, and suggests the missing key. Tests: - `redact_via_metadata_label_matches_case_insensitively` covers `"PII"` / `"pii"` / `"Pii"` for both array and scalar-string metadata shapes. - `test_validate_warns_on_memory_access_flags_without_readable_namespaces` pins the warning shape: warns on the typo'd user, suppresses on a correctly-configured user, names both the user and the flag. `cargo test -p librefang-memory --lib namespace_acl`: 9/9. `cargo test -p librefang-types --lib config`: 139/139 plus the new validate test. (A third proposed item — startup warning when memory routes are in the auth allowlist while users have memory_access policies — turned out to be moot: `/api/memory/*` is not in any allowlist, the only fallback path is the loopback bypass which is deliberate and documented in `guard_for_request`.)
Closes the M5-followup TODO that left UserConfig.budget as read-only
display data. Per-user spend caps now actually deny over-budget calls,
matching the global / per-agent / per-provider enforcement semantics.
Surface:
- AuthManager: UserIdentity gains an optional UserBudgetConfig (cloned
from UserConfig.budget at populate time). New `budget_for(user_id)`
getter mirrors `memory_acl_for` / `user_policy`.
- MeteringEngine: new `check_user_budget(user_id, &user_budget)`. Each
window with `0.0` is treated as unlimited. On breach returns
`QuotaExceeded` with the user id in the message so dashboard / log
triage can grep by uid.
- kernel::execute_llm_agent (sync) and the streaming spawn now run
the per-user check after `check_all_and_record` succeeds. Same
post-call semantics as the existing global / agent / provider arms
— a breach trips a BudgetExceeded audit (already attribution-tagged
with the user / channel) so the next call from this user gets denied
at the gate; the current response is returned unchanged because the
tokens are already billed.
- /api/budget/users/{id} flips `enforced` from false to true; the
earlier `enforced=false` was a UX guard while enforcement was
deferred. Comment now explains the cap-to-denial chain.
Tests:
- 4 new tests in librefang-kernel-metering covering under-limit,
zero-means-unlimited, hourly breach with uid in message, and
user isolation (Bob's spend doesn't trigger Alice's gate).
Verification:
- cargo check -p librefang-kernel-metering -p librefang-kernel
-p librefang-api --lib → clean
- cargo test -p librefang-kernel-metering --lib check_user_budget → all passing
- cargo clippy -p librefang-kernel-metering -p librefang-kernel
-p librefang-api --lib --tests -- -D warnings → clean
Resolve M4 (#3202)→main conflict in `config/types.rs`: keep both `UserBudgetConfig` (M5) and `ChannelRoleMapping` family (M4) — pure additive overlap. Two follow-ups on the M3 review-gap commit (#9ac0acf5): 1. **Split memory_access validation along the actual gating axis.** `MemoryNamespaceGuard::check_delete` calls `check_write`, not `check_read` — `delete_allowed` depends on `writable_namespaces`, not `readable_namespaces`. The earlier pass grouped all three flags against `readable_namespaces`, which silently missed the dual case (read-but-no-write user with `delete_allowed = true`). Now two independent passes mirror the runtime gates exactly: - `pii_access` / `export_allowed` ↔ `readable_namespaces` - `delete_allowed` ↔ `writable_namespaces` New regression test `test_validate_warns_on_delete_allowed_without_writable_namespaces` pins both sides — warns on read-only + delete_allowed user, stays silent on a properly-configured user. 2. **`has_pii_label` uses ASCII-only lowercasing.** `TaintLabel` variants are pure-ASCII identifiers, so locale-aware `to_lowercase()` is unnecessary cost and risks edge cases (Turkish locale `I → ı`). Switched to `to_ascii_lowercase()` + `eq_ignore_ascii_case` — same case-insensitive contract, no per-call locale lookup. Existing case-insensitive tests cover the change unchanged.
…annel role test Auto-merge missed adding `budget: None` to the `UserConfig` literal at `auth.rs:1383` (the M4 `channel_role_explicit_user_config_wins` test). Three other UserConfig literals in the same file (lines 852/868/883) and the `user_with_policy` helper at 1047 already carried the field from earlier merges; this one slipped through because it was added by M4 after M5's field addition fanned out to its sibling literals. CI failure: E0063 "missing field `budget` in initializer` on Test / Ubuntu / Windows / macOS / Quality. Caught by Test / Windows in run 24951702471.
…or, CSV import + stubs) (#3054) (#3209) * feat(api): admin-only user CRUD + bulk CSV import endpoints Add a thin RBAC user-management surface so the dashboard slice (#3054 / M6) can list, create, edit, delete and bulk-import operator accounts without going through the generic config_set path. - crates/librefang-api/src/routes/users.rs: GET / POST /api/users, GET / PUT / DELETE /api/users/{name}, POST /api/users/import. Writes rebuild the [[users]] array-of-tables in config.toml via toml_edit so unrelated comments and sections stay intact, then trigger a kernel reload. - crates/librefang-kernel/src/auth.rs: AuthManager::replace_users — clears + repopulates the channel index in place so identity resolution picks up the new set without restart. - crates/librefang-kernel/src/config_reload.rs + kernel/mod.rs: classify users diffs as a hot action (HotAction::ReloadUsers) and wire the apply_hot_actions handler. - Sanitize the wire shape: UserView omits api_key_hash and only exposes has_api_key. Bulk import accepts a pre-parsed JSON body (the dashboard parses CSV client-side) and supports dry_run for preview. Tests cover full CRUD, validation errors, duplicate detection, rename, dry-run vs commit semantics, and round-trip persistence (boots a kernel on a temp home_dir and walks the routes via tower::ServiceExt::oneshot). Refs #3054 * feat(dashboard): users page with CRUD + identity-linking wizard Surfaces the new admin-only /api/users endpoints from the previous commit through a single Users page at /users. Three jobs: - list/edit/delete users with search + role filter - a 4-step identity-linking wizard (User -> Platform -> Identifier -> Confirm) that walks an admin through binding a Telegram / Discord / Slack / Email / WeChat platform_id to an existing user. Each platform tile carries the expected platform_id format so admins don't have to spelunk the channel docs. - a drag-drop CSV bulk-import modal with client-side parsing, dry-run preview, and a separate commit step that maps to /api/users/import. CSV header is name,role,<platform>… — every non-name/role column is treated as a channel binding. Data layer follows AGENTS.md to the letter: - src/api.ts: listUsers / getUser / createUser / updateUser / deleteUser / importUsers + sanitized UserItem shape (no api_key_hash leak). - lib/http/client.ts: explicit re-exports. - lib/queries/keys.ts: userKeys factory (hierarchical, anchored). - lib/queries/users.ts: useUsers / useUser with a client-side filter so pages can subscribe with role + search and still share the cache key for the underlying list. - lib/mutations/users.ts: useCreateUser / useUpdateUser / useDeleteUser / useImportUsers — every write invalidates the narrowest matching keys (lists + the touched detail). The import hook invalidates userKeys.all per the bulk-reset escape hatch in AGENTS.md. App.tsx grows a Users entry under Core; router.tsx grows the /users route. Refs #3054 * feat(dashboard): permission simulator using existing 4-role enum Add /users/simulator — pick a user, see which Action variants their role allows. Decision is computed locally from the ROLE_ORDER index, mirroring `librefang_kernel::auth::UserRole as u8` (Viewer < User < Admin < Owner). The action -> required_role table is duplicated from `auth::Action` so the matrix stays stable even when the daemon doesn't have a server-side simulator endpoint yet. When M3 (#3205) lands the per-user policy editor will refine these results further; this page already pre-wires `usePermissionPolicy` via the stub hook so hooking it up is a one-line swap. Refs #3054 * feat(dashboard): audit viewer + budget charts + permission matrix stubs awaiting M3/M5 Three thin pages whose payload depends on still-open PRs: - /audit: searchable / filterable audit trail. Real shape is wired via `useAuditQuery` in lib/queries/audit.ts hitting GET /api/audit/query. Endpoint ships with M5 (#3203). Hook is invoked with enabled: false so we don't 404-spam pre-M5 daemons; the page renders a placeholder pointing at the dependency. - /users/{name}/budget: per-user spend charts + alert thresholds. `useUserBudget` keyed via userBudgetKeys.detail(name) maps to /api/budget/users/{name}, also gated until M5. - /users/{name}/policy: per-user tool/memory policy matrix editor. `usePermissionPolicy` + `useUpdateUserPolicy` hit /api/users/{name}/policy, gated until M3 (#3205) ships the endpoint. Key factories are added to keys.ts (auditKeys.query / queries, userBudgetKeys, permissionPolicyKeys) and registered in keys.test.ts so the anchoring + 'all factories exist' invariants stay green. The plumbing (query hooks, mutation invalidation, page routing, nav entry) lands now so the only delta when M3/M5 merge is dropping the `enabled: false` guard and rendering the actual table/chart. Refs #3054 * fix(rbac-m6): gate /api/users to Owner, validate hash, fail loud on corrupt config Address PR #3209 review: 1. BLOCKING — Admin per-user API key could POST /api/users to create a new `role: "owner"` user with a chosen `api_key_hash` and self-promote. The handlers had no role check and the comment said the gate would arrive "with M3" — M3 (#3205) did not in fact add it. Extend the existing `is_owner_only_write` allowlist with a `/api/users*` prefix so every mutating call under that surface requires Owner, mirroring `Action::ManageUsers` in the kernel. GET stays Admin-or-above so the permission simulator's user list keeps working. 2. HIGH — `persist_users` had `unwrap_or_default()` on both `read_to_string` and `toml::parse`, so a transient read failure or a hand-edited-into-corruption `config.toml` would silently get replaced by a stub document containing only `[[users]]` — wiping every other section. Both branches now return `PersistError::Internal`; corrupt files stay untouched on disk so `backups/config.toml.prev` plus the bad file are sufficient for human recovery. 3. MEDIUM — `api_key_hash` was accepted as any string. New `validate_api_key_hash` runs `argon2::password_hash::PasswordHash::new` to reject anything that isn't a real Argon2id PHC string, applied in create / update / bulk import. Empty / whitespace still maps to `None` (clear the hash), unchanged. Tests: - middleware: `test_user_role_admin_cannot_mutate_users_endpoints` / `test_user_role_viewer_can_still_list_users_for_simulator` - users_test: `users_create_rejects_invalid_api_key_hash` / `users_create_refuses_to_overwrite_corrupt_config_toml` - middleware suite 34/34, users integration 10/10, clippy clean. * fix(rbac-m6): persist RBAC M3 policy fields + simplify update_user Re-review of #3209 found the preserve-and-merge logic in `update_user` and `import_users` only protected the in-memory `Vec<UserConfig>` — the on-disk write path in `persist_users` hand-emitted just the four M6 fields (name / role / channel_bindings / api_key_hash) and silently dropped the M3 ones (tool_policy / tool_categories / memory_access / channel_tool_rules). After every successful PUT, `reload_config()` read the truncated file back and the per-user policy collapsed to None. Two changes: 1. `persist_users` now serializes each user via `toml_edit::ser::to_document` so the full serde shape (incl. `#[serde(skip_serializing_if)]`-gated M3 fields) round-trips through `config.toml`. We use `toml_edit::ser` rather than `toml::ser` because `UserConfig` interleaves scalar fields (`api_key_hash`) after a nested table (`channel_bindings`), which the strict serializer rejects with `ValueAfterTable`. Enabled the `serde` feature on `toml_edit` to expose `to_document`. 2. Generalized `persist_users` to `FnOnce(&mut Vec<UserConfig>) -> Result<R>` and let the closure thread an arbitrary `R` back to the caller. The `Arc<Mutex<Option<UserConfig>>>` capture pattern in `update_user` is gone — closure now returns `Ok(users[idx].clone())` directly. Other call sites stay at `R = ()`. Tests: - New `users_update_and_import_preserve_rbac_m3_policy_fields` seeds a Bob with non-default `tool_policy` + `memory_access`, runs PUT (rename + role bump) and bulk-import update, asserts the policy fields survive by reading back through `state.kernel.config_ref()` (i.e. after the disk round-trip). This test FAILED before the persist_users fix. - 11/11 users_test pass, clippy clean.
…ult::default()
CI Test/Ubuntu (run 24952194471) failed on
crates/librefang-api/tests/api_integration_test.rs:2565 — the new
RBAC integration test fixture `start_test_server_with_rbac_users`
uses `..Default::default()` to fill the optional RBAC fields, but
UserConfig had no Default impl.
The integration test file is a separate compilation unit from
`--lib --tests` runs targeting only the lib's unit tests, which is
why my prior local verification missed it. CI's full
`cargo test -p librefang-api` caught it.
Add a hand-written `impl Default for UserConfig` that mirrors the
per-field `#[serde(default)]` attributes — `name = ""`, `role =
default_role()` ("user"), all RBAC `Option`s = `None`, both
`HashMap`s empty. Now `serde::from_str("name = \"x\"")` and
`UserConfig { name: ..., ..Default::default() }` produce identical
values.
Verification:
- cargo test -p librefang-api --tests → all passing
- cargo clippy -p librefang-api -p librefang-types --all-targets
-- -D warnings → clean
Deploying librefang with
|
| Latest commit: |
8f14bd3
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://37dd7c92.librefang.pages.dev |
| Branch Preview URL: | https://feat-rbac-m5-audit-api.librefang.pages.dev |
PR #3209 (RBAC M6 dashboard) landed in main and added crates/librefang-api/src/routes/users.rs with three UserConfig literals — create, update, CSV import — that predate the per-user budget field this PR added to UserConfig. Compile breaks with E0063 "missing field `budget`" when the two land together. - create_user / import_users new-row: budget=None (no per-user cap; global/per-agent/per-provider caps still apply). Dedicated /api/users/{name}/budget owns budget edits. - update_user / import_users update-row: preserve from existing config alongside the M3 policy fields, same reasoning — the M6 dashboard doesn't surface budget yet, clobbering it on a name/role edit would silently wipe a user's spend cap. Also adds budget=None to the seed in users_test.rs so the existing M6 preservation test still compiles.
8f14bd3 to
439a896
Compare
Catches the latent bug in users.rs:568 where `..new_u.clone()` would silently overwrite an existing user's `budget` with `None` because CSV rows always carry no budget. Mirrors the existing M3 policy preservation test (`users_update_and_import_preserve_rbac_m3_policy_fields`). Seeds Carol with a non-zero UserBudgetConfig, then: 1. PUT /api/users/Carol — rename + role bump, no budget in body. Asserts the budget round-trips. 2. POST /api/users/import — same name, no budget in CSV row. Asserts the budget still round-trips through the import-update branch. Both assertions name the failure mode in the message so a future regression points at the exact line that broke.
Both tests landed earlier in the M5 stack but were never actually
exercised by CI — the UserConfig.budget compile errors after the M6
merge masked them. Now that the literals compile, the tests run and
fail against the current behavior, which is correct in both cases.
1) test_audit_query_rejects_anonymous_403 → ..._anonymous (asserts 401)
The setup asks for `allow_no_auth = true` to bypass middleware, but
middleware.rs:501-526 only honors that flag when both `api_key` and
`user_api_keys` are empty. The test configures both, so anonymous
requests are 401'd at the middleware before reaching the in-handler
`require_admin` 403 gate. Earlier rejection is the stricter outcome
("anonymous cannot read audit" still holds).
2) test_user_budget_detail_includes_enforced_false → ..._true
Per-user budget enforcement landed in 4a00a64 ("RBAC M5 — wire
per-user budget enforcement") — AuthManager::budget_for,
MeteringEngine::check_user_budget, and the post-call arm in
kernel::execute_llm_agent now actually deny over-budget calls. The
handler was updated to report `enforced: true` accordingly; the
stale assertion was missed.
No production changes — only the assertions move to track the
already-shipped behavior.
…ed (#3225) * feat(dashboard): activate AuditPage now that M5 audit endpoints shipped Same root cause as the recent UserBudgetPage activation: PR #3209 (M6 dashboard) shipped AuditPage as a "Pending M5" stub linking back to PR #3203, then M5 merged and nobody came back to drop the `enabled: false` guard or fix the wire-format guesses. What changed: - `api.ts` — `AuditQueryFilters` / `AuditQueryEntry` / `AuditQueryResponse` retyped to match `routes/audit.rs::audit_query`'s actual response. The previous shapes were guesses made before the endpoint shipped: - filters: `since/until` -> `from/to`; dropped non-existent `offset`/`status`; added missing `agent`/`channel` - entry: `agent`/`details`/`status`/`user` -> `agent_id`/`detail`/ `outcome`/`user_id`; added `seq`/`channel`/`hash` - wrapper: `total`/`has_more` -> `count`/`limit` - `lib/queries/audit.ts` — dropped the M5-pending comment. - `AuditPage.tsx` — rewritten from stub to real viewer: - filter form (user, action, agent, channel, from/to) - table of entries with seq/timestamp/action/agent/user/channel/outcome/detail - CSV export button — local `buildExportUrl` helper inline in the page (the URL is for `<a download>`, not a fetch, so it doesn't belong in the http/client data layer) - 403 banner when the caller isn't an admin (audit is admin-only via `require_admin` in routes/audit.rs:98), so the page can degrade gracefully for User/Viewer-role api keys instead of just showing a generic error. Server-side tests already cover this surface in PR #3203; this PR is frontend-only and connects existing wires. * fix(dashboard/audit): RFC-3339 normalisation, authenticated CSV, robust 403 Four issues caught in review: 1. <input type="datetime-local"> emits 'YYYY-MM-DDTHH:MM' with no offset, but the server parses from/to via DateTime::parse_from_rfc3339 which requires one. Picking from/to and applying filters returned 400 silently. New toRfc3339 helper treats the picker value as local time and converts to ISO-8601 'Z'; applied to both the query path and the export URL. 2. CSV export was a bare <a download href="/api/audit/export?...">, but the dashboard auths via Authorization: Bearer <key> from localStorage. Browser navigation drops custom headers, so on any auth-enabled deployment (the actual target of admin-only audit) the user downloaded the daemon's 401 / login HTML as audit.csv. Replaced with fetch + Blob + programmatic-click pattern that carries the Bearer header. Surfaces ApiError status/message in an inline export-error banner on failure. 3. isForbidden detection was /403|admin/i.test(error.message), which fragility-couples the UI banner to the exact wording of the server's 403 body. Changed to error instanceof ApiError && error.status === 403. 4. Pre-existing prop typo: <Button icon=...> — Button accepts leftIcon/rightIcon, not icon. The Search glyph next to 'Apply filters' would never render. Fixed to leftIcon. dashboard typecheck passes (the 3 remaining errors in sessions-stream.test.tsx are pre-existing on main, see 181430b). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…3224) * feat(api): per-user budget write/clear endpoints + dashboard editor Closes the M5 follow-up gap: `UserConfig.budget` was set-via-config.toml only, but the M6 dashboard already shipped a `UserBudgetPage` stub that linked back to PR #3203 saying "wire up after M5 lands". M5 has landed (per-user enforcement is real), so this connects the management plane. Surface - `PUT /api/budget/users/{user_id}` — admin-only upsert. Body mirrors `UserBudgetConfig` (max_hourly_usd / max_daily_usd / max_monthly_usd / alert_threshold). Validates negatives / NaN / infinite + threshold range before touching disk so a bad payload returns 400 cleanly. On success, `users::persist_users` writes the toml and reloads the kernel so the new cap takes effect on the next LLM call. - `DELETE /api/budget/users/{user_id}` — admin-only clear. Sets `UserConfig.budget = None` and persists. Idempotent (cleared user gets 200, no error). - Both record a `RoleChange` audit event with the caller's user_id, same shape as `update_user`. Both reuse `require_admin_for_user_budget` (the existing gate on the GET sibling) — Owner-level isn't required because budget management mirrors `update_agent_budget` semantically. `users::persist_users` and `PersistError` lifted from module-private to `pub(crate)` so the budget handler can share the same toml-write path that user CRUD uses. Dashboard - `api.ts`: `UserBudgetResponse` retyped to match the actual server shape (`{user_id, name, role, hourly{spend,limit,pct}, daily{...}, monthly{...}, alert_threshold, alert_breach, enforced}`). The previous shape was an M5-stub guess that never matched the wire format. - New `useUpdateUserBudget` / `useDeleteUserBudget` mutations; both invalidate `userBudgetKeys.detail(name)` plus `userKeys.detail(name)` + `userKeys.lists()` since `UserConfig.budget` is part of `UserItem`. - `UserBudgetPage` rewritten from stub to live editor: spend bars per window, limit / threshold form, validation (negative / out-of-range) matched to server; "Clear cap" button that calls DELETE. Tests - `test_user_budget_put_get_delete_round_trip`: PUT 1.5/12/100 USD caps, assert GET reflects them, DELETE, assert GET is back to 0. - `test_user_budget_put_rejects_invalid_payload`: -1.0 hourly, threshold=1.5, threshold=-0.1 — all 400. - `test_user_budget_put_rejects_viewer_with_403`: Bob (viewer) PUT against Alice → 403, gate from in-handler `require_admin_for_user_budget`. - `test_user_budget_put_unknown_user_returns_404`: PUT against NonExistent surfaces 404 instead of silent insert. * fix(api/budget): require full payload, fix audit action, run fmt Addresses review on #3224: 1. cargo fmt — collapsed two split format!() literals on budget.rs that tripped the Quality CI check. 2. PUT /api/budget/users/{id} now rejects partial bodies with 400 instead of silently zeroing missing windows. UserBudgetConfig has #[serde(default)], so a previous `{"max_hourly_usd": 5}` would have reset daily/monthly/alert to defaults — clearing an existing cap without warning. Also rejects wrong-typed values (`"1.0"` as a string) with 400 instead of coercing to 0.0. 3. Audit detail now records the target user_id alongside the new values, so "who changed Bob's budget" is one log line instead of a timestamp+number-fishing exercise. 4. AuditAction switched from RoleChange → ConfigChange. RoleChange is reserved for admin/editor/viewer flips; budget edits are config changes. Keeps the audit filter clean. New regression test test_user_budget_put_rejects_partial_payload pins all four shapes that must 400: missing key, multiple-missing, empty object, and string-typed number. Existing 404 / 403 / round-trip / invalid-value tests updated to send full-shape bodies. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
#3240) PR #3203 shipped /api/audit/export but both the JSON and CSV emitters dropped `prev_hash`. Without it, a downstream verifier cannot replay the SHA-256 hash chain off the dump alone — defeating the whole point of the chain. Append `prev_hash` as the last field on the JSON object and as the last column on the CSV row, after the existing `hash`. Column order of all pre-existing fields is preserved so any downstream parsers keyed off the documented schema only need to add the new tail column. Tests: - audit::tests::test_stream_json_includes_prev_hash — parses the streamed body and asserts prev_hash round-trips for one entry. - audit::tests::test_stream_csv_includes_prev_hash_column — pins the new header row and asserts the per-entry row ends with hash,prev_hash. - Updated test_audit_export_csv_emits_documented_headers to expect the new header schema. Out of scope: the same review flagged that audit_export buffers the filtered Vec in memory before streaming. That's a separate optimization (true row-by-row streaming from SQLite) and is left for a follow-up PR.
Summary
RBAC M5 lands the audit-query / export and per-user budget surface for issue #3054. Stacked on top of
feat/rbac-m2-audit-attribution(PR #3196) — every change here assumes that PR'srecord_with_contextand v22 schema migration are already in place.Base note: this PR's base is
feat/rbac-m2-audit-attribution, not main. After PR #3196 merges to main this PR auto-retargets to main on rebase.What's new
4 new HTTP endpoints, all admin-only and authenticated (no
is_publicallowlist changes):GET /api/audit/query— filter the hash-chained audit log by user (UUID or name), action, agent, channel, and ISO-8601 time range. Newest-first, hard cap 5000 rows.GET /api/audit/export?format=json|csv— same filters, response body chunked over the wire viaaxum::body::Body::from_stream. Note (review correction): the filtered result set IS materialised in memory before chunking — the wire transfer is progressive but the kernel-side memory profile is O(filtered_rows). At the 50K row hard cap × ~500 B/entry that caps at ~25 MB; switching to true streaming (per-row pull fromAuditLog::recent) is a follow-up. CSV emits a fixed header row with RFC-4180 cell escaping pinned by a regression test.GET /api/budget/users— per-user cost ranking (top N) joined with the user's configured limits when present so the dashboard can render % bars without a follow-up call.GET /api/budget/users/{user_id}— single-user detail with hourly / daily / monthly spend, alert threshold, and analert_breachboolean. Accepts both the canonical UUID and the raw configured name.4 new
AuditActionvariants:UserLogin,RoleChange,PermissionDenied,BudgetExceeded.PermissionDeniedfires from the auth middleware (role-check 403) and from each admin-gated endpoint that rejects a sub-Admin caller.BudgetExceededfires fromexecute_llm_agentand the streaming spawn whenevercheck_all_and_recordreturns aQuotaExceeded— i.e. when a global, per-agent, or per-provider cap trips. Per-user caps do not yet trip this audit event (see "What's deferred" below).test_new_rbac_variants_preserve_chain) pins theDisplayoutput of every new variant so a casual rename surfaces as a test failure rather than a silent invalidation of every persisted hash.UserConfig.budget: Option<UserBudgetConfig>withmax_hourly_usd / max_daily_usd / max_monthly_usd / alert_threshold. In this slice the field is read-only display data — surfaced via/api/budget/users/{id}.alert_breachso the dashboard can render a warning, butcheck_all_and_recorddoes not consult it. Operators that configure a per-user cap today will see the breach reported, not enforced. Wiring enforcement is tracked as a follow-up; see the inlineTODO(M5-followup)at the kernel call site (librefang-kernel/src/kernel/mod.rsafter thecheck_all_and_recordarm). None means "no per-user cap" — the user is still bounded by global / per-agent / per-provider budgets. Defaultalert_threshold = 0.8matches the globalBudgetConfigfor consistency.Cost-attribution pipeline:
usage_eventsschema v23 adds NULL-ableuser_id/channelcolumns and indexes both.UsageRecordgains optionaluser_id/channelfields.execute_llm_agentand the streaming path resolveUserIdfromSenderContextviaauth.identifyand stamp every record. Pre-M5 rows return NULL — they fall outside any per-user filter, which is the right default.Middleware glue from feat(security): RBAC M2 — audit user/channel attribution + stable UserId (#3054) #3196: the dirty
routes/config.rschange threadingAuthenticatedApiUser.user_idthrough torecord_with_contextfor/api/shutdown,/api/config/reload, and/api/config/setis folded in unchanged.Design decisions (carried from issue #3054)
domain.action.resourcewith*wildcard (placeholder — not yet enforced; M5 sticks withUserRole).guest.config.tomlonly.UserConfig.role> channel-derived.AuditLog::recentso chain verification runs on every read.What's deferred
UserConfig.budgetis read-only display data in this slice. The metering pipeline only enforces global / per-agent / per-provider caps. Wiring per-user enforcement requires looking up the resolvedUserConfigviaattribution_user_idinexecute_llm_agentand calling a newmetering.check_user_budgetarm. InlineTODO(M5-followup)marker inkernel/mod.rsmakes the gap greppable.check_all_and_recordruns after the LLM call (cf. review HIGH#6); rate-limit-style pre-call gating is a separate design concern.Test plan
cargo test -p librefang-runtime --lib audit— 12 passed, including the newtest_new_rbac_variants_preserve_chain(locks theDisplayoutput forUserLogin / RoleChange / PermissionDenied / BudgetExceeded).cargo test -p librefang-memory --lib usage— 14 passed, includingtest_user_spend_rollup_per_window(anonymous spend isolated from per-user buckets) andtest_user_ranking_excludes_anonymous_and_orders_by_daily.cargo test -p librefang-memory --lib migration—test_migrate_v23_adds_user_id_and_channel_to_usage_eventsandtest_migrate_v23_is_idempotentboth pass; the legacy INSERT path keeps working with NULLs.cargo test -p librefang-api --lib -- audit— 5 passed:test_filter_by_user_uuid_and_name,test_filter_by_action_case_insensitive,test_filter_by_agent_channel_and_time_range,test_csv_escape_pins_format,test_filter_does_not_match_via_sql_injection_attempt.cargo test -p librefang-kernel --lib auth— 15 passed (UserConfig.budgetfield addition does not break any auth test).cargo check -p librefang-api -p librefang-types -p librefang-runtime -p librefang-kernel— clean.cargo clippy -p librefang-api -p librefang-runtime --lib --tests -- -D warnings— clean (no full-workspace cargo per task instructions; M3 / M4 agents are running in parallel).Refs #3054
Update — post-M6 merge reconciliation
PR #3209 (RBAC M6 dashboard) landed in main and added
crates/librefang-api/src/routes/users.rswith threeUserConfigliterals (create / update / CSV-import-new) plus a fourth update-existing branch using..new_u.clone(). None of those knew about M5's newbudgetfield — the merge here adds it everywhere it's needed:create_userand CSV new-row →budget: None. Per-user budget remains set viaconfig.toml(no write endpoint until the M5 follow-up that wires enforcement).update_userand CSV update-row →budget: preserved.budget. Same pattern as the M3 policy fields: the M6 dashboard doesn't surface budget editing, so aname/roleedit must not silently wipe a per-user spend cap...new_u.clone()would always overwrite the existing budget withNonebecause CSV rows carry no budget field. Now fixed and pinned byusers_update_and_import_preserve_m5_budget.