Skip to content

perf(hot-path): cut Vec/Arc clones, regex compiles, and N+1 SUMs#4129

Merged
houko merged 5 commits into
mainfrom
perf/hot-path-batch3
Apr 30, 2026
Merged

perf(hot-path): cut Vec/Arc clones, regex compiles, and N+1 SUMs#4129
houko merged 5 commits into
mainfrom
perf/hot-path-batch3

Conversation

@houko

@houko houko commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Four independent hot-path optimizations that all fire on every LLM turn:

Test plan

  • CI: cargo build --workspace --lib
  • CI: cargo test --workspace
  • CI: cargo clippy --workspace --all-targets -- -D warnings
  • Live: send a real LLM message and confirm response cost is metered
    (/api/budget reflects spend; per-window quotas still trigger).
  • Live: subscribe to /api/events SSE while another agent publishes;
    payload arrives intact.

@github-actions github-actions Bot added area/channels Messaging channel adapters area/runtime Agent loop, LLM drivers, WASM sandbox area/kernel Core kernel (scheduling, RBAC, workflows) size/L 250-999 lines changed labels Apr 29, 2026
@github-actions github-actions Bot added the has-conflicts PR has merge conflicts that need resolution label Apr 30, 2026
houko and others added 5 commits May 1, 2026 07:40
Four hot-path optimizations on every LLM turn:

* messages: Arc<Vec<Message>> in CompletionRequest. retry / driver
  hand-off now bumps a refcount instead of deep-copying 200-600 KB of
  history per turn (#3766).
* EventBus broadcasts Arc<Event>. publishing to N subscribers no
  longer deep-clones the payload N times (#3380).
* check_all_with_provider_and_record collapses up to 10 SUM queries
  into 4 by combining agent / global / provider sums into one
  conditional aggregate per time window (#3382).
* Cache compiled regexes via LazyLock / shared cache for the LLM
  function-call recovery parser, the QQ markdown stripper, the
  structured-output JSON extractor, the kernel-router phrase matcher,
  and QualityCheck::MatchesRegex (#3491).

Closes #3766
Closes #3380
Closes #3382
Closes #3491
`request.messages` became `Arc<Vec<Message>>` (#3766) but
`preprocess_moonshot_files` still did `for msg in &mut request.messages`,
which fails to compile because `Arc<Vec<_>>` only derefs to `&Vec<_>`.
Use `Arc::make_mut` to obtain an exclusive `&mut Vec<Message>` —
O(1) when the Arc isn't shared (the common case for a fresh request),
and clones once on a shared Arc, which is unavoidable since we must
mutate the message list to swap image/file blocks for upload markers.
The hot-path batch promoted request.messages from Vec<Message> to
Arc<Vec<Message>> to avoid clones, but six driver call sites still
wrote `for msg in &request.messages`. `&Arc<Vec<T>>` doesn't
implement IntoIterator, so build broke on every test target.

Use `request.messages.iter()` — Arc derefs to Vec, method resolution
picks Vec::iter, no clone or extra deref needed.

Also picks up the rustfmt drift fmt-on-precommit produced earlier.
The hot-path swap from regex to regex_lite kept the static fallback
pattern `(?!x)x`, which uses negative look-ahead — regex_lite rejects
that ("look-around is not supported"). The expect() then panics on
every invalid user pattern, breaking test_quality_check_invalid_regex
on macOS runners.

Use `[^\s\S]` instead — the universe of chars negated to the empty
set, no look-around required.
@houko
houko force-pushed the perf/hot-path-batch3 branch from a23fd7d to 479a891 Compare April 30, 2026 22:41
@houko
houko enabled auto-merge (squash) April 30, 2026 22:41
@houko
houko merged commit c91b11b into main Apr 30, 2026
12 of 13 checks passed
@houko
houko deleted the perf/hot-path-batch3 branch April 30, 2026 22:41
@github-actions github-actions Bot added ready-for-review PR is ready for maintainer review and removed has-conflicts PR has merge conflicts that need resolution labels Apr 30, 2026
houko added a commit that referenced this pull request May 1, 2026
The rebase brought in a bunch of pre-existing breakage on main that
this PR has to clear before CI can pass:

- recv_event_skipping_lag still took Receiver<Event> but the Arc<Event>
  rework left desktop calling it with Receiver<Arc<Event>>; flip the
  helper to Arc<Event> so the only caller compiles.
- librefang-llm-driver and tests/common/mod.rs still built
  CompletionRequest with messages: vec![...] but the Arc<Vec<Message>>
  rework requires Arc::new(vec![...]).
- clippy 1.94 added manual_contains; rewrite EXACT.iter().any(|p| ...)
  as EXACT.contains(&path).
- kernel approval test ignored a Result return value; add let _ = to
  acknowledge the must_use.

These are not part of the security fix proper but the security commit
cannot land until the workspace clippy / build is green again.
houko added a commit that referenced this pull request May 1, 2026
…4126)

* fix(security): tighten audit, sandbox, and spawn deniability holes

Bundle of five security fixes that all touch tamper-resistance or
denial-of-wallet boundaries:

- spawn: validate AgentManifest `module` field — reject absolute paths,
  `..` traversal, and Windows drive prefixes so a hostile agent.toml
  cannot smuggle `module = "python:/etc/passwd.py"` and have the host
  Python interpreter exec it under the agent's capabilities. (#3533)
- migrations: v13 / v17 / v18 applied DDL but skipped the
  `INSERT INTO migrations (...)` audit row, so `user_version` and the
  audit trail drifted by three. Add the missing rows and a startup
  consistency check that logs an `error!` on drift. (#3538)
- audit: lock in #4080's fix that drops an in-memory chain push when
  the SQLite INSERT fails, and add a dup-of regression note. (#3497)
- wasm sandbox: charge per-method fuel reservations on `host_call` so
  a guest cannot fan out unlimited `agent_send` / `agent_spawn` /
  `net_fetch` / `shell_exec` calls under a budget meant to cap CPU. The
  default 1 M fuel ceiling now bounds host LLM calls to ~10/run. (#3532)
- middleware: lock the `/a2a/tasks/{id}/status` auth gate with an
  explicit regression test (dup of #3781 fix). (#3473)

* ci: retrigger CI after auto-update merge with GITHUB_TOKEN

* fix(security): address review feedback on PR #4126

- validate_module_string: normalize backslashes before component scan
  (Linux Path::components treated '..\..\etc' as one filename),
  reject NUL/control characters in payload.
- wasm host_call fuel: fail closed if fuel meter is unavailable or
  set_fuel errors instead of silently swallowing — the reservation is
  the only DOW guard.
- migration drift check: surface query errors with error! instead of
  hiding them behind unwrap_or(-1).
- Strengthened tests: backslash-traversal + NUL cases for
  validate_module_string; relative-ordering assertions for fuel cost
  table.

* fix(security): close 3 #3533 module-path bypasses + auto-heal #3538 audit drift

Address review of #4126:

- #3533: the original PR added validate_module_string only to spawn_agent_inner,
  but three other entry points feed manifests into the registry without going
  through it: reload_agent_from_disk (hot-reload), update_manifest (caller-
  supplied manifest update), and the boot loop's SQLite restore path. A
  hostile agent.toml on disk + restart, or a disk-edit + reload, would still
  install module = "python:/etc/passwd.py" into a live agent.

  Centralised the check in `validate_manifest_module_path` and wired it into
  all four call sites. Boot restore logs error! and skips the agent (refusing
  to boot the daemon for one bad manifest would turn a CVE into a DoS); the
  other paths return Err so the caller sees the rejection.

- #3538: the post-merge consistency check only logged error! on drift but
  never repaired it, so any pre-fix prod DB would spam the log on every
  restart forever. Replaced with a backfill that inserts INSERT OR IGNORE
  audit rows for any version <= user_version that's missing, logs a single
  warn! summarising the rescue, and is idempotent on clean DBs.

  New regression test: simulate v13/v17/v18 drift by deleting their audit
  rows after migrate, re-run, assert rows are restored, then re-run a third
  time and assert the row count is unchanged.

* chore: fix incidental rebase breakage on top of #4129 / clippy 1.94

The rebase brought in a bunch of pre-existing breakage on main that
this PR has to clear before CI can pass:

- recv_event_skipping_lag still took Receiver<Event> but the Arc<Event>
  rework left desktop calling it with Receiver<Arc<Event>>; flip the
  helper to Arc<Event> so the only caller compiles.
- librefang-llm-driver and tests/common/mod.rs still built
  CompletionRequest with messages: vec![...] but the Arc<Vec<Message>>
  rework requires Arc::new(vec![...]).
- clippy 1.94 added manual_contains; rewrite EXACT.iter().any(|p| ...)
  as EXACT.contains(&path).
- kernel approval test ignored a Result return value; add let _ = to
  acknowledge the must_use.

These are not part of the security fix proper but the security commit
cannot land until the workspace clippy / build is green again.

* test(security): add e2e regressions for #3533 spawn boundary and #3532 fuel guard

- spawn_agent rejects absolute and '..' traversal module paths at the
  kernel boundary (locks the wiring on top of the pure-fn validate_module_string
  unit tests in librefang-runtime).
- host_call denial-of-wallet guard fires before dispatch when
  remaining fuel < per-method reservation cost.
- approval.rs: incidental cargo fmt fixup pulled in by pre-commit hook.
@houko houko mentioned this pull request May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/channels Messaging channel adapters area/kernel Core kernel (scheduling, RBAC, workflows) area/runtime Agent loop, LLM drivers, WASM sandbox ready-for-review PR is ready for maintainer review size/L 250-999 lines changed

Projects

None yet

1 participant