perf(hot-path): cut Vec/Arc clones, regex compiles, and N+1 SUMs#4129
Merged
Conversation
3 tasks
Four hot-path optimizations on every LLM turn: * messages: Arc<Vec<Message>> in CompletionRequest. retry / driver hand-off now bumps a refcount instead of deep-copying 200-600 KB of history per turn (#3766). * EventBus broadcasts Arc<Event>. publishing to N subscribers no longer deep-clones the payload N times (#3380). * check_all_with_provider_and_record collapses up to 10 SUM queries into 4 by combining agent / global / provider sums into one conditional aggregate per time window (#3382). * Cache compiled regexes via LazyLock / shared cache for the LLM function-call recovery parser, the QQ markdown stripper, the structured-output JSON extractor, the kernel-router phrase matcher, and QualityCheck::MatchesRegex (#3491). Closes #3766 Closes #3380 Closes #3382 Closes #3491
`request.messages` became `Arc<Vec<Message>>` (#3766) but `preprocess_moonshot_files` still did `for msg in &mut request.messages`, which fails to compile because `Arc<Vec<_>>` only derefs to `&Vec<_>`. Use `Arc::make_mut` to obtain an exclusive `&mut Vec<Message>` — O(1) when the Arc isn't shared (the common case for a fresh request), and clones once on a shared Arc, which is unavoidable since we must mutate the message list to swap image/file blocks for upload markers.
The hot-path batch promoted request.messages from Vec<Message> to Arc<Vec<Message>> to avoid clones, but six driver call sites still wrote `for msg in &request.messages`. `&Arc<Vec<T>>` doesn't implement IntoIterator, so build broke on every test target. Use `request.messages.iter()` — Arc derefs to Vec, method resolution picks Vec::iter, no clone or extra deref needed. Also picks up the rustfmt drift fmt-on-precommit produced earlier.
The hot-path swap from regex to regex_lite kept the static fallback
pattern `(?!x)x`, which uses negative look-ahead — regex_lite rejects
that ("look-around is not supported"). The expect() then panics on
every invalid user pattern, breaking test_quality_check_invalid_regex
on macOS runners.
Use `[^\s\S]` instead — the universe of chars negated to the empty
set, no look-around required.
houko
force-pushed
the
perf/hot-path-batch3
branch
from
April 30, 2026 22:41
a23fd7d to
479a891
Compare
houko
enabled auto-merge (squash)
April 30, 2026 22:41
houko
added a commit
that referenced
this pull request
May 1, 2026
The rebase brought in a bunch of pre-existing breakage on main that this PR has to clear before CI can pass: - recv_event_skipping_lag still took Receiver<Event> but the Arc<Event> rework left desktop calling it with Receiver<Arc<Event>>; flip the helper to Arc<Event> so the only caller compiles. - librefang-llm-driver and tests/common/mod.rs still built CompletionRequest with messages: vec![...] but the Arc<Vec<Message>> rework requires Arc::new(vec![...]). - clippy 1.94 added manual_contains; rewrite EXACT.iter().any(|p| ...) as EXACT.contains(&path). - kernel approval test ignored a Result return value; add let _ = to acknowledge the must_use. These are not part of the security fix proper but the security commit cannot land until the workspace clippy / build is green again.
houko
added a commit
that referenced
this pull request
May 1, 2026
…4126) * fix(security): tighten audit, sandbox, and spawn deniability holes Bundle of five security fixes that all touch tamper-resistance or denial-of-wallet boundaries: - spawn: validate AgentManifest `module` field — reject absolute paths, `..` traversal, and Windows drive prefixes so a hostile agent.toml cannot smuggle `module = "python:/etc/passwd.py"` and have the host Python interpreter exec it under the agent's capabilities. (#3533) - migrations: v13 / v17 / v18 applied DDL but skipped the `INSERT INTO migrations (...)` audit row, so `user_version` and the audit trail drifted by three. Add the missing rows and a startup consistency check that logs an `error!` on drift. (#3538) - audit: lock in #4080's fix that drops an in-memory chain push when the SQLite INSERT fails, and add a dup-of regression note. (#3497) - wasm sandbox: charge per-method fuel reservations on `host_call` so a guest cannot fan out unlimited `agent_send` / `agent_spawn` / `net_fetch` / `shell_exec` calls under a budget meant to cap CPU. The default 1 M fuel ceiling now bounds host LLM calls to ~10/run. (#3532) - middleware: lock the `/a2a/tasks/{id}/status` auth gate with an explicit regression test (dup of #3781 fix). (#3473) * ci: retrigger CI after auto-update merge with GITHUB_TOKEN * fix(security): address review feedback on PR #4126 - validate_module_string: normalize backslashes before component scan (Linux Path::components treated '..\..\etc' as one filename), reject NUL/control characters in payload. - wasm host_call fuel: fail closed if fuel meter is unavailable or set_fuel errors instead of silently swallowing — the reservation is the only DOW guard. - migration drift check: surface query errors with error! instead of hiding them behind unwrap_or(-1). - Strengthened tests: backslash-traversal + NUL cases for validate_module_string; relative-ordering assertions for fuel cost table. * fix(security): close 3 #3533 module-path bypasses + auto-heal #3538 audit drift Address review of #4126: - #3533: the original PR added validate_module_string only to spawn_agent_inner, but three other entry points feed manifests into the registry without going through it: reload_agent_from_disk (hot-reload), update_manifest (caller- supplied manifest update), and the boot loop's SQLite restore path. A hostile agent.toml on disk + restart, or a disk-edit + reload, would still install module = "python:/etc/passwd.py" into a live agent. Centralised the check in `validate_manifest_module_path` and wired it into all four call sites. Boot restore logs error! and skips the agent (refusing to boot the daemon for one bad manifest would turn a CVE into a DoS); the other paths return Err so the caller sees the rejection. - #3538: the post-merge consistency check only logged error! on drift but never repaired it, so any pre-fix prod DB would spam the log on every restart forever. Replaced with a backfill that inserts INSERT OR IGNORE audit rows for any version <= user_version that's missing, logs a single warn! summarising the rescue, and is idempotent on clean DBs. New regression test: simulate v13/v17/v18 drift by deleting their audit rows after migrate, re-run, assert rows are restored, then re-run a third time and assert the row count is unchanged. * chore: fix incidental rebase breakage on top of #4129 / clippy 1.94 The rebase brought in a bunch of pre-existing breakage on main that this PR has to clear before CI can pass: - recv_event_skipping_lag still took Receiver<Event> but the Arc<Event> rework left desktop calling it with Receiver<Arc<Event>>; flip the helper to Arc<Event> so the only caller compiles. - librefang-llm-driver and tests/common/mod.rs still built CompletionRequest with messages: vec![...] but the Arc<Vec<Message>> rework requires Arc::new(vec![...]). - clippy 1.94 added manual_contains; rewrite EXACT.iter().any(|p| ...) as EXACT.contains(&path). - kernel approval test ignored a Result return value; add let _ = to acknowledge the must_use. These are not part of the security fix proper but the security commit cannot land until the workspace clippy / build is green again. * test(security): add e2e regressions for #3533 spawn boundary and #3532 fuel guard - spawn_agent rejects absolute and '..' traversal module paths at the kernel boundary (locks the wiring on top of the pure-fn validate_module_string unit tests in librefang-runtime). - host_call denial-of-wallet guard fires before dispatch when remaining fuel < per-method reservation cost. - approval.rs: incidental cargo fmt fixup pulled in by pre-commit hook.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four independent hot-path optimizations that all fire on every LLM turn:
messages: Arc<Vec<Message>>inCompletionRequest—call_with_retry'srequest.clone()and the trait-handoff intodriver.complete(request)nowbump a refcount instead of deep-copying a 200-600 KB message history every
turn. Retry on rate-limit / overload was particularly bad before.
Closes Hot path clones whole Vec<Message> + AgentManifest every turn — Cow / Arc would eliminate 200-600KB copies on long sessions #3766.
EventBusbroadcastsArc<Event>—publishwraps the event in anArconce and hands every subscriber a refcount-bumped clone. The hotEventTarget::Broadcastpath used to deep-clone the full payload once forthe global sender plus once per per-agent channel. Closes EventBus::publish deep-clones the Event for every subscriber #3380.
check_all_with_provider_and_recordruns 4 queries instead of up to 10 —collapses the per-window agent / global / provider
SUM(cost_usd)queriesinto a single
SUM(CASE WHEN ...)row per time window, keeping the tokenbudget aggregate as its own (different columns) query. Same semantics, same
early-exit ordering. Closes usage.rs runs 8 SUM aggregate queries per LLM turn inside a single transaction under global mutex #3382.
Cached regexes on five hot paths —
LazyLock/ shared mutex cache forthe LLM
<function name=... />recovery parser, the QQ markdown stripper(10 regexes per outbound message), the structured-output JSON code-block
extractor, the kernel-router phrase matcher (now reuses the existing
REGEX_CACHE), andQualityCheck::MatchesRegexfor workflow gates.Closes Multiple regex patterns recompiled per call in hot paths (QQ, agent_loop, orchestration) #3491.
Test plan
cargo build --workspace --libcargo test --workspacecargo clippy --workspace --all-targets -- -D warnings(
/api/budgetreflects spend; per-window quotas still trigger)./api/eventsSSE while another agent publishes;payload arrives intact.