refactor(kernel): decompose LibreFangKernel god struct into 13 subsystems (#3565)#4756
Merged
Conversation
… 1/13)
Move audit_log + metering engine + budget_config off the LibreFangKernel
god struct into a new field-owning MeteringSubsystem under
kernel/subsystems/. Three slots become one. Method bodies stay on the
kernel and reach into the subsystem via self.metering.{audit_log,engine,budget_config}.
Refs #3565.
…2/13) Move process_manager + process_registry off LibreFangKernel into ProcessSubsystem under kernel/subsystems/. Two slots become one; references migrate from self.process_manager / self.process_registry to self.processes.manager / self.processes.registry. Refs #3565.
… 3/13)
Move auth + pairing + vault_cache + vault_recovery_codes_mutex off
LibreFangKernel into SecuritySubsystem under kernel/subsystems/. Four
slots become one. Field references migrate to self.security.{auth,pairing,
vault_cache,vault_recovery_codes_mutex}.
Refs #3565.
…4/13) Move web_ctx + browser_ctx + media_engine + tts_engine + media_drivers off LibreFangKernel into MediaSubsystem under kernel/subsystems/. Five slots become one. References migrate to self.media.<field>. Refs #3565.
Move default_driver + aux_client + embedding_driver + driver_cache + model_catalog + default_model_override off LibreFangKernel into LlmSubsystem under kernel/subsystems/. Six slots become one. References migrate to self.llm.<field>. Refs #3565.
… 6/13) Move workflow engine + template_registry + triggers + background + cron_scheduler + command_queue off LibreFangKernel into WorkflowSubsystem under kernel/subsystems/. Six slots become one. Inner WorkflowEngine field renamed to engine to avoid the self.workflows.workflows collision. Refs #3565.
…7/13) Move skill_registry + hand_registry + skill_generation + skill_review_cooldowns + skill_review_concurrency off LibreFangKernel into SkillsSubsystem under kernel/subsystems/. Five slots become one. References migrate to self.skills.<field>; the CachedToolList.skill_generation field stays put because it's the private cache-invalidation cursor, not the kernel one. Refs #3565.
Move mcp_connections + mcp_auth_states + mcp_oauth_provider + mcp_tools + mcp_summary_cache + mcp_catalog + mcp_health + effective_mcp_servers + mcp_generation off LibreFangKernel into McpSubsystem under kernel/subsystems/. Nine slots become one. Inner field names keep their mcp_ prefix so the migration is purely mechanical (self.mcp_X to self.mcp.mcp_X). Refs #3565.
…/13) Move a2a_task_store + a2a_external_agents + delivery_tracker + bindings + broadcast + peer_registry + peer_node + channel_adapters off LibreFangKernel into MeshSubsystem under kernel/subsystems/. Eight slots become one. References migrate to self.mesh.<field>. Refs #3565.
…3565 10/13) Move approval_manager + hooks + external_hooks + approval_sweep_started + task_board_sweep_started off LibreFangKernel into GovernanceSubsystem under kernel/subsystems/. Five slots become one. References migrate to self.governance.<field>. session_stream_hub_gc_started stays put — it will move with the session stream hub into EventSubsystem. Refs #3565.
…11/13) Move event_bus + session_lifecycle_bus + session_stream_hub + injection_senders + injection_receivers + assistant_routes + route_divergence + session_stream_hub_gc_started off LibreFangKernel into EventSubsystem under kernel/subsystems/. Eight slots become one. References migrate to self.events.<field>. AssistantRouteTarget visibility widened to pub(crate) so the subsystem can name it. Refs #3565.
…12/13) Move memory + wiki_vault + proactive_memory + proactive_memory_extractor + prompt_store off LibreFangKernel into MemorySubsystem under kernel/subsystems/. Five slots become one. Inner MemorySubstrate field renamed to substrate to avoid the self.memory.memory collision. Refs #3565.
…13/13) Move registry + agent_identities + capabilities + scheduler + supervisor + running_tasks + session_interrupts + agent_msg_locks + session_msg_locks + agent_concurrency + hand_runtime_override_locks + decision_traces + agent_watchers off LibreFangKernel into AgentSubsystem under kernel/subsystems/. Thirteen slots become one — the largest cluster. References migrate to self.agents.<field>. Final state: 13 subsystems own all of LibreFangKernel's previously-flat field surface. The kernel struct itself is now a thin facade over typed subsystem handles plus a small residual of cross-cutting state (boot dirs, config, wasm sandbox, context engine, etc.). Refs #3565.
follow-up #1) Each subsystem now owns its trivial accessor methods. The kernel struct keeps the same public API but every accessor body becomes a one-line delegate to the subsystem method that owns the underlying field. Subsystem methods added (all trivial, all tested via the existing kernel-side delegates): - AgentSubsystem: registry_ref, identities_ref, scheduler_ref, supervisor_ref, traces. - EventSubsystem: event_bus_ref, lifecycle_bus, injection_senders_ref. - GovernanceSubsystem: approvals, hook_registry. - LlmSubsystem: catalog_swap, catalog_load, catalog_update, clear_driver_cache, embedding, default_model_override_ref. - McpSubsystem: catalog_swap, catalog_load, health, connections_ref, auth_states_ref, oauth_provider_ref, tools_ref, effective_servers_ref. - MediaSubsystem: web_tools, browser, engine, tts, drivers. - MemorySubsystem: substrate_ref, proactive_store. - MeshSubsystem: a2a_tasks, a2a_agents, channel_adapters_ref, bindings_ref, broadcast_ref, delivery, peer_registry_ref, peer_node_ref. - MeteringSubsystem: audit_log, engine, current_budget, update_budget. - ProcessSubsystem: manager, registry. - SecuritySubsystem: auth_ref, pairing_ref. - SkillsSubsystem: registry_ref, hand_registry_ref. - WorkflowSubsystem: engine_ref, templates_ref, triggers_ref, cron_ref, command_queue_ref. Non-trivial methods (vault_*, mcp_catalog_reload, spawn_*_sweep, gc_sweep, etc.) keep their bodies on LibreFangKernel for now — those involve cross-subsystem coordination and will move in a later follow-up if they fit cleanly. Refs #3565.
…#2) Each of the 13 subsystems now has its own focused trait — AgentSubsystemApi, EventSubsystemApi, GovernanceSubsystemApi, LlmSubsystemApi, McpSubsystemApi, MediaSubsystemApi, MemorySubsystemApi, MeshSubsystemApi, MeteringSubsystemApi, ProcessSubsystemApi, SecuritySubsystemApi, SkillsSubsystemApi, WorkflowSubsystemApi. Each trait pulls the trivial accessors that follow-up #1 added onto the corresponding subsystem struct. Generic mutators (catalog_update, update_budget) keep their inherent-method form because trait methods cannot accept impl Fn / FnMut arguments without dyn boxing — and the overhead would matter on the hot path. Re-exports added in subsystems/mod.rs so external crates can name the trait without referencing the per-subsystem submodule. accessors.rs imports the bundle so the kernel-side delegate methods compile against the trait surface rather than reaching into struct fields. Refs #3565.
follow-up #3) Add a forwarding impl block per focused trait (AgentSubsystemApi, EventSubsystemApi, GovernanceSubsystemApi, etc.) for LibreFangKernel itself in a new kernel/subsystem_forwards.rs module. Each method delegates to the matching subsystem instance (self.agents.<method>(), self.metering.<method>(), …). This lets new callers — and tests / mocks — bind against a focused trait surface (Arc<dyn AgentSubsystemApi>, &dyn MeteringSubsystemApi) without dragging in the entire 50+ method KernelApi trait. Existing Arc<dyn KernelApi> consumers continue to work unchanged. Together with follow-ups #1 and #2, this completes the issue's 'thin facade with subsystem handles, each owning its own state and exposing a focused trait' goal: - #1 made each subsystem own its trivial accessor bodies. - #2 defined a focused per-subsystem trait and implemented it on the subsystem struct. - #3 forwards those same traits onto LibreFangKernel itself, so the kernel can be used either through its fat KernelApi facade or through any subset of focused subsystem traits. Refs #3565.
…s on LibreFangKernel (#3565 follow-up #4) Code review on PR #4756 flagged that several *SubsystemApi traits used the same short method name, so once LibreFangKernel implements all 13 focused traits at once, kernel.catalog_swap() becomes ambiguous and needs UFCS to disambiguate. The forwarding impls were already working around this with McpSubsystemApi::catalog_swap(&self.mcp) etc., which is a yellow flag, not a green one. Rename trait methods so every kernel.<method>() call resolves to a single trait without UFCS: - LlmSubsystemApi::catalog_swap/load -> model_catalog_swap/load - McpSubsystemApi::catalog_swap/load -> mcp_catalog_swap/load - MeteringSubsystemApi::engine -> metering_engine - MediaSubsystemApi::engine -> media_engine - AgentSubsystemApi::registry_ref -> agent_registry_ref - SkillsSubsystemApi::registry_ref -> skill_registry_ref - ProcessSubsystemApi::manager/registry -> process_manager_ref/registry_ref The new names also match the convention LibreFangKernel's existing inherent accessors already follow (mcp_catalog, mcp_health, event_bus_ref, session_lifecycle_bus, …), so binding through the focused trait now reads exactly like binding through the kernel. UFCS workarounds in subsystem_forwards.rs are dropped now that names are unique. accessors.rs delegate calls updated to the new method names. The kernel-public API (LibreFangKernel inherent methods like audit(), metering_ref(), processes(), agent_registry()) is unchanged — only the trait surface is renamed. Refs #3565.
houko
force-pushed
the
refactor/kernel-subsystems-3565
branch
from
May 7, 2026 12:01
2dae971 to
5a4e6d2
Compare
…variant (#3565) Two doc-only fixes after the subsystem extraction: * Restore the `running_tasks` rationale comment (parallel `session_mode = "new"` triggers / `agent_send` fan-out / parallel channel chats; pre-#3172 rekey history) on the field's new home in `AgentSubsystem` — the comment was truncated to a one-liner during the move. * Drop the orphan `running_tasks` doc comment that was left dangling on `LibreFangKernel` after the field migrated to `AgentSubsystem`. * Add an explicit "Status" + "Shutdown ordering invariant" section to `kernel::subsystems` — clarifies that focused traits and forward impls have already landed (follow-ups #2 + #3) while method-body migration is still pending, and documents that graceful shutdown is broadcast-based via `shutdown_tx` + `supervisor.shutdown()` + `workflows.engine.drain_on_shutdown()`, so reorganising fields between subsystem structs is safe with respect to Rust drop order. No behavior change.
…temApi (#3565) The focused per-subsystem traits added in follow-up #2 had no consumers in this PR (the API layer still binds against `Arc<dyn KernelApi>`). Add `#[cfg(test)]` boundary tests next to the trait impls to: 1. **Pin the trait shape** — assert object-safety (`&dyn`) and `Send + Sync` bounds at compile time. Drift in lifetimes / bounds breaks here, not at the first real caller. 2. **Validate routing** — call methods through `&dyn ProcessSubsystemApi` / `&dyn MeteringSubsystemApi` and verify the returned `&Arc<...>` handles are pointer-equal to the ones the constructor stored, with no hidden clone or rewrapping. 3. **Demonstrate the mock pattern** — `ProcessSubsystemApi` ships with a minimal `StubProcesses` impl proving the trait is implementable independently of `LibreFangKernel` boot, so future tests can mock the subsystem without spinning up the full kernel. 4. **Cover both return shapes** — `MeteringSubsystemApi` returns both borrowed `&Arc<T>` (audit_log, metering_engine) and an owned snapshot (current_budget); the test exercises both. Two subsystems are covered as exemplars; the same pattern applies to the remaining 11 traits when consumers materialise. cargo test -p librefang-kernel --lib subsystems:: → 4 passed; 0 failed
… test The pre-existing sanity assertion expected `boot_with_config()` to leave the baseline `default_model.model` untouched, but boot legitimately rewrites it via the auto-detect fallback in `kernel/boot.rs` when the requested provider has no usable credentials. On dev machines that have OPENAI_API_KEY set or Claude Code / Copilot CLI logged in, the fallback fires and the test panics deterministically with `gpt-5` / `copilot/gpt-4o` instead of `user-picked-model`. The regression actually being guarded here is "invalid TOML reload must not mutate the live config" — it does not depend on the post-boot value being exactly the baseline. Snapshot whatever `boot_with_config()` settles on and assert that snapshot survives the failed reload, so the test is deterministic regardless of ambient credentials. Behaviour under test is unchanged; CI was already green because its runners have none of those credentials configured.
The refactor itself is internal — no user-visible behaviour change — but the structural shift (~70 flat fields → 13 typed subsystem handles plus focused-trait forwarding) is large enough to warrant a Maintenance-level trace in the changelog so future reviewers walking `git log` for "why did `LibreFangKernel` shape change?" find the rationale and the remaining-follow-up notes (method-body migration + KernelApi trait carving) without diving into 19 individual commits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3565.
Summary
LibreFangKernelwas a flat-field god struct (~70 fields) thatconcentrated ownership of every subsystem the kernel touches. This PR
decomposes it into 13 typed subsystem handles under
crates/librefang-kernel/src/kernel/subsystems/. The kernel structitself shrinks from ~70 fields to 33 fields (13 subsystem handles
context engine, log reloader, etc.).
What ships in this PR
The PR was originally scoped as field extraction only (commits 1–13);
review feedback pulled in three of the four follow-ups so future
callers can already bind against focused traits without waiting for a
second round-trip:
Field extraction (commits 1–13): mechanical rename —
self.X→self.<sub>.Xacross ~600 internal call-sites. Three inner-namecollisions resolved (
metering.enginewas the originalmetering,workflow.enginewasworkflows,memory.substratewasmemory).Trivial accessor migration (commit 14): bodies of single-line
getters (e.g.
model_catalog_load,clear_driver_cache) movedinside the matching subsystem. Non-trivial method bodies still live
on
LibreFangKerneland reach into subsystems viaself.<sub>.<field>— full method-body migration is the remainingfollow-up.
Focused per-subsystem traits (commit 15): each subsystem exposes
a
*SubsystemApitrait so new callers can bind&dyn MeteringSubsystemApiinstead of dragging in the entireKernelApisurface.Forward impls on the kernel (commit 16):
LibreFangKernelimplements every
*SubsystemApitrait by delegating to itssubsystem field — existing
Arc<dyn KernelApi>flows continue towork unchanged.
Naming disambiguation (commit 17): trait method names made
globally unambiguous so the forward impls compile without UFCS.
Polish + boundary tests (commits 18–19, post-review): preserve
the original
running_tasksdoc, document the explicitshutdown-coordination invariant in
kernel::subsystems, and add#[cfg(test)]boundary tests next toProcessSubsystemApiandMeteringSubsystemApithat:Send + Syncat compile time;&dynand verify routing;StubProcessesmock proving the trait shape isimplementable without
LibreFangKernel.The remaining 11 traits will follow the same pattern as consumers
materialise.
Subsystems
Each subsystem lives in its own file under
kernel/subsystems/:MeteringSubsystemmetering.rsProcessSubsystemprocesses.rsSecuritySubsystemsecurity.rsMediaSubsystemmedia.rsLlmSubsystemllm.rsWorkflowSubsystemworkflow.rsSkillsSubsystemskills.rsMcpSubsystemmcp.rsMeshSubsystemmesh.rsGovernanceSubsystemgovernance.rsEventSubsystemevents.rsMemorySubsystemmemory.rsAgentSubsystemagents.rsDrop ordering
Field reorganisation could in principle reshuffle Rust's implicit drop
order, but graceful shutdown does not depend on it.
LibreFangKernel::shutdownis broadcast-based:
shutdown_tx.send(true)signals every long-running subscriber(cron tick, background sweeps, approval expiry, session-stream-hub
GC, auto-dream scheduler, inbox watcher) to exit its loop.
agents.supervisor.shutdown()stops accepting new work.workflows.engine.drain_on_shutdown()pauses anyRunning/Pendingworkflow runs and persists them with a resume token.Suspended).Documented inline at the top of
kernel::subsystems.Verification
cargo check -p librefang-kernel --lib --tests✅cargo clippy -p librefang-kernel --lib --tests -- -D warnings✅cargo clippy --workspace --all-targets -- -D warnings(pre-push hook) ✅cargo test -p librefang-kernel --lib✅ — 833 passed (829 pre-existingcargo test -p librefang-api --lib✅ — 638 passed, 0 failedcargo fmt --check✅ (pre-commit hook on every commit)What's still deferred
impl LibreFangKernelmethod that touched the moved fields stillhas
self.Xrewritten toself.<sub>.Xrather than living insidean
impl <Subsystem>block. Trivial accessors are migrated in thisPR; moving the rest is the next refactor and shrinks
mod.rstothe "thin facade" goal in the issue.
KernelApitrait carving — the 50+ methodKernelApitraitis still monolithic. Split into per-subsystem traits, with the
fat trait either deprecated or rebuilt as a blanket impl over the
focused ones.
as
ProcessSubsystemApi/MeteringSubsystemApi; will be added asconsumers of each focused trait land.
Commit structure
Test plan