Skip to content

refactor(arch): KernelApi trait + Arc<dyn KernelApi> AppState (#3566)#4726

Merged
houko merged 5 commits into
mainfrom
refactor/3566-kernel-api
May 7, 2026
Merged

refactor(arch): KernelApi trait + Arc<dyn KernelApi> AppState (#3566)#4726
houko merged 5 commits into
mainfrom
refactor/3566-kernel-api

Conversation

@houko

@houko houko commented May 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • New pub trait KernelApi: KernelHandle + Send + Sync in librefang-kernel (crates/librefang-kernel/src/kernel_api.rs) — the single explicit contract between the HTTP API layer and the kernel.
  • AppState.kernel: Arc<LibreFangKernel>Arc<dyn KernelApi>. Routes never see the concrete kernel struct again; the only place Arc<LibreFangKernel> survives is server::build_router (entry point from the daemon, which has the real kernel) — it implicitly upcasts to Arc<dyn KernelApi> at the AppState struct literal.
  • KernelApi extends KernelHandle (the runtime-side super-trait that bundles every KernelHandle is a 50+ method god trait with 30 silent default Err(...) methods #3746 / librefang-api directly imports kernel internal types at 88+ sites — KernelHandle trait boundary unenforced #3744 role trait), so Arc<dyn KernelApi> upcasts to Arc<dyn KernelHandle> and to any individual role trait. Removes the 7 as Arc<dyn KernelHandle> non-primitive casts that openai_compat / agents / network / workflows / ws / tools_sessions used to perform.
  • channel_bridge::KernelBridgeAdapter, start_channel_bridge, start_channel_bridge_with_config migrated to Arc<dyn KernelApi>. Routes no longer import LibreFangKernel.

Trait shape

  • ~115 method signatures: subsystem accessors (audit, agent_registry, approvals, hands, memory, MCP, templates, supervisor, browser, media, tts, web_tools, …), *_ref accessors (config, model_catalog, mcp_tools, event_bus, peer_registry, …), config / lifecycle (reload_config, validate_config_for_reload, shutdown), full agent lifecycle, messaging (send_message + handle / blocks / sender_context / incognito / streaming variants), hand ops, MCP connect/disconnect/retry/reload/reconnect (Arc<Self> receivers), triggers + workflows + events, bindings, vault, auto-dream, a2a tasks/agents, embedding/tts/web/browser/media engines, and data_dir / install_peer_registry_for_test / set_self_handle for integration tests.
  • Naming conflicts with role-trait methods get _typed suffixes:
    • spawn_agent_typed(AgentManifest) -> AgentId vs AgentControl::spawn_agent(&str, Option<&str>) -> (String, String)
    • kill_agent_typed(AgentId) vs AgentControl::kill_agent(&str)
    • run_workflow_typed(WorkflowId, String) -> (WorkflowRunId, String) vs WorkflowRunner::run_workflow(&str, &str) -> (String, String)
    • publish_typed_event(Event) -> Vec<TriggerMatch> vs EventBus::publish_event(&str, Value) -> Result<(), _>
    • Routes/tests migrated to the typed names (agents.rs, workflows.rs, channel_bridge.rs, agent_kv_authz_integration.rs, agents_routes_integration.rs).
  • The two generic-closure inherent kernel methods become dyn-safe trait methods:
    • update_budget_config(&dyn Fn(&mut BudgetConfig)) (callers wrap as &|b| { ... })
    • model_catalog_update(&mut dyn FnMut(&mut ModelCatalog)) returning (). The 8 providers/registry/server callers that needed the closure's return value now capture it via &mut Option<R> from the surrounding scope.
  • Receiver mechanics: every Arc<Self>-receiver method (connect_/disconnect_/retry_/reconnect_/reload_mcp_servers, send_message_streaming_with_*, start_background_for_agent, spawn_key_validation, auto_dream_trigger_manual, probe_local_provider, set_self_handle) takes self: Arc<Self> on the trait. Call sites consume one strong ref per call (state.kernel.clone().method(...)) — Arc clones are cheap.
  • Bumps librefang-kernel's recursion_limit to 256 — the #[async_trait] expansion across this many methods pushes the default 128-step layout pass over the edge.
  • Module-level #![allow(clippy::too_many_arguments)] on kernel_api.rs because two methods (register_trigger_with_target, send_message_with_incognito) mirror the kernel-inherent 8-arg signatures verbatim — splitting into a builder would diverge the trait surface from the inherent surface and break the "thin facade" invariant.

Verification

  • cargo check --workspace --all-targets passes
  • cargo clippy --workspace --all-targets -- -D warnings passes
  • cargo test -p librefang-api --lib — 632 passed
  • cargo test -p librefang-api --test daemon_lifecycle_test — 7 passed
  • cargo test -p librefang-api --test agents_routes_integration — 19 passed
  • cargo test -p librefang-api --test api_integration_test — 72 passed
  • cargo test -p librefang-api --test config_routes_integration --test network_routes_integration --test load_test — 13 + 12 + … passed
  • (Full cargo test -p librefang-api ran into a docker-host OOM during the test-binary link step; ran the suites individually instead. CI has more headroom.)

Out-of-scope follow-ups

  • Banning librefang_kernel::LibreFangKernel imports from librefang-api at the compiler level — nothing in librefang-api/src references it any more, but a cargo deny / clippy disallowed_types rule would harden the boundary against regression. Deferred to a follow-up so this PR stays focused on the structural swap.

Closes #3566.

houko added 4 commits May 7, 2026 00:13
Foundation for #3566: define an empty 'pub trait KernelApi: Send + Sync'
in librefang-kernel and an 'impl KernelApi for LibreFangKernel {}'.
The trait is the single explicit contract between the API layer and the
kernel — subsequent commits widen it method-by-method until AppState's
'pub kernel: Arc<LibreFangKernel>' can be replaced with
'pub kernel: Arc<dyn KernelApi>'.

This is the API-side counterpart to the runtime-side 'KernelHandle'
role traits introduced in #3746 and widened in #3744.
…#3566 2/N)

Adds ~110 method signatures to KernelApi covering subsystem accessors
(audit, agent_registry, approvals, event_bus, hands, memory, MCP,
templates, supervisor, …), '_ref' accessors that expose internal locks,
config / lifecycle hooks (reload_config, validate_config_for_reload,
shutdown), vault, inbox, auto-dream, full agent lifecycle (spawn / kill /
suspend / resume / compact / sessions / manifest / skills / mcp servers /
tool filters), messaging (send_message, send_message_ephemeral), hands
(activate / deactivate / pause / resume / reload / persist), MCP
connection ops (connect / disconnect / retry / reload / reconnect),
triggers + workflows + events, agent bindings, model-catalog access, and
'start_background_for_agent'.

The 'Arc<Self>'-receiver methods (connect_mcp_servers, retry_mcp_connection,
reload_mcp_servers, reconnect_mcp_server, start_background_for_agent) take
'self: Arc<Self>' on the trait so 'Arc<dyn KernelApi>' can dispatch to
them — call sites consume one strong ref per call (cloning the AppState
arc costs nothing).

The two generic-closure inherent methods are exposed as dyn-safe trait
methods:
  - update_budget_config takes '&dyn Fn(&mut BudgetConfig)'
  - model_catalog_update takes '&mut dyn FnMut(&mut ModelCatalog)' and
    returns (); callers needing the closure's return value capture it
    via '&mut Option<R>' from the surrounding scope (the AppState swap
    commit migrates the affected providers / registry / budget routes).

Bumps 'librefang-kernel' recursion_limit to 256 because the
'#[async_trait]' expansion nests pinned futures past the default 128
when the trait is this wide.
…erage (#3566 3/N)

Final structural change for #3566: 'AppState.kernel' is now
'Arc<dyn KernelApi>'. Routes interact with the kernel exclusively
through the trait; the concrete 'LibreFangKernel' is no longer reachable
from 'AppState'.

Trait widening
  - 'KernelApi' now extends 'KernelHandle' (the runtime-facing
    super-trait that bundles every role trait via #3746/#3744). This
    lets 'Arc<dyn KernelApi>' upcast to 'Arc<dyn KernelHandle>' (and to
    any individual role trait) for the runtime-call paths the dashboard
    uses (channel send, prompt store, task queue, approval gate, …) —
    no separate trait object needed.
  - Adds the kernel-inherent surface routes/WS/server/channel_bridge
    actually use that the role traits don't cover: subsystem accessors
    (audit, agent_registry, approvals, hands, memory, MCP, templates,
    supervisor, browser, media, tts, web_tools, …), '_ref' accessors,
    config/lifecycle ops (reload_config, validate_config_for_reload,
    shutdown), full agent lifecycle (spawn/kill/suspend/resume/compact/
    sessions/manifest/skills/MCP servers/tool filters), messaging
    (send_message + handle / blocks / sender_context / incognito /
    streaming variants), hand ops, MCP connect/disconnect/retry/reload/
    reconnect (Arc<Self> receivers), triggers + workflows + events,
    bindings, model-catalog access, vault, auto-dream, a2a tasks/agents,
    embedding/tts/web/browser/media engines, and 'data_dir' /
    'install_peer_registry_for_test' / 'set_self_handle' for
    integration tests.

Naming conflicts with role-trait methods are resolved by suffixing the
typed-arg variants on KernelApi:
  - 'spawn_agent_typed(AgentManifest) -> AgentId' (vs.
    'AgentControl::spawn_agent(&str, Option<&str>) -> (String, String)')
  - 'kill_agent_typed(AgentId)' (vs. 'AgentControl::kill_agent(&str)')
  - 'run_workflow_typed(WorkflowId, String) -> (WorkflowRunId, String)'
    (vs. 'WorkflowRunner::run_workflow(&str, &str) -> (String, String)')
  - 'publish_typed_event(Event) -> Vec<TriggerMatch>' (vs.
    'EventBus::publish_event(&str, Value) -> Result<(), _>')
Routes/tests are migrated to the typed names.

Generic-closure inherent methods become dyn-safe trait methods:
  - 'update_budget_config(&dyn Fn(&mut BudgetConfig))' (callers wrap as
    '&|b| { ... }')
  - 'model_catalog_update(&mut dyn FnMut(&mut ModelCatalog))' returning
    '()'; the providers/registry/server callers that needed the
    closure's return value now capture it via '&mut Option<R>' from the
    surrounding scope.

Receiver mechanics: the seven Arc<Self>-receiver methods
('connect_mcp_servers', 'disconnect_mcp_server' [&self], 'retry_/reconnect_/
reload_mcp_servers', 'send_message_streaming_with_*',
'start_background_for_agent', 'spawn_key_validation',
'auto_dream_trigger_manual', 'probe_local_provider', 'set_self_handle')
take 'self: Arc<Self>' on the trait. Call sites consume one strong ref
per call ('state.kernel.clone().method(...)') — Arc clones are cheap.

channel_bridge / server.rs migration:
  - 'KernelBridgeAdapter' / 'start_channel_bridge' /
    'start_channel_bridge_with_config' now take 'Arc<dyn KernelApi>'.
  - 'server::build_router' still takes 'Arc<LibreFangKernel>' (the
    daemon-side caller has the concrete kernel); it implicitly coerces
    to 'Arc<dyn KernelApi>' at the AppState struct literal.
  - 'as Arc<dyn KernelHandle>' explicit casts at 7 call sites are
    replaced by implicit upcast via the type annotation
    ('let kh: Arc<dyn KernelHandle> = state.kernel.clone();') — works
    because of the new supertrait.

Recursion limit on librefang-kernel is bumped to 256 ('lib.rs') because
the '#[async_trait]' expansion across this many methods pushes the
default 128-step layout pass over the edge.

Verification
  cargo check --workspace --all-targets  passes locally
  (next commits: clippy zero-warnings, then 'cargo test -p librefang-api').
'register_trigger_with_target' (8 args) and 'send_message_with_incognito'
(8 args) mirror the kernel-inherent signatures verbatim. Splitting the
trait method into a builder would diverge the trait surface from the
inherent surface and break the 'trait is a thin facade' invariant. The
inherent kernel methods are themselves over the threshold; clippy is
silenced at the file level rather than per-method to keep the trait body
uncluttered.
@github-actions github-actions Bot added area/kernel Core kernel (scheduling, RBAC, workflows) size/XL 1000+ lines changed labels May 7, 2026
Addresses every concrete item from the self-review pass:

P2 #2 — `build_router` now takes `Arc<dyn KernelApi>` instead of
`Arc<LibreFangKernel>`. The "stub the kernel for tests" goal of #3566
required this final step; embedders that previously held the concrete
type get implicit upcasting via Rust's coercion. `set_self_handle`'s
`self: Arc<Self>` receiver forces a `kernel.clone().set_self_handle()`
at the boot site so the surrounding scope keeps its handle for
`start_background_agents` and friends.

P2 #3 — Removed dead `as_dyn` helper. With `build_router` accepting
`Arc<dyn KernelApi>` directly, callers either upcast at construction
(implicit coerce on `Arc::new(LibreFangKernel { .. })`) or use
`as Arc<dyn KernelApi>` explicitly; the helper was never wired.

P2 #4 — Confirmed by-suite test runs for the 5 highest-risk integration
suites the PR body's run skipped:
  - mcp_oauth_flow_test                        — 7 pass
  - auth_public_allowlist                      — 9 pass
  - pairing_test                               — 8 pass
  - tools_invoke_test                          — 8 pass
  - users_test                                 — 27 pass
mcp_oauth_flow specifically exercises the `Arc<Self>` MCP retry/reconnect
surface that the trait migration touched; clean.

P3 #1 — Module-level `#![allow(clippy::too_many_arguments)]` collapsed
to per-method `#[allow]` on the only two methods that need it
(`register_trigger_with_target` + `send_message_with_incognito`,
both in trait def and `LibreFangKernel` impl). Unrelated methods
that creep over 7 args will trip the lint instead of being silently
masked.

P3 #2 — Inherent `LibreFangKernel::run_workflow` now carries a
docstring pointing at the typed trait method
`KernelApi::run_workflow_typed` so readers crossing the seam don't
mistake the role-trait `String`-shape variant for the typed path.

Note on P2 #1 (RCU retry safety in `routes/providers.rs`): re-read
of the inherent `model_catalog_update` confirms each rcu attempt
clones `(**cat).clone()` into a fresh `next` and runs the closure
on that fork — closures never observe their own previous-attempt
writes. The two flagged callers (`set_model_overrides`,
`delete_custom_model`) are retry-safe; no change needed.

Verification

- `cargo check -p librefang-api -p librefang-kernel --all-targets`
  on host (docker image lacks libdbus / gdk for sibling crates;
  CI Linux runner covers them)
- 5 by-suite tests above
@houko
houko merged commit 13e8bb9 into main May 7, 2026
28 checks passed
@houko
houko deleted the refactor/3566-kernel-api branch May 7, 2026 02:46
houko added a commit that referenced this pull request May 7, 2026
Pull in 7 commits from main since merge-base dbd9042, including the
two Rust changes that landed mid-split:

- #4725 (a6f1f91) — workflow drain on shutdown (#3335). Adds an
  18-line drain block + 1-line trace log between `supervisor.shutdown()`
  and the agent-state-suspend block. Migrated to its post-split home in
  `kernel/background_lifecycle.rs::shutdown` (the original mod.rs site
  was extracted in phase 3a).
- #4726 (13e8bb9) — KernelApi trait + Arc<dyn KernelApi> AppState
  (#3566). Auto-merged cleanly because all touched files outside
  mod.rs (api crate routes / extensions / extractors / kernel_api.rs)
  hit no overlap with the split. The doc comment added on
  `Kernel::run_workflow` referencing `KernelApi::run_workflow_typed`
  was migrated to its post-split home in
  `kernel/triggers_and_workflow.rs`.

The mod.rs conflict was the entire 16,460-line region that the split
moved out into ~28 sibling files. Resolution: keep HEAD (split version,
~2k lines), drop the origin/main reproduction of the pre-split block —
all of that code now lives under sibling submodules with the same
visibility (descendants of `kernel`).

Verified locally:
- `cargo check -p librefang-kernel -p librefang-api --lib` — clean (35s).
- `cargo check --workspace --lib` — clean (2m 10s).
@houko houko mentioned this pull request May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/kernel Core kernel (scheduling, RBAC, workflows) size/XL 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

arch: AppState leaks the entire kernel surface; routes depend on 116 kernel methods

1 participant