Skip to content

Commit 25f9a63

Browse files
drewelezar
authored andcommitted
refactor(compute): register compiled drivers
Signed-off-by: Drew Newberry <[email protected]>
1 parent f067e60 commit 25f9a63

8 files changed

Lines changed: 858 additions & 222 deletions

File tree

‎.agents/skills/debug-openshell-cluster/SKILL.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,13 @@ The target deployment flow is:
1919
4. The CLI registers a reachable gateway endpoint with `openshell gateway add`.
2020
5. The gateway creates sandboxes through the selected compute driver.
2121

22+
The standard gateway binary explicitly installs its compiled Docker, Podman,
23+
Kubernetes, and VM registrations at startup. With no configured driver, the
24+
gateway probes only installed registrations in priority order (Kubernetes,
25+
Podman, then Docker); VM has no probe and remains opt-in. A custom gateway
26+
binary may install a different set, so confirm the binary's registered drivers
27+
when auto-detection reports that no suitable driver is available.
28+
2229
For local evaluation only, TLS may be disabled and the gateway can be reached through `http://127.0.0.1:<port>`.
2330

2431
## Prerequisites

‎architecture/compute-runtimes.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,28 @@ The gateway records driver identity and version from the startup capability
9191
response. Elevated gateway info reports that initialized driver snapshot instead
9292
of re-querying drivers on each request.
9393

94+
## Compiled Driver Selection
95+
96+
The gateway binary explicitly installs the compute drivers compiled into that
97+
binary before entering server startup. The server selects a configured driver
98+
by normalized registry name. When no driver is configured, it evaluates only
99+
the installed drivers' probes in registered priority order, records every
100+
available registration, and selects the first. Drivers without a probe,
101+
including VM, remain opt-in.
102+
103+
Startup computes this selection once after merging configuration. The same
104+
selection drives authentication defaults and runtime construction, so a probe
105+
result cannot change which driver is constructed later in startup.
106+
107+
This follows the same composition model as SQLx's `Any` drivers: the binary
108+
defines the available implementation set, while the runtime consumes a generic
109+
registry. Adding or removing a compiled driver therefore changes registration
110+
rather than the server's selection flow. Alternate gateway binaries can install
111+
their own `ComputeDriverFactory` registrations and hand the completed registry
112+
to `run_cli_with_compute_drivers`; factories receive merged driver config and
113+
finish through the same in-process runtime adapter. A configured UDS endpoint
114+
still takes precedence over a compiled registration with the same name.
115+
94116
## Stop and Start Lifecycle
95117

96118
The gateway persists lifecycle intent before mutating compute:

‎crates/openshell-core/src/config.rs‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,9 @@ pub fn detect_driver() -> Option<ComputeDriverKind> {
208208
None
209209
}
210210

211-
fn is_podman_available() -> bool {
211+
/// Return whether a responsive local Podman API socket is available.
212+
#[must_use]
213+
pub fn is_podman_available() -> bool {
212214
detect_podman_socket().is_some()
213215
}
214216

@@ -266,7 +268,9 @@ fn podman_socket_candidates_from_env(
266268
candidates
267269
}
268270

269-
fn is_docker_available() -> bool {
271+
/// Return whether a responsive local Docker API socket is available.
272+
#[must_use]
273+
pub fn is_docker_available() -> bool {
270274
detect_docker_socket().is_some()
271275
}
272276

‎crates/openshell-server/src/cli.rs‎

Lines changed: 141 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,10 @@ use crate::certgen;
1717
use crate::compute::driver_config::GuestTlsPaths;
1818
use crate::config_file::{self, ConfigFile, GatewayFileSection};
1919
use crate::defaults::{self, LocalTlsPaths};
20-
use crate::{ServerStartupConfig, run_server, tracing_bus::TracingLogBus};
20+
use crate::{
21+
ComputeDriverRegistry, ServerStartupConfig, install_default_compute_drivers, run_server,
22+
tracing_bus::TracingLogBus,
23+
};
2124

2225
/// `OpenShell` gateway process - gRPC and HTTP server with protocol multiplexing.
2326
///
@@ -220,6 +223,11 @@ pub fn command() -> Command {
220223
}
221224

222225
pub async fn run_cli() -> Result<()> {
226+
run_cli_with_compute_drivers(install_default_compute_drivers()).await
227+
}
228+
229+
/// Run the gateway CLI with the compute drivers linked by the binary.
230+
pub async fn run_cli_with_compute_drivers(compute_drivers: ComputeDriverRegistry) -> Result<()> {
223231
rustls::crypto::ring::default_provider()
224232
.install_default()
225233
.map_err(|e| miette::miette!("failed to install rustls crypto provider: {e:?}"))?;
@@ -229,11 +237,15 @@ pub async fn run_cli() -> Result<()> {
229237

230238
match cli.command {
231239
Some(Commands::GenerateCerts(args)) => certgen::run(args).await,
232-
None => Box::pin(run_from_args(cli.run, matches)).await,
240+
None => Box::pin(run_from_args(cli.run, matches, compute_drivers)).await,
233241
}
234242
}
235243

236-
fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<ServerStartupConfig> {
244+
fn prepare_server_config(
245+
args: &mut RunArgs,
246+
matches: &ArgMatches,
247+
compute_drivers: &ComputeDriverRegistry,
248+
) -> Result<ServerStartupConfig> {
237249
// Load TOML when explicitly requested, or from the default XDG location
238250
// when that file exists. Missing default config is not an error: runtime
239251
// defaults and OPENSHELL_* env vars are enough for package-managed starts.
@@ -247,6 +259,10 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser
247259
merge_file_into_args(args, &file.openshell.gateway, matches);
248260
}
249261
normalize_compute_driver_socket_args(args, matches)?;
262+
let compute_driver = compute_drivers
263+
.select(&args.drivers)
264+
.map_err(|error| miette::miette!("{error}"))?;
265+
let compute_driver_kind = compute_driver.name().parse::<ComputeDriverKind>().ok();
250266

251267
let local_tls = apply_runtime_defaults(args)?;
252268
let guest_tls = local_tls.as_ref().map(GuestTlsPaths::from);
@@ -256,7 +272,8 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser
256272

257273
let has_client_ca = args.tls_client_ca.is_some();
258274
let has_oidc = args.oidc_issuer.is_some();
259-
let mtls_auth_enabled = resolve_mtls_auth_enabled(args, matches, file.as_ref());
275+
let mtls_auth_enabled =
276+
resolve_mtls_auth_enabled(args, matches, file.as_ref(), compute_driver_kind);
260277

261278
if args.disable_tls && has_client_ca {
262279
return Err(miette::miette!(
@@ -273,12 +290,7 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser
273290
"mTLS user authentication requires --tls-client-ca so client certificates can be verified."
274291
));
275292
}
276-
if mtls_auth_enabled
277-
&& matches!(
278-
effective_single_driver(args),
279-
Some(ComputeDriverKind::Kubernetes)
280-
)
281-
{
293+
if mtls_auth_enabled && matches!(compute_driver_kind, Some(ComputeDriverKind::Kubernetes)) {
282294
return Err(miette::miette!(
283295
"mTLS user authentication is not supported with the Kubernetes compute driver. Configure OIDC or a trusted fronting proxy for user authentication."
284296
));
@@ -466,11 +478,16 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser
466478
config,
467479
config_file: file,
468480
guest_tls,
481+
compute_driver,
469482
})
470483
}
471484

472-
async fn run_from_args(mut args: RunArgs, matches: ArgMatches) -> Result<()> {
473-
let prepared = prepare_server_config(&mut args, &matches)?;
485+
async fn run_from_args(
486+
mut args: RunArgs,
487+
matches: ArgMatches,
488+
compute_drivers: ComputeDriverRegistry,
489+
) -> Result<()> {
490+
let prepared = prepare_server_config(&mut args, &matches, &compute_drivers)?;
474491

475492
let tracing_log_bus = TracingLogBus::new();
476493
let otlp_config = prepared
@@ -538,7 +555,7 @@ async fn run_from_args(mut args: RunArgs, matches: ArgMatches) -> Result<()> {
538555

539556
info!(bind = %prepared.config.bind_address, "Starting OpenShell server");
540557

541-
let result = Box::pin(run_server(prepared, tracing_log_bus)).await;
558+
let result = Box::pin(run_server(prepared, tracing_log_bus, compute_drivers)).await;
542559

543560
tracing_handle.shutdown();
544561

@@ -774,17 +791,9 @@ fn normalize_compute_driver_socket_args(args: &mut RunArgs, matches: &ArgMatches
774791
}
775792
}
776793

777-
fn effective_single_driver(args: &RunArgs) -> Option<ComputeDriverKind> {
778-
match args.drivers.as_slice() {
779-
[] => openshell_core::config::detect_driver(),
780-
[driver] => driver.parse().ok(),
781-
_ => None,
782-
}
783-
}
784-
785-
fn is_singleplayer_driver(args: &RunArgs) -> bool {
794+
fn is_singleplayer_driver(driver: Option<ComputeDriverKind>) -> bool {
786795
matches!(
787-
effective_single_driver(args),
796+
driver,
788797
Some(ComputeDriverKind::Docker | ComputeDriverKind::Podman | ComputeDriverKind::Vm)
789798
)
790799
}
@@ -793,6 +802,7 @@ fn resolve_mtls_auth_enabled(
793802
args: &RunArgs,
794803
matches: &ArgMatches,
795804
file: Option<&ConfigFile>,
805+
compute_driver: Option<ComputeDriverKind>,
796806
) -> bool {
797807
let file_configured = file
798808
.and_then(|f| f.openshell.gateway.mtls_auth.as_ref())
@@ -805,7 +815,7 @@ fn resolve_mtls_auth_enabled(
805815
return false;
806816
}
807817

808-
is_singleplayer_driver(args)
818+
is_singleplayer_driver(compute_driver)
809819
}
810820

811821
#[cfg(test)]
@@ -814,6 +824,43 @@ mod tests {
814824
use crate::TEST_ENV_LOCK as ENV_LOCK;
815825
use clap::Parser;
816826
use std::net::{IpAddr, Ipv4Addr};
827+
use std::sync::atomic::{AtomicUsize, Ordering};
828+
829+
static REGISTRY_DETECTION_CALLS: AtomicUsize = AtomicUsize::new(0);
830+
831+
fn detect_registered_docker() -> bool {
832+
REGISTRY_DETECTION_CALLS.fetch_add(1, Ordering::SeqCst);
833+
true
834+
}
835+
836+
#[derive(Clone, Copy)]
837+
struct TestComputeDriverFactory;
838+
839+
#[async_trait::async_trait]
840+
impl crate::ComputeDriverFactory for TestComputeDriverFactory {
841+
async fn build(
842+
&self,
843+
_context: crate::ComputeDriverBuildContext<'_>,
844+
) -> openshell_core::Result<crate::ComputeDriverBuildOutput> {
845+
unreachable!("configuration tests do not construct the driver")
846+
}
847+
}
848+
849+
fn detected_docker_registry() -> crate::ComputeDriverRegistry {
850+
let mut registry = crate::ComputeDriverRegistry::new();
851+
registry
852+
.install(
853+
crate::ComputeDriverRegistration::new(
854+
"docker",
855+
100,
856+
Some(detect_registered_docker),
857+
TestComputeDriverFactory,
858+
)
859+
.unwrap(),
860+
)
861+
.unwrap();
862+
registry
863+
}
817864

818865
struct EnvVarGuard {
819866
key: &'static str,
@@ -1281,7 +1328,47 @@ mod tests {
12811328
"/tmp/ca.crt",
12821329
]);
12831330

1284-
assert!(super::resolve_mtls_auth_enabled(&args, &matches, None));
1331+
assert!(super::resolve_mtls_auth_enabled(
1332+
&args,
1333+
&matches,
1334+
None,
1335+
Some(openshell_core::ComputeDriverKind::Docker)
1336+
));
1337+
}
1338+
1339+
#[test]
1340+
fn registry_detection_drives_auth_defaults_once() {
1341+
let _lock = ENV_LOCK
1342+
.lock()
1343+
.unwrap_or_else(std::sync::PoisonError::into_inner);
1344+
let state = tempfile::tempdir().unwrap();
1345+
let config = tempfile::tempdir().unwrap();
1346+
let _state = EnvVarGuard::set("XDG_STATE_HOME", state.path().to_str().unwrap());
1347+
let _config = EnvVarGuard::set("XDG_CONFIG_HOME", config.path().to_str().unwrap());
1348+
let _kubernetes = EnvVarGuard::set("KUBERNETES_SERVICE_HOST", "10.0.0.1");
1349+
let _mtls = EnvVarGuard::remove("OPENSHELL_ENABLE_MTLS_AUTH");
1350+
let _drivers = EnvVarGuard::remove("OPENSHELL_DRIVERS");
1351+
REGISTRY_DETECTION_CALLS.store(0, Ordering::SeqCst);
1352+
1353+
let (mut args, matches) = parse_with_args(&[
1354+
"openshell-gateway",
1355+
"--db-url",
1356+
"sqlite::memory:",
1357+
"--tls-cert",
1358+
"/tmp/server.crt",
1359+
"--tls-key",
1360+
"/tmp/server.key",
1361+
"--tls-client-ca",
1362+
"/tmp/ca.crt",
1363+
]);
1364+
let registry = detected_docker_registry();
1365+
1366+
let prepared = super::prepare_server_config(&mut args, &matches, &registry).unwrap();
1367+
1368+
assert_eq!(prepared.compute_driver.name(), "docker");
1369+
assert!(prepared.config.compute_drivers.is_empty());
1370+
assert!(prepared.config.mtls_auth.enabled);
1371+
assert_eq!(REGISTRY_DETECTION_CALLS.load(Ordering::SeqCst), 1);
12851372
}
12861373

12871374
#[test]
@@ -1305,7 +1392,12 @@ mod tests {
13051392
"/tmp/ca.crt",
13061393
]);
13071394

1308-
assert!(!super::resolve_mtls_auth_enabled(&args, &matches, None));
1395+
assert!(!super::resolve_mtls_auth_enabled(
1396+
&args,
1397+
&matches,
1398+
None,
1399+
Some(openshell_core::ComputeDriverKind::Kubernetes)
1400+
));
13091401
}
13101402

13111403
#[test]
@@ -1340,7 +1432,8 @@ enabled = false
13401432
assert!(!super::resolve_mtls_auth_enabled(
13411433
&args,
13421434
&matches,
1343-
Some(&file)
1435+
Some(&file),
1436+
Some(openshell_core::ComputeDriverKind::Docker)
13441437
));
13451438
}
13461439

@@ -1568,37 +1661,21 @@ ssh_session_ttl_secs = 1234
15681661

15691662
#[test]
15701663
fn singleplayer_driver_matches_only_one_local_driver() {
1571-
for driver in ["docker", "podman", "vm"] {
1572-
let (args, _) = parse_with_args(&[
1573-
"openshell-gateway",
1574-
"--db-url",
1575-
"sqlite::memory:",
1576-
"--drivers",
1577-
driver,
1578-
]);
1664+
for driver in [
1665+
openshell_core::ComputeDriverKind::Docker,
1666+
openshell_core::ComputeDriverKind::Podman,
1667+
openshell_core::ComputeDriverKind::Vm,
1668+
] {
15791669
assert!(
1580-
super::is_singleplayer_driver(&args),
1670+
super::is_singleplayer_driver(Some(driver)),
15811671
"{driver} should be singleplayer"
15821672
);
15831673
}
15841674

1585-
let (k8s, _) = parse_with_args(&[
1586-
"openshell-gateway",
1587-
"--db-url",
1588-
"sqlite::memory:",
1589-
"--drivers",
1590-
"kubernetes",
1591-
]);
1592-
assert!(!super::is_singleplayer_driver(&k8s));
1593-
1594-
let (multi, _) = parse_with_args(&[
1595-
"openshell-gateway",
1596-
"--db-url",
1597-
"sqlite::memory:",
1598-
"--drivers",
1599-
"docker,podman",
1600-
]);
1601-
assert!(!super::is_singleplayer_driver(&multi));
1675+
assert!(!super::is_singleplayer_driver(Some(
1676+
openshell_core::ComputeDriverKind::Kubernetes
1677+
)));
1678+
assert!(!super::is_singleplayer_driver(None));
16021679
}
16031680

16041681
#[test]
@@ -1624,7 +1701,11 @@ ssh_session_ttl_secs = 1234
16241701
Some(std::path::Path::new("/run/openshell/kyma.sock"))
16251702
);
16261703
assert_eq!(args.drivers, ["kyma"]);
1627-
assert!(super::effective_single_driver(&args).is_none());
1704+
assert!(
1705+
args.drivers[0]
1706+
.parse::<openshell_core::ComputeDriverKind>()
1707+
.is_err()
1708+
);
16281709
}
16291710

16301711
#[test]
@@ -1804,8 +1885,12 @@ mem_mib = "not-a-number"
18041885
"--disable-tls",
18051886
]);
18061887

1807-
let prepared =
1808-
super::prepare_server_config(&mut args, &matches).expect("server config is prepared");
1888+
let prepared = super::prepare_server_config(
1889+
&mut args,
1890+
&matches,
1891+
&crate::install_default_compute_drivers(),
1892+
)
1893+
.expect("server config is prepared");
18091894

18101895
assert_eq!(prepared.config.compute_drivers, vec!["podman".to_string()]);
18111896
assert_eq!(

0 commit comments

Comments
 (0)