Skip to content

Commit 2c31ac7

Browse files
authored
feat(network): support additional destination CAs (#3292)
* feat(network): support additional destination CAs Signed-off-by: Jesse Jaggars <[email protected]> * fix(network): harden additional CA propagation Signed-off-by: Jesse Jaggars <[email protected]> * fix(network): narrow additional CA review scope Signed-off-by: Jesse Jaggars <[email protected]> * fix(network): harden additional CA rotation Signed-off-by: Jesse Jaggars <[email protected]> * feat(network): complete additional CA delivery Signed-off-by: Jesse Jaggars <[email protected]> * fix(docker): remove stale runtime fixture field Signed-off-by: Jesse Jaggars <[email protected]> * fix(network): adapt additional CA series to upstream main Resolve rebase fallout against current upstream main after the series was rebased 214 commits forward: - follow upstream #3110: supervisor→gateway TLS is CA-only, so configure_gateway_tls and its isolation tests drop the client identity - follow upstream tls_secret_names reduction to [String; 1] in the Podman driver while keeping the network trust bundle delivery - port [openshell.supervisor.network] rendering into the openshell.gatewayConfigToml serializer in _toml.tpl - adapt tests to upstream struct/function changes (DriverStartupContext, SandboxRunConfig, IsolationSpecInput, supervisor_pod, VM bootstrap image) - drop conformance-artifact-prefix plumbing removed upstream Signed-off-by: Jesse Jaggars <[email protected]> * refactor(network): adopt upstream additional CA bundle surface Unify the additional destination CA series on the mechanism merged to main in #3996: one operator pre-assembled PEM bundle, the --additional-ca-bundle supervisor flag, and the /etc/openshell/tls/additional/ca-bundle.pem staging path. - Replace the gateway-level additional_ca_cert_paths list with a single additional_ca_bundle file validated and staged at gateway startup. - Restore the upstream supervisor argument path and fold-into-roots behavior; drop the --network-additional-ca-bundle and --network-additional-ca-digest contracts. - Keep gateway normalization bounds, content-addressed staging, digest-based driver reconciliation, and stop/start rotation; the freshness check now runs at the gateway/driver boundary instead of the supervisor boundary. - Reject additional_ca_bundle in operator driver JSON config; standalone drivers self-source the bundle from --sandbox-additional-ca-bundle with the same validation. - Extend the Kubernetes ConfigMap name digest segment to keep the full 63-byte budget after the prefix rename. - Box the persisted-sandbox reconciliation futures so callers stay under the clippy::large_futures threshold. Updates Docker, Podman, Kubernetes, and VM delivery, Helm chart and tests, e2e wiring, gateway configuration docs, and the cluster debugging skill to the single mechanism. Signed-off-by: Jesse Jaggars <[email protected]> * fix(ci): make gateway startup tests wrap-tolerant and refresh example lockfile The network trust startup tests asserted on raw substrings of gateway diagnostics, but the CLI renderer wraps output at a fixed width with continuation markers and hard-breaks long paths. Compare assertions on whitespace- and continuation-squashed text so they hold regardless of wrap position. Format the e2e workspace (separate cargo workspace, not covered by the root cargo fmt run) and refresh examples/isolation-backend-v8/Cargo.lock for the libc dependency added to the network trust crate. Signed-off-by: Jesse Jaggars <[email protected]> * fix(podman): reuse TLS CA secret during stopped-pair trust replacement The additional CA e2e never ran in CI before the first maintainer test dispatch, and its local container inspection was broken in two ways: it located the supervisor container through e2e network membership (the supervisor is host-networked or runtime-bridged, so the filter matched the CI job container instead), and it required the gateway mTLS CA as a bind mount (Podman delivers it as a container secret when user namespaces remap UIDs). Select the supervisor by managed labels and accept the secret delivery form. The lifecycle leg then exposed a driver bug: stopped-pair trust replacement re-created the per-sandbox TLS CA secret, colliding with the secret still owned by the renamed stopped pair. Mirror the token and resolver secret handling: require and reuse the existing secret during replacement instead of creating it. Validated end-to-end with the Podman additional CA e2e lane. Signed-off-by: Jesse Jaggars <[email protected]> * fix(server): restore upstream recovery semantics for start-request failures Recovery no longer marks sandboxes Error/InvalidPersistedSandbox when driver_start_request fails during the startup or lifecycle sweeps. Gateway configuration problems (for example a missing launch signer) must not permanently fail unrelated sandbox records; the sweep logs a warning and skips them, matching the behavior before the additional CA work. Sandbox-record failures surface through the normal start path. Signed-off-by: Jesse Jaggars <[email protected]> * refactor(podman): refresh stale supervisors without replacing workloads Replace the stopped-pair trust reconciliation with supervisor-only recreation. The destination-trust generation label moves from the workload to the supervisor container, and start_sandbox recreates only the supervisor when its generation is missing or stale. The workload container, its volumes, its secrets, and its workspace data are never replaced: the additional CA bundle is mounted only into the supervisor. Running sandboxes are never gated on trust labels, and recovery without fresh launch credentials keeps the existing supervisor. The intentional -removal fence machinery in the watcher existed only for workload replacement and is removed with it. Signed-off-by: Jesse Jaggars <[email protected]> * refactor(docker): drop workload replacement for destination-trust changes Delete the stopped-workload trust reconciliation path and all of its machinery: the trust-generation label on workload containers, the generation match check in the start fence, the stopped-sandbox replacement, the bounded workspace archive copy with its 512 MiB cap and regular-file-only validation, and the boundary-state rollback helpers. No replacement machinery is needed: the trusted supervisor container is already re-created from the gateway's current configuration on every start via ensure_control_process_for_container, so stop/start picks up the current additional CA bundle while the workload container, its volumes, and its workspace data are never replaced. Signed-off-by: Jesse Jaggars <[email protected]> * docs(configuration): drop reconciliation wording from additional CA lifecycle The stop/start lifecycle description already matches the supervisor-only refresh model; remove the two remaining references to restart reconciliation of workload containers. Signed-off-by: Jesse Jaggars <[email protected]> * fix(network): port additional CA test fixtures to the crypto interface Route the gateway TLS isolation test PKI, the destination CA source fixtures, and the gateway startup trust tests through openshell_crypto::pki now that rcgen access is centralized there: generate_keypair/self_signed/signed_by replace the direct rcgen calls and CertifiedKey field access follows the wrapper's shape. Signed-off-by: Jesse Jaggars <[email protected]> * fix(e2e): repair additional CA lane for Docker and non-container builds The first maintainer test dispatch revealed two latent issues in the additional CA e2e that no local run had exercised: the Podman gateway CA secret helper referenced ContainerEngine without the container feature gate, breaking Kubernetes and VM builds of the test, and the Docker branch asserted a client ca.crt bind mount that the Docker driver never creates. Gate the helper like its callers, and assert Docker's actual delivery path instead: the gateway mTLS CA staged as tls/ca.pem inside the read-only supervisor state volume, selected through the OPENSHELL_TLS_CA environment pointer. Signed-off-by: Jesse Jaggars <[email protected]> * fix(helm): restore alphabetical README ordering for supervisor keys The helm:docs check regenerates the chart README with helm-docs, which orders rows alphabetically; a conflict resolution had placed supervisor.network.additionalCaConfigMapName after the supervisor.podSecurityContext rows. Regenerate with mise run helm:docs. Signed-off-by: Jesse Jaggars <[email protected]> * chore: refresh testkit lockfile for openshell-core libc dependency Signed-off-by: Jesse Jaggars <[email protected]> * fix(driver-podman): stage operator CA bundle as driver-owned artifact Standalone Podman drivers accepting --sandbox-additional-ca-bundle previously kept the operator's source path as the trust artifact path. The immutable-artifact verification performed on every create/start requires exactly mode 0444, which an operator-owned source path cannot guarantee, so a readable-but-writable source failed reconciliation. Read the operator bundle with the same bounded, no-symlink validation, then stage the normalized material into a driver-owned artifact inside a private tempdir held for the driver process lifetime. The artifact is created 0600, synced, and only then restricted to 0444 so no observer ever sees a writable artifact. The operator source and its permissions are never referenced after startup. Addresses GATOR-291f43b2-02. Signed-off-by: Jesse Jaggars <[email protected]> * fix(driver-podman): pin retained workload image during supervisor refresh Supervisor-only trust reconciliation re-entered create_sandbox_with_existing_token, which re-pulled and re-resolved the workload image tag even though the workload container is retained. If the tag moved to a different image between stop and start, identity resolution and child-environment construction would reflect the new image while the retained workload still runs the original one, leaving mode-0600 bootstrap files the retained user cannot read. During a refresh with a retained workload, inspect the stopped container and pin to the immutable image ID it was created from (Config.Image); never pull or resolve the request tag. The requested reference is still used for child metadata so the retained workload's environment is byte-identical to its original launch. Addresses GATOR-291f43b2-01. Signed-off-by: Jesse Jaggars <[email protected]> * test(e2e): verify host-only VM trust delivery and drop curl dependency The additional-CA lane's VM assertions read the destination CA out of the guest overlay and a guest-side supervisor-argv marker — artifacts from the guest-supervisor design that no longer exist now that the VM driver stages the CA host-only and runs the supervisor on the host. Inspect the driver-staged host artifact (digest-compared against the source), assert the guest overlay contains no CA material at the old mount paths, and keep the serial-console no-disclosure check. The workload validation scripts also ran curl, which exists only in the container-engine fixture image; the Kubernetes and VM lanes use the uv-based workload image and died instantly with "canonical main process already finished". Rewrite the main-process, exec, and lifecycle validation scripts in python3 (present in every lane's workload image). The network-policy binary identity is the resolved /proc/<pid>/exe target, so the versioned python3.12 binaries are allowlisted rather than the unversioned symlinks. The hostname mismatch check keeps its semantics: same reachable fixture, mismatched TLS SNI, and the failure must specifically be hostname verification. Addresses GATOR-291f43b2-04. Signed-off-by: Jesse Jaggars <[email protected]> * fix(kubernetes): allow admission-added labels on destination-trust ConfigMaps The destination-trust ConfigMap validation compared the whole label map for equality. A mutating admission webhook that attaches an organizational label to the created ConfigMap made the create response fail validation, and the retained ConfigMap then failed every subsequent create/start through the existing-object path, blocking sandbox launches despite the ownership labels, immutable flag, digest, and certificate bytes remaining correct. Validate the gateway-owned label pairs individually and allow unrelated labels; the exact trust-material checks are unchanged. Covered by a regression test with an admission-added label and a missing gateway-owned label. Signed-off-by: Jesse Jaggars <[email protected]> * fix(podman): close supervisor-refresh rollback gaps The supervisor-only trust refresh renamed the previous supervisor to a random rollback name and then performed the token-secret lookup. A transient lookup failure returned with the supervisor still renamed, where a normal delete could never find it, and the rollback rename and final rollback removal ignored their own failures, leaving orphans with attached control and channel resources. Complete every fallible preparation before the rename. On a failed replacement, restore the canonical name; if the restore itself fails, return an error naming the rollback container so the orphan stays identifiable — a later retry still succeeds because the canonical name is absent and the replacement path recreates a fresh supervisor. Treat not-found during the final rollback removal as success and name the orphan in both the log and the error on other removal failures; a retry observes the fresh canonical supervisor and resumes the normal start path. Covered by failure-injection tests for each transaction boundary: preparation failure before any rename, replacement failure restoring the canonical name, and restore failure naming the orphaned rollback container. Signed-off-by: Jesse Jaggars <[email protected]> * docs: address destination-trust review corrections The gateway-configuration RFC documented an additional_ca_cert_paths list that the strict configuration parser never accepted; document the additional_ca_bundle single-path setting the parser actually enforces. The debug-openshell-cluster skill named the wrong generation annotation (openshell.ai/network-supervisor-trust-generation) and described digest argv and network-init canonicalization checks the implementation does not perform. Correct it to openshell.ai/additional-ca-generation and describe the actual delivery: the supervisor Pod mounts the immutable ConfigMap read-only and receives --additional-ca-bundle for the mount path, with the driver validating the digest annotation and exact bytes per creation. Classify the destination-trust generation hashes as deterministic identity derivation rather than authenticity proofs, following the openshell-crypto exclusions table: trust decisions come from the gateway's normalization, immutable staging, exact-byte validation, and TLS verification. Record the classification in the crypto README and at each hashing site. Signed-off-by: Jesse Jaggars <[email protected]> * fix(network): reconcile additional CA tests with suspend/resume and auth changes Upstream's suspend/resume lifecycle took StartSandboxRequest fields 7 and 8, so the durable-snapshot field moves from 7 to 9; update the wire-format test to assert the field-9 key (0x4a) and initialize the new resume controls. Repair the cli test module after the auth opt-in test merged alongside the environment-guard helpers, and initialize the gateway driver-startup context's network-trust fields in the shared supervision test. Signed-off-by: Jesse Jaggars <[email protected]> * fix(server): box start-recovery futures for clippy large-futures The durable-snapshot field grows StartSandboxRequest enough that the start-recovery match and the recovery-expiry test exceed the large-futures limit under -D warnings. Box::pin both futures, matching the pattern the recovery path used before the suspend/resume rework. Signed-off-by: Jesse Jaggars <[email protected]> --------- Signed-off-by: Jesse Jaggars <[email protected]>
1 parent 26134dc commit 2c31ac7

71 files changed

Lines changed: 8024 additions & 434 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/branch-e2e.yml‎

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,19 @@ jobs:
328328
suite-matrix: >-
329329
[{"suite":"ci","runner":"ubuntu-26.04","podman_major":"5","podman_package_version":"5.7.0+ds2-3build1","conmon_package_version":"2.1.13+ds1-2","cmd":"mise run --no-deps --skip-deps e2e:podman:ci"}]
330330
331+
podman-additional-ca-e2e:
332+
needs: [pr_metadata, build-binaries, build-images]
333+
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
334+
permissions:
335+
actions: read
336+
contents: read
337+
packages: read
338+
uses: ./.github/workflows/e2e-podman-test.yml
339+
with:
340+
image-tag: ${{ github.sha }}
341+
suite-matrix: >-
342+
[{"suite":"additional-ca","runner":"ubuntu-26.04","podman_major":"5","podman_package_version":"5.7.0+ds2-3build1","conmon_package_version":"2.1.13+ds1-2","cmd":"OPENSHELL_E2E_PODMAN_TEST=additional_ca mise run --no-deps --skip-deps e2e:podman"}]
343+
331344
vm-e2e:
332345
needs: [pr_metadata, build-binaries, build-vm-driver]
333346
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
@@ -337,6 +350,18 @@ jobs:
337350
packages: read
338351
uses: ./.github/workflows/e2e-vm-test.yml
339352

353+
vm-additional-ca-e2e:
354+
needs: [pr_metadata, build-binaries, build-vm-driver]
355+
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
356+
permissions:
357+
actions: read
358+
contents: read
359+
packages: read
360+
uses: ./.github/workflows/e2e-vm-test.yml
361+
with:
362+
suite-name: additional-ca
363+
test-name: additional_ca
364+
340365
docker-external-driver-e2e:
341366
needs: [pr_metadata, build-binaries, build-gateway-plain, build-external-drivers, build-images]
342367
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
@@ -401,17 +426,26 @@ jobs:
401426
include:
402427
- agent_sandbox_api: v1beta1
403428
agent_sandbox_version: v1.0.3
429+
job_name: Kubernetes E2E (Rust smoke, Agent Sandbox v1beta1)
430+
test_name: ""
404431
- agent_sandbox_api: v1alpha1
405432
agent_sandbox_version: v0.4.6
433+
job_name: Kubernetes E2E (Rust smoke, Agent Sandbox v1alpha1)
434+
test_name: ""
435+
- agent_sandbox_api: v1beta1
436+
agent_sandbox_version: v0.5.0
437+
job_name: Kubernetes E2E (additional destination CA, Agent Sandbox v1beta1)
438+
test_name: additional_ca
406439
permissions:
407440
actions: read
408441
contents: read
409442
packages: read
410443
uses: ./.github/workflows/e2e-kubernetes-test.yml
411444
with:
412445
image-tag: ${{ github.sha }}
413-
job-name: Kubernetes E2E (Rust smoke, Agent Sandbox ${{ matrix.agent_sandbox_api }})
446+
job-name: ${{ matrix.job_name }}
414447
agent-sandbox-version: ${{ matrix.agent_sandbox_version }}
448+
test-name: ${{ matrix.test_name }}
415449

416450
kubernetes-workspace-managed-e2e:
417451
needs: [pr_metadata, build-binaries, build-images]
@@ -494,7 +528,9 @@ jobs:
494528
- driver-specific-integration
495529
- docker-e2e
496530
- podman-e2e
531+
- podman-additional-ca-e2e
497532
- vm-e2e
533+
- vm-additional-ca-e2e
498534
- docker-external-driver-e2e
499535
- podman-external-driver-e2e
500536
- vm-external-driver-e2e

‎.github/workflows/e2e-docker-test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ on:
2929
required: false
3030
type: string
3131
default: >-
32-
[{"suite":"python","cmd":"mise run --no-deps --skip-deps e2e:python","apt_packages":"","python_proto":true,"mcp":false},{"suite":"oidc-python","cmd":"mise run --no-deps --skip-deps e2e:oidc-python:docker","apt_packages":"","python_proto":true,"mcp":false},{"suite":"oidc-pkce-docker","cmd":"mise run --no-deps --skip-deps e2e:oidc-pkce:docker","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"rust-docker","cmd":"mise run --no-deps --skip-deps e2e:rust","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"rust-docker-push","cmd":"mise run --no-deps --skip-deps e2e:rust:push","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"oci-image","cmd":"mise run --no-deps --skip-deps e2e:oci-image","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"mcp","cmd":"mise run --no-deps --skip-deps e2e:mcp","apt_packages":"","python_proto":false,"mcp":true}]
32+
[{"suite":"python","cmd":"mise run --no-deps --skip-deps e2e:python","apt_packages":"","python_proto":true,"mcp":false},{"suite":"oidc-python","cmd":"mise run --no-deps --skip-deps e2e:oidc-python:docker","apt_packages":"","python_proto":true,"mcp":false},{"suite":"oidc-pkce-docker","cmd":"mise run --no-deps --skip-deps e2e:oidc-pkce:docker","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"rust-docker","cmd":"mise run --no-deps --skip-deps e2e:rust","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"rust-docker-push","cmd":"mise run --no-deps --skip-deps e2e:rust:push","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"oci-image","cmd":"mise run --no-deps --skip-deps e2e:oci-image","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"additional-ca","cmd":"OPENSHELL_E2E_DOCKER_TEST=additional_ca mise run --no-deps --skip-deps e2e:docker","apt_packages":"openssh-client","python_proto":false,"mcp":false},{"suite":"mcp","cmd":"mise run --no-deps --skip-deps e2e:mcp","apt_packages":"","python_proto":false,"mcp":true}]
3333
3434
permissions:
3535
actions: read

‎.github/workflows/e2e-vm-test.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@ on:
2222
required: false
2323
type: string
2424
default: e2e:vm
25+
test-name:
26+
required: false
27+
type: string
28+
default: ""
2529

2630
permissions:
2731
actions: read
@@ -55,4 +59,5 @@ jobs:
5559
- name: Run tests
5660
env:
5761
E2E_TASK: ${{ inputs.e2e-task }}
62+
OPENSHELL_E2E_VM_TEST: ${{ inputs.test-name }}
5863
run: mise run --no-deps --skip-deps "$E2E_TASK"

‎Cargo.lock‎

Lines changed: 7 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎crates/openshell-core/Cargo.toml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ openshell-otel = { path = "../openshell-otel", optional = true }
4646
[target.'cfg(unix)'.dependencies]
4747
nix = { workspace = true }
4848
rustix = { workspace = true }
49+
libc = "0.2"
4950

5051
[features]
5152
default = ["telemetry"]
@@ -66,9 +67,10 @@ tonic-prost-build = { workspace = true }
6667
protoc-bin-vendored = { workspace = true }
6768

6869
[dev-dependencies]
70+
h2 = "0.4"
71+
rcgen = { workspace = true }
6972
tempfile = "3"
7073
tracing-subscriber = { workspace = true }
71-
rcgen = { workspace = true }
7274
tokio-rustls = { workspace = true }
7375

7476
[lints]

‎crates/openshell-core/src/driver_utils.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -545,7 +545,7 @@ fn read_regular_file_bounded(path: &str, max_bytes: u64) -> Result<String, Bound
545545
use std::os::unix::fs::OpenOptionsExt as _;
546546
std::fs::OpenOptions::new()
547547
.read(true)
548-
.custom_flags(nix::libc::O_NONBLOCK)
548+
.custom_flags(libc::O_NONBLOCK)
549549
.open(path)
550550
};
551551
#[cfg(not(unix))]

‎crates/openshell-core/src/grpc_client.rs‎

Lines changed: 143 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -370,6 +370,24 @@ impl tonic::service::Interceptor for AuthInterceptor {
370370
}
371371
}
372372

373+
/// Configure server-authenticated TLS for the gateway control-plane endpoint.
374+
///
375+
/// Trust only the configured gateway CA. Destination, system, and proxy
376+
/// trust are intentionally not parameters to this control-plane boundary.
377+
fn configure_gateway_tls(
378+
ep: Endpoint,
379+
ca_pem: Vec<u8>,
380+
server_name: Option<&str>,
381+
) -> Result<Endpoint> {
382+
let mut tls_config = ClientTlsConfig::new().ca_certificate(Certificate::from_pem(ca_pem));
383+
if let Some(server_name) = server_name.filter(|name| !name.is_empty()) {
384+
tls_config = tls_config.domain_name(server_name);
385+
}
386+
ep.tls_config(tls_config)
387+
.into_diagnostic()
388+
.wrap_err("failed to configure TLS")
389+
}
390+
373391
/// Build the plain (un-intercepted) gRPC channel.
374392
///
375393
/// When the endpoint uses `https://`, server-authenticated TLS is configured
@@ -405,27 +423,12 @@ async fn build_plain_channel(endpoint: &str) -> Result<Channel> {
405423
.into_diagnostic()
406424
.wrap_err_with(|| format!("failed to read CA cert from {ca_path}"))?;
407425

408-
// Trust only the configured CA — this is the chart's internal CA
409-
// that signs the gateway's internal server certificate. The gateway uses SNI-based
410-
// certificate selection to present this internal cert to supervisor
411-
// connections, so no public root trust is needed here.
412-
//
413-
// Do NOT add `.with_native_roots()` or `.with_webpki_roots()` here:
414-
// the supervisor runs inside the user-selected sandbox image
415-
// (Docker/Podman drivers), and broadening the trust store would let
416-
// an attacker who controls the image + DNS present a publicly valid
417-
// certificate and intercept the supervisor→gateway TLS connection.
418-
let mut tls_config = ClientTlsConfig::new().ca_certificate(Certificate::from_pem(ca_pem));
419-
if let Ok(server_name) = std::env::var(sandbox_env::GATEWAY_TLS_SERVER_NAME)
420-
&& !server_name.is_empty()
421-
{
422-
tls_config = tls_config.domain_name(server_name);
423-
}
424-
425-
ep = ep
426-
.tls_config(tls_config)
427-
.into_diagnostic()
428-
.wrap_err("failed to configure TLS")?;
426+
// The gateway uses SNI-based certificate selection to present its
427+
// internal cert to supervisor connections. Do not add native,
428+
// webpki, destination, or child-process roots here: an image that can
429+
// influence DNS must not broaden supervisor→gateway authentication.
430+
let server_name = std::env::var(sandbox_env::GATEWAY_TLS_SERVER_NAME).ok();
431+
ep = configure_gateway_tls(ep, ca_pem, server_name.as_deref())?;
429432
}
430433

431434
ep.connect()
@@ -462,6 +465,125 @@ impl Drop for ClientSpanStatus {
462465
}
463466
}
464467

468+
#[cfg(test)]
469+
mod gateway_tls_isolation_tests {
470+
use super::*;
471+
use rcgen::{CertificateParams, IsCa};
472+
use rustls::pki_types::{CertificateDer, PrivateKeyDer};
473+
use std::io::Cursor;
474+
use tokio::net::TcpListener;
475+
use tokio_rustls::TlsAcceptor;
476+
477+
#[allow(clippy::struct_field_names)]
478+
struct TestPki {
479+
ca_pem: Vec<u8>,
480+
server_cert_pem: Vec<u8>,
481+
server_key_pem: Vec<u8>,
482+
}
483+
484+
fn test_pki(server_name: &str) -> TestPki {
485+
let mut ca_params = CertificateParams::new(Vec::<String>::new()).unwrap();
486+
ca_params.is_ca = IsCa::Ca(rcgen::BasicConstraints::Unconstrained);
487+
ca_params.key_usages = vec![
488+
rcgen::KeyUsagePurpose::KeyCertSign,
489+
rcgen::KeyUsagePurpose::CrlSign,
490+
];
491+
let ca_key = openshell_crypto::pki::generate_keypair().unwrap();
492+
let ca = openshell_crypto::pki::self_signed(ca_params, &ca_key).unwrap();
493+
494+
let server_key = openshell_crypto::pki::generate_keypair().unwrap();
495+
let server_params = CertificateParams::new(vec![server_name.to_string()]).unwrap();
496+
let server =
497+
openshell_crypto::pki::signed_by(server_params, &server_key, &ca, &ca_key).unwrap();
498+
TestPki {
499+
ca_pem: ca.pem().into_bytes(),
500+
server_cert_pem: server.pem().into_bytes(),
501+
server_key_pem: server_key.serialize_pem().unwrap().into_bytes(),
502+
}
503+
}
504+
505+
async fn start_h2_tls_server(pki: &TestPki) -> std::net::SocketAddr {
506+
let certificates = rustls_pemfile::certs(&mut Cursor::new(&pki.server_cert_pem))
507+
.collect::<std::result::Result<Vec<CertificateDer<'static>>, _>>()
508+
.unwrap();
509+
let key: PrivateKeyDer<'static> =
510+
rustls_pemfile::private_key(&mut Cursor::new(&pki.server_key_pem))
511+
.unwrap()
512+
.unwrap();
513+
let mut config = rustls::ServerConfig::builder()
514+
.with_no_client_auth()
515+
.with_single_cert(certificates, key)
516+
.unwrap();
517+
config.alpn_protocols = vec![b"h2".to_vec()];
518+
let acceptor = TlsAcceptor::from(Arc::new(config));
519+
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
520+
let address = listener.local_addr().unwrap();
521+
tokio::spawn(async move {
522+
let Ok((stream, _)) = listener.accept().await else {
523+
return;
524+
};
525+
let Ok(stream) = acceptor.accept(stream).await else {
526+
return;
527+
};
528+
if let Ok(mut connection) = h2::server::handshake(stream).await {
529+
while let Some(request) = connection.accept().await {
530+
if request.is_err() {
531+
break;
532+
}
533+
}
534+
}
535+
});
536+
address
537+
}
538+
539+
async fn connect_with_gateway_ca(
540+
address: std::net::SocketAddr,
541+
pki: &TestPki,
542+
server_name: &str,
543+
) -> Result<Channel> {
544+
let endpoint = Endpoint::from_shared(format!("https://{address}")).into_diagnostic()?;
545+
configure_gateway_tls(endpoint, pki.ca_pem.clone(), Some(server_name))?
546+
.connect()
547+
.await
548+
.into_diagnostic()
549+
}
550+
551+
#[tokio::test]
552+
async fn destination_ca_cannot_authenticate_gateway() {
553+
const GATEWAY_NAME: &str = "gateway.internal.test";
554+
let gateway = test_pki(GATEWAY_NAME);
555+
let destination = test_pki(GATEWAY_NAME);
556+
557+
let real_address = start_h2_tls_server(&gateway).await;
558+
tokio::time::timeout(
559+
Duration::from_secs(5),
560+
connect_with_gateway_ca(real_address, &gateway, GATEWAY_NAME),
561+
)
562+
.await
563+
.expect("real gateway connection timed out")
564+
.expect("configured gateway CA should authenticate the gateway");
565+
566+
let fake_address = start_h2_tls_server(&destination).await;
567+
let error = tokio::time::timeout(
568+
Duration::from_secs(5),
569+
connect_with_gateway_ca(fake_address, &gateway, GATEWAY_NAME),
570+
)
571+
.await
572+
.expect("fake gateway rejection timed out")
573+
.expect_err("destination-signed fake gateway must be rejected");
574+
assert!(format!("{error:?}").contains("certificate"), "{error:?}");
575+
576+
let mismatch_address = start_h2_tls_server(&gateway).await;
577+
tokio::time::timeout(
578+
Duration::from_secs(5),
579+
connect_with_gateway_ca(mismatch_address, &gateway, "wrong.internal.test"),
580+
)
581+
.await
582+
.expect("hostname mismatch rejection timed out")
583+
.expect_err("normal server-name verification must remain enabled");
584+
}
585+
}
586+
465587
/// Build a Bearer-authenticated channel to the gateway with one sandbox
466588
/// instance's credentials. The first connection starts that instance's
467589
/// renewal task.

‎crates/openshell-core/src/lib.rs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ pub mod mcp;
4040
pub mod metadata;
4141
pub mod middleware;
4242
pub mod net;
43+
pub mod network_trust;
4344
#[cfg(feature = "oauth")]
4445
pub mod oauth;
4546
pub mod paths;
@@ -79,6 +80,7 @@ pub use error::{ComputeDriverError, Error, Result};
7980
pub use metadata::{
8081
GetResourceVersion, ObjectId, ObjectLabels, ObjectName, ObjectWorkspace, SetResourceVersion,
8182
};
83+
pub use network_trust::NetworkSupervisorTrustBundle;
8284
pub use sandbox_session::{SandboxSessionId, SandboxSessionIdError};
8385

8486
/// Build version string derived from git metadata.

0 commit comments

Comments
 (0)