Skip to content

Commit 21cdda1

Browse files
committed
feat(sandbox): validate configuration before workload activation
Closes #3145 Signed-off-by: John Myers <[email protected]>
1 parent 12a7a35 commit 21cdda1

37 files changed

Lines changed: 2918 additions & 967 deletions

File tree

‎architecture/sandbox.md‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -595,6 +595,44 @@ the structured 403 and authors the narrowest rule. Mechanistically mapping L7
595595
would either over-broaden rules or require path-templating logic that rots
596596
quickly.
597597

598+
## Configuration Admission
599+
600+
Gateway-managed supervisors reconcile configuration before launching the main
601+
process or exposing workload services. Admission covers the effective policy,
602+
provider layers, credential bindings, and gateway-derived provenance. Explicit
603+
user and global policy precedence is unchanged; an image without a policy uses
604+
the restrictive baseline. An invalid image policy does not become a launchable
605+
default.
606+
607+
The gateway tracks configuration admission independently of compute health.
608+
A blocked startup remains `Provisioning` with a `ConfigurationInvalid` readiness
609+
condition, even when the container backend reports readiness. Gateway management
610+
operations remain available. Replacing the policy or repairing providers allows
611+
the same supervisor to reconcile and launch; it does not recreate the sandbox.
612+
Static policy fields can be replaced before the first accepted activation.
613+
Admission validates policy composition; image and host setup failures, such as
614+
an unresolved OCI user or unavailable isolation facilities, retain their existing
615+
startup error behavior.
616+
617+
Acceptance identifies the effective policy hash/version, configuration revision,
618+
provider-environment revision, and reporting supervisor instance. Startup captures
619+
the matching provider environment and constructs the runtime before reporting
620+
acceptance. Live reconciliation begins only after the main process has spawned,
621+
so it cannot replace the configuration captured for that launch. Restart resets
622+
admission and requires a fresh accepted configuration.
623+
624+
Policy and provider refreshes are prepared before publication. Publication
625+
invalidates prior policy guards before exposing new provider material and swaps
626+
the policy under the same publication locks. Rejected candidates cannot install
627+
their credentials alongside the previous policy. Existing runtime fail-closed
628+
checks remain necessary for in-flight traffic and invalid live updates.
629+
630+
In sidecar topology, the authenticated process supervisor supplies discovery
631+
from the workload image over the existing control socket. The network supervisor
632+
withholds bootstrap until admission succeeds, then sends the accepted policy and
633+
child environment together. Subsequent configuration messages carry both parts
634+
and an ordered generation; older messages cannot restore stale child credentials.
635+
598636
## Policy Revision Acknowledgement
599637

600638
When the supervisor loads a sandbox-scoped policy from the gateway, it retains
@@ -630,12 +668,11 @@ outages cannot block policy polling, enforcement, settings, or provider
630668
refreshes and cannot permanently lose the initial acknowledgement.
631669

632670
Only sandbox-scoped revisions (`PolicySource::Sandbox`, version greater than
633-
zero) are acknowledged. Global policies and local-file development policies do
634-
not use the sandbox revision API and produce no acknowledgement. When explicit
635-
local Rego and data files are provisioned into the supervisor, it continues
636-
polling the gateway for settings and provider refreshes but never replaces the
637-
local OPA engine with a gateway policy revision. Workload image files and
638-
environment variables do not configure the separately isolated supervisor.
671+
zero) use the policy revision acknowledgement API. Global policies use the
672+
configuration admission contract without a sandbox policy revision acknowledgement.
673+
Local Rego/data overrides remain available for standalone development; combining
674+
them with a gateway-managed sandbox is rejected because the gateway cannot admit
675+
the runtime policy it would enforce.
639676

640677
## Failure Behavior
641678

‎architecture/security-policy.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,14 @@ flag defaults to `false` and is security-flagged in policy approval flows.
142142
Incremental merges only ever add the flag to a matching endpoint; clearing it
143143
requires removing the endpoint or replacing the policy.
144144

145+
Image discovery may persist a desired policy for repair, but does not authorize
146+
workload activation. The gateway applies the credential gate after full provider
147+
composition and provenance derivation. A rejected effective configuration keeps
148+
startup blocked with a bounded diagnostic; the supervisor waits for management
149+
repair instead of launching with connection-time denials or a fallback policy.
150+
Accepted runtime state includes the matching provider-environment revision, so
151+
policy and credential updates cannot activate independently.
152+
145153
The network supervisor independently enforces the same boundary. Credentialed
146154
WebSocket upgrades use the parsed relay, binary frames fail closed, and text
147155
placeholders require rewrite. REST bodies continue streaming when body rewrite is disabled. The relay holds

‎crates/openshell-cli/src/run.rs‎

Lines changed: 103 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1510,11 +1510,23 @@ pub async fn sandbox_get(
15101510
sandbox.object_id().to_string()
15111511
};
15121512

1513-
let config = client
1513+
let config_result = client
15141514
.get_sandbox_config(GetSandboxConfigRequest { sandbox_id })
1515-
.await
1516-
.into_diagnostic()?
1517-
.into_inner();
1515+
.await;
1516+
let config = match config_result {
1517+
Ok(response) => response.into_inner(),
1518+
Err(_) if !policy_only && configuration_failure_message(&sandbox).is_some() => {
1519+
// An invalid desired policy must not hide the status needed to
1520+
// repair it. Keep payload-only reads strict.
1521+
GetSandboxConfigResponse {
1522+
configuration_error: configuration_failure_message(&sandbox)
1523+
.unwrap_or_default()
1524+
.to_string(),
1525+
..Default::default()
1526+
}
1527+
}
1528+
Err(error) => return Err(error).into_diagnostic(),
1529+
};
15181530

15191531
if policy_only {
15201532
let Some(ref policy) = config.policy else {
@@ -1548,6 +1560,22 @@ pub async fn sandbox_get(
15481560
println!(" {} {}", "Id:".dimmed(), id);
15491561
println!(" {} {}", "Name:".dimmed(), name);
15501562
println!(" {} {}", "Phase:".dimmed(), phase_name(sandbox.phase()));
1563+
if let Some(status) = sandbox.status.as_ref() {
1564+
for condition in &status.conditions {
1565+
if matches!(
1566+
condition.r#type.as_str(),
1567+
"ConfigurationReady" | "DesiredConfigurationReady"
1568+
) && condition.status.eq_ignore_ascii_case("false")
1569+
{
1570+
println!(
1571+
" {} {}: {}",
1572+
"Configuration:".dimmed(),
1573+
condition.reason,
1574+
condition.message
1575+
);
1576+
}
1577+
}
1578+
}
15511579
if let Some(exit_code) = sandbox.status.as_ref().and_then(|status| status.exit_code) {
15521580
println!(" {} {}", "Exit Code:".dimmed(), exit_code);
15531581
}
@@ -2405,7 +2433,19 @@ pub async fn sandbox_list(
24052433
Ok(())
24062434
}
24072435

2436+
fn configuration_failure_message(sandbox: &Sandbox) -> Option<&str> {
2437+
sandbox
2438+
.status
2439+
.as_ref()?
2440+
.configuration_admission
2441+
.as_ref()
2442+
.map(|admission| admission.error.as_str())
2443+
.filter(|message| !message.is_empty())
2444+
}
2445+
24082446
fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
2447+
use openshell_core::proto::ConfigurationAdmissionState;
2448+
24092449
let meta = sandbox.metadata.as_ref();
24102450
let labels = meta.map_or_else(|| serde_json::json!({}), |m| serde_json::json!(m.labels));
24112451
let annotations = meta.map_or_else(
@@ -2445,6 +2485,25 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
24452485
})
24462486
.collect::<Vec<_>>()
24472487
});
2488+
let admission = sandbox
2489+
.status
2490+
.as_ref()
2491+
.and_then(|status| status.configuration_admission.as_ref())
2492+
.map(|admission| {
2493+
serde_json::json!({
2494+
"state": match ConfigurationAdmissionState::try_from(admission.state) {
2495+
Ok(ConfigurationAdmissionState::Pending) => "pending",
2496+
Ok(ConfigurationAdmissionState::Accepted) => "accepted",
2497+
Ok(ConfigurationAdmissionState::Rejected) => "rejected",
2498+
_ => "unknown",
2499+
},
2500+
"error": admission.error,
2501+
"policy_version": admission.policy_version,
2502+
"policy_hash": admission.policy_hash,
2503+
"config_revision": admission.config_revision,
2504+
"provider_env_revision": admission.provider_env_revision,
2505+
})
2506+
});
24482507
serde_json::json!({
24492508
"id": sandbox.object_id(),
24502509
"name": sandbox.object_name(),
@@ -2458,6 +2517,7 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
24582517
"exit_code": sandbox.status.as_ref().and_then(|status| status.exit_code),
24592518
"conditions": conditions,
24602519
"endpoint_statuses": endpoint_statuses,
2520+
"configuration_admission": admission,
24612521
"created_from_workload_template": created_from_workload_template,
24622522
})
24632523
}
@@ -7217,6 +7277,45 @@ mod tests {
72177277
assert_eq!(json["resources"]["gpu"], 2);
72187278
}
72197279

7280+
#[test]
7281+
fn sandbox_json_exposes_repair_diagnostic_and_accepted_generation() {
7282+
use openshell_core::proto::{ConfigurationAdmissionState, SandboxConfigurationAdmission};
7283+
7284+
let mut sandbox = Sandbox::default();
7285+
sandbox.set_phase(SandboxPhase::Provisioning as i32);
7286+
let status = sandbox.status.as_mut().unwrap();
7287+
status.configuration_admission = Some(SandboxConfigurationAdmission {
7288+
state: ConfigurationAdmissionState::Rejected as i32,
7289+
error: "rule image_api requires L7 inspection".to_string(),
7290+
policy_hash: "candidate-hash".to_string(),
7291+
..Default::default()
7292+
});
7293+
status.conditions.push(SandboxCondition {
7294+
r#type: "ConfigurationReady".to_string(),
7295+
status: "False".to_string(),
7296+
reason: "ConfigurationInvalid".to_string(),
7297+
message: "rule image_api requires L7 inspection".to_string(),
7298+
..Default::default()
7299+
});
7300+
let json = super::sandbox_to_json(&sandbox);
7301+
assert_eq!(json["configuration_admission"]["state"], "rejected");
7302+
assert_eq!(json["conditions"][0]["reason"], "ConfigurationInvalid");
7303+
assert_eq!(
7304+
super::configuration_failure_message(&sandbox),
7305+
Some("rule image_api requires L7 inspection")
7306+
);
7307+
sandbox
7308+
.status
7309+
.as_mut()
7310+
.unwrap()
7311+
.configuration_admission
7312+
.as_mut()
7313+
.unwrap()
7314+
.error
7315+
.clear();
7316+
assert_eq!(super::configuration_failure_message(&sandbox), None);
7317+
}
7318+
72207319
#[test]
72217320
fn sandbox_detail_to_json_includes_policy_fields() {
72227321
let mut sandbox = Sandbox {

‎crates/openshell-cli/tests/ensure_providers_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -560,6 +560,13 @@ impl OpenShell for TestOpenShell {
560560
Err(Status::unimplemented("not implemented in test"))
561561
}
562562

563+
async fn report_sandbox_configuration(
564+
&self,
565+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
566+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
567+
Err(Status::unimplemented("not implemented in test"))
568+
}
569+
563570
async fn report_policy_status(
564571
&self,
565572
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/mtls_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,13 @@ impl OpenShell for TestOpenShell {
417417
Err(Status::unimplemented("not implemented in test"))
418418
}
419419

420+
async fn report_sandbox_configuration(
421+
&self,
422+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
423+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
424+
Err(Status::unimplemented("not implemented in test"))
425+
}
426+
420427
async fn report_policy_status(
421428
&self,
422429
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/provider_commands_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -990,6 +990,13 @@ impl OpenShell for TestOpenShell {
990990
Err(Status::unimplemented("not implemented in test"))
991991
}
992992

993+
async fn report_sandbox_configuration(
994+
&self,
995+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
996+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
997+
Err(Status::unimplemented("not implemented in test"))
998+
}
999+
9931000
async fn report_policy_status(
9941001
&self,
9951002
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -827,6 +827,13 @@ impl OpenShell for TestOpenShell {
827827
Err(Status::unimplemented("not implemented in test"))
828828
}
829829

830+
async fn report_sandbox_configuration(
831+
&self,
832+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
833+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
834+
Err(Status::unimplemented("not implemented in test"))
835+
}
836+
830837
async fn report_policy_status(
831838
&self,
832839
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_name_fallback_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -505,6 +505,13 @@ impl OpenShell for TestOpenShell {
505505
Err(Status::unimplemented("not implemented in test"))
506506
}
507507

508+
async fn report_sandbox_configuration(
509+
&self,
510+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
511+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
512+
Err(Status::unimplemented("not implemented in test"))
513+
}
514+
508515
async fn report_policy_status(
509516
&self,
510517
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-core/src/grpc_client.rs‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -939,6 +939,40 @@ pub async fn sync_policy_and_fetch_snapshot(
939939
fetch_settings_snapshot_with_client(&mut client, sandbox_id).await
940940
}
941941

942+
/// Report an exact runtime configuration generation. Pending registration uses
943+
/// the snapshot's instance fence; retain that snapshot across registration retries.
944+
pub async fn report_sandbox_configuration(
945+
endpoint: &str,
946+
sandbox_id: &str,
947+
instance_id: &str,
948+
snapshot: Option<&SettingsPollResult>,
949+
state: crate::proto::ConfigurationAdmissionState,
950+
error: &str,
951+
) -> Result<()> {
952+
let mut client = connect(endpoint).await?;
953+
client
954+
.report_sandbox_configuration(crate::proto::ReportSandboxConfigurationRequest {
955+
sandbox_id: sandbox_id.to_string(),
956+
expected_instance_id: snapshot.map_or_else(String::new, |snapshot| {
957+
snapshot.configuration_instance_id.clone()
958+
}),
959+
admission: Some(crate::proto::SandboxConfigurationAdmission {
960+
instance_id: instance_id.to_string(),
961+
state: state.into(),
962+
policy_version: snapshot.map_or(0, |snapshot| snapshot.version),
963+
policy_hash: snapshot
964+
.map_or_else(String::new, |snapshot| snapshot.policy_hash.clone()),
965+
config_revision: snapshot.map_or(0, |snapshot| snapshot.config_revision),
966+
provider_env_revision: snapshot
967+
.map_or(0, |snapshot| snapshot.provider_env_revision),
968+
error: error.to_string(),
969+
}),
970+
})
971+
.await
972+
.into_diagnostic()?;
973+
Ok(())
974+
}
975+
942976
/// Fetch provider environment variables for a sandbox from `OpenShell` server via gRPC.
943977
///
944978
/// Returns a map of environment variable names to values derived from provider
@@ -1037,6 +1071,9 @@ pub struct CachedOpenShellClient {
10371071
/// Settings poll result returned by [`CachedOpenShellClient::poll_settings`].
10381072
#[derive(Clone, Debug)]
10391073
pub struct SettingsPollResult {
1074+
pub configuration_instance_id: String,
1075+
pub configuration_admitted: bool,
1076+
pub configuration_error: String,
10401077
pub policy: Option<ProtoSandboxPolicy>,
10411078
pub version: u32,
10421079
pub policy_hash: String,
@@ -1058,6 +1095,9 @@ pub struct SettingsPollResult {
10581095

10591096
fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> SettingsPollResult {
10601097
SettingsPollResult {
1098+
configuration_instance_id: inner.configuration_instance_id,
1099+
configuration_admitted: inner.configuration_admitted,
1100+
configuration_error: inner.configuration_error,
10611101
policy: inner.policy,
10621102
version: inner.version,
10631103
policy_hash: inner.policy_hash,

0 commit comments

Comments
 (0)