Skip to content

Commit 3ac61d1

Browse files
committed
feat(sandbox): validate configuration before workload activation
Closes #3145 Signed-off-by: John Myers <[email protected]>
1 parent c502be9 commit 3ac61d1

37 files changed

Lines changed: 2968 additions & 1017 deletions

File tree

‎architecture/sandbox.md‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -595,6 +595,44 @@ the structured 403 and authors the narrowest rule. Mechanistically mapping L7
595595
would either over-broaden rules or require path-templating logic that rots
596596
quickly.
597597

598+
## Configuration Admission
599+
600+
Gateway-managed supervisors reconcile configuration before launching the main
601+
process or exposing workload services. Admission covers the effective policy,
602+
provider layers, credential bindings, and gateway-derived provenance. Explicit
603+
user and global policy precedence is unchanged; an image without a policy uses
604+
the restrictive baseline. An invalid image policy does not become a launchable
605+
default.
606+
607+
The gateway tracks configuration admission independently of compute health.
608+
A blocked startup remains `Provisioning` with a `ConfigurationInvalid` readiness
609+
condition, even when the container backend reports readiness. Gateway management
610+
operations remain available. Replacing the policy or repairing providers allows
611+
the same supervisor to reconcile and launch; it does not recreate the sandbox.
612+
Static policy fields can be replaced before the first accepted activation.
613+
Admission validates policy composition; image and host setup failures, such as
614+
an unresolved OCI user or unavailable isolation facilities, retain their existing
615+
startup error behavior.
616+
617+
Acceptance identifies the effective policy hash/version, configuration revision,
618+
provider-environment revision, and reporting supervisor instance. Startup captures
619+
the matching provider environment and constructs the runtime before reporting
620+
acceptance. Live reconciliation begins only after the main process has spawned,
621+
so it cannot replace the configuration captured for that launch. Restart resets
622+
admission and requires a fresh accepted configuration.
623+
624+
Policy and provider refreshes are prepared before publication. Publication
625+
invalidates prior policy guards before exposing new provider material and swaps
626+
the policy under the same publication locks. Rejected candidates cannot install
627+
their credentials alongside the previous policy. Existing runtime fail-closed
628+
checks remain necessary for in-flight traffic and invalid live updates.
629+
630+
In sidecar topology, the authenticated process supervisor supplies discovery
631+
from the workload image over the existing control socket. The network supervisor
632+
withholds bootstrap until admission succeeds, then sends the accepted policy and
633+
child environment together. Subsequent configuration messages carry both parts
634+
and an ordered generation; older messages cannot restore stale child credentials.
635+
598636
## Policy Revision Acknowledgement
599637

600638
When the supervisor loads a sandbox-scoped policy from the gateway, it retains
@@ -630,12 +668,11 @@ outages cannot block policy polling, enforcement, settings, or provider
630668
refreshes and cannot permanently lose the initial acknowledgement.
631669

632670
Only sandbox-scoped revisions (`PolicySource::Sandbox`, version greater than
633-
zero) are acknowledged. Global policies and local-file development policies do
634-
not use the sandbox revision API and produce no acknowledgement. When explicit
635-
local Rego and data files are provisioned into the supervisor, it continues
636-
polling the gateway for settings and provider refreshes but never replaces the
637-
local OPA engine with a gateway policy revision. Workload image files and
638-
environment variables do not configure the separately isolated supervisor.
671+
zero) use the policy revision acknowledgement API. Global policies use the
672+
configuration admission contract without a sandbox policy revision acknowledgement.
673+
Local Rego/data overrides remain available for standalone development; combining
674+
them with a gateway-managed sandbox is rejected because the gateway cannot admit
675+
the runtime policy it would enforce.
639676

640677
## Failure Behavior
641678

‎architecture/security-policy.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,14 @@ flag defaults to `false` and is security-flagged in policy approval flows.
161161
Incremental merges only ever add the flag to a matching endpoint; clearing it
162162
requires removing the endpoint or replacing the policy.
163163

164+
Image discovery may persist a desired policy for repair, but does not authorize
165+
workload activation. The gateway applies the credential gate after full provider
166+
composition and provenance derivation. A rejected effective configuration keeps
167+
startup blocked with a bounded diagnostic; the supervisor waits for management
168+
repair instead of launching with connection-time denials or a fallback policy.
169+
Accepted runtime state includes the matching provider-environment revision, so
170+
policy and credential updates cannot activate independently.
171+
164172
The network supervisor independently enforces the same boundary. Credentialed
165173
WebSocket upgrades use the parsed relay, binary frames fail closed, and text
166174
placeholders require rewrite. REST bodies continue streaming when body rewrite is disabled. The relay holds

‎crates/openshell-cli/src/run.rs‎

Lines changed: 103 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1559,11 +1559,23 @@ where
15591559
sandbox.object_id().to_string()
15601560
};
15611561

1562-
let config = client
1562+
let config_result = client
15631563
.get_sandbox_config(GetSandboxConfigRequest { sandbox_id })
1564-
.await
1565-
.into_diagnostic()?
1566-
.into_inner();
1564+
.await;
1565+
let config = match config_result {
1566+
Ok(response) => response.into_inner(),
1567+
Err(_) if !policy_only && configuration_failure_message(&sandbox).is_some() => {
1568+
// An invalid desired policy must not hide the status needed to
1569+
// repair it. Keep payload-only reads strict.
1570+
GetSandboxConfigResponse {
1571+
configuration_error: configuration_failure_message(&sandbox)
1572+
.unwrap_or_default()
1573+
.to_string(),
1574+
..Default::default()
1575+
}
1576+
}
1577+
Err(error) => return Err(error).into_diagnostic(),
1578+
};
15671579

15681580
if policy_only {
15691581
let Some(ref policy) = config.policy else {
@@ -1597,6 +1609,22 @@ where
15971609
println!(" {} {}", "Id:".dimmed(), id);
15981610
println!(" {} {}", "Name:".dimmed(), name);
15991611
println!(" {} {}", "Phase:".dimmed(), phase_name(sandbox.phase()));
1612+
if let Some(status) = sandbox.status.as_ref() {
1613+
for condition in &status.conditions {
1614+
if matches!(
1615+
condition.r#type.as_str(),
1616+
"ConfigurationReady" | "DesiredConfigurationReady"
1617+
) && condition.status.eq_ignore_ascii_case("false")
1618+
{
1619+
println!(
1620+
" {} {}: {}",
1621+
"Configuration:".dimmed(),
1622+
condition.reason,
1623+
condition.message
1624+
);
1625+
}
1626+
}
1627+
}
16001628
if let Some(exit_code) = sandbox.status.as_ref().and_then(|status| status.exit_code) {
16011629
println!(" {} {}", "Exit Code:".dimmed(), exit_code);
16021630
}
@@ -2459,7 +2487,19 @@ pub async fn sandbox_list(
24592487
Ok(())
24602488
}
24612489

2490+
fn configuration_failure_message(sandbox: &Sandbox) -> Option<&str> {
2491+
sandbox
2492+
.status
2493+
.as_ref()?
2494+
.configuration_admission
2495+
.as_ref()
2496+
.map(|admission| admission.error.as_str())
2497+
.filter(|message| !message.is_empty())
2498+
}
2499+
24622500
fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
2501+
use openshell_core::proto::ConfigurationAdmissionState;
2502+
24632503
let meta = sandbox.metadata.as_ref();
24642504
let labels = meta.map_or_else(|| serde_json::json!({}), |m| serde_json::json!(m.labels));
24652505
let annotations = meta.map_or_else(
@@ -2499,6 +2539,25 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
24992539
})
25002540
.collect::<Vec<_>>()
25012541
});
2542+
let admission = sandbox
2543+
.status
2544+
.as_ref()
2545+
.and_then(|status| status.configuration_admission.as_ref())
2546+
.map(|admission| {
2547+
serde_json::json!({
2548+
"state": match ConfigurationAdmissionState::try_from(admission.state) {
2549+
Ok(ConfigurationAdmissionState::Pending) => "pending",
2550+
Ok(ConfigurationAdmissionState::Accepted) => "accepted",
2551+
Ok(ConfigurationAdmissionState::Rejected) => "rejected",
2552+
_ => "unknown",
2553+
},
2554+
"error": admission.error,
2555+
"policy_version": admission.policy_version,
2556+
"policy_hash": admission.policy_hash,
2557+
"config_revision": admission.config_revision,
2558+
"provider_env_revision": admission.provider_env_revision,
2559+
})
2560+
});
25022561
serde_json::json!({
25032562
"id": sandbox.object_id(),
25042563
"name": sandbox.object_name(),
@@ -2512,6 +2571,7 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
25122571
"exit_code": sandbox.status.as_ref().and_then(|status| status.exit_code),
25132572
"conditions": conditions,
25142573
"endpoint_statuses": endpoint_statuses,
2574+
"configuration_admission": admission,
25152575
"created_from_workload_template": created_from_workload_template,
25162576
})
25172577
}
@@ -7332,6 +7392,45 @@ mod tests {
73327392
assert_eq!(json["resources"]["gpu"], 2);
73337393
}
73347394

7395+
#[test]
7396+
fn sandbox_json_exposes_repair_diagnostic_and_accepted_generation() {
7397+
use openshell_core::proto::{ConfigurationAdmissionState, SandboxConfigurationAdmission};
7398+
7399+
let mut sandbox = Sandbox::default();
7400+
sandbox.set_phase(SandboxPhase::Provisioning as i32);
7401+
let status = sandbox.status.as_mut().unwrap();
7402+
status.configuration_admission = Some(SandboxConfigurationAdmission {
7403+
state: ConfigurationAdmissionState::Rejected as i32,
7404+
error: "rule image_api requires L7 inspection".to_string(),
7405+
policy_hash: "candidate-hash".to_string(),
7406+
..Default::default()
7407+
});
7408+
status.conditions.push(SandboxCondition {
7409+
r#type: "ConfigurationReady".to_string(),
7410+
status: "False".to_string(),
7411+
reason: "ConfigurationInvalid".to_string(),
7412+
message: "rule image_api requires L7 inspection".to_string(),
7413+
..Default::default()
7414+
});
7415+
let json = super::sandbox_to_json(&sandbox);
7416+
assert_eq!(json["configuration_admission"]["state"], "rejected");
7417+
assert_eq!(json["conditions"][0]["reason"], "ConfigurationInvalid");
7418+
assert_eq!(
7419+
super::configuration_failure_message(&sandbox),
7420+
Some("rule image_api requires L7 inspection")
7421+
);
7422+
sandbox
7423+
.status
7424+
.as_mut()
7425+
.unwrap()
7426+
.configuration_admission
7427+
.as_mut()
7428+
.unwrap()
7429+
.error
7430+
.clear();
7431+
assert_eq!(super::configuration_failure_message(&sandbox), None);
7432+
}
7433+
73357434
#[test]
73367435
fn sandbox_detail_to_json_includes_policy_fields() {
73377436
let mut sandbox = Sandbox {

‎crates/openshell-cli/tests/ensure_providers_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -565,6 +565,13 @@ impl OpenShell for TestOpenShell {
565565
Err(Status::unimplemented("not implemented in test"))
566566
}
567567

568+
async fn report_sandbox_configuration(
569+
&self,
570+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
571+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
572+
Err(Status::unimplemented("not implemented in test"))
573+
}
574+
568575
async fn report_policy_status(
569576
&self,
570577
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/mtls_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -420,6 +420,13 @@ impl OpenShell for TestOpenShell {
420420
Err(Status::unimplemented("not implemented in test"))
421421
}
422422

423+
async fn report_sandbox_configuration(
424+
&self,
425+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
426+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
427+
Err(Status::unimplemented("not implemented in test"))
428+
}
429+
423430
async fn report_policy_status(
424431
&self,
425432
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/provider_commands_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1011,6 +1011,13 @@ impl OpenShell for TestOpenShell {
10111011
Err(Status::unimplemented("not implemented in test"))
10121012
}
10131013

1014+
async fn report_sandbox_configuration(
1015+
&self,
1016+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
1017+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
1018+
Err(Status::unimplemented("not implemented in test"))
1019+
}
1020+
10141021
async fn report_policy_status(
10151022
&self,
10161023
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -834,6 +834,13 @@ impl OpenShell for TestOpenShell {
834834
Err(Status::unimplemented("not implemented in test"))
835835
}
836836

837+
async fn report_sandbox_configuration(
838+
&self,
839+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
840+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
841+
Err(Status::unimplemented("not implemented in test"))
842+
}
843+
837844
async fn report_policy_status(
838845
&self,
839846
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_name_fallback_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -510,6 +510,13 @@ impl OpenShell for TestOpenShell {
510510
Err(Status::unimplemented("not implemented in test"))
511511
}
512512

513+
async fn report_sandbox_configuration(
514+
&self,
515+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
516+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
517+
Err(Status::unimplemented("not implemented in test"))
518+
}
519+
513520
async fn report_policy_status(
514521
&self,
515522
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-core/src/grpc_client.rs‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1001,6 +1001,40 @@ pub async fn sync_policy_and_fetch_snapshot(
10011001
fetch_settings_snapshot_with_client(&mut client, sandbox_id).await
10021002
}
10031003

1004+
/// Report an exact runtime configuration generation. Pending registration uses
1005+
/// the snapshot's instance fence; retain that snapshot across registration retries.
1006+
pub async fn report_sandbox_configuration(
1007+
endpoint: &str,
1008+
sandbox_id: &str,
1009+
instance_id: &str,
1010+
snapshot: Option<&SettingsPollResult>,
1011+
state: crate::proto::ConfigurationAdmissionState,
1012+
error: &str,
1013+
) -> Result<()> {
1014+
let mut client = connect(endpoint).await?;
1015+
client
1016+
.report_sandbox_configuration(crate::proto::ReportSandboxConfigurationRequest {
1017+
sandbox_id: sandbox_id.to_string(),
1018+
expected_instance_id: snapshot.map_or_else(String::new, |snapshot| {
1019+
snapshot.configuration_instance_id.clone()
1020+
}),
1021+
admission: Some(crate::proto::SandboxConfigurationAdmission {
1022+
instance_id: instance_id.to_string(),
1023+
state: state.into(),
1024+
policy_version: snapshot.map_or(0, |snapshot| snapshot.version),
1025+
policy_hash: snapshot
1026+
.map_or_else(String::new, |snapshot| snapshot.policy_hash.clone()),
1027+
config_revision: snapshot.map_or(0, |snapshot| snapshot.config_revision),
1028+
provider_env_revision: snapshot
1029+
.map_or(0, |snapshot| snapshot.provider_env_revision),
1030+
error: error.to_string(),
1031+
}),
1032+
})
1033+
.await
1034+
.into_diagnostic()?;
1035+
Ok(())
1036+
}
1037+
10041038
/// Fetch provider environment variables for a sandbox from `OpenShell` server via gRPC.
10051039
///
10061040
/// Returns a map of environment variable names to values derived from provider
@@ -1117,6 +1151,9 @@ pub struct CachedOpenShellClient {
11171151
/// Settings poll result returned by [`CachedOpenShellClient::poll_settings`].
11181152
#[derive(Clone, Debug)]
11191153
pub struct SettingsPollResult {
1154+
pub configuration_instance_id: String,
1155+
pub configuration_admitted: bool,
1156+
pub configuration_error: String,
11201157
pub policy: Option<ProtoSandboxPolicy>,
11211158
pub version: u32,
11221159
pub policy_hash: String,
@@ -1138,6 +1175,9 @@ pub struct SettingsPollResult {
11381175

11391176
fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> SettingsPollResult {
11401177
SettingsPollResult {
1178+
configuration_instance_id: inner.configuration_instance_id,
1179+
configuration_admitted: inner.configuration_admitted,
1180+
configuration_error: inner.configuration_error,
11411181
policy: inner.policy,
11421182
version: inner.version,
11431183
policy_hash: inner.policy_hash,

0 commit comments

Comments
 (0)