Skip to content

Commit 8ee7e23

Browse files
committed
feat(sandbox): validate configuration before workload activation
Closes #3145 Signed-off-by: John Myers <[email protected]>
1 parent d68b706 commit 8ee7e23

37 files changed

Lines changed: 2955 additions & 1004 deletions

File tree

‎architecture/sandbox.md‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -595,6 +595,44 @@ the structured 403 and authors the narrowest rule. Mechanistically mapping L7
595595
would either over-broaden rules or require path-templating logic that rots
596596
quickly.
597597

598+
## Configuration Admission
599+
600+
Gateway-managed supervisors reconcile configuration before launching the main
601+
process or exposing workload services. Admission covers the effective policy,
602+
provider layers, credential bindings, and gateway-derived provenance. Explicit
603+
user and global policy precedence is unchanged; an image without a policy uses
604+
the restrictive baseline. An invalid image policy does not become a launchable
605+
default.
606+
607+
The gateway tracks configuration admission independently of compute health.
608+
A blocked startup remains `Provisioning` with a `ConfigurationInvalid` readiness
609+
condition, even when the container backend reports readiness. Gateway management
610+
operations remain available. Replacing the policy or repairing providers allows
611+
the same supervisor to reconcile and launch; it does not recreate the sandbox.
612+
Static policy fields can be replaced before the first accepted activation.
613+
Admission validates policy composition; image and host setup failures, such as
614+
an unresolved OCI user or unavailable isolation facilities, retain their existing
615+
startup error behavior.
616+
617+
Acceptance identifies the effective policy hash/version, configuration revision,
618+
provider-environment revision, and reporting supervisor instance. Startup captures
619+
the matching provider environment and constructs the runtime before reporting
620+
acceptance. Live reconciliation begins only after the main process has spawned,
621+
so it cannot replace the configuration captured for that launch. Restart resets
622+
admission and requires a fresh accepted configuration.
623+
624+
Policy and provider refreshes are prepared before publication. Publication
625+
invalidates prior policy guards before exposing new provider material and swaps
626+
the policy under the same publication locks. Rejected candidates cannot install
627+
their credentials alongside the previous policy. Existing runtime fail-closed
628+
checks remain necessary for in-flight traffic and invalid live updates.
629+
630+
In sidecar topology, the authenticated process supervisor supplies discovery
631+
from the workload image over the existing control socket. The network supervisor
632+
withholds bootstrap until admission succeeds, then sends the accepted policy and
633+
child environment together. Subsequent configuration messages carry both parts
634+
and an ordered generation; older messages cannot restore stale child credentials.
635+
598636
## Policy Revision Acknowledgement
599637

600638
When the supervisor loads a sandbox-scoped policy from the gateway, it retains
@@ -630,12 +668,11 @@ outages cannot block policy polling, enforcement, settings, or provider
630668
refreshes and cannot permanently lose the initial acknowledgement.
631669

632670
Only sandbox-scoped revisions (`PolicySource::Sandbox`, version greater than
633-
zero) are acknowledged. Global policies and local-file development policies do
634-
not use the sandbox revision API and produce no acknowledgement. When explicit
635-
local Rego and data files are provisioned into the supervisor, it continues
636-
polling the gateway for settings and provider refreshes but never replaces the
637-
local OPA engine with a gateway policy revision. Workload image files and
638-
environment variables do not configure the separately isolated supervisor.
671+
zero) use the policy revision acknowledgement API. Global policies use the
672+
configuration admission contract without a sandbox policy revision acknowledgement.
673+
Local Rego/data overrides remain available for standalone development; combining
674+
them with a gateway-managed sandbox is rejected because the gateway cannot admit
675+
the runtime policy it would enforce.
639676

640677
## Failure Behavior
641678

‎architecture/security-policy.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,14 @@ flag defaults to `false` and is security-flagged in policy approval flows.
161161
Incremental merges only ever add the flag to a matching endpoint; clearing it
162162
requires removing the endpoint or replacing the policy.
163163

164+
Image discovery may persist a desired policy for repair, but does not authorize
165+
workload activation. The gateway applies the credential gate after full provider
166+
composition and provenance derivation. A rejected effective configuration keeps
167+
startup blocked with a bounded diagnostic; the supervisor waits for management
168+
repair instead of launching with connection-time denials or a fallback policy.
169+
Accepted runtime state includes the matching provider-environment revision, so
170+
policy and credential updates cannot activate independently.
171+
164172
The network supervisor independently enforces the same boundary. Credentialed
165173
WebSocket upgrades use the parsed relay, binary frames fail closed, and text
166174
placeholders require rewrite. REST bodies continue streaming when body rewrite is disabled. The relay holds

‎crates/openshell-cli/src/run.rs‎

Lines changed: 103 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1558,11 +1558,23 @@ where
15581558
sandbox.object_id().to_string()
15591559
};
15601560

1561-
let config = client
1561+
let config_result = client
15621562
.get_sandbox_config(GetSandboxConfigRequest { sandbox_id })
1563-
.await
1564-
.into_diagnostic()?
1565-
.into_inner();
1563+
.await;
1564+
let config = match config_result {
1565+
Ok(response) => response.into_inner(),
1566+
Err(_) if !policy_only && configuration_failure_message(&sandbox).is_some() => {
1567+
// An invalid desired policy must not hide the status needed to
1568+
// repair it. Keep payload-only reads strict.
1569+
GetSandboxConfigResponse {
1570+
configuration_error: configuration_failure_message(&sandbox)
1571+
.unwrap_or_default()
1572+
.to_string(),
1573+
..Default::default()
1574+
}
1575+
}
1576+
Err(error) => return Err(error).into_diagnostic(),
1577+
};
15661578

15671579
if policy_only {
15681580
let Some(ref policy) = config.policy else {
@@ -1596,6 +1608,22 @@ where
15961608
println!(" {} {}", "Id:".dimmed(), id);
15971609
println!(" {} {}", "Name:".dimmed(), name);
15981610
println!(" {} {}", "Phase:".dimmed(), phase_name(sandbox.phase()));
1611+
if let Some(status) = sandbox.status.as_ref() {
1612+
for condition in &status.conditions {
1613+
if matches!(
1614+
condition.r#type.as_str(),
1615+
"ConfigurationReady" | "DesiredConfigurationReady"
1616+
) && condition.status.eq_ignore_ascii_case("false")
1617+
{
1618+
println!(
1619+
" {} {}: {}",
1620+
"Configuration:".dimmed(),
1621+
condition.reason,
1622+
condition.message
1623+
);
1624+
}
1625+
}
1626+
}
15991627
if let Some(exit_code) = sandbox.status.as_ref().and_then(|status| status.exit_code) {
16001628
println!(" {} {}", "Exit Code:".dimmed(), exit_code);
16011629
}
@@ -2458,7 +2486,19 @@ pub async fn sandbox_list(
24582486
Ok(())
24592487
}
24602488

2489+
fn configuration_failure_message(sandbox: &Sandbox) -> Option<&str> {
2490+
sandbox
2491+
.status
2492+
.as_ref()?
2493+
.configuration_admission
2494+
.as_ref()
2495+
.map(|admission| admission.error.as_str())
2496+
.filter(|message| !message.is_empty())
2497+
}
2498+
24612499
fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
2500+
use openshell_core::proto::ConfigurationAdmissionState;
2501+
24622502
let meta = sandbox.metadata.as_ref();
24632503
let labels = meta.map_or_else(|| serde_json::json!({}), |m| serde_json::json!(m.labels));
24642504
let annotations = meta.map_or_else(
@@ -2498,6 +2538,25 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
24982538
})
24992539
.collect::<Vec<_>>()
25002540
});
2541+
let admission = sandbox
2542+
.status
2543+
.as_ref()
2544+
.and_then(|status| status.configuration_admission.as_ref())
2545+
.map(|admission| {
2546+
serde_json::json!({
2547+
"state": match ConfigurationAdmissionState::try_from(admission.state) {
2548+
Ok(ConfigurationAdmissionState::Pending) => "pending",
2549+
Ok(ConfigurationAdmissionState::Accepted) => "accepted",
2550+
Ok(ConfigurationAdmissionState::Rejected) => "rejected",
2551+
_ => "unknown",
2552+
},
2553+
"error": admission.error,
2554+
"policy_version": admission.policy_version,
2555+
"policy_hash": admission.policy_hash,
2556+
"config_revision": admission.config_revision,
2557+
"provider_env_revision": admission.provider_env_revision,
2558+
})
2559+
});
25012560
serde_json::json!({
25022561
"id": sandbox.object_id(),
25032562
"name": sandbox.object_name(),
@@ -2511,6 +2570,7 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
25112570
"exit_code": sandbox.status.as_ref().and_then(|status| status.exit_code),
25122571
"conditions": conditions,
25132572
"endpoint_statuses": endpoint_statuses,
2573+
"configuration_admission": admission,
25142574
"created_from_workload_template": created_from_workload_template,
25152575
})
25162576
}
@@ -7306,6 +7366,45 @@ mod tests {
73067366
assert_eq!(json["resources"]["gpu"], 2);
73077367
}
73087368

7369+
#[test]
7370+
fn sandbox_json_exposes_repair_diagnostic_and_accepted_generation() {
7371+
use openshell_core::proto::{ConfigurationAdmissionState, SandboxConfigurationAdmission};
7372+
7373+
let mut sandbox = Sandbox::default();
7374+
sandbox.set_phase(SandboxPhase::Provisioning as i32);
7375+
let status = sandbox.status.as_mut().unwrap();
7376+
status.configuration_admission = Some(SandboxConfigurationAdmission {
7377+
state: ConfigurationAdmissionState::Rejected as i32,
7378+
error: "rule image_api requires L7 inspection".to_string(),
7379+
policy_hash: "candidate-hash".to_string(),
7380+
..Default::default()
7381+
});
7382+
status.conditions.push(SandboxCondition {
7383+
r#type: "ConfigurationReady".to_string(),
7384+
status: "False".to_string(),
7385+
reason: "ConfigurationInvalid".to_string(),
7386+
message: "rule image_api requires L7 inspection".to_string(),
7387+
..Default::default()
7388+
});
7389+
let json = super::sandbox_to_json(&sandbox);
7390+
assert_eq!(json["configuration_admission"]["state"], "rejected");
7391+
assert_eq!(json["conditions"][0]["reason"], "ConfigurationInvalid");
7392+
assert_eq!(
7393+
super::configuration_failure_message(&sandbox),
7394+
Some("rule image_api requires L7 inspection")
7395+
);
7396+
sandbox
7397+
.status
7398+
.as_mut()
7399+
.unwrap()
7400+
.configuration_admission
7401+
.as_mut()
7402+
.unwrap()
7403+
.error
7404+
.clear();
7405+
assert_eq!(super::configuration_failure_message(&sandbox), None);
7406+
}
7407+
73097408
#[test]
73107409
fn sandbox_detail_to_json_includes_policy_fields() {
73117410
let mut sandbox = Sandbox {

‎crates/openshell-cli/tests/ensure_providers_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -556,6 +556,13 @@ impl OpenShell for TestOpenShell {
556556
Err(Status::unimplemented("not implemented in test"))
557557
}
558558

559+
async fn report_sandbox_configuration(
560+
&self,
561+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
562+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
563+
Err(Status::unimplemented("not implemented in test"))
564+
}
565+
559566
async fn report_policy_status(
560567
&self,
561568
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/mtls_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,13 @@ impl OpenShell for TestOpenShell {
417417
Err(Status::unimplemented("not implemented in test"))
418418
}
419419

420+
async fn report_sandbox_configuration(
421+
&self,
422+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
423+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
424+
Err(Status::unimplemented("not implemented in test"))
425+
}
426+
420427
async fn report_policy_status(
421428
&self,
422429
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/provider_commands_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -990,6 +990,13 @@ impl OpenShell for TestOpenShell {
990990
Err(Status::unimplemented("not implemented in test"))
991991
}
992992

993+
async fn report_sandbox_configuration(
994+
&self,
995+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
996+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
997+
Err(Status::unimplemented("not implemented in test"))
998+
}
999+
9931000
async fn report_policy_status(
9941001
&self,
9951002
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -827,6 +827,13 @@ impl OpenShell for TestOpenShell {
827827
Err(Status::unimplemented("not implemented in test"))
828828
}
829829

830+
async fn report_sandbox_configuration(
831+
&self,
832+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
833+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
834+
Err(Status::unimplemented("not implemented in test"))
835+
}
836+
830837
async fn report_policy_status(
831838
&self,
832839
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_name_fallback_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -505,6 +505,13 @@ impl OpenShell for TestOpenShell {
505505
Err(Status::unimplemented("not implemented in test"))
506506
}
507507

508+
async fn report_sandbox_configuration(
509+
&self,
510+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
511+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
512+
Err(Status::unimplemented("not implemented in test"))
513+
}
514+
508515
async fn report_policy_status(
509516
&self,
510517
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-core/src/grpc_client.rs‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1001,6 +1001,40 @@ pub async fn sync_policy_and_fetch_snapshot(
10011001
fetch_settings_snapshot_with_client(&mut client, sandbox_id).await
10021002
}
10031003

1004+
/// Report an exact runtime configuration generation. Pending registration uses
1005+
/// the snapshot's instance fence; retain that snapshot across registration retries.
1006+
pub async fn report_sandbox_configuration(
1007+
endpoint: &str,
1008+
sandbox_id: &str,
1009+
instance_id: &str,
1010+
snapshot: Option<&SettingsPollResult>,
1011+
state: crate::proto::ConfigurationAdmissionState,
1012+
error: &str,
1013+
) -> Result<()> {
1014+
let mut client = connect(endpoint).await?;
1015+
client
1016+
.report_sandbox_configuration(crate::proto::ReportSandboxConfigurationRequest {
1017+
sandbox_id: sandbox_id.to_string(),
1018+
expected_instance_id: snapshot.map_or_else(String::new, |snapshot| {
1019+
snapshot.configuration_instance_id.clone()
1020+
}),
1021+
admission: Some(crate::proto::SandboxConfigurationAdmission {
1022+
instance_id: instance_id.to_string(),
1023+
state: state.into(),
1024+
policy_version: snapshot.map_or(0, |snapshot| snapshot.version),
1025+
policy_hash: snapshot
1026+
.map_or_else(String::new, |snapshot| snapshot.policy_hash.clone()),
1027+
config_revision: snapshot.map_or(0, |snapshot| snapshot.config_revision),
1028+
provider_env_revision: snapshot
1029+
.map_or(0, |snapshot| snapshot.provider_env_revision),
1030+
error: error.to_string(),
1031+
}),
1032+
})
1033+
.await
1034+
.into_diagnostic()?;
1035+
Ok(())
1036+
}
1037+
10041038
/// Fetch provider environment variables for a sandbox from `OpenShell` server via gRPC.
10051039
///
10061040
/// Returns a map of environment variable names to values derived from provider
@@ -1117,6 +1151,9 @@ pub struct CachedOpenShellClient {
11171151
/// Settings poll result returned by [`CachedOpenShellClient::poll_settings`].
11181152
#[derive(Clone, Debug)]
11191153
pub struct SettingsPollResult {
1154+
pub configuration_instance_id: String,
1155+
pub configuration_admitted: bool,
1156+
pub configuration_error: String,
11201157
pub policy: Option<ProtoSandboxPolicy>,
11211158
pub version: u32,
11221159
pub policy_hash: String,
@@ -1138,6 +1175,9 @@ pub struct SettingsPollResult {
11381175

11391176
fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> SettingsPollResult {
11401177
SettingsPollResult {
1178+
configuration_instance_id: inner.configuration_instance_id,
1179+
configuration_admitted: inner.configuration_admitted,
1180+
configuration_error: inner.configuration_error,
11411181
policy: inner.policy,
11421182
version: inner.version,
11431183
policy_hash: inner.policy_hash,

0 commit comments

Comments
 (0)