Skip to content

Commit 0b11017

Browse files
committed
feat(sandbox): validate configuration before workload activation
Closes #3145 Signed-off-by: John Myers <[email protected]>
1 parent fd3fd9c commit 0b11017

39 files changed

Lines changed: 4259 additions & 1376 deletions

File tree

‎architecture/sandbox.md‎

Lines changed: 43 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -507,6 +507,44 @@ the structured 403 and authors the narrowest rule. Mechanistically mapping L7
507507
would either over-broaden rules or require path-templating logic that rots
508508
quickly.
509509

510+
## Configuration Admission
511+
512+
Gateway-managed supervisors reconcile configuration before launching the main
513+
process or exposing workload services. Admission covers the effective policy,
514+
provider layers, credential bindings, and gateway-derived provenance. Explicit
515+
user and global policy precedence is unchanged; an image without a policy uses
516+
the restrictive baseline. An invalid image policy does not become a launchable
517+
default.
518+
519+
The gateway tracks configuration admission independently of compute health.
520+
A blocked startup remains `Provisioning` with a `ConfigurationInvalid` readiness
521+
condition, even when the container backend reports readiness. Gateway management
522+
operations remain available. Replacing the policy or repairing providers allows
523+
the same supervisor to reconcile and launch; it does not recreate the sandbox.
524+
Static policy fields can be replaced before the first accepted activation.
525+
Admission validates policy composition; image and host setup failures, such as
526+
an unresolved OCI user or unavailable isolation facilities, retain their existing
527+
startup error behavior.
528+
529+
Acceptance identifies the effective policy hash/version, configuration revision,
530+
provider-environment revision, and reporting supervisor instance. Startup captures
531+
the matching provider environment and constructs the runtime before reporting
532+
acceptance. Live reconciliation begins only after the main process has spawned,
533+
so it cannot replace the configuration captured for that launch. Restart resets
534+
admission and requires a fresh accepted configuration.
535+
536+
Policy and provider refreshes are prepared before publication. Publication
537+
invalidates prior policy guards before exposing new provider material and swaps
538+
the policy under the same publication locks. Rejected candidates cannot install
539+
their credentials alongside the previous policy. Existing runtime fail-closed
540+
checks remain necessary for in-flight traffic and invalid live updates.
541+
542+
In sidecar topology, the authenticated process supervisor supplies discovery
543+
from the workload image over the existing control socket. The network supervisor
544+
withholds bootstrap until admission succeeds, then sends the accepted policy and
545+
child environment together. Subsequent configuration messages carry both parts
546+
and an ordered generation; older messages cannot restore stale child credentials.
547+
510548
## Policy Revision Acknowledgement
511549

512550
When the supervisor loads a sandbox-scoped policy from the gateway, it retains
@@ -542,11 +580,11 @@ outages cannot block policy polling, enforcement, settings, or provider
542580
refreshes and cannot permanently lose the initial acknowledgement.
543581

544582
Only sandbox-scoped revisions (`PolicySource::Sandbox`, version greater than
545-
zero) are acknowledged. Global policies and local-file development policies do
546-
not use the sandbox revision API and produce no acknowledgement. When explicit
547-
local Rego and data files are configured, the supervisor continues polling the
548-
gateway for settings and provider refreshes but never replaces the local OPA
549-
engine with a gateway policy revision.
583+
zero) use the policy revision acknowledgement API. Global policies use the
584+
configuration admission contract without a sandbox policy revision acknowledgement.
585+
Local Rego/data overrides remain available for standalone development; combining
586+
them with a gateway-managed sandbox is rejected because the gateway cannot admit
587+
the runtime policy it would enforce.
550588

551589
## Failure Behavior
552590

‎architecture/security-policy.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,14 @@ flag defaults to `false` and is security-flagged in policy approval flows.
142142
Incremental merges only ever add the flag to a matching endpoint; clearing it
143143
requires removing the endpoint or replacing the policy.
144144

145+
Image discovery may persist a desired policy for repair, but does not authorize
146+
workload activation. The gateway applies the credential gate after full provider
147+
composition and provenance derivation. A rejected effective configuration keeps
148+
startup blocked with a bounded diagnostic; the supervisor waits for management
149+
repair instead of launching with connection-time denials or a fallback policy.
150+
Accepted runtime state includes the matching provider-environment revision, so
151+
policy and credential updates cannot activate independently.
152+
145153
The network supervisor independently enforces the same boundary. Credentialed
146154
WebSocket upgrades use the parsed relay, binary frames fail closed, and text
147155
placeholders require rewrite. REST bodies continue streaming when body rewrite is disabled. The relay holds

‎crates/openshell-cli/src/run.rs‎

Lines changed: 103 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1510,11 +1510,23 @@ pub async fn sandbox_get(
15101510
sandbox.object_id().to_string()
15111511
};
15121512

1513-
let config = client
1513+
let config_result = client
15141514
.get_sandbox_config(GetSandboxConfigRequest { sandbox_id })
1515-
.await
1516-
.into_diagnostic()?
1517-
.into_inner();
1515+
.await;
1516+
let config = match config_result {
1517+
Ok(response) => response.into_inner(),
1518+
Err(_) if !policy_only && configuration_failure_message(&sandbox).is_some() => {
1519+
// An invalid desired policy must not hide the status needed to
1520+
// repair it. Keep payload-only reads strict.
1521+
GetSandboxConfigResponse {
1522+
configuration_error: configuration_failure_message(&sandbox)
1523+
.unwrap_or_default()
1524+
.to_string(),
1525+
..Default::default()
1526+
}
1527+
}
1528+
Err(error) => return Err(error).into_diagnostic(),
1529+
};
15181530

15191531
if policy_only {
15201532
let Some(ref policy) = config.policy else {
@@ -1548,6 +1560,22 @@ pub async fn sandbox_get(
15481560
println!(" {} {}", "Id:".dimmed(), id);
15491561
println!(" {} {}", "Name:".dimmed(), name);
15501562
println!(" {} {}", "Phase:".dimmed(), phase_name(sandbox.phase()));
1563+
if let Some(status) = sandbox.status.as_ref() {
1564+
for condition in &status.conditions {
1565+
if matches!(
1566+
condition.r#type.as_str(),
1567+
"ConfigurationReady" | "DesiredConfigurationReady"
1568+
) && condition.status.eq_ignore_ascii_case("false")
1569+
{
1570+
println!(
1571+
" {} {}: {}",
1572+
"Configuration:".dimmed(),
1573+
condition.reason,
1574+
condition.message
1575+
);
1576+
}
1577+
}
1578+
}
15511579
if let Some(exit_code) = sandbox.status.as_ref().and_then(|status| status.exit_code) {
15521580
println!(" {} {}", "Exit Code:".dimmed(), exit_code);
15531581
}
@@ -2405,7 +2433,19 @@ pub async fn sandbox_list(
24052433
Ok(())
24062434
}
24072435

2436+
fn configuration_failure_message(sandbox: &Sandbox) -> Option<&str> {
2437+
sandbox
2438+
.status
2439+
.as_ref()?
2440+
.configuration_admission
2441+
.as_ref()
2442+
.map(|admission| admission.error.as_str())
2443+
.filter(|message| !message.is_empty())
2444+
}
2445+
24082446
fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
2447+
use openshell_core::proto::ConfigurationAdmissionState;
2448+
24092449
let meta = sandbox.metadata.as_ref();
24102450
let labels = meta.map_or_else(|| serde_json::json!({}), |m| serde_json::json!(m.labels));
24112451
let annotations = meta.map_or_else(
@@ -2445,6 +2485,25 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
24452485
})
24462486
.collect::<Vec<_>>()
24472487
});
2488+
let admission = sandbox
2489+
.status
2490+
.as_ref()
2491+
.and_then(|status| status.configuration_admission.as_ref())
2492+
.map(|admission| {
2493+
serde_json::json!({
2494+
"state": match ConfigurationAdmissionState::try_from(admission.state) {
2495+
Ok(ConfigurationAdmissionState::Pending) => "pending",
2496+
Ok(ConfigurationAdmissionState::Accepted) => "accepted",
2497+
Ok(ConfigurationAdmissionState::Rejected) => "rejected",
2498+
_ => "unknown",
2499+
},
2500+
"error": admission.error,
2501+
"policy_version": admission.policy_version,
2502+
"policy_hash": admission.policy_hash,
2503+
"config_revision": admission.config_revision,
2504+
"provider_env_revision": admission.provider_env_revision,
2505+
})
2506+
});
24482507
serde_json::json!({
24492508
"id": sandbox.object_id(),
24502509
"name": sandbox.object_name(),
@@ -2458,6 +2517,7 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
24582517
"exit_code": sandbox.status.as_ref().and_then(|status| status.exit_code),
24592518
"conditions": conditions,
24602519
"endpoint_statuses": endpoint_statuses,
2520+
"configuration_admission": admission,
24612521
"created_from_workload_template": created_from_workload_template,
24622522
})
24632523
}
@@ -7217,6 +7277,45 @@ mod tests {
72177277
assert_eq!(json["resources"]["gpu"], 2);
72187278
}
72197279

7280+
#[test]
7281+
fn sandbox_json_exposes_repair_diagnostic_and_accepted_generation() {
7282+
use openshell_core::proto::{ConfigurationAdmissionState, SandboxConfigurationAdmission};
7283+
7284+
let mut sandbox = Sandbox::default();
7285+
sandbox.set_phase(SandboxPhase::Provisioning as i32);
7286+
let status = sandbox.status.as_mut().unwrap();
7287+
status.configuration_admission = Some(SandboxConfigurationAdmission {
7288+
state: ConfigurationAdmissionState::Rejected as i32,
7289+
error: "rule image_api requires L7 inspection".to_string(),
7290+
policy_hash: "candidate-hash".to_string(),
7291+
..Default::default()
7292+
});
7293+
status.conditions.push(SandboxCondition {
7294+
r#type: "ConfigurationReady".to_string(),
7295+
status: "False".to_string(),
7296+
reason: "ConfigurationInvalid".to_string(),
7297+
message: "rule image_api requires L7 inspection".to_string(),
7298+
..Default::default()
7299+
});
7300+
let json = super::sandbox_to_json(&sandbox);
7301+
assert_eq!(json["configuration_admission"]["state"], "rejected");
7302+
assert_eq!(json["conditions"][0]["reason"], "ConfigurationInvalid");
7303+
assert_eq!(
7304+
super::configuration_failure_message(&sandbox),
7305+
Some("rule image_api requires L7 inspection")
7306+
);
7307+
sandbox
7308+
.status
7309+
.as_mut()
7310+
.unwrap()
7311+
.configuration_admission
7312+
.as_mut()
7313+
.unwrap()
7314+
.error
7315+
.clear();
7316+
assert_eq!(super::configuration_failure_message(&sandbox), None);
7317+
}
7318+
72207319
#[test]
72217320
fn sandbox_detail_to_json_includes_policy_fields() {
72227321
let mut sandbox = Sandbox {

‎crates/openshell-cli/tests/ensure_providers_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -560,6 +560,13 @@ impl OpenShell for TestOpenShell {
560560
Err(Status::unimplemented("not implemented in test"))
561561
}
562562

563+
async fn report_sandbox_configuration(
564+
&self,
565+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
566+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
567+
Err(Status::unimplemented("not implemented in test"))
568+
}
569+
563570
async fn report_policy_status(
564571
&self,
565572
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/mtls_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,13 @@ impl OpenShell for TestOpenShell {
417417
Err(Status::unimplemented("not implemented in test"))
418418
}
419419

420+
async fn report_sandbox_configuration(
421+
&self,
422+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
423+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
424+
Err(Status::unimplemented("not implemented in test"))
425+
}
426+
420427
async fn report_policy_status(
421428
&self,
422429
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/provider_commands_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -990,6 +990,13 @@ impl OpenShell for TestOpenShell {
990990
Err(Status::unimplemented("not implemented in test"))
991991
}
992992

993+
async fn report_sandbox_configuration(
994+
&self,
995+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
996+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
997+
Err(Status::unimplemented("not implemented in test"))
998+
}
999+
9931000
async fn report_policy_status(
9941001
&self,
9951002
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -827,6 +827,13 @@ impl OpenShell for TestOpenShell {
827827
Err(Status::unimplemented("not implemented in test"))
828828
}
829829

830+
async fn report_sandbox_configuration(
831+
&self,
832+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
833+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
834+
Err(Status::unimplemented("not implemented in test"))
835+
}
836+
830837
async fn report_policy_status(
831838
&self,
832839
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_name_fallback_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -505,6 +505,13 @@ impl OpenShell for TestOpenShell {
505505
Err(Status::unimplemented("not implemented in test"))
506506
}
507507

508+
async fn report_sandbox_configuration(
509+
&self,
510+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
511+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
512+
Err(Status::unimplemented("not implemented in test"))
513+
}
514+
508515
async fn report_policy_status(
509516
&self,
510517
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-core/src/grpc_client.rs‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -824,6 +824,40 @@ pub async fn sync_policy_and_fetch_snapshot(
824824
fetch_settings_snapshot_with_client(&mut client, sandbox_id).await
825825
}
826826

827+
/// Report an exact runtime configuration generation. Pending registration uses
828+
/// the snapshot's instance fence; retain that snapshot across registration retries.
829+
pub async fn report_sandbox_configuration(
830+
endpoint: &str,
831+
sandbox_id: &str,
832+
instance_id: &str,
833+
snapshot: Option<&SettingsPollResult>,
834+
state: crate::proto::ConfigurationAdmissionState,
835+
error: &str,
836+
) -> Result<()> {
837+
let mut client = connect(endpoint).await?;
838+
client
839+
.report_sandbox_configuration(crate::proto::ReportSandboxConfigurationRequest {
840+
sandbox_id: sandbox_id.to_string(),
841+
expected_instance_id: snapshot.map_or_else(String::new, |snapshot| {
842+
snapshot.configuration_instance_id.clone()
843+
}),
844+
admission: Some(crate::proto::SandboxConfigurationAdmission {
845+
instance_id: instance_id.to_string(),
846+
state: state.into(),
847+
policy_version: snapshot.map_or(0, |snapshot| snapshot.version),
848+
policy_hash: snapshot
849+
.map_or_else(String::new, |snapshot| snapshot.policy_hash.clone()),
850+
config_revision: snapshot.map_or(0, |snapshot| snapshot.config_revision),
851+
provider_env_revision: snapshot
852+
.map_or(0, |snapshot| snapshot.provider_env_revision),
853+
error: error.to_string(),
854+
}),
855+
})
856+
.await
857+
.into_diagnostic()?;
858+
Ok(())
859+
}
860+
827861
/// Fetch provider environment variables for a sandbox from `OpenShell` server via gRPC.
828862
///
829863
/// Returns a map of environment variable names to values derived from provider
@@ -922,6 +956,9 @@ pub struct CachedOpenShellClient {
922956
/// Settings poll result returned by [`CachedOpenShellClient::poll_settings`].
923957
#[derive(Clone, Debug)]
924958
pub struct SettingsPollResult {
959+
pub configuration_instance_id: String,
960+
pub configuration_admitted: bool,
961+
pub configuration_error: String,
925962
pub policy: Option<ProtoSandboxPolicy>,
926963
pub version: u32,
927964
pub policy_hash: String,
@@ -943,6 +980,9 @@ pub struct SettingsPollResult {
943980

944981
fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> SettingsPollResult {
945982
SettingsPollResult {
983+
configuration_instance_id: inner.configuration_instance_id,
984+
configuration_admitted: inner.configuration_admitted,
985+
configuration_error: inner.configuration_error,
946986
policy: inner.policy,
947987
version: inner.version,
948988
policy_hash: inner.policy_hash,

0 commit comments

Comments
 (0)