Skip to content

Commit e79edbf

Browse files
committed
feat(sandbox): validate configuration before workload activation
Closes #3145 Signed-off-by: John Myers <[email protected]>
1 parent cb93f62 commit e79edbf

34 files changed

Lines changed: 1707 additions & 113 deletions

File tree

‎architecture/sandbox.md‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -618,6 +618,44 @@ the structured 403 and authors the narrowest rule. Mechanistically mapping L7
618618
would either over-broaden rules or require path-templating logic that rots
619619
quickly.
620620

621+
## Configuration Admission
622+
623+
Gateway-managed supervisors reconcile configuration before launching the main
624+
process or exposing workload services. Admission covers the effective policy,
625+
provider layers, credential bindings, and gateway-derived provenance. Explicit
626+
user and global policy precedence is unchanged; an image without a policy uses
627+
the restrictive baseline. An invalid image policy does not become a launchable
628+
default.
629+
630+
The gateway tracks configuration admission independently of compute health.
631+
A blocked startup remains `Provisioning` with a `ConfigurationInvalid` readiness
632+
condition, even when the container backend reports readiness. Gateway management
633+
operations remain available. Replacing the policy or repairing providers allows
634+
the same supervisor to reconcile and launch; it does not recreate the sandbox.
635+
Static policy fields can be replaced before the first accepted activation.
636+
Admission validates policy composition; image and host setup failures, such as
637+
an unresolved OCI user or unavailable isolation facilities, retain their existing
638+
startup error behavior.
639+
640+
Acceptance identifies the effective policy hash/version, configuration revision,
641+
provider-environment revision, and reporting supervisor instance. Startup captures
642+
the matching provider environment and constructs the runtime before reporting
643+
acceptance. Live reconciliation begins only after the main process has spawned,
644+
so it cannot replace the configuration captured for that launch. Restart resets
645+
admission and requires a fresh accepted configuration.
646+
647+
Policy and provider refreshes are prepared before publication. Publication
648+
invalidates prior policy guards before exposing new provider material and swaps
649+
the policy under the same publication locks. Rejected candidates cannot install
650+
their credentials alongside the previous policy. Existing runtime fail-closed
651+
checks remain necessary for in-flight traffic and invalid live updates.
652+
653+
In sidecar topology, the authenticated process supervisor supplies discovery
654+
from the workload image over the existing control socket. The network supervisor
655+
withholds bootstrap until admission succeeds, then sends the accepted policy and
656+
child environment together. Subsequent configuration messages carry both parts
657+
and an ordered generation; older messages cannot restore stale child credentials.
658+
621659
## Policy Revision Acknowledgement
622660

623661
When the supervisor loads a sandbox-scoped policy from the gateway, it retains
@@ -653,12 +691,11 @@ outages cannot block policy polling, enforcement, settings, or provider
653691
refreshes and cannot permanently lose the initial acknowledgement.
654692

655693
Only sandbox-scoped revisions (`PolicySource::Sandbox`, version greater than
656-
zero) are acknowledged. Global policies and local-file development policies do
657-
not use the sandbox revision API and produce no acknowledgement. When explicit
658-
local Rego and data files are provisioned into the supervisor, it continues
659-
polling the gateway for settings and provider refreshes but never replaces the
660-
local OPA engine with a gateway policy revision. Workload image files and
661-
environment variables do not configure the separately isolated supervisor.
694+
zero) use the policy revision acknowledgement API. Global policies use the
695+
configuration admission contract without a sandbox policy revision acknowledgement.
696+
Local Rego/data overrides remain available for standalone development; combining
697+
them with a gateway-managed sandbox is rejected because the gateway cannot admit
698+
the runtime policy it would enforce.
662699

663700
## Failure Behavior
664701

‎architecture/security-policy.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,14 @@ flag defaults to `false` and is security-flagged in policy approval flows.
160160
Incremental merges only ever add the flag to a matching endpoint; clearing it
161161
requires removing the endpoint or replacing the policy.
162162

163+
Image discovery may persist a desired policy for repair, but does not authorize
164+
workload activation. The gateway applies the credential gate after full provider
165+
composition and provenance derivation. A rejected effective configuration keeps
166+
startup blocked with a bounded diagnostic; the supervisor waits for management
167+
repair instead of launching with connection-time denials or a fallback policy.
168+
Accepted runtime state includes the matching provider-environment revision, so
169+
policy and credential updates cannot activate independently.
170+
163171
The network supervisor independently enforces the same boundary. Credentialed
164172
WebSocket upgrades use the parsed relay, binary frames fail closed, and text
165173
placeholders require rewrite. REST bodies continue streaming when body rewrite is disabled. The relay holds

‎crates/openshell-cli/src/run.rs‎

Lines changed: 103 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1561,11 +1561,23 @@ where
15611561
sandbox.object_id().to_string()
15621562
};
15631563

1564-
let config = client
1564+
let config_result = client
15651565
.get_sandbox_config(GetSandboxConfigRequest { sandbox_id })
1566-
.await
1567-
.into_diagnostic()?
1568-
.into_inner();
1566+
.await;
1567+
let config = match config_result {
1568+
Ok(response) => response.into_inner(),
1569+
Err(_) if !policy_only && configuration_failure_message(&sandbox).is_some() => {
1570+
// An invalid desired policy must not hide the status needed to
1571+
// repair it. Keep payload-only reads strict.
1572+
GetSandboxConfigResponse {
1573+
configuration_error: configuration_failure_message(&sandbox)
1574+
.unwrap_or_default()
1575+
.to_string(),
1576+
..Default::default()
1577+
}
1578+
}
1579+
Err(error) => return Err(error).into_diagnostic(),
1580+
};
15691581

15701582
if policy_only {
15711583
let Some(ref policy) = config.policy else {
@@ -1599,6 +1611,22 @@ where
15991611
println!(" {} {}", "Id:".dimmed(), id);
16001612
println!(" {} {}", "Name:".dimmed(), name);
16011613
println!(" {} {}", "Phase:".dimmed(), phase_name(sandbox.phase()));
1614+
if let Some(status) = sandbox.status.as_ref() {
1615+
for condition in &status.conditions {
1616+
if matches!(
1617+
condition.r#type.as_str(),
1618+
"ConfigurationReady" | "DesiredConfigurationReady"
1619+
) && condition.status.eq_ignore_ascii_case("false")
1620+
{
1621+
println!(
1622+
" {} {}: {}",
1623+
"Configuration:".dimmed(),
1624+
condition.reason,
1625+
condition.message
1626+
);
1627+
}
1628+
}
1629+
}
16021630
if let Some(exit_code) = sandbox.status.as_ref().and_then(|status| status.exit_code) {
16031631
println!(" {} {}", "Exit Code:".dimmed(), exit_code);
16041632
}
@@ -2461,7 +2489,19 @@ pub async fn sandbox_list(
24612489
Ok(())
24622490
}
24632491

2492+
fn configuration_failure_message(sandbox: &Sandbox) -> Option<&str> {
2493+
sandbox
2494+
.status
2495+
.as_ref()?
2496+
.configuration_admission
2497+
.as_ref()
2498+
.map(|admission| admission.error.as_str())
2499+
.filter(|message| !message.is_empty())
2500+
}
2501+
24642502
fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
2503+
use openshell_core::proto::ConfigurationAdmissionState;
2504+
24652505
let meta = sandbox.metadata.as_ref();
24662506
let labels = meta.map_or_else(|| serde_json::json!({}), |m| serde_json::json!(m.labels));
24672507
let annotations = meta.map_or_else(
@@ -2501,6 +2541,25 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
25012541
})
25022542
.collect::<Vec<_>>()
25032543
});
2544+
let admission = sandbox
2545+
.status
2546+
.as_ref()
2547+
.and_then(|status| status.configuration_admission.as_ref())
2548+
.map(|admission| {
2549+
serde_json::json!({
2550+
"state": match ConfigurationAdmissionState::try_from(admission.state) {
2551+
Ok(ConfigurationAdmissionState::Pending) => "pending",
2552+
Ok(ConfigurationAdmissionState::Accepted) => "accepted",
2553+
Ok(ConfigurationAdmissionState::Rejected) => "rejected",
2554+
_ => "unknown",
2555+
},
2556+
"error": admission.error,
2557+
"policy_version": admission.policy_version,
2558+
"policy_hash": admission.policy_hash,
2559+
"config_revision": admission.config_revision,
2560+
"provider_env_revision": admission.provider_env_revision,
2561+
})
2562+
});
25042563
serde_json::json!({
25052564
"id": sandbox.object_id(),
25062565
"name": sandbox.object_name(),
@@ -2514,6 +2573,7 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
25142573
"exit_code": sandbox.status.as_ref().and_then(|status| status.exit_code),
25152574
"conditions": conditions,
25162575
"endpoint_statuses": endpoint_statuses,
2576+
"configuration_admission": admission,
25172577
"created_from_workload_template": created_from_workload_template,
25182578
})
25192579
}
@@ -7349,6 +7409,45 @@ mod tests {
73497409
assert_eq!(json["resources"]["gpu"], 2);
73507410
}
73517411

7412+
#[test]
7413+
fn sandbox_json_exposes_repair_diagnostic_and_accepted_generation() {
7414+
use openshell_core::proto::{ConfigurationAdmissionState, SandboxConfigurationAdmission};
7415+
7416+
let mut sandbox = Sandbox::default();
7417+
sandbox.set_phase(SandboxPhase::Provisioning as i32);
7418+
let status = sandbox.status.as_mut().unwrap();
7419+
status.configuration_admission = Some(SandboxConfigurationAdmission {
7420+
state: ConfigurationAdmissionState::Rejected as i32,
7421+
error: "rule image_api requires L7 inspection".to_string(),
7422+
policy_hash: "candidate-hash".to_string(),
7423+
..Default::default()
7424+
});
7425+
status.conditions.push(SandboxCondition {
7426+
r#type: "ConfigurationReady".to_string(),
7427+
status: "False".to_string(),
7428+
reason: "ConfigurationInvalid".to_string(),
7429+
message: "rule image_api requires L7 inspection".to_string(),
7430+
..Default::default()
7431+
});
7432+
let json = super::sandbox_to_json(&sandbox);
7433+
assert_eq!(json["configuration_admission"]["state"], "rejected");
7434+
assert_eq!(json["conditions"][0]["reason"], "ConfigurationInvalid");
7435+
assert_eq!(
7436+
super::configuration_failure_message(&sandbox),
7437+
Some("rule image_api requires L7 inspection")
7438+
);
7439+
sandbox
7440+
.status
7441+
.as_mut()
7442+
.unwrap()
7443+
.configuration_admission
7444+
.as_mut()
7445+
.unwrap()
7446+
.error
7447+
.clear();
7448+
assert_eq!(super::configuration_failure_message(&sandbox), None);
7449+
}
7450+
73527451
#[test]
73537452
fn sandbox_detail_to_json_includes_policy_fields() {
73547453
let mut sandbox = Sandbox {

‎crates/openshell-cli/tests/ensure_providers_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -586,6 +586,13 @@ impl OpenShell for TestOpenShell {
586586
Err(Status::unimplemented("not implemented in test"))
587587
}
588588

589+
async fn report_sandbox_configuration(
590+
&self,
591+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
592+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
593+
Err(Status::unimplemented("not implemented in test"))
594+
}
595+
589596
async fn report_policy_status(
590597
&self,
591598
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/mtls_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -438,6 +438,13 @@ impl OpenShell for TestOpenShell {
438438
Err(Status::unimplemented("not implemented in test"))
439439
}
440440

441+
async fn report_sandbox_configuration(
442+
&self,
443+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
444+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
445+
Err(Status::unimplemented("not implemented in test"))
446+
}
447+
441448
async fn report_policy_status(
442449
&self,
443450
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/provider_commands_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1318,6 +1318,13 @@ impl OpenShell for TestOpenShell {
13181318
Err(Status::unimplemented("not implemented in test"))
13191319
}
13201320

1321+
async fn report_sandbox_configuration(
1322+
&self,
1323+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
1324+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
1325+
Err(Status::unimplemented("not implemented in test"))
1326+
}
1327+
13211328
async fn report_policy_status(
13221329
&self,
13231330
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -852,6 +852,13 @@ impl OpenShell for TestOpenShell {
852852
Err(Status::unimplemented("not implemented in test"))
853853
}
854854

855+
async fn report_sandbox_configuration(
856+
&self,
857+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
858+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
859+
Err(Status::unimplemented("not implemented in test"))
860+
}
861+
855862
async fn report_policy_status(
856863
&self,
857864
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_name_fallback_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -528,6 +528,13 @@ impl OpenShell for TestOpenShell {
528528
Err(Status::unimplemented("not implemented in test"))
529529
}
530530

531+
async fn report_sandbox_configuration(
532+
&self,
533+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
534+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
535+
Err(Status::unimplemented("not implemented in test"))
536+
}
537+
531538
async fn report_policy_status(
532539
&self,
533540
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-core/src/grpc_client.rs‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1022,6 +1022,40 @@ pub async fn sync_policy_and_fetch_snapshot(
10221022
fetch_settings_snapshot_with_client(&mut client, sandbox_id).await
10231023
}
10241024

1025+
/// Report an exact runtime configuration generation. Pending registration uses
1026+
/// the snapshot's instance fence; retain that snapshot across registration retries.
1027+
pub async fn report_sandbox_configuration(
1028+
endpoint: &str,
1029+
sandbox_id: &str,
1030+
instance_id: &str,
1031+
snapshot: Option<&SettingsPollResult>,
1032+
state: crate::proto::ConfigurationAdmissionState,
1033+
error: &str,
1034+
) -> Result<()> {
1035+
let mut client = connect(endpoint).await?;
1036+
client
1037+
.report_sandbox_configuration(crate::proto::ReportSandboxConfigurationRequest {
1038+
sandbox_id: sandbox_id.to_string(),
1039+
expected_instance_id: snapshot.map_or_else(String::new, |snapshot| {
1040+
snapshot.configuration_instance_id.clone()
1041+
}),
1042+
admission: Some(crate::proto::SandboxConfigurationAdmission {
1043+
instance_id: instance_id.to_string(),
1044+
state: state.into(),
1045+
policy_version: snapshot.map_or(0, |snapshot| snapshot.version),
1046+
policy_hash: snapshot
1047+
.map_or_else(String::new, |snapshot| snapshot.policy_hash.clone()),
1048+
config_revision: snapshot.map_or(0, |snapshot| snapshot.config_revision),
1049+
provider_env_revision: snapshot
1050+
.map_or(0, |snapshot| snapshot.provider_env_revision),
1051+
error: error.to_string(),
1052+
}),
1053+
})
1054+
.await
1055+
.into_diagnostic()?;
1056+
Ok(())
1057+
}
1058+
10251059
/// Fetch provider environment variables for a sandbox from `OpenShell` server via gRPC.
10261060
///
10271061
/// Returns the credential snapshot and its exact readiness identity. An empty
@@ -1218,6 +1252,9 @@ pub struct CachedOpenShellClient {
12181252
/// Settings poll result returned by [`CachedOpenShellClient::poll_settings`].
12191253
#[derive(Clone, Debug)]
12201254
pub struct SettingsPollResult {
1255+
pub configuration_instance_id: String,
1256+
pub configuration_admitted: bool,
1257+
pub configuration_error: String,
12211258
pub policy: Option<ProtoSandboxPolicy>,
12221259
pub version: u32,
12231260
pub policy_hash: String,
@@ -1241,6 +1278,9 @@ pub struct SettingsPollResult {
12411278

12421279
fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> SettingsPollResult {
12431280
SettingsPollResult {
1281+
configuration_instance_id: inner.configuration_instance_id,
1282+
configuration_admitted: inner.configuration_admitted,
1283+
configuration_error: inner.configuration_error,
12441284
policy: inner.policy,
12451285
version: inner.version,
12461286
policy_hash: inner.policy_hash,

0 commit comments

Comments
 (0)