Skip to content

Commit 198906a

Browse files
committed
fix(server): key inherited credential findings per port and revalidate on commit
Review of the differential credential gate found two reachable gaps. An uninspected credentialed endpoint's identity collapsed a multi-port endpoint onto its first port. Keeping an inherited port and adding a second one to the same endpoint therefore matched the baseline and was admitted, even though the added port had never been classified. Identity is now one key per effective port, so widening `[443]` to `[443, 8443]` introduces `8443`, while reordering or narrowing the port set carries the endpoint forward. Full policy replacement classified the edit before its persistence loop. Between that check and the write, another writer could remove the violation the edit was inheriting; the replacement then restored it without ever being classified. The gate now runs inside each persistence attempt, against the revision that attempt commits against. The write claims `latest.version + 1`, so any revision landing after that read loses the unique-version race and the retry reclassifies against the newer policy. Refs: #2998 Signed-off-by: Philippe Martin <[email protected]>
1 parent b4f81f7 commit 198906a

2 files changed

Lines changed: 245 additions & 31 deletions

File tree

‎architecture/security-policy.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -142,8 +142,13 @@ the uninspected credentialed endpoints it introduces, while endpoints the
142142
sandbox's current policy already carries are logged and admitted. An endpoint's
143143
identity for this comparison is its rule name, host, port, and uninspected mode,
144144
so moving an inherited endpoint from L4-only to `tls: skip` is a fresh authoring
145-
act rather than the same finding carried forward. Stamping is unaffected and
146-
stays a full recomputation.
145+
act rather than the same finding carried forward. A multi-port endpoint has one
146+
identity per port, so keeping an inherited port and adding another one is
147+
likewise newly authored. The baseline is the policy revision the edit is about
148+
to be committed against, evaluated inside the persistence attempt: a baseline
149+
read earlier could name a violation a concurrent writer has since removed, and
150+
the edit would restore it without ever being classified. Stamping is unaffected
151+
and stays a full recomputation.
147152

148153
The gate has to work this way because a supervisor syncing the policy baked into
149154
a sandbox image reaches the store without passing it — rejecting that delivery

‎crates/openshell-server/src/grpc/policy.rs‎

Lines changed: 238 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -2143,21 +2143,30 @@ pub(super) async fn validate_candidate_sandbox_credential_policy(
21432143
/// so an operator can round-trip `policy get --base` through `policy set` — and
21442144
/// can edit the offending endpoint — instead of being locked out by a rule the
21452145
/// sandbox image supplied.
2146+
///
2147+
/// `current_base` must be the policy revision the replacement is about to be
2148+
/// committed against, so callers evaluate it inside their persistence attempt:
2149+
/// a baseline read before the attempt could name a violation another writer has
2150+
/// since removed, and the replacement would restore it unclassified.
21462151
pub(super) async fn validate_updated_sandbox_credential_policy(
21472152
state: &ServerState,
21482153
workspace: &str,
2149-
sandbox: &Sandbox,
2154+
current_base: &ProtoSandboxPolicy,
21502155
provider_names: &[String],
21512156
policy: &ProtoSandboxPolicy,
21522157
) -> Result<(), Status> {
21532158
let catalog = state
21542159
.provider_profile_sources
21552160
.snapshot_catalog(state.store.as_ref(), workspace)
21562161
.await?;
2157-
let current_base = current_base_policy_for_sandbox(state.store.as_ref(), sandbox).await?;
2158-
let current_effective =
2159-
effective_policy_for_source(state, &catalog, workspace, provider_names, current_base)
2160-
.await?;
2162+
let current_effective = effective_policy_for_source(
2163+
state,
2164+
&catalog,
2165+
workspace,
2166+
provider_names,
2167+
current_base.clone(),
2168+
)
2169+
.await?;
21612170
let inherited = uninspected_credentialed_endpoint_keys(&current_effective);
21622171

21632172
let effective =
@@ -2967,7 +2976,9 @@ impl UninspectedCredentialedEndpoint {
29672976

29682977
/// Identity of an uninspected credentialed endpoint. `mode` participates: moving
29692978
/// an inherited L4-only endpoint to `tls: skip` is a fresh authoring act, not the
2970-
/// same finding carried forward.
2979+
/// same finding carried forward. A multi-port endpoint contributes one key per
2980+
/// port, so widening `[443]` to `[443, 8443]` introduces `8443` while reordering
2981+
/// the same ports carries the whole endpoint forward.
29712982
#[derive(Clone, PartialEq, Eq, Hash)]
29722983
struct UninspectedCredentialedEndpointKey {
29732984
rule_name: String,
@@ -3009,15 +3020,25 @@ fn collect_uninspected_credentialed_endpoints(
30093020
continue;
30103021
}
30113022

3012-
violations.push(UninspectedCredentialedEndpoint {
3013-
rule_name: rule_name.clone(),
3014-
host: endpoint.host.clone(),
3015-
port: endpoint_ports(endpoint)
3016-
.first()
3017-
.copied()
3018-
.unwrap_or(endpoint.port),
3019-
mode,
3020-
});
3023+
// One violation per effective port. Collapsing a multi-port
3024+
// endpoint onto its first port would let an edit keep an inherited
3025+
// port and add another one under the same identity.
3026+
let mut ports = endpoint_ports(endpoint);
3027+
if ports.is_empty() {
3028+
// Unconstrained endpoint: it stands for every port, and `0`
3029+
// is the identity the message reports.
3030+
ports.push(endpoint.port);
3031+
}
3032+
ports.sort_unstable();
3033+
ports.dedup();
3034+
for port in ports {
3035+
violations.push(UninspectedCredentialedEndpoint {
3036+
rule_name: rule_name.clone(),
3037+
host: endpoint.host.clone(),
3038+
port,
3039+
mode,
3040+
});
3041+
}
30213042
}
30223043
}
30233044
violations
@@ -3778,20 +3799,6 @@ async fn handle_update_config_inner(
37783799
&effective_policy,
37793800
)
37803801
.await?;
3781-
// Sandbox-authored syncs replay a policy the supervisor already discovered
3782-
// on disk. Rejecting it here would crash-loop the sandbox instead of
3783-
// surfacing an operator decision, so only operator-authored updates gate.
3784-
if !sandbox_caller {
3785-
validate_updated_sandbox_credential_policy(
3786-
state,
3787-
&workspace,
3788-
&sandbox,
3789-
&spec.providers,
3790-
&new_policy,
3791-
)
3792-
.await?;
3793-
}
3794-
37953802
let _sandbox_sync_guard = if backfill_policy.is_some() {
37963803
Some(state.compute.sandbox_sync_guard().await)
37973804
} else {
@@ -3809,6 +3816,31 @@ async fn handle_update_config_inner(
38093816
.await
38103817
.map_err(|e| Status::internal(format!("fetch latest policy failed: {e}")))?;
38113818

3819+
// Sandbox-authored syncs replay a policy the supervisor already
3820+
// discovered on disk. Rejecting it here would crash-loop the sandbox
3821+
// instead of surfacing an operator decision, so only
3822+
// operator-authored updates gate. The baseline is this attempt's
3823+
// revision: the write below claims `latest.version + 1` and loses
3824+
// the unique-version race to any revision that landed after this
3825+
// read, so a retry reclassifies against the newer policy.
3826+
if !sandbox_caller {
3827+
let current_base = match latest {
3828+
Some(ref record) => ProtoSandboxPolicy::decode(
3829+
record.policy_payload.as_slice(),
3830+
)
3831+
.map_err(|e| Status::internal(format!("decode current policy failed: {e}")))?,
3832+
None => spec.policy.clone().unwrap_or_default(),
3833+
};
3834+
validate_updated_sandbox_credential_policy(
3835+
state,
3836+
&workspace,
3837+
&current_base,
3838+
&spec.providers,
3839+
&new_policy,
3840+
)
3841+
.await?;
3842+
}
3843+
38123844
if let Some(ref current) = latest
38133845
&& current.policy_hash == hash
38143846
&& current.provenance == req.annotations
@@ -7278,6 +7310,62 @@ mod tests {
72787310
);
72797311
}
72807312

7313+
#[test]
7314+
fn a_port_added_to_an_inherited_endpoint_is_newly_authored() {
7315+
let endpoint = |ports: Vec<u32>| NetworkEndpoint {
7316+
host: "github.com".to_string(),
7317+
ports,
7318+
provider_credentialed: true,
7319+
..Default::default()
7320+
};
7321+
let policy = |endpoint: NetworkEndpoint| ProtoSandboxPolicy {
7322+
network_policies: std::iter::once((
7323+
"pypi".to_string(),
7324+
NetworkPolicyRule {
7325+
endpoints: vec![endpoint],
7326+
..Default::default()
7327+
},
7328+
))
7329+
.collect(),
7330+
..Default::default()
7331+
};
7332+
7333+
// Keeping the inherited port and adding another one exposes the added
7334+
// port for the first time, so the endpoint cannot be carried forward
7335+
// wholesale on the strength of its first port.
7336+
let single_port = uninspected_credentialed_endpoint_keys(&policy(endpoint(vec![443])));
7337+
let error = validate_uninspected_credentialed_endpoints_excluding(
7338+
&policy(endpoint(vec![443, 8443])),
7339+
&single_port,
7340+
)
7341+
.expect_err("a port added to an inherited endpoint must be rejected");
7342+
assert!(
7343+
error.message().contains("github.com:8443"),
7344+
"{}",
7345+
error.message()
7346+
);
7347+
7348+
let both_ports = uninspected_credentialed_endpoint_keys(&policy(endpoint(vec![443, 8443])));
7349+
assert_eq!(both_ports.len(), 2);
7350+
// Reordering exposes nothing new.
7351+
assert!(
7352+
validate_uninspected_credentialed_endpoints_excluding(
7353+
&policy(endpoint(vec![8443, 443])),
7354+
&both_ports
7355+
)
7356+
.is_ok()
7357+
);
7358+
// Neither does dropping a port: narrowing an inherited endpoint is the
7359+
// repair the operator has to be able to make.
7360+
assert!(
7361+
validate_uninspected_credentialed_endpoints_excluding(
7362+
&policy(endpoint(vec![443])),
7363+
&both_ports
7364+
)
7365+
.is_ok()
7366+
);
7367+
}
7368+
72817369
#[test]
72827370
fn credentialed_l4_and_tls_skip_require_explicit_opt_in() {
72837371
let endpoint = |protocol: &str, tls: &str, allow: bool| NetworkEndpoint {
@@ -9432,6 +9520,127 @@ mod tests {
94329520
assert!(!stored.network_policies.contains_key("pypi"));
94339521
}
94349522

9523+
#[tokio::test]
9524+
async fn update_config_rejects_a_port_added_to_an_inherited_uninspected_endpoint() {
9525+
let (state, image_policy) = inherited_uninspected_gating_state().await;
9526+
9527+
// The same endpoint, restated through `ports`: nothing new is exposed.
9528+
let mut restated = image_policy.clone();
9529+
restated.network_policies.get_mut("pypi").unwrap().endpoints[0].ports = vec![443];
9530+
handle_update_config(
9531+
&state,
9532+
with_user(Request::new(UpdateConfigRequest {
9533+
name: "inherited-gating".to_string(),
9534+
workspace: "default".to_string(),
9535+
policy: Some(restated),
9536+
..Default::default()
9537+
})),
9538+
)
9539+
.await
9540+
.expect("restating the inherited port must be admitted");
9541+
9542+
// Keeping the inherited port and adding a second one to the same
9543+
// endpoint authors a new uninspected credentialed port.
9544+
let mut widened = image_policy;
9545+
widened.network_policies.get_mut("pypi").unwrap().endpoints[0].ports = vec![443, 8443];
9546+
let error = handle_update_config(
9547+
&state,
9548+
with_user(Request::new(UpdateConfigRequest {
9549+
name: "inherited-gating".to_string(),
9550+
workspace: "default".to_string(),
9551+
policy: Some(widened),
9552+
..Default::default()
9553+
})),
9554+
)
9555+
.await
9556+
.expect_err("a port added to an inherited endpoint must be rejected");
9557+
assert_eq!(error.code(), Code::FailedPrecondition);
9558+
assert!(
9559+
error.message().contains("github.com:8443"),
9560+
"{}",
9561+
error.message()
9562+
);
9563+
}
9564+
9565+
#[tokio::test]
9566+
async fn update_config_rejects_a_replacement_that_restores_a_removed_uninspected_endpoint() {
9567+
let (state, image_policy) = inherited_uninspected_gating_state().await;
9568+
9569+
let mut fixed = image_policy.clone();
9570+
fixed.network_policies.remove("pypi");
9571+
handle_update_config(
9572+
&state,
9573+
with_user(Request::new(UpdateConfigRequest {
9574+
name: "inherited-gating".to_string(),
9575+
workspace: "default".to_string(),
9576+
policy: Some(fixed),
9577+
..Default::default()
9578+
})),
9579+
)
9580+
.await
9581+
.expect("removing the offending rule must be possible");
9582+
9583+
// A replacement composed against the older revision. The endpoint is no
9584+
// longer inherited from the policy this write commits against, so it is
9585+
// newly authored and must not slip back in unclassified.
9586+
let error = handle_update_config(
9587+
&state,
9588+
with_user(Request::new(UpdateConfigRequest {
9589+
name: "inherited-gating".to_string(),
9590+
workspace: "default".to_string(),
9591+
policy: Some(image_policy),
9592+
..Default::default()
9593+
})),
9594+
)
9595+
.await
9596+
.expect_err("restoring a removed uninspected endpoint must be rejected");
9597+
assert_eq!(error.code(), Code::FailedPrecondition);
9598+
assert!(error.message().contains("pypi"), "{}", error.message());
9599+
}
9600+
9601+
#[tokio::test]
9602+
async fn concurrent_full_replacements_do_not_restore_a_removed_uninspected_endpoint() {
9603+
let (state, image_policy) = inherited_uninspected_gating_state().await;
9604+
9605+
let mut fixed = image_policy.clone();
9606+
fixed.network_policies.remove("pypi");
9607+
9608+
// Whichever order these interleave in, the inherited endpoint must not
9609+
// survive a replacement classified against a revision that no longer
9610+
// carries it.
9611+
let (restore, remove) = tokio::join!(
9612+
handle_update_config(
9613+
&state,
9614+
with_user(Request::new(UpdateConfigRequest {
9615+
name: "inherited-gating".to_string(),
9616+
workspace: "default".to_string(),
9617+
policy: Some(image_policy),
9618+
..Default::default()
9619+
})),
9620+
),
9621+
handle_update_config(
9622+
&state,
9623+
with_user(Request::new(UpdateConfigRequest {
9624+
name: "inherited-gating".to_string(),
9625+
workspace: "default".to_string(),
9626+
policy: Some(fixed),
9627+
..Default::default()
9628+
})),
9629+
)
9630+
);
9631+
remove.expect("removing the offending rule must be possible");
9632+
9633+
if restore.is_ok() {
9634+
let stored = get_sandbox_policy(&state, "sb-inherited-gating").await;
9635+
assert!(
9636+
!stored.network_policies.contains_key("pypi"),
9637+
"a replacement validated against a stale revision restored the endpoint"
9638+
);
9639+
} else {
9640+
assert_eq!(restore.unwrap_err().code(), Code::FailedPrecondition);
9641+
}
9642+
}
9643+
94359644
#[tokio::test]
94369645
async fn update_config_rejects_sigv4_without_credential_source_before_persisting_revision() {
94379646
let state = test_server_state().await;

0 commit comments

Comments
 (0)