Repository navigation
Conversation
Validate authored files before gateway selection or authentication. Share creation safety checks and report file-only violations together while deferring checks that require live gateway state. Closes NVIDIA#4035 Signed-off-by: Shiju <[email protected]>
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. api-behavior — No explicit risk acceptance foundCreateSandbox size rejections after MCP defaulting previously included BadRequest(spec.policy) and ErrorInfo(INVALID_ARGUMENT), but now return bare INVALID_ARGUMENT.
Review details
Advisory review; publication gates are unchanged. |
74dcaf5 to
b49c040
Compare
Signed-off-by: Shiju <[email protected]>
shvma
left a comment
There was a problem hiding this comment.
I ran 11 authored policies through openshell policy lint and through CreateSandbox on the in-process gateway from openshell_server::test_support. For the four built-in middleware cases I compared against MiddlewareRegistry::validate_policy_configs with the built-in services instead (see the note at the end). Lint agreed everywhere except the external middleware case, which lint skips by design, as the docs say. Two small things, both with patches below.
-
The pair order in endpoint ambiguity messages isn't stable. With two conflicting rules
aandb,openshell policy lintprinted'a' … and 'b'in 10 of 20 runs and'b' … and 'a'in the other 10.find_endpoint_ambiguitieswalks thenetwork_policiesmap without sorting. The code predates this PR, and the gateway message has the same problem. Lint makes it more visible because its output goes to CI logs and scripts. The crate already states the rule in lib.rs: "Sort rule keys so callers receive stable validation errors for equivalent policy inputs." Patch 1 sorts the keys the same way and adds a unit test that fails without the change (one run got[("mid", "alpha"), ("alpha", "zeta")]) and passes with it.cargo test -p openshell-policy --libis 300/300 and clippy with-D warningsis clean. Fine as a follow-up if you'd rather keep this PR to lint. -
authored_policy_lint_matches_gateway_file_validationcovers identity, reserved keys and credential bindings.policy_lint_integration.rscovers ambiguity on the lint side only, and I didn't find a test thatCreateSandboxrejects the same authored file. The gateway rejects an ambiguous file onCreateSandboxeven without providers, withFailedPrecondition(policy.rs), so it doesn't fit the existing loop that expectsInvalidArgument. Patch 2 adds a separate test on the same harness. It uses one rule with two endpoints, so it doesn't depend on the ordering in point 1 and works with or without Patch 1. It passes, and clippy is clean.
Not for this PR, just context for why the parity test can't cover built-in middleware: gateway_service_with_driver doesn't register the built-ins that the real gateway does (openshell-server lib.rs), so openshell/regex is always "not registered" there.
Patch 1: sort ambiguity pairs by rule key
diff --git a/crates/openshell-policy/src/ambiguity.rs b/crates/openshell-policy/src/ambiguity.rs
index efb846a..0eb9739 100644
--- a/crates/openshell-policy/src/ambiguity.rs
+++ b/crates/openshell-policy/src/ambiguity.rs
@@ -58,9 +58,12 @@ struct EndpointRef<'a> {
/// request-specific metadata) path selectors can match the same request.
#[must_use]
pub fn find_endpoint_ambiguities(policy: &SandboxPolicy) -> Vec<EndpointAmbiguity> {
- let endpoints = policy
- .network_policies
- .iter()
+ // Protobuf maps do not preserve iteration order. Sort rule keys so callers
+ // receive the same ambiguity pairs, in the same order, for equivalent input.
+ let mut network_policies: Vec<_> = policy.network_policies.iter().collect();
+ network_policies.sort_by_key(|(key, _)| *key);
+ let endpoints = network_policies
+ .into_iter()
.flat_map(|(key, rule)| {
let policy_name = if rule.name.is_empty() {
key.as_str()
@@ -775,6 +778,43 @@ mod tests {
assert!(find_endpoint_ambiguities(&policy_with(left, right)).is_empty());
}
+ #[test]
+ fn ambiguities_are_reported_in_rule_key_order() {
+ let pairs = |policy: &SandboxPolicy| {
+ find_endpoint_ambiguities(policy)
+ .into_iter()
+ .map(|ambiguity| (ambiguity.left_policy, ambiguity.right_policy))
+ .collect::<Vec<_>>()
+ };
+ let build = || {
+ let mut policy = SandboxPolicy::default();
+ for (key, tls) in [
+ ("zeta", NetworkTlsMode::Skip),
+ ("alpha", NetworkTlsMode::Unspecified),
+ ("mid", NetworkTlsMode::Skip),
+ ] {
+ let mut endpoint = endpoint("api.example.com", 443);
+ endpoint.tls = tls as i32;
+ policy.network_policies.insert(
+ key.to_string(),
+ NetworkPolicyRule {
+ endpoints: vec![endpoint],
+ ..Default::default()
+ },
+ );
+ }
+ policy
+ };
+ let expected = vec![
+ ("alpha".to_string(), "mid".to_string()),
+ ("alpha".to_string(), "zeta".to_string()),
+ ];
+ // Each map gets its own hash seed, so repeat to catch unordered iteration.
+ for _ in 0..32 {
+ assert_eq!(pairs(&build()), expected);
+ }
+ }
+
#[test]
fn compatible_request_rules_may_overlap() {
let mut left = endpoint("api.example.com", 443);Patch 2: parity test for endpoint ambiguity
diff --git a/crates/openshell-cli/tests/policy_only_process_identity_integration.rs b/crates/openshell-cli/tests/policy_only_process_identity_integration.rs
index 559534e..90b77ca 100644
--- a/crates/openshell-cli/tests/policy_only_process_identity_integration.rs
+++ b/crates/openshell-cli/tests/policy_only_process_identity_integration.rs
@@ -183,3 +183,35 @@ async fn authored_policy_lint_matches_gateway_file_validation() {
}
}
}
+
+#[tokio::test]
+async fn authored_policy_lint_matches_gateway_endpoint_ambiguity() {
+ let server = run_server().await;
+ let policy = openshell_policy::parse_sandbox_policy(
+ "version: 1\nnetwork_policies:\n conflict:\n endpoints:\n - {host: example.com, port: 443}\n - {host: example.com, port: 443, tls: skip}\n",
+ )
+ .unwrap();
+ let ambiguities = openshell_policy::find_endpoint_ambiguities(&policy);
+ assert_eq!(ambiguities.len(), 1, "{ambiguities:?}");
+ let mut client = grpc_client(&server.endpoint, &server.tls).await.unwrap();
+ let error = client
+ .create_sandbox(CreateSandboxRequest {
+ name: "ambiguity".to_string(),
+ spec: Some(SandboxSpec {
+ policy: Some(policy),
+ ..Default::default()
+ }),
+ workspace_scope: Some(openshell_core::proto::workspace_selector(
+ "default".to_string(),
+ )),
+ ..Default::default()
+ })
+ .await
+ .expect_err("an ambiguous authored file must be rejected by the gateway too");
+ assert_eq!(error.code(), tonic::Code::FailedPrecondition);
+ assert!(
+ error.message().contains(&ambiguities[0].to_string()),
+ "{error} did not report {}",
+ ambiguities[0]
+ );
+}
Summary
Add
openshell policy lint <file>to validate authored policies without a gateway. After parsing, it reports file-only semantic violations together and identifies deferred live checks.Related Issue
Closes #4035
Changes
Release Notes
openshell policy lint policy.yamlvalidates authored files offline. Success does not establish that a sandbox can start.Testing
-D warnings.Checklist