Skip to content

Commit d7f1f80

Browse files
committed
feat(prover): add OpenShell Policy Prover (OPP) to Python SDK
Integrate Z3-based formal verification for sandbox policies as openshell.prover. Detects data exfiltration paths, write bypass attacks, L7 enforcement gaps, overpermissive methods, and privilege escalation via binary capability chains. Closes #699 Signed-off-by: Alexander Watson <[email protected]>
1 parent 0ac1fbd commit d7f1f80

39 files changed

Lines changed: 3772 additions & 3 deletions
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
name: harden-sandbox-policy
3+
description: Harden sandbox policies to minimum-access using the OpenShell Policy Prover (OPP). Fetches policies from local files or GitHub repos, runs formal verification to find gaps (L4-only endpoints, overpermissive methods, data exfiltration paths, write bypass attacks), and suggests concrete policy changes. Trigger keywords - harden policy, verify policy, prove policy, policy gaps, policy audit, opp, policy prover.
4+
---
5+
6+
# Harden Sandbox Policy
7+
8+
Run the OpenShell Policy Prover (OPP) against a sandbox policy to find security gaps and suggest hardening changes.
9+
10+
## Prerequisites
11+
12+
- Python 3.12+ with the prover dependencies installed: `uv pip install 'openshell[prover]'`
13+
- A sandbox policy YAML file
14+
- A credential descriptor YAML file
15+
16+
## Workflow
17+
18+
### Step 1: Locate inputs
19+
20+
Find or receive the following files:
21+
1. **Policy YAML** — the sandbox network/filesystem policy to verify
22+
2. **Credentials YAML** — describes what credentials the sandbox has and their scopes
23+
3. **Accepted risks YAML** (optional) — previously acknowledged risks to filter from results
24+
25+
If the user provides a GitHub URL or repo path, fetch the policy from there.
26+
If the user points to a running sandbox, use `openshell policy get <name> --full` to export the current policy.
27+
28+
### Step 2: Run the prover
29+
30+
```bash
31+
python3 -m openshell.prover.cli prove \
32+
--policy <policy.yaml> \
33+
--credentials <credentials.yaml> \
34+
[--accepted-risks <accepted.yaml>] \
35+
--compact
36+
```
37+
38+
Or via the Rust CLI:
39+
```bash
40+
openshell policy prove \
41+
--policy <policy.yaml> \
42+
--credentials <credentials.yaml> \
43+
[--accepted-risks <accepted.yaml>] \
44+
--compact
45+
```
46+
47+
### Step 3: Interpret findings
48+
49+
The prover returns findings at these risk levels:
50+
- **CRITICAL** — data exfiltration via L4-only or wire protocol bypass
51+
- **HIGH** — write bypass violating read-only intent, overpermissive HTTP methods, inconsistent L7 enforcement
52+
- **MEDIUM** — L4-only endpoints without sibling L7, binary inheritance chains
53+
- **ADVISORY** — inference.local side channel (architectural, always reported)
54+
55+
Exit code: 0 = PASS (advisories only), 1 = FAIL (critical/high gaps found), 2 = input error.
56+
57+
### Step 4: Suggest hardening changes
58+
59+
For each finding, suggest concrete policy YAML changes:
60+
61+
- **L4-only endpoints**: Add `protocol: rest` with `enforcement: enforce`, `tls: terminate`, and explicit `access: read-only` or method rules
62+
- **Overpermissive methods**: Replace `method: "*"` with specific methods (e.g., `POST` for inference, `GET` for package registries)
63+
- **Write bypass via wire protocol**: Remove binaries like `git`, `ssh`, `nc` from the policy if write access is not intended, or restrict credential scopes
64+
- **Binary inheritance**: Review spawn chains — if a parent binary (e.g., `python3`) spawns children with L7-bypass capabilities, consider using a more specific binary path
65+
66+
### Step 5: Generate report (optional)
67+
68+
For a shareable HTML report with interactive Mermaid diagrams:
69+
```bash
70+
python3 -m openshell.prover.cli prove \
71+
--policy <policy.yaml> \
72+
--credentials <credentials.yaml> \
73+
--html report.html
74+
```
75+
76+
## Workflow Chain
77+
78+
This skill complements `generate-sandbox-policy`:
79+
1. `generate-sandbox-policy` — author a policy from requirements
80+
2. `harden-sandbox-policy` — verify and tighten the authored policy
81+
3. Iterate until the prover returns PASS
82+
83+
## Output Format
84+
85+
Report findings as a structured summary:
86+
```
87+
## Policy Verification Results
88+
89+
**Verdict:** FAIL — 3 critical/high gaps
90+
91+
### Findings
92+
1. **CRITICAL** Data Exfiltration Paths Detected
93+
- ...
94+
2. **HIGH** Write Bypass — Read-Only Intent Violated
95+
- ...
96+
97+
### Suggested Changes
98+
- [ ] Add `protocol: rest` to `github.com:443` endpoint
99+
- [ ] Remove `/usr/bin/git` from `github_readonly` binaries
100+
- ...
101+
```

‎AGENTS.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
2222
- Spike explores feasibility, then build executes once `state:agent-ready` is applied by a human.
2323
- **Security:** `review-security-issue` → `fix-security-issue`
2424
- Review produces a severity assessment and remediation plan. Fix implements it. Both require the `topic:security` label; fix also requires `state:agent-ready`.
25-
- **Policy iteration:** `openshell-cli` → `generate-sandbox-policy`
26-
- CLI manages the sandbox lifecycle; policy generation authors the YAML constraints.
25+
- **Policy iteration:** `openshell-cli` → `generate-sandbox-policy` → `harden-sandbox-policy`
26+
- CLI manages the sandbox lifecycle; policy generation authors the YAML constraints; policy hardening formally verifies and tightens the policy using the OpenShell Policy Prover (OPP).
2727

2828
## Architecture Overview
2929

@@ -39,6 +39,7 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
3939
| `crates/openshell-providers/` | Provider management | Credential provider backends |
4040
| `crates/openshell-tui/` | Terminal UI | Ratatui-based dashboard for monitoring |
4141
| `python/openshell/` | Python SDK | Python bindings and CLI packaging |
42+
| `python/openshell/prover/` | Policy Prover (OPP) | Z3-based formal verification for sandbox policies |
4243
| `proto/` | Protobuf definitions | gRPC service contracts |
4344
| `deploy/` | Docker, Helm, K8s | Dockerfiles, Helm chart, manifests |
4445
| `.agents/skills/` | Agent skills | Workflow automation for development |

‎crates/openshell-cli/src/main.rs‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,8 @@ const POLICY_EXAMPLES: &str = "\x1b[1mALIAS\x1b[0m
253253
$ openshell policy set --global --policy policy.yaml
254254
$ openshell policy delete --global
255255
$ openshell policy list my-sandbox
256+
$ openshell policy prove --policy policy.yaml --credentials creds.yaml
257+
$ openshell policy prove --policy policy.yaml --credentials creds.yaml --compact
256258
";
257259

258260
const SETTINGS_EXAMPLES: &str = "\x1b[1mEXAMPLES\x1b[0m
@@ -1416,6 +1418,38 @@ enum PolicyCommands {
14161418
#[arg(long)]
14171419
yes: bool,
14181420
},
1421+
1422+
/// Formally verify a policy using the OpenShell Policy Prover (OPP).
1423+
///
1424+
/// Runs Z3-based verification queries against a sandbox policy to detect
1425+
/// data exfiltration paths, write bypass attacks, L7 enforcement gaps,
1426+
/// and privilege escalation via binary capabilities.
1427+
#[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")]
1428+
Prove {
1429+
/// Path to the sandbox policy YAML file.
1430+
#[arg(long, value_hint = ValueHint::FilePath)]
1431+
policy: String,
1432+
1433+
/// Path to the credential descriptor YAML file.
1434+
#[arg(long, value_hint = ValueHint::FilePath)]
1435+
credentials: String,
1436+
1437+
/// Path to the capability registry directory (default: bundled).
1438+
#[arg(long, value_hint = ValueHint::DirPath)]
1439+
registry: Option<String>,
1440+
1441+
/// Path to accepted risks YAML.
1442+
#[arg(long, value_hint = ValueHint::FilePath)]
1443+
accepted_risks: Option<String>,
1444+
1445+
/// One-line-per-finding output for CI.
1446+
#[arg(long)]
1447+
compact: bool,
1448+
1449+
/// Output self-contained HTML report to this path.
1450+
#[arg(long, value_hint = ValueHint::FilePath)]
1451+
html: Option<String>,
1452+
},
14191453
}
14201454

14211455
#[derive(Subcommand, Debug)]
@@ -1901,6 +1935,38 @@ async fn main() -> Result<()> {
19011935
}
19021936
run::gateway_setting_delete(&ctx.endpoint, "policy", yes, &tls).await?;
19031937
}
1938+
PolicyCommands::Prove {
1939+
policy,
1940+
credentials,
1941+
registry,
1942+
accepted_risks,
1943+
compact,
1944+
html,
1945+
} => {
1946+
let mut cmd = std::process::Command::new("python3");
1947+
cmd.args(["-m", "openshell.prover.cli", "prove"]);
1948+
cmd.args(["--policy", &policy]);
1949+
cmd.args(["--credentials", &credentials]);
1950+
if let Some(ref reg) = registry {
1951+
cmd.args(["--registry", reg]);
1952+
}
1953+
if let Some(ref risks) = accepted_risks {
1954+
cmd.args(["--accepted-risks", risks]);
1955+
}
1956+
if compact {
1957+
cmd.arg("--compact");
1958+
}
1959+
if let Some(ref html_path) = html {
1960+
cmd.args(["--html", html_path]);
1961+
}
1962+
let status = cmd
1963+
.status()
1964+
.into_diagnostic()
1965+
.wrap_err("failed to execute policy prover — is the 'prover' optional dependency installed? (uv pip install 'openshell[prover]')")?;
1966+
if !status.success() {
1967+
std::process::exit(status.code().unwrap_or(1));
1968+
}
1969+
}
19041970
}
19051971
}
19061972

‎pyproject.toml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,13 @@ classifiers = [
3636
]
3737
keywords = ["agent", "sandbox", "nvidia"]
3838

39+
[project.optional-dependencies]
40+
prover = [
41+
"z3-solver>=4.12",
42+
"pyyaml>=6.0",
43+
"rich>=13.0",
44+
]
45+
3946
[project.urls]
4047
source = "https://github.com/NVIDIA/OpenShell"
4148
documentation = "https://docs.nvidia.com/openshell/latest/"
@@ -51,6 +58,9 @@ dev = [
5158
"maturin>=1.5,<2.0",
5259
"setuptools-scm>=8",
5360
"grpcio-tools>=1.60",
61+
"z3-solver>=4.12",
62+
"pyyaml>=6.0",
63+
"rich>=13.0",
5464
]
5565
docs = [
5666
"sphinx<=7.5",

‎python/conftest.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""Root conftest for Python tests.
5+
6+
Pre-register openshell as a namespace package so subpackage tests (e.g.,
7+
the prover) can run without the full Rust build that generates proto stubs.
8+
"""
9+
10+
from __future__ import annotations
11+
12+
import sys
13+
import types
14+
from pathlib import Path
15+
16+
if "openshell" not in sys.modules:
17+
_pkg = types.ModuleType("openshell")
18+
_pkg.__path__ = [str(Path(__file__).resolve().parent / "openshell")]
19+
_pkg.__package__ = "openshell"
20+
sys.modules["openshell"] = _pkg
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""OpenShell Policy Prover (OPP) — formal verification for AI agent sandbox policies."""
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
from .cli import main
5+
6+
main()
Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""Load and match accepted risk annotations against findings."""
5+
6+
from __future__ import annotations
7+
8+
from dataclasses import dataclass
9+
from typing import TYPE_CHECKING
10+
11+
import yaml
12+
13+
from .queries import Finding
14+
15+
if TYPE_CHECKING:
16+
from pathlib import Path
17+
18+
19+
@dataclass
20+
class AcceptedRisk:
21+
query: str
22+
reason: str
23+
accepted_by: str = ""
24+
binary: str = ""
25+
endpoint: str = ""
26+
27+
28+
def load_accepted_risks(path: Path) -> list[AcceptedRisk]:
29+
"""Load accepted risks from YAML."""
30+
with open(path) as f: # noqa: PTH123
31+
raw = yaml.safe_load(f)
32+
33+
return [
34+
AcceptedRisk(
35+
query=r.get("query", ""),
36+
reason=r.get("reason", ""),
37+
accepted_by=r.get("accepted_by", ""),
38+
binary=r.get("binary", ""),
39+
endpoint=r.get("endpoint", ""),
40+
)
41+
for r in (raw.get("accepted_risks") or [])
42+
]
43+
44+
45+
def _path_matches_risk(path, risk: AcceptedRisk) -> bool:
46+
"""Check if a single finding path matches an accepted risk."""
47+
if risk.binary:
48+
path_binary = getattr(path, "binary", "") or getattr(path, "parent", "")
49+
if path_binary != risk.binary:
50+
return False
51+
52+
if risk.endpoint:
53+
endpoint_host = getattr(path, "endpoint_host", "")
54+
if endpoint_host != risk.endpoint:
55+
return False
56+
57+
return True
58+
59+
60+
def apply_accepted_risks(
61+
findings: list[Finding],
62+
accepted: list[AcceptedRisk],
63+
) -> list[Finding]:
64+
"""Mark findings as accepted where they match accepted risk annotations.
65+
66+
A finding is accepted if ALL of its paths match at least one accepted risk
67+
entry for that query. If only some paths match, the finding stays active
68+
with the unmatched paths.
69+
"""
70+
if not accepted:
71+
return findings
72+
73+
result = []
74+
for finding in findings:
75+
matching_risks = [r for r in accepted if r.query == finding.query]
76+
if not matching_risks:
77+
result.append(finding)
78+
continue
79+
80+
if not finding.paths:
81+
# Pathless finding (e.g., inference relay) — accept if query matches
82+
accepted_finding = Finding(
83+
query=finding.query,
84+
title=finding.title,
85+
description=finding.description,
86+
risk=finding.risk,
87+
paths=finding.paths,
88+
remediation=finding.remediation,
89+
accepted=True,
90+
accepted_reason=matching_risks[0].reason,
91+
)
92+
result.append(accepted_finding)
93+
continue
94+
95+
unmatched_paths = []
96+
matched_reason = ""
97+
for path in finding.paths:
98+
path_accepted = False
99+
for risk in matching_risks:
100+
if _path_matches_risk(path, risk):
101+
path_accepted = True
102+
matched_reason = risk.reason
103+
break
104+
if not path_accepted:
105+
unmatched_paths.append(path)
106+
107+
if not unmatched_paths:
108+
accepted_finding = Finding(
109+
query=finding.query,
110+
title=finding.title,
111+
description=finding.description,
112+
risk=finding.risk,
113+
paths=finding.paths,
114+
remediation=finding.remediation,
115+
accepted=True,
116+
accepted_reason=matched_reason,
117+
)
118+
result.append(accepted_finding)
119+
elif len(unmatched_paths) < len(finding.paths):
120+
result.append(
121+
Finding(
122+
query=finding.query,
123+
title=finding.title,
124+
description=finding.description,
125+
risk=finding.risk,
126+
paths=unmatched_paths,
127+
remediation=finding.remediation,
128+
)
129+
)
130+
else:
131+
result.append(finding)
132+
133+
return result

0 commit comments

Comments
 (0)