Skip to content

Commit e78f779

Browse files
committed
feat(sandbox): support live policy updates, history, and policy-aware logs (!55)
Closes #78 ## What You Can Do ### Live policy updates Update a running sandbox's network policy without recreating it: ```bash nav sandbox policy set <sandbox> --policy new-policy.yaml --wait --timeout 60 ``` The sandbox hot-reloads the policy within 30s (configurable). On failure, the previous policy stays active (last-known-good). Idempotent — submitting the same policy twice returns the existing version: ``` ✓ Policy version 3 submitted (hash: a1b2c3d4e5f6) $ nav sandbox policy set test --policy same.yaml · Policy unchanged (version 3, hash: a1b2c3d4e5f6) ``` ### Policy history & inspection ```bash nav sandbox policy list <sandbox> # version history with status nav sandbox policy get <sandbox> # current policy metadata nav sandbox policy get <sandbox> --full # print full policy as YAML nav sandbox policy get <sandbox> --rev 2 --full # specific revision as YAML ``` ### Sandbox logs Stream logs from both the gateway and sandbox supervisor in one view: ```bash nav sandbox logs <sandbox> # one-shot, last 2000 lines nav sandbox logs <sandbox> --tail # live streaming nav sandbox logs <sandbox> --source sandbox # supervisor logs only nav sandbox logs <sandbox> --source gateway # gateway logs only nav sandbox logs <sandbox> --level warn # warnings and errors only nav sandbox logs <sandbox> --since 5m # last 5 minutes nav sandbox logs <sandbox> --tail --source sandbox --level info ``` Each log line is tagged with its source and includes structured fields: ``` [1772055394.673] [sandbox] [INFO ] [navigator_sandbox::proxy] CONNECT action=allow dst_host=api.anthropic.com dst_port=443 policy=claude_code [1772055061.005] [gateway] [INFO ] [navigator_server::grpc] GetSandboxPolicy served from policy history ``` --- ## Implementation ### Proto changes - 7 new RPCs: `UpdateSandboxPolicy`, `GetSandboxPolicyStatus`, `ListSandboxPolicies`, `ReportPolicyStatus`, `GetSandboxLogs`, `PushSandboxLogs` (client-streaming) - `SandboxLogLine`: added `source` (gateway/sandbox), `fields` (structured key-value map) - `WatchSandboxRequest`: added `log_since_ms`, `log_sources`, `log_min_level` - `GetSandboxLogsRequest`: added `sources`, `min_level` - `PolicyStatus` enum, `SandboxPolicyRevision` message - `Sandbox.current_policy_version` field ### Server - **Policy persistence**: New `sandbox_policies` table (SQLite + Postgres) with per-sandbox monotonic versions, status tracking, and policy hash - **UpdateSandboxPolicy**: Validates static field immutability (filesystem/landlock/process), network mode consistency (Block↔Proxy), deterministic hash comparison for idempotent updates - **Lazy backfill**: First `GetSandboxPolicy` call creates version 1 from `spec.policy` for existing sandboxes - **Log broker**: `TracingLogBus::publish_external()` injects sandbox-pushed logs into the same broadcast channel + tail buffer (2000 lines). Server forces `source="sandbox"` and `sandbox_id` on all pushed logs - **Source/level filtering**: Applied server-side in both `GetSandboxLogs` and `WatchSandbox` streams - **Version supersession**: When a new version is loaded, all older pending+loaded versions are marked superseded ### Sandbox - **`OpaEngine::reload_from_proto()`**: Full `from_proto()` pipeline (L7 validation, access preset expansion) with atomic engine swap. On failure, previous engine untouched (LKG) - **Policy poll loop**: Background task polls every 30s (configurable via `NAVIGATOR_POLICY_POLL_INTERVAL_SECS`), reports status via `ReportPolicyStatus` RPC - **`LogPushLayer`**: Tracing layer captures events at INFO+ (configurable via `NAVIGATOR_LOG_PUSH_LEVEL`), sends structured fields via `PushSandboxLogs` client-streaming RPC. Background task batches 50 lines / flushes every 500ms. Best-effort (drops on full channel, never blocks) - **`CachedNavigatorClient`**: Persistent mTLS channel for both policy polling and log push ### Database migration - `002_create_sandbox_policies.sql` (SQLite + Postgres) ## Tests - **Unit**: 8 policy persistence tests (put/get/list/status/supersede/isolation) - **Integration**: 4 test files updated with new RPC stubs - **E2E**: `test_live_policy_update_and_logs` — full lifecycle: create → set same (unchanged) → push new → wait for load → verify connectivity → push same (unchanged) → verify history → fetch logs ## Documentation - `architecture/sandbox.md`: Log streaming architecture, LogPushLayer, push task, server broker, source tagging, structured fields, CLI filtering, failure modes - `architecture/security-policy.md`: Live update semantics, deterministic hashing, CLI filter flags, policy inspection - `architecture/plans/issue-78-sandbox-log-streaming.md`: Design plan for log streaming ## Security - Trust boundary documented: shared mTLS cert model (per-sandbox auth tracked in #80) - Server forces `source="sandbox"` and `sandbox_id` on pushed logs (can't impersonate gateway or other sandboxes) - Per-batch line cap (100) prevents flooding
1 parent efa3d5b commit e78f779

33 files changed

Lines changed: 3497 additions & 39 deletions

‎.claude/agent-memory/arch-doc-writer/MEMORY.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,33 @@
7979
- PTY I/O: 3 std::threads (writer, reader, exit) with reader_done sync for SSH protocol ordering
8080
- SSH daemon: russh server, ephemeral Ed25519 key, pre_exec: setsid -> TIOCSCTTY -> setns -> drop_privileges -> sandbox::apply
8181

82+
## Policy Reload Details
83+
- Poll loop: `run_policy_poll_loop()` in lib.rs, spawned after child process, gRPC mode only
84+
- `OpaEngine::reload_from_proto()`: reuses `from_proto()` pipeline, atomically swaps inner engine, LKG on failure
85+
- `CachedNavigatorClient` in grpc_client.rs: persistent mTLS channel for poll + status report (mirrors CachedInferenceClient)
86+
- Dynamic domains: network_policies, inference (OPA engine swap). Static domains: filesystem, landlock, process (pre_exec, immutable)
87+
- Server-side: `UpdateSandboxPolicy` RPC rejects changes to static fields or network mode changes
88+
- Server-side validation: `validate_static_fields_unchanged()` + `validate_network_mode_unchanged()` in grpc.rs
89+
- Poll interval: `NAVIGATOR_POLICY_POLL_INTERVAL_SECS` env var (default 30), no CLI flag
90+
- Version tracking: monotonic i64 per sandbox, `GetSandboxPolicyResponse` has version + policy_hash
91+
- Version 1 backfill: lazy on first `GetSandboxPolicy` from spec.policy if no policy_revisions row exists
92+
- `supersede_pending_policies()`: marks older pending revisions as superseded when new version persisted
93+
- Status reporting: `ReportPolicyStatus` RPC with `PolicyStatus` enum (PENDING, LOADED, FAILED, SUPERSEDED)
94+
- `report_policy_status()` updates `sandbox.current_policy_version` on LOADED, notifies watch bus
95+
- Proto files: `ReportPolicyStatusRequest`/`Response` in navigator.proto, `GetSandboxPolicyResponse` in sandbox.proto
96+
- `Sandbox.current_policy_version` (uint32) in datamodel.proto -- tracks active loaded version
97+
- Persistence: `PolicyRecord` in persistence/mod.rs (id, sandbox_id, version, policy_payload, policy_hash, status, load_error, timestamps)
98+
- CLI: `PolicyCommands` enum in main.rs (~line 516): Set, Get, List subcommands
99+
- CLI: `sandbox_policy_set()` in run.rs (~line 2901): loads YAML, calls UpdateSandboxPolicy, optionally polls for status
100+
- CLI: `sandbox_policy_get()` in run.rs (~line 3015): supports --rev N (version=0 means latest) and --full (YAML output via policy_to_yaml)
101+
- CLI: `sandbox_logs()` in run.rs (~line 3124): --source (all/gateway/sandbox) and --level (error/warn/info/debug/trace) filters
102+
- Deterministic hashing: `deterministic_policy_hash()` in grpc.rs (~line 1133): sorts network_policies by key, hashes fields individually
103+
- Idempotent UpdateSandboxPolicy: compares hash of new policy to latest stored hash, returns existing version if match
104+
- `policy_to_yaml()` in run.rs (~line 1623): converts proto to YAML via `PolicyYaml` struct (uses BTreeMap for ordered keys)
105+
- `policy_record_to_revision()` in grpc.rs (~line 1234): `include_policy` param controls whether full proto is included
106+
- Server-side log filtering: `source_matches()` + `level_matches()` in grpc.rs, applied in both get_sandbox_logs and watch_sandbox
107+
- Standalone `proxy_inference()` was removed; proxy uses `CachedInferenceClient` via `InferenceContext.grpc_client` (OnceCell)
108+
82109
## Policy System Details
83110
- YAML data file top-level keys: filesystem_policy, landlock, process, network_policies, inference
84111
- Proto message field `filesystem` maps to YAML key `filesystem_policy` (different names!)
@@ -112,6 +139,26 @@
112139
- Patterns: POST /v1/chat/completions, /v1/completions, /v1/responses, /v1/messages
113140
- Dev sandbox: `mise run sandbox -e VAR_NAME` forwards host env vars; NVIDIA_API_KEY always passed
114141

142+
## Log Streaming Details
143+
- LogPushLayer: `crates/navigator-sandbox/src/log_push.rs` -- tracing layer + spawn_log_push_task()
144+
- Initialized in main.rs before run_sandbox(), gRPC mode only
145+
- mpsc channel: 1024 lines (bounded), try_send (best-effort, never blocks)
146+
- Background task: batches up to 50 lines, flushes every 500ms via PushSandboxLogs client-streaming RPC
147+
- Secondary channel to gRPC call: mpsc::channel::<PushSandboxLogsRequest>(32) wrapped in ReceiverStream
148+
- CachedNavigatorClient.raw_client() returns clone of inner NavigatorClient for direct RPC calls
149+
- NAVIGATOR_LOG_PUSH_LEVEL env var (default INFO), parsed in LogPushLayer::new()
150+
- Server handler: push_sandbox_logs in grpc.rs, caps 100 lines/batch, forces source="sandbox" + sandbox_id
151+
- TracingLogBus.publish_external(): injects into same broadcast + tail buffer as SandboxLogLayer
152+
- Tail buffer: DEFAULT_TAIL = 2000 lines per sandbox (was 200, increased with log push)
153+
- SandboxLogLayer (server tracing layer): sets source="gateway", only publishes events with sandbox_id field
154+
- CLI: --source (gateway/sandbox/all), --level (error/warn/info/debug/trace) on `sandbox logs`
155+
- Source filter: "all" normalized to empty list (no filter) in run.rs
156+
- level_matches(): numeric ranking ERROR=0..TRACE=4, unknown levels always pass
157+
- Create-watch filter: log_sources: ["gateway"] to prevent sandbox logs from blocking stop_on_terminal
158+
- Proto: SandboxLogLine.source (string), SandboxLogLine.fields (map<string,string>)
159+
- Proto: PushSandboxLogsRequest/Response, GetSandboxLogsRequest (sources, min_level fields)
160+
- Proto: WatchSandboxRequest (log_sources, log_min_level fields)
161+
115162
## Naming Conventions
116163
- The project name "Navigator" appears in code but docs should use generic terms per user preference
117164
- CLI binary: `navigator` (aliased as `nav` in dev via mise)
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Principal Engineer Reviewer Memory
2+
3+
## Project Structure
4+
- Proto definitions: `proto/navigator.proto`, `proto/sandbox.proto`, `proto/sandbox_policy.proto`
5+
- Server gRPC handlers: `crates/navigator-server/src/grpc.rs`
6+
- TracingLogBus (log broadcast): `crates/navigator-server/src/tracing_bus.rs`
7+
- Sandbox watch bus: `crates/navigator-server/src/sandbox_watch.rs`
8+
- Server state: `crates/navigator-server/src/lib.rs` (ServerState struct)
9+
- Sandbox main: `crates/navigator-sandbox/src/main.rs`
10+
- Sandbox library: `crates/navigator-sandbox/src/lib.rs`
11+
- Sandbox gRPC client: `crates/navigator-sandbox/src/grpc_client.rs`
12+
- CLI commands: `crates/navigator-cli/src/main.rs` (clap defs), `crates/navigator-cli/src/run.rs` (impl)
13+
- Python SDK: `python/navigator/`
14+
- Plans go in: `architecture/plans/`
15+
16+
## Key Patterns
17+
- TracingLogBus: per-sandbox broadcast::channel(1024) + VecDeque tail buffer (200 lines)
18+
- CachedNavigatorClient: reusable mTLS gRPC channel for sandbox->server calls
19+
- SandboxLogLayer: tracing Layer that captures events with sandbox_id field
20+
- Sandbox logging: stdout (ANSI, configurable level) + /var/log/navigator.log (info, no ANSI, non-blocking)
21+
- WatchSandbox: server-streaming with select! loop over status_rx, log_rx, platform_rx
22+
- Proto codegen: `mise run proto`
23+
- Build: `mise run sandbox` for sandbox infra
24+
25+
## Review Preferences (observed)
26+
- Plans stored as markdown in architecture/plans/
27+
- Conventional commits required
28+
- No AI attribution in commits

‎.claude/settings.local.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"sandbox": {
3+
"enabled": true,
4+
"autoAllowBashIfSandboxed": true
5+
},
6+
"permissions": {
7+
"allow": [
8+
"WebFetch(domain:code.claude.com)"
9+
]
10+
}
11+
}

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -187,4 +187,5 @@ mise.local.toml
187187
architecture/plans
188188

189189
# Claude
190-
.claude/settings.local.json
190+
.claude/settings.local.json.claude/worktrees/
191+
.claude/worktrees/

‎.gitlab-ci.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,8 @@ python_e2e_sandbox_test:
228228
rules:
229229
- if: $CI_COMMIT_TAG
230230
when: never
231-
- when: on_success
231+
- if: $CI_COMMIT_BRANCH == "main"
232+
- when: never
232233
services:
233234
- docker:24-dind
234235
tags:

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)