Repository navigation
feat(telemetry): add anonymous opt-out OpenShell usage telemetry - #1433
Conversation
4aa8e3c to
fab5164
Compare
|
Looks like the Rust telemetry helper shells out to Suggestion: we might want to implement the telemetry publisher fully in Rust for better portability. That would keep the runtime behavior inside the shipped binaries, avoid depending on Python/script packaging, and make the opt-out/publish path easier to test across gateway, supervisor, and package installs. |
|
@kirit93 Can you add a short telemetry section to the primary README as part of this PR? |
20df3af to
96c8730
Compare
dc0aa8a to
dbab5e8
Compare
russellb
left a comment
There was a problem hiding this comment.
Something seems to be wrong with your branch. It appears to be reverting a lot of changes from main.
|
@russellb yeah i'm looking at this as well |
dbab5e8 to
5b2e26f
Compare
5b2e26f to
bdc660f
Compare
Signed-off-by: Kirit93 <[email protected]>
Signed-off-by: Kirit Thadaka <[email protected]>
Signed-off-by: Kirit93 <[email protected]>
43047b3 to
dd42c38
Compare
|
Label |
* fix(ci): eliminate image-tag race between concurrent workflows (#1413) - Add publish-manifest input to docker-build.yml (default true); single-arch branch callers set it false so the merge job is skipped and the shared bare :SHA tag in GHCR is never written by branch workflows - branch-kubernetes-e2e: retag :SHA-amd64 to :SHA before kind load so Helm's image.tag matches what is loaded in kind containerd - branch-e2e: pass image-tag as :SHA-arm64 to e2e-test so the arch-specific GHCR tag is used directly without depending on the bare tag - bare :SHA in GHCR is now written only by test-gpu.yml (multi-arch build), eliminating the last-writer-wins race across concurrent workflows * test(server): cover service endpoint plaintext security (#1352) * test(server): cover service endpoint plaintext security * test(server): align tls test with from_files Option<&Path> signature TlsAcceptor::from_files now accepts the client CA path as Option<&Path> (per the require_client_auth refactor on main). Wrap the helper's CA path in Some(...) so the new plaintext-service-http tests compile after rebasing onto current main. --------- Co-authored-by: Taylor Mutch <[email protected]> * fix(cli): add auth and TLS support to completion client (#1489) * fix(scripts): use portable lowercase in normalize_bool for Bash 3.2 (#1493) * refactor(server): extract shared relay-await and sandbox-scan helpers (#1495) * fix(sandbox): skip fork-exec socket ambiguity test on SELinux-enforcing hosts (#1449) Exec'ing /bin/sleep (SELinux label bin_t) from a user_home_t test binary causes /proc/<pid>/exe readlink to return ENOENT on SELinux-enforcing hosts due to the cross-domain boundary. Skip the test at runtime when getenforce reports Enforcing. Also adds a ChildGuard drop guard for safe child cleanup on panic and increases the exec-detection deadline from 2s to 5s. Signed-off-by: Derek Carr <[email protected]> * fix(sandbox): allow first-label L7 host wildcards (#1304) * fix(sandbox): allow first-label L7 host wildcards * docs(sandbox): document L7 host wildcard contract + add OPA runtime tests - Add Host Wildcards section to architecture/security-policy.md describing accepted (first-label *, **, intra-label *-X) and rejected (bare, TLD, non-first-label, recursive-in-label) forms, and noting that wildcards never cross '.' boundaries. - Expand the policy-schema.mdx 'host' field description to reflect the same contract instead of only mentioning '*.example.com'. - Add OPA runtime tests asserting '*-aiplatform.googleapis.com' matches 'us-central1-aiplatform.googleapis.com' and does not match 'us-central1.aiplatform.googleapis.com' (cross-dot boundary). Locks validator/runtime alignment for intra-label wildcards. * chore: update mise lockfile * test(server): tolerate serialized inference upserts --------- Co-authored-by: John Myers <[email protected]> * feat(cli): add JSON/YAML output format to gateway list (#1500) Add -o/--output flag to `openshell gateway list` matching the existing sandbox list pattern, enabling machine-readable output for scripting. Signed-off-by: Florent Benoit <[email protected]> * refactor: deduplicate repeated patterns across crates (#1499) Remove ~280 lines of duplicated code across 30 files in 5 areas: - centered_rect: consolidate 5 identical TUI layout helpers into a single pub fn in openshell-tui/src/ui/mod.rs - server test helpers: replace ~100 inline Store::connect() calls with local test_store() helpers; deduplicate test_server_state() in grpc/service.rs to use the shared test_support version - rogue PKI: extract 20-line rogue CA+client cert generation block (duplicated in two integration tests) into generate_rogue_pki() in tests/common/mod.rs - provider tests: replace 8 identical 28-line test modules with a single macro_rules! test_discovers_env_credential! invocation - label constants: centralize openshell.ai/ container label keys in openshell-core::driver_utils; update Docker and Kubernetes drivers to import from there instead of redefining them locally * fix(ci): resolve mirror gate statuses for fork PRs (#1504) Signed-off-by: Piotr Mlocek <[email protected]> * fix(server): respect OPENSHELL_PODMAN_SOCKET env var in embedded driver (#1483) The env var was only wired up via clap in the standalone openshell-driver-podman binary. When the Podman driver runs embedded in the gateway, config came exclusively from TOML deserialization and the env var was never consulted. Apply it as a post-deserialization override, matching the existing OPENSHELL_K8S_WORKSPACE_DEFAULT_STORAGE_SIZE pattern. Closes #1446 * refactor(sandbox,driver-vm): Start moving to rustix (esp over libc unsafe) (#1505) In the Rust ecosystem there's largely three ways to do system calls: - raw libc - nix - rustix Of the three, libc is almost all `unsafe` and really 95% of use cases should be either nix or rustix. nix is the original one, but after having looked at the code of both, I think rustix is just better designed and organized. It's also reached 1.0, whereas nix is still making semver-breaking changes (in fact we're behind here in this project). Now in practice, we have both *transitively* in the depchain already, and that's true for quite a lot of projects. But I think rustix is better, so let's add rustix as a workspace dependency (process feature) and migrate a few use cases to it - it's especially better than the raw libc which is suprisingly widespread. If we agree to do this, then many other calls can be ported. Signed-off-by: Colin Walters <[email protected]> * fix(packaging): add upgrade migration docs and podman socket retry (#1507) After #1415 ships, users upgrading from previous releases need guidance on the gateway.env deprecation, port/bind/database path changes, and the podman.socket restart requirement. - docs(rpm): add 'Migrating from gateway.env' section to TROUBLESHOOTING covering backward compatibility, env-to-TOML key mapping, and three breaking changes (default port 8080->17670, bind address 0.0.0.0->127.0.0.1, database path move). Add podman.socket restart step to upgrade procedure. - docs(rpm): add upgrade callout to CONFIGURATION.md pointing at migration section. - fix(podman): retry PodmanComputeDriver ping up to 5 times with 2s delay to tolerate transient socket unavailability after package upgrades. The systemd unit uses Wants=podman.socket (not Requires) so the gateway can start while the socket is briefly re-activating after an RPM upgrade changes its unit file on disk. - chore(rpm): update EnvironmentFile comment in RPM spec to explain backward-compatibility intent. Signed-off-by: Adam Miller <[email protected]> * ci: deduplicate e2e workflows (#1512) Signed-off-by: Taylor Mutch <[email protected]> * feat(auth): per-sandbox authentication to gateway (#1404) * docs(sandboxes): add policy advisor guide (#1480) Signed-off-by: John Myers <[email protected]> * fix(docker): use host-gateway callbacks on macOS (#1516) * ci(e2e): load single-arch images into kind (#1518) Signed-off-by: Taylor Mutch <[email protected]> * docs(rfc): add sandbox resource requirements proposal (#1360) * docs(rfc): add sandbox resource requirements proposal Signed-off-by: Evan Lezar <[email protected]> * docs(rfc): finalize sandbox resource requirements --------- Signed-off-by: Evan Lezar <[email protected]> * ci(canary): keep helm jwt secret generation enabled (#1521) Signed-off-by: Taylor Mutch <[email protected]> * fix(cli): add json output for policy get (#1410) * fix(cli): add json output for policy get * test(cli): cover policy get full json output * fix(cli): address policy get json clippy --------- Co-authored-by: John Myers <[email protected]> * feat(providers): derive discovery from profiles (#1503) * feat(providers): derive discovery from profiles Signed-off-by: John Myers <[email protected]> * fix(providers): keep v2 discovery profile-only Signed-off-by: John Myers <[email protected]> * docs(providers): update providers v2 behavior Signed-off-by: John Myers <[email protected]> * fix(providers): make github profile read-only Signed-off-by: John Myers <[email protected]> --------- Signed-off-by: John Myers <[email protected]> * docs: update NemoClaw/OpenClaw references (#1529) * ci: seed shared Rust caches from main (#1530) * fix(release): build host Linux binaries with glibc floor (#1490) * fix(homebrew): repair local driver bootstrap state (#1527) * fix(homebrew): repair local driver bootstrap state * fix(bootstrap): satisfy default SAN doc lint * ci: install cargo-zigbuild from release binaries (#1533) * fix(cli): propagate --gateway-insecure to OIDC auth flows (#1535) Thread the gateway_insecure flag through gateway_add(), gateway_login(), and all OIDC HTTP clients so that --gateway-insecure and OPENSHELL_GATEWAY_INSECURE apply to OIDC discovery, token exchange, and token refresh requests. Previously, the flag only affected gRPC connections to the gateway. OIDC HTTP clients (reqwest::get and http_client) always verified TLS certificates, causing gateway registration and login to fail when the OIDC issuer used a self-signed certificate (common on OpenShift with edge-terminated routes). Fixes #1534 Signed-off-by: Adel Zaalouk <[email protected]> * ci(release): smoke test rpm artifacts on fedora (#1558) Signed-off-by: Piotr Mlocek <[email protected]> * chore(deps): bump docker/login-action from 4.1.0 to 4.2.0 (#1554) Bumps [docker/login-action](https://github.com/docker/login-action) from 4.1.0 to 4.2.0. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/4907a6ddec9925e35a0a9e82d7399ccc52663121...650006c6eb7dba73a995cc03b0b2d7f5ca915bee) --- updated-dependencies: - dependency-name: docker/login-action dependency-version: 4.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(helm): add missing SPDX header to gateway-config template (#1545) * chore(helm): add missing SPDX header to gateway-config template * chore(scripts): remove helm templates from license header exclusions The bypass had no known rationale. Removing it ensures the header script covers deploy/helm/openshell/templates uniformly going forward. Signed-off-by: mesutoezdil <[email protected]> --------- Signed-off-by: mesutoezdil <[email protected]> * ci(release): skip python rpm in gateway smoke test (#1559) Signed-off-by: Piotr Mlocek <[email protected]> * ci: pin azure/setup-helm and helm/kind-action to commit SHAs (#1544) * ci: pin azure/setup-helm and helm/kind-action to commit SHAs * chore(python): add py.typed marker for PEP 561 compliance * ci: use full semver in pinned action version comments Signed-off-by: mesutoezdil <[email protected]> --------- Signed-off-by: mesutoezdil <[email protected]> * refactor: deduplicate shared code across ocsf builders and driver crates (#1526) Extract repeated patterns into shared helpers: - Add impl_builder_setters! macro to openshell-ocsf/builders that generates the identical severity(), status(), and message() setter methods present on all 7 OCSF event builders - Add SandboxContext::apply_common_fields() to consolidate the four-line build() finalization (set_status, set_message, set_device, set_container) repeated in every builder - Add driver_utils::sandbox_token_path() to centralize the XDG state path construction for sandbox JWT files used by both the Docker and Podman drivers - Add driver_utils::build_capabilities_response() to eliminate the identical GetCapabilitiesResponse struct literal repeated across the Docker, Podman, and Kubernetes compute drivers * fix(python): raise SandboxError instead of FileNotFoundError or KeyError (#1547) * fix(python): raise SandboxError instead of FileNotFoundError or KeyError * fix(python): suppress exception chaining in SandboxError raises Add `from None` to both `raise SandboxError(...)` calls inside `except FileNotFoundError` blocks to satisfy ruff B904. * fix(scripts): replace mapfile with bash 3.2-compatible read loop in helm-k3s-local (#1539) macOS ships bash 3.2 which lacks mapfile/readarray. Replace all three occurrences in configure_ghcr_credentials, cluster_has_image, and cluster_image_platform with a portable while-read loop, consistent with the fix applied to docker-build-image.sh in #1334. * docs: add macOS compiler troubleshooting (#1569) Signed-off-by: Ann Marie Fred <[email protected]> * fix(gateway): configure local dev auth (#1575) This makes it so you can run the dev gateway and sandbox with: ``` mise run gateway # in another shell mise run sandbox ``` Signed-off-by: Kris Hicks <[email protected]> * docs: add Pi as supported sandbox (#1572) * fix(sandbox): add mechanistic smoke test for L4 deny and document the L4/L7 split (#1412) * fix(sandbox): add mechanistic smoke test for L4 deny and document the L4/L7 split The old smoke script exercised an L7 PUT which hung because the denial aggregator is only wired to L4 CONNECT denies, not L7 enforcement. Add mechanistic-smoke.sh which triggers an L4 deny, waits for the aggregator to flush, and asserts a pending chunk appears under openshell rule get --status pending. Document the intentional L4-only scope of the mechanistic mapper in architecture/sandbox.md. Fixes #1333 Signed-off-by: mesutoezdil <[email protected]> * refactor(smoke): remove redundant variable inits and merge double step call Signed-off-by: mesutoezdil <[email protected]> * fix(smoke): wire mechanistic smoke into mise and guard TMP_DIR - Initialize TMP_DIR before trap to prevent unbound variable on early exit - Add e2e:mechanistic-smoke mise task with gateway setup - Document mechanistic smoke in policy-advisor README * test(proxy): verify L4 deny enqueues a DenialEvent Signed-off-by: mesutoezdil <[email protected]> * fix(proxy): remove unnecessary path qualifications in L4 denial smoke test --------- Signed-off-by: mesutoezdil <[email protected]> * docs(readme): whitespace (#1578) Signed-off-by: Kris Hicks <[email protected]> * fix(cli): replace outdated name reference (#1582) Signed-off-by: Kris Hicks <[email protected]> * fix(sandbox): probe Landlock before build, skip on unsupported kernels (#1585) On kernels without Landlock (e.g. gVisor's sentry returns ENOSYS for syscall 444), the previous best_effort path still logged "Applying Landlock" + "Landlock ruleset built" events even though no enforcement was happening. Probe at the top of `landlock::prepare` and short-circuit with a single High-severity "Sandbox Unavailable" finding. Signed-off-by: Davanum Srinivas <[email protected]> * fix(sandbox): decouple GPU baseline from network policy (#1524) Signed-off-by: Evan Lezar <[email protected]> * docs(kubernetes): note that Sandbox volumeClaimTemplates is immutable (#1543) * fix(sandbox): use succinct endpoint denial reason (#1584) Signed-off-by: Kris Hicks <[email protected]> * feat(docker): add provisioning progress events (#1567) * docs(kubernetes): add RBAC section to setup page (#1540) Documents the ServiceAccount, Role, and ClusterRole created by the Helm chart inline on the setup page, per reviewer feedback on #1250. Reflects the current chart templates including pods/get for sandbox identity and tokenreviews/create for projected token validation. Closes #1018 * fix(sandbox): delegate PID limits to runtimes (#1497) Signed-off-by: mjamiv <[email protected]> * fix(gateway): make readiness health checks dependency-aware (#1328) * feat(gateway): add readiness probe metrics and test-only store close Emit Prometheus readiness metrics for database probes (healthy gauge and outcome-labeled latency histogram) with coverage in health HTTP tests. Restrict Store::close behind test support cfg to prevent accidental runtime pool shutdown under live traffic. Signed-off-by: Adrien Langou <[email protected]> * test(e2e): add simple e2e test with kubernetes to test /readyz Signed-off-by: Adrien Langou <[email protected]> --------- Signed-off-by: Adrien Langou <[email protected]> * fix(vm): scope rootfs cache by openshell version (#1587) Signed-off-by: Drew Newberry <[email protected]> * fix(cli): preserve symlinks during sandbox upload (#1595) * fix(cli): preserve symlinks during sandbox upload * docs(sandboxes): document upload symlink behavior * fix(core): preserve SSH gateway default ports (#1602) Signed-off-by: Taylor Mutch <[email protected]> * feat(server): declare gRPC auth (mode + scope + role) at the handler, enforce at the router (#1596) * feat(server): per-handler gRPC auth annotations Move scope, role, and auth-mode metadata to the handler definition site via #[rpc_authz] + #[rpc_auth] proc macros. The previously hand-maintained SCOPED_METHODS, ADMIN_METHODS, UNAUTHENTICATED_METHODS, and ALLOWED_SANDBOX_METHODS tables are now generated from per-method annotations on the tonic service impls, with canonical gRPC paths derived from the service name and method name. Adds a new openshell-server-macros proc-macro crate, an aggregator in auth/method_authz.rs, and an exhaustiveness test that decodes the protobuf FileDescriptorSet (now emitted by openshell-core/build.rs) and verifies every RPC has an annotation. Signed-off-by: Mrunal Patel <[email protected]> * refactor(server): rename `sandbox-secret` auth mode to `sandbox` PR #1404 replaced the shared sandbox secret with per-sandbox gateway-minted JWTs. A handler marked `sandbox` now authenticates as a specific `Principal::Sandbox`, not as a holder of a shared credential. Rename `auth = "sandbox-secret"` to `auth = "sandbox"` and `AuthMode::SandboxSecret` to `AuthMode::Sandbox` so the name matches the post-#1404 identity model. Signed-off-by: Mrunal Patel <[email protected]> * fix(server): enforce per-handler AuthMode at the router Addresses review feedback on the per-handler auth-annotation work. - Router-level enforcement of #[rpc_auth] auth mode (HIGH). The previous router only checked is_sandbox_callable() for Principal::Sandbox; user principals still flowed into AuthzPolicy::check() and bypassed the per-handler declaration. A user with `openshell:all` could therefore reach `sandbox`-only handlers like GetSandboxProviderEnvironment, ReportPolicyStatus, PushSandboxLogs, and SubmitPolicyAnalysis even though their annotations said sandbox-only. Adds an is_user_callable() predicate and rejects User principals at the router for `sandbox` / `unauthenticated` methods. - Proc macro now errors on duplicate keys in #[rpc_auth(...)] (LOW). A second `auth`, `scope`, or `role` previously silently overwrote the first value; now it fails to compile. - Regression tests: a unit test for is_user_callable() and a router test that proves a user with admin role + openshell:all cannot reach the nine sandbox-only handlers. Signed-off-by: Mrunal Patel <[email protected]> * docs(server): finish renaming sandbox-secret to sandbox in method_authz doc comments Signed-off-by: Mrunal Patel <[email protected]> * refactor(server-macros): drop standalone `rpc_auth` stub The stub was a safety net that fired only when a method had `#[rpc_auth(...)]` without an enclosing `#[rpc_authz]`. Triggering it required `rpc_auth` to be imported, which is why both call sites carried `#[allow(unused_imports)] use openshell_server_macros::{rpc_auth, rpc_authz};`. Drop the stub and the unused-import workaround. A missing `#[rpc_authz]` now surfaces as rustc's standard "cannot find attribute `rpc_auth` in this scope" — clear enough, and one fewer import + lint exception. Addresses review comment on PR #1596. Signed-off-by: Mrunal Patel <[email protected]> * refactor(server-macros): emit fixed `AUTH_METADATA` const per service The previous trait-derived const name turned `OpenShell` into `OPEN_SHELL_AUTH_METADATA`, splitting the project name across an underscore. Each impl already lives in its own module (`crate::grpc::`, `crate::inference::`), so the module path is enough to disambiguate between services — a fixed `AUTH_METADATA` name reads more naturally. Aggregator in `auth/method_authz.rs` now references `crate::grpc::AUTH_METADATA` and `crate::inference::AUTH_METADATA` directly. Addresses review comment on PR #1596. Signed-off-by: Mrunal Patel <[email protected]> * docs(server-macros): fix typo in AUTH_METADATA_CONST doc comment OpenShell is one word; reference name in the doc should be OPENSHELL_AUTH_METADATA, not OPEN_SHELL_AUTH_METADATA. Addresses review nit on PR #1596. Signed-off-by: Mrunal Patel <[email protected]> --------- Signed-off-by: Mrunal Patel <[email protected]> * ci(snap): add snap release pipeline (#1600) * docs: refresh landing terminal demo and apply NVIDIA fern theme (#1615) - Extract landing-page terminal demo into a reusable <CommandTerminal /> component with inline styles (no global CSS dependency) - Animate a second command line cycling through claude/opencode/codex via @keyframes scoped inside the component - Inline BadgeLinks layout styles so the component renders correctly without relying on .badge-links from main.css - Add jsx.d.ts shim so editors do not flag the React global in component TSX files - Switch fern instance to global-theme: nvidia with multi-source enabled - Bump fern CLI to 5.40.0 and drop the basepath-aware experimental flag - Register fern/components/ as a second mdx-components directory - Remove the unused Adobe analytics script tag * build(macos): remove unused import of tracing::warn (#1619) Signed-off-by: Calum Murray <[email protected]> * chore: align .python-version with mise.toml (#1618) Signed-off-by: Calum Murray <[email protected]> * feat(helm): add optional PostgreSQL backing store (#1579) * feat(helm): add optional PostgreSQL backing store with Secret-based credentials - Add postgres.enabled and postgres.deploy values to control database backend (SQLite vs PostgreSQL) and subchart deployment independently. - Introduce db-secret.yaml template for Opaque Secret with assembled postgresql:// connection string injected via OPENSHELL_DB_URL env var. - Add Bitnami PostgreSQL as optional subchart dependency keyed on postgres.deploy to prevent subchart deployment in external mode. - Externalize JWT signing key file mode via sandboxJwt.secretDefaultMode with 0400 default matching upstream. - Add validation guard for postgres.deploy=true without postgres.enabled. - Add helm unit tests covering internal, external, URL-override, special character encoding, and misconfiguration error paths. - Update README with Kubernetes and OpenShift install examples for bundled and external PostgreSQL configurations. - Add helm dependency build to lint and unittest tasks. * fix(helm): add database backend docs to README.md.gotmpl and regenerate The helm-docs CI check failed because the Database backend section was added directly to README.md instead of README.md.gotmpl. Move the content to the template and regenerate so the check passes. * fix(helm): use Secret-based DB credentials and support existingSecret Replace the inline db-url stringData pattern with a proper Secret containing individual fields plus a uri key. When postgres.deploy=true the Bitnami service-binding secret is referenced directly; when deploy=false users can supply postgres.external.existingSecret to bring their own Secret, or let the chart generate one from the external field values. Also restructures the README database section for clarity, adds helm-unittest coverage for the new secret resolution paths, and fixes a markdown lint issue in the root README. * refactor(helm): move OpenShift e2e script to e2e/rust/ and add mise task Move test-openshift-scenarios.sh from deploy/helm/openshell/ci/ to e2e/rust/e2e-openshift.sh, matching the existing e2e script naming convention. Register it as `e2e:openshift` in tasks/test.toml — not wired into the `test` or `e2e` aggregates so it only runs on explicit invocation against a live OpenShift cluster. * feat(e2e): add database backend scenarios to Kubernetes e2e Extend with-kube-gateway.sh with an optional multi-scenario loop gated by OPENSHELL_E2E_KUBE_DB_SCENARIOS=1. When enabled, the script installs the Helm chart three times — SQLite (default), bundled PostgreSQL, and external PostgreSQL with existingSecret — running the full test suite against each backend. When unset, existing single-install behavior is unchanged. Also adds helm dependency build before helm install, fixing CI failures caused by the missing PostgreSQL subchart dependency. * refactor(helm): simplify PostgreSQL config to two orthogonal controls Replace postgres.deploy and postgres.external.* with two simple controls: - postgres.enabled: deploy the bundled Bitnami PostgreSQL subchart - server.externalDbSecret: name of a pre-existing Secret with a uri key Delete db-secret.yaml — the chart no longer generates Secrets from individual credential fields. Users either get the Bitnami service-binding secret (bundled) or bring their own via server.externalDbSecret. Add validation that postgres.serviceBindings.enabled must stay true when using bundled PostgreSQL, preventing a confusing runtime failure. * docs(config): update gateway config reference (#1624) * feat(flake): add Nix development shell (#1592) * feat(build): add simple nix flake with formatter for nix code * feat(flake): setup rust toolchain, able to build and run unit tests * feat(flake): add support for arm linux and macos * feat(toolchain): add rust-src and rust-analyzer to the toolchain * refactor(proto): move phase and current_policy_version into status (#1565) * refactor(proto): move phase and current_policy_version into SandboxStatus Move phase and current_policy_version from SandboxSpec into SandboxStatus to correctly model mutable runtime state. Update all callers in the gateway server, TUI, and Python SDK to read and write these fields through SandboxStatus accessors. Signed-off-by: Derek Carr <[email protected]> * fix(server): preserve sandbox status on statusless driver updates When a driver update arrives without a status payload (e.g. before Kubernetes populates the status subresource), preserve the stored phase, conditions, and current policy version instead of resetting them. Adds a regression test covering the edge case. Signed-off-by: Derek Carr <[email protected]> --------- Signed-off-by: Derek Carr <[email protected]> * feat(python-sdk): support OIDC Bearer auth on SandboxClient (#1621) * feat(python-sdk): support OIDC Bearer auth on SandboxClient PR #1596 hardened the gateway side of the OIDC story; the Python SDK was the remaining gap — it only supported plaintext or mTLS, with no Bearer metadata anywhere. Deployments with OIDC enabled (the recommended posture since PR #935 / PR #1404) were unreachable from the SDK. Adds: - `bearer_token: str | Callable[[], str] | None` kwarg on `SandboxClient`. Static strings or zero-arg callables (the latter is invoked per RPC, so callers can drop in a refresh loop or token-file watcher without reconstructing the client). Composes with `tls` for OIDC-over-mTLS deployments. - `_BearerAuthInterceptor` implementing all four `grpc.{Unary,Stream}{Unary,Stream}ClientInterceptor` types. Appends `authorization: Bearer <token>` to outgoing metadata. Implemented as an interceptor (not call credentials) so it works on both plaintext (`disableTls=true` dev) and TLS channels without `grpc.composite_channel_credentials`. - `TlsConfig` ergonomics: all three fields (`ca_path`, `cert_path`, `key_path`) are now optional with `cert_path` / `key_path` required-together-or-not-at-all (enforced in `__post_init__`). This unlocks three transport profiles from one dataclass: * full mTLS (all three) * CA-only trust (`ca_path` only) * system roots (`TlsConfig()` — for OIDC gateways behind a public CA) - `from_active_cluster` mirrors `crates/openshell-tui/src/lib.rs` `build_oidc_channel`: * For any `https://` gateway, always build a secure channel. Pick the strongest TLS profile available in `mtls/` (full mTLS → CA-only → system roots). No more `insecure_channel` fallback for HTTPS. * Gate OIDC bearer attachment on `metadata.json["auth_mode"] == "oidc"`. Matches `crates/openshell-cli/src/main.rs:132` and the TUI; a stale `oidc_token.json` next to a non-OIDC gateway no longer causes the SDK to attach a bearer. - `_OidcRefresher` — thread-safe, in-process native OAuth2 refresh modeled on `google.oauth2.credentials.Credentials` and `botocore.tokens.SSOTokenProvider`. Lazily checks expiry on every RPC; when stale, re-reads disk first (the CLI may have rotated the bundle), and only then exchanges the refresh_token against the IdP's token endpoint discovered via OIDC discovery (`/.well-known/openid-configuration`, cached after first call). Concurrent RPCs share a single refresh via `threading.Lock` (no IdP stampede). Honors refresh-token rotation. Surfaces IdP failures as `SandboxError` with the RFC 6749 error body included for diagnostics. Mirrors the Rust CLI's HTTP-policy posture from `crates/openshell-cli/src/oidc_auth.rs`: * `follow_redirects=False` so a 3xx during discovery can't steer us to an attacker-controlled token endpoint. * Discovery `issuer` is validated against the configured issuer; a discovery document claiming a different issuer is rejected, preventing the SDK from POSTing the refresh_token to a malicious endpoint. * `insecure: bool` flag plumbed through to httpx's `verify=` so self-signed-cert deployments work the same way they do in the Rust CLI. Built on `httpx` (chosen over `urllib` specifically for follow_redirects + verify control as kwargs). The OAuth2 refresh-token grant itself (RFC 6749 §6) is one form-encoded POST — handled inline rather than via a dedicated OAuth library; tried `authlib`'s `OAuth2Client` first but it auto-injects an Authorization header on every request, which breaks the unauthenticated discovery GET. - `_make_cluster_bearer_provider(..., auto_refresh=True, write_back=True, insecure=False)` factory. Defaults to the refresher path with write-back enabled; `auto_refresh=False` falls back to the read-only fail-closed behavior for callers that don't want the SDK to make outbound HTTP calls to the IdP. `write_back=True` is the default (changed from the first round of review): IdPs with refresh-token rotation (Keycloak with rotation, Entra in strict mode) invalidate the old refresh_token on each refresh, so an in-memory-only refresh would leave the on-disk bundle pointing at an invalidated value — any second process starting from disk would `invalid_grant`. With write-back enabled by default, the SDK keeps the shared cache consistent with the IdP. - `from_active_cluster` exposes `auto_refresh`, `write_back`, and `insecure` kwargs (defaults: True / True / False). The high-level `Sandbox` context manager surfaces the same three kwargs and forwards them through, so callers using the wrapper have parity with `SandboxClient` for OIDC-protected gateways. - `SandboxClient.close()` chains to a `_bearer_close` hook so the `_OidcRefresher`'s underlying `httpx.Client` is released deterministically instead of leaking sockets/FDs until GC runs `__del__`. Idempotent. - `_OidcRefresher._write_to_disk` uses `tempfile.mkstemp` (PID + random suffix) instead of a fixed `.oidc_token.json.tmp` path, so two writers racing on the same gateway directory don't trample each other's tmp content. Success path atomically replaces; failure path unlinks the orphan. OAuth2 refresh policy and write-back semantics deliberately mirror what the major Python SDKs do — see github.com/googleapis/google-auth-library-python (`Credentials`) and github.com/boto/botocore (`SSOTokenProvider`): | Library | Native refresh | Writes back | |-------------------------------|----------------|-------------| | google-auth Credentials | yes | no | | botocore SSOTokenProvider | yes | yes | | openshell SandboxClient (here)| yes (opt-out) | yes (opt-out)| OpenShell sits between the two; chose write-back-by-default because the rotation invariant matters more for our deployments than the "CLI is the only writer" assumption that fits google-auth. Adds `httpx>=0.27` as a runtime dependency. No new OAuth2 library — the refresh grant is a single POST. Tested: - 42 sandbox_test.py tests pass (5 pre-existing + 37 new across the bearer interceptor, fail-closed provider, refresher behavior, TlsConfig validation, from_active_cluster auth ladder, security-review regressions, Sandbox-wrapper kwarg forwarding, and lifecycle / concurrency probes). `mise run test:python` → 47 passed total across the python suite. - `mise run python:lint` (ruff) clean. - End-to-end against a Keycloak-protected gateway on OpenShift: * unauthenticated `Health` bypass works * admin + `openshell:all` reaches user-callable methods * reader (`sandbox:read`) denied on `CreateSandbox` by scope * admin + `openshell:all` denied on PR #1596 sandbox-only methods at the router (the new gate is honored from the SDK) * full provider CRUD lifecycle via the SDK * callable token provider rotates per RPC as expected - Regression-probed against three pre-review security findings: * **Discovery issuer validation** — a discovery document claiming a different `issuer` than the configured one is rejected with a clear `SandboxError` before any refresh POST can reach the attacker-controlled endpoint. * **Redirect during discovery** — `follow_redirects=False` on the underlying httpx client means a 3xx during discovery surfaces as a SandboxError rather than silently chasing the redirect. * **Cross-process rotation** — a two-process simulation shows process B starting from disk and successfully refreshing with the rotated refresh_token, because process A's write-back updated the shared cache. - Refresher unit tests cover: cached-fresh fast path, disk-rotated re-read before refresh, OAuth2 exchange against the discovered token endpoint, refresh-token rotation, atomic write-back at 0600 mode (default), default-on write_back proven by test, concurrent N-thread coordination (one refresh shared across 8 threads), IdP failure surfaced with the error body, the client_credentials / no-refresh_token error path, issuer- mismatch rejection, redirect-during-discovery rejection, insecure flag plumbing. - Lifecycle / concurrency regression tests added: `close()` invokes the `_bearer_close` hook (idempotent), the refresher's `httpx.Client` is marked closed after `SandboxClient.close()`, and 16 concurrent writers don't leave orphan tmp files behind while producing a valid final bundle. The `Sandbox` wrapper has direct forwarding tests proving `auto_refresh`, `write_back`, and `insecure` reach `from_active_cluster` (both explicit values and defaults). - End-to-end against a real OpenShift + Keycloak cluster from inside a pod: real OIDC discovery against `keycloak.keycloak.svc.cluster.local:8080`, refresh-token grant POST, atomic write-back of the rotated bundle at 0600, and a follow-up RPC reusing the freshly-rotated in-memory token — full round-trip in ~170ms. Signed-off-by: Mrunal Patel <[email protected]> * fix(python-sdk): adopt newer on-disk OIDC bundle before refreshing _OidcRefresher.current_access_token() only adopted the on-disk oidc_token.json when its access token was still fresh; otherwise it refreshed using the in-memory bundle. With refresh-token rotation enabled (Keycloak with rotation, Entra strict mode), this let a process keep using an invalidated refresh_token: 1. Process A holds a stale in-memory bundle with refresh_token=r1. 2. Process B refreshes first and writes a rotated (r2) but now near-expiry bundle to disk. 3. Process A re-reads disk, sees the access token is not fresh, ignores the disk bundle, and POSTs the stale r1 — which the IdP has already invalidated, yielding invalid_grant. Fix: when the cached bundle is stale, adopt the on-disk bundle if it was refreshed more recently than ours, even when its access token is also stale. "More recently" is decided by expires_at — a refresh mints a new access token with a forward expiry alongside the rotated refresh_token, so the later expiry carries the newest refresh_token. Comparing by expiry (rather than unconditionally preferring disk) preserves the write_back=False case, where the in-memory bundle has already rotated past the on-disk copy and must not be clobbered. When the adopted bundle's issuer differs, the cached token endpoint is reset so the refresh re-discovers against the new issuer. Adds regression tests for the cross-process rotation race and the issuer-change re-discovery path. * fix(python-sdk): recover from invalid_grant on lost rotation race The expiry-based disk re-read narrows but does not fully close the cross-process refresh-token rotation race: two processes sharing a gateway directory can both enter their refresh window, both POST their copy of the refresh_token, and with rotation enabled the IdP invalidates the loser's token (invalid_grant). Neither google-auth nor botocore close this window without an OS file lock; a Python-only flock would not coordinate with the Rust CLI/TUI that also write oidc_token.json, so locking is not worth its cost here. Recover instead of prevent: distinguish an OAuth2 invalid_grant (the refresh_token was rejected) from transport/5xx failures via a private _InvalidGrantError, and on invalid_grant re-read oidc_token.json once. If a peer wrote a different refresh_token (it won the race), adopt and retry with it — returning early if it is already fresh — so the loser succeeds transparently instead of forcing a re-authenticate. If disk offers no new token, the rejection is genuine and surfaces the re-authenticate hint as before. The retry is single-shot; a second invalid_grant propagates. Adds tests for the peer-rotation recovery and the genuine-rejection (no-retry) paths. --------- Signed-off-by: Mrunal Patel <[email protected]> * fix(helm): vendor chart dependencies before release packaging (#1627) Signed-off-by: Taylor Mutch <[email protected]> * fix(driver-podman): bind gateway to 0.0.0.0 in rootless mode (#1623) Rootless Podman sandbox containers reach the host through pasta's local connection bypass, which translates L2 frames to L4 host sockets. The dev gateway script binds to 127.0.0.1 by default, which is not routable through pasta. Auto-detect rootless mode and bind to 0.0.0.0 so sandbox containers can connect to the gateway. - Auto-detect rootless Podman in gateway.sh and export OPENSHELL_BIND_ADDRESS=0.0.0.0 when not explicitly set - Add e2e:podman:rootless mise task and CI matrix entry to validate rootless Podman networking end-to-end - CI creates a non-root user inside the privileged container to trigger Podman's rootless code paths (pasta, user namespace isolation) Signed-off-by: Naveen Malik <[email protected]> * docs(providers): note that ANTHROPIC_API_KEY requires an API account, not a subscription (#1542) * docs(providers): note that ANTHROPIC_API_KEY requires an API account, not a subscription Anthropic subscription users authenticate via OAuth, not an API key, causing a silent failure when creating the provider. Adds a Note callout in the provider type table and quickstart guide directing subscription users to generate an API key from console.anthropic.com. Closes #620 * docs(providers): fix Note placement and remove subscription brand names Move the Note callout in manage-providers.mdx to after the complete provider type table so it does not break table rendering. Remove subscription brand names from both Note callouts. * fix(podman): avoid host-gateway on macOS machines (#1637) Closes #1307 Default the Podman host gateway alias override to gvproxy's host-loopback IP on macOS while preserving host-gateway resolution on Linux. Wire the setting through Podman config, gateway TOML inheritance, and the standalone driver, and document the platform behavior. Signed-off-by: Taylor Mutch <[email protected]> * chore(vm): generalize crate for multi-device PCIe passthrough (#1573) * generalize crate for multi-device PCIe passthrough Signed-off-by: Patrick Riel <[email protected]> * add adopt apis which allow for devices already bound to vfio-pci during restart reconciliation, without rebinding or mutating sysfs. Signed-off-by: Patrick Riel <[email protected]> * refactor(vfio): generalize GPU passthrough sysfs handling Signed-off-by: Patrick Riel <[email protected]> * fix(vfio): centralize vfio ID refcounting Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Patrick Riel <[email protected]> Signed-off-by: Evan Lezar <[email protected]> Co-authored-by: Evan Lezar <[email protected]> * fix(sandbox): trust exact declared private endpoints (#1560) * fix(sandbox): trust exact declared private endpoints * fix(sandbox): preserve advisor endpoint provenance * fix(sandbox): repair advisor provenance lint failures --------- Co-authored-by: John Myers <[email protected]> * feat(policy): add agentic approval loop (#1528) * fix(e2e): clean up temp files in sandbox-runner on exit (#1647) * ci(kubernetes): add HA e2e workflow (#1598) Signed-off-by: Taylor Mutch <[email protected]> * ci(release): use bundled Z3 for macOS gateway build (#1658) * fix(gateway): align package TLS bootstrap path (#1601) * fix(gateway): align package TLS bootstrap path Closes #1593 Default package-managed gateway services to a stable local TLS directory and use that same value for certificate generation and runtime startup. Signed-off-by: Taylor Mutch <[email protected]> * test(packaging): validate package asset paths exist Signed-off-by: Taylor Mutch <[email protected]> * ci(e2e): pin mise in kubernetes job Signed-off-by: Taylor Mutch <[email protected]> --------- Signed-off-by: Taylor Mutch <[email protected]> * feat(tui): add PageUp/PageDown scrolling to all panes (#1656) Add PageUp/PageDown key support to the policy, logs, and draft/rules views. All three panes now scroll by one viewport height per keypress. Also fix scroll_policy() clamping to stop at the last viewport of content instead of the last line, preventing a blank-screen overshoot on G and PageDown. Signed-off-by: Major Hayden <[email protected]> * feat(telemetry): add anonymous opt-out OpenShell usage telemetry (#1433) * feat(telemetry): add anonymous opt-out usage telemetry Signed-off-by: Kirit93 <[email protected]> * Removed enums from schema Signed-off-by: Kirit Thadaka <[email protected]> * Updated telemetry URL Signed-off-by: Kirit93 <[email protected]> * ci(kubernetes): pin mise installer for e2e --------- Signed-off-by: Kirit93 <[email protected]> Signed-off-by: Kirit Thadaka <[email protected]> Co-authored-by: John Myers <[email protected]> * ci(release): gate helm/oci artifact publishing on release (#1662) release-helm and tag-ghcr-release now depend on the release job. This is to prevent a GHCR image or helm chart from being published when some other aspect of the release fails. Signed-off-by: Kris Hicks <[email protected]> * ci(kubernetes): stabilize HA e2e setup (#1659) * ci(kubernetes): pin mise in e2e workflow Signed-off-by: Taylor Mutch <[email protected]> * ci(kubernetes): mirror postgres image for ha e2e Signed-off-by: Taylor Mutch <[email protected]> * ci(kubernetes): reuse e2e workflow for ha Signed-off-by: Taylor Mutch <[email protected]> --------- Signed-off-by: Taylor Mutch <[email protected]> * fix(gateway): place supervisor_image under podman driver TOML table (#1661) The gateway.sh script appended supervisor_image after the [openshell.gateway.gateway_jwt] table header, so TOML parsed it as a gateway_jwt field. The Podman driver never saw the override and fell back to the default ghcr.io/nvidia/openshell/supervisor:latest. Move supervisor_image into [openshell.drivers.podman] where the driver config deserializer expects it. * refactor: deduplicate shared utilities across driver crates (#1660) Move three duplicated definitions into openshell-core so every consumer has a single canonical source: - format_bytes: identical 14-line function existed in docker, kubernetes, and vm drivers. Moved to openshell-core::progress where all three already imported from. - DEFAULT_SANDBOX_PIDS_LIMIT: i64 constant (2048) duplicated in docker driver and podman config. Moved to openshell-core::config alongside other shared defaults. Podman re-exports it for internal call-site compatibility. - current_time_ms: secrets.rs in openshell-sandbox reimplemented the same logic as openshell-core::time::now_ms(). Remove the local copy and call now_ms() directly via the existing dep. * fix(config): reject unknown fields in nested gateway config tables (#1666) * fix(config): reject unknown fields in nested gateway config tables The gateway TOML loader silently ignored keys placed under the wrong table header. PR #1661 fixed one instance of this (supervisor_image under [openshell.gateway.gateway_jwt]) but the root cause remained: the nested gateway config tables did not deny unknown fields, so a misplaced key was accepted and dropped instead of erroring. Concretely, tasks/scripts/gateway.sh emitted `sandbox_namespace` right after the [openshell.gateway.gateway_jwt] heredoc, so it landed inside the gateway_jwt table rather than [openshell.gateway]. The k8s driver already receives the namespace via [openshell.drivers.kubernetes], so the stray line was dead config that parsed without complaint. Changes: - Add #[serde(deny_unknown_fields)] to the nested gateway config tables that are part of the config-file parse tree: TlsConfig, OidcConfig, MtlsAuthConfig, GatewayAuthConfig, GatewayJwtConfig. - Remove the misplaced sandbox_namespace line from gateway.sh. - Drop the unused Serialize/Deserialize derives from Config and ServiceRoutingConfig (see below). - Add a regression test asserting a key under the wrong nested table is rejected. * feat(kubernetes): support sandbox image pull secrets (#1671) * refactor(driver): trim compute capability response (#1402) Signed-off-by: Evan Lezar <[email protected]> * feat(providers): add Google Vertex AI inference provider (#1568) * feat(providers): add Google Vertex AI provider Adds Vertex AI provider profiles, routing, credential refresh plumbing, CLI support, docs, and regression coverage. Keeps the related NETLINK_ROUTE seccomp allowance needed by Vertex client tooling that calls getifaddrs. * docs: add Vertex AI sandbox usage for Claude Code and OpenCode Cover the full end-to-end setup for running Claude Code and OpenCode inside an OpenShell sandbox via inference.local with a Vertex AI backend: - google-vertex-ai.mdx: add 'Use from a Sandbox' section with tabbed examples for Claude Code (--bare flag, no /v1 suffix) and OpenCode (/v1 suffix required). Add providers_v2_enabled prerequisite and --no-verify note for global region. Document policy proposals table covering metadata.google.internal (always blocked), downloads.claude.ai, and storage.googleapis.com. - inference-routing.mdx: expand 'Use the Local Endpoint' section with tabbed examples for Claude Code, OpenCode, Python OpenAI SDK, and Python Anthropic SDK. Add notes explaining the /v1 path suffix difference between clients. - supported-agents.mdx: update Claude Code and OpenCode rows to mention inference.local support and correct base URL requirements. * fix: address vertex review findings * test(sandbox): retry on spurious Ok in fork-exec ambiguity test On arm64 under heavy CI load, the /proc fd scan in find_socket_inode_owners can transiently miss the parent process's socket fd entry, returning only the child as an owner. This causes resolve_process_identity to return Ok (single owner, no ambiguity check fires) instead of the expected ambiguous-ownership Err. Extend the retry loop to also handle unexpected Ok results, mirroring the existing retry for transient Err results. 10 retries at 50ms gives a 500ms settling window, which is sufficient for procfs to stabilize on loaded arm64 runners. * fix: address vertex review regressions * docs(router): clarify stream_response semantics for Vertex rawPredict routing Document the three call sites of prepare_backend_request and their stream_response values in a caller table: - send_backend_request: false → :rawPredict (unary endpoint) - send_backend_request_streaming: true → :streamRawPredict - verify_backend_endpoint: explicitly false to probe the unary endpoint Cross-reference the table from build_provider_url and is_vertex_anthropic_rawpredict_route so the stream_response=true guard in the suffix upgrade branch is understood in full context. Also note that is_vertex_anthropic_rawpredict_route is a structural predicate (model_in_path + anthropic_messages + :rawPredict suffix), not a named-provider check, so any future provider with the same route shape inherits the transforms automatically. * fix: correct example paths in local-inference README (#1676) * fix: correct example paths in local-inference README * fix: correct example paths in local-inference routes.yaml * ci(release): bring Fedora RPM canary to parity (#1688) The RPM canary needs to exercise the install.sh user-service path, but a GitHub Actions job container does not boot with systemd as PID 1. The Fedora RPM canary needs to exercise the install.sh user-service path, but a GitHub Actions job container does not boot with systemd as PID 1. This means the Fedora RPM canary was incomplete as compared to the others. With this change, we run Fedora as a nested privileged systemd container instead, wait for systemd to become reachable, then start the root user manager so systemctl --user works for the RPM gateway unit, achieving parity with the other canary tests. Signed-off-by: Kris Hicks <[email protected]> * fix: update RFC link in agent-driven-policy-management README (#1677) * feat(providers): add profile-backed policy visibility (#1640) * chore: wip providers v2 tui and codex profile * chore: wip effective policy get and codex profile * chore: wip provider profiles and tui detail views * feat(tui): annotate policy proposal review status * ci(release): fix Ubuntu Snap canary install and registration (#1699) Install the Snap built by the triggering Release Dev workflow by setting merge-multiple: true on the artifact download. actions/download-artifact otherwise extracts each artifact into its own subdirectory, leaving the package at release/snap-linux-amd64/*.snap, so the install glob ./release/*.snap matched nothing. Merging flattens the artifact's contents directly into release/ where the dangerous local snap install expects it. Harden the Snap canary setup by enabling snapd.socket, waiting for snap seeding (snap wait system seed.loaded), and running every step with strict shell options (set -euo pipefail) so failures surface immediately. Register the snapped gateway with the CLI as the documented local plaintext snap-docker gateway, and print version and snap services, before running openshell status so the canary verifies a configured and reachable gateway instead of only the install. Signed-off-by: Kris Hicks <[email protected]> * feat(snap): add openshell.term desktop app (#1693) Add a desktop launcher for the OpenShell TUI so users can launch "openshell term" from their desktop environment application menu. The change adds three files: - snap/local/term.desktop: desktop entry file for the application launcher - snap/local/icon.png: application icon (copied from snap store data) - snapcraft.yaml: new "term" app entry that runs "openshell term" with home, network, ssh-keys, and system-observe plugs, plus install rules to stage the desktop file and icon under meta/gui/ The desktop file references the icon via ${SNAP} which is resolved at runtime to the snap installation directory. The term app reuses the same connection plugs as the main openshell app. Signed-off-by: Zygmunt Krynicki <[email protected]> * fix(sandbox): restore GPU procfs baseline (#1522) Signed-off-by: Evan Lezar <[email protected]> * fix(gateway): try harder to detect Podman (#1536) Auto-detection previously treated Podman as available only when the podman CLI was visible on PATH. However, package manager services can run with a restricted PATH, which lets Docker be selected even when a Podman API socket is reachable. Additionally, podman may symlink /var/run/docker.sock to podman's machine unix socket, which would be incorrectly detected as Docker. Worse still: the podman machine may not even be running. This replaces the Podman binary check with a functional HTTP probe against the standard Podman socket paths. The probe requires /_ping to answer with a Libpod-Api-Version header before treating the socket as Podman, which lets the gateway select the embedded Podman driver only when the API is usable. Signed-off-by: Kris Hicks <[email protected]> * chore(mise): refresh tool lockfile (#1712) Signed-off-by: Kris Hicks <[email protected]> * ci(release): authenticate snap canary artifact download (#1711) The Ubuntu Snap canary downloads its artifact from a different workflow run (the triggering Release Dev run) via run-id. Cross-run downloads require authentication, so pass github.token to actions/download-artifact. Signed-off-by: Kris Hicks <[email protected]> * docs(container-gateway): fix Docker driver setup for containerized gateway (#1419) The existing docs omitted or misstated several requirements when running the gateway as a container with the Docker compute driver: - OPENSHELL_GRPC_ENDPOINT is required; the Docker driver uses only the scheme (http/https) — host and port are substituted automatically with host.openshell.internal and the gateway's own bind port - Supervisor binary must be extracted to a host path before starting the gateway; bind-mount sources are resolved by the host Docker daemon so the path must be identical inside and outside the gateway container - Docker socket access requires adding the docker group (UID 1000 default) - Port binding should remain 127.0.0.1; Docker driver adds a bridge listener automatically - add --server-san host.openshell.internal to generate-certs for mTLS - Complete the mTLS docker run with all Docker driver requirements - Add deploy/docker/gateway.toml — TOML config for the Docker driver - Add deploy/docker/docker-compose.yml referencing the TOML - Add docs/get-started/tutorials/docker-compose.mdx tutorial page - Remote gateway registration instructions (--remote flag) Address reviewer feedback: - Move Docker Compose tutorials card to the bottom of the list - Replace inline YAML snippet in Docker Compose section with a reference to deploy/docker/ to avoid drift - Clarify OPENSHELL_DB_URL is safe in compose.yml (plain SQLite path, no credentials); the TOML block targets credential-bearing DSNs - Note that ./ in source: resolves relative to the compose file directory - Clarify that only the scheme from OPENSHELL_GRPC_ENDPOINT matters - Add note that the tilde volume mount resolves to the same absolute path on both host and container * refactor(server): deduplicate test helpers and grpc utilities (#1708) Remove three groups of copy-pasted code in openshell-server: 1. grpc/mod.rs had a private current_time_ms() wrapper identical to the one already exported from persistence/mod.rs. Remove the duplicate and update the three grpc sub-modules (policy, sandbox, service) to import directly from crate::persistence. 2. test_store() was repeated verbatim in seven #[cfg(test)] blocks. Promote a single canonical version to persistence/mod.rs (cfg-gated) and replace all copies with crate::persistence::test_store() calls or a thin Arc wrapper in supervisor_session. 3. grpc_client_mtls() and build_tls_root() were copy-pasted across edge_tunnel_auth.rs and multiplex_tls_integration.rs. Move both into the existing tests/common/mod.rs shared module and import from there. * fix(gateway): allow local sandbox jwt to not expire (#1721) * fix(helm): create sandbox JWT secret when cert-manager is enabled (#1700) * fix(helm): create sandbox JWT secret under cert-manager The cert-manager install path (certManager.enabled=true, pkiInitJob.enabled=false) left the gateway StatefulSet unable to start because nothing created the openshell-jwt-keys Secret: cert-manager owns TLS Secrets but does not mint the sandbox JWT signing key, and the certgen hook only rendered when pkiInitJob.enabled was true. Separate JWT signing-key provisioning from TLS PKI provisioning: - certgen: add a --jwt-only mode that creates only the Opaque JWT signing Secret, for use when another controller owns TLS Secrets. - certgen.yaml: render the hook when pkiInitJob.enabled OR certManager.enabled is true. cert-manager takes precedence and runs the hook with --jwt-only even if pkiInitJob.enabled remains true. Remove the mutual-exclusion failure between the two values. - _helpers.tpl: add openshell.sandboxJwtSecretName, shared by the hook and the StatefulSet mount. - Update values, README, docs, architecture, and the debug-openshell-cluster skill to reflect the new precedence; the documented cert-manager install no longer needs pkiInitJob.enabled=false. Closes #1691 * fix(helm): honor cert-manager precedence for client CA volume The client CA volume logic treated pkiInitJob.enabled as proof that built-in PKI owns the client CA. With cert-manager precedence now allowing certManager.enabled=true alongside the default pkiInitJob.enabled=true, that assumption mounts the server TLS cert secret as the client CA and ignores certManager.clientCaFromServerTlsSecret=false, which can break mTLS or trust the wrong CA. Gate the pkiInitJob.enabled term with (not certManager.enabled) in all three client CA conditions (volume mount, volume definition, and secret selection) so cert-manager owns TLS when enabled. Add a Helm test suite covering built-in PKI, cert-manager shared CA, the regression config (cert-manager + clientCaFromServerTlsSecret=false + default pkiInitJob), and the no-client-CA case. * feat(k8s-driver): add default_runtime_class_name config for sandbox pods (#1729) Allow operators to configure a default Kubernetes runtimeClassName that is applied to sandbox pods when the CreateSandbox request does not specify one. This avoids requiring every API caller to explicitly set the runtime class for clusters that always need a specific RuntimeClass (e.g. kata-containers, nvidia). The fallback is applied in the Kubernetes driver only — per-request values still take priority, and an empty default (the built-in) preserves existing behavior (field omitted, cluster default applies). * docs: add Hermes Agent to supported agents (#1735) * fix(cli): roll back gateway registration when auth fails during gateway add (#1538) * refactor: deduplicate shared driver and TUI helpers (#1741) * feat(cli): support multiple --upload flags on sandbox create (#1635) (#1645) Closes #1635 Signed-off-by: Philippe Martin <[email protected]> * updates for new containers --------- Signed-off-by: Derek Carr <[email protected]> Signed-off-by: Florent Benoit <[email protected]> Signed-off-by: Piotr Mlocek <[email protected]> Signed-off-by: Colin Walters <[email protected]> Signed-off-by: Adam Miller <[email protected]> Signed-off-by: Taylor Mutch <[email protected]> Signed-off-by: John Myers <[email protected]> Signed-off-by: Evan Lezar <[email protected]> Signed-off-by: Adel Zaalouk <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: mesutoezdil <[email protected]> Signed-off-by: Ann Marie Fred <[email protected]> Signed-off-by: Kris Hicks <[email protected]> Signed-off-by: Davanum Srinivas <[email protected]> Signed-off-by: mjamiv <[email protected]> Signed-off-by: Adrien Langou <[email protected]> Signed-off-by: Drew Newberry <[email protected]> Signed-off-by: Mrunal Patel <[email protected]> Signed-off-by: Calum Murray <[email protected]> Signed-off-by: Naveen Malik <[email protected]> Signed-off-by: Patrick Riel <[email protected]> Signed-off-by: Major Hayden <[email protected]> Signed-off-by: Kirit93 <[email protected]> Signed-off-by: Kirit Thadaka <[email protected]> Signed-off-by: Zygmunt Krynicki <[email protected]> Signed-off-by: Philippe Martin <[email protected]> Co-authored-by: Mesut Oezdil <[email protected]> Co-authored-by: Drew Newberry <[email protected]> Co-authored-by: Taylor Mutch <[email protected]> Co-authored-by: Seth Jennings <[email protected]> Co-authored-by: Florent BENOIT <[email protected]> Co-authored-by: Eric Curtin <[email protected]> Co-authored-by: Derek Carr <[email protected]> Co-authored-by: mjamiv <[email protected]> Co-authored-by: John Myers <[email protected]> Co-authored-by: Piotr Mlocek <[email protected]> Co-authored-by: Russell Bryant <[email protected]> Co-authored-by: Colin Walters <[email protected]> Co-authored-by: Adam Miller <[email protected]> Co-authored-by: Taylor Mutch <[email protected]> Co-authored-by: Evan Lezar <[email protected]> Co-authored-by: Adel Zaalouk <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ann Marie Fred <[email protected]> Co-authored-by: krishicks <[email protected]> Co-authored-by: Vegard Stikbakke <[email protected]> Co-authored-by: krishicks <[email protected]> Co-authored-by: Davanum Srinivas <[email protected]> Co-authored-by: alangou <[email protected]> Co-authored-by: Mrunal Patel <[email protected]> Co-authored-by: Andrew Schilling <[email protected]> Co-authored-by: Calum Murray <[email protected]> Co-authored-by: Saurabh Agarwal <[email protected]> Co-authored-by: Simon Scatton <[email protected]> Co-authored-by: Naveen Malik <[email protected]> Co-authored-by: Patrick Riel <[email protected]> Co-authored-by: Alexander Watson <[email protected]> Co-authored-by: Major Hayden <[email protected]> Co-authored-by: Kirit Thadaka <[email protected]> Co-authored-by: Jesse Jaggars <[email protected]> Co-authored-by: Zygmunt Krynicki <[email protected]> Co-authored-by: shannonsands <[email protected]> Co-authored-by: Philippe Martin <[email protected]>
Clarify that all-time figures are cumulative from #1433, so readers know when the all-time counts begin. Signed-off-by: Kirit Thadaka <[email protected]>
Clarify that all-time figures are cumulative from #1433, so readers know when the all-time counts begin. Signed-off-by: Kirit Thadaka <[email protected]>
* docs(telemetry): add community telemetry reports page Add telemetry/README.md to publish aggregate usage trends every two weeks, and link to it from the Telemetry section of the main README. First report covers the July 8, 2026 window. Signed-off-by: Kirit Thadaka <[email protected]> * docs(telemetry): note telemetry start date (June 1, 2026) Clarify that all-time figures are cumulative from #1433, so readers know when the all-time counts begin. Signed-off-by: Kirit Thadaka <[email protected]> --------- Signed-off-by: Kirit Thadaka <[email protected]>
* refactor(server): normalize compute driver config acquisition (#1974) * refactor(server): remove unused compute runtime constructor parameter Signed-off-by: Evan Lezar <[email protected]> * refactor(server): normalize compute driver type imports Signed-off-by: Evan Lezar <[email protected]> * refactor(server): key driver config tables by name Signed-off-by: Evan Lezar <[email protected]> * refactor(server): normalize compute driver config acquisition Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Evan Lezar <[email protected]> * test(e2e): run gpu workloads from manifest (#1709) * test(e2e): add workload manifest build flow Signed-off-by: Evan Lezar <[email protected]> * test(e2e): add gpu workload validation tests Signed-off-by: Evan Lezar <[email protected]> * ci(e2e): build gpu workloads before gpu e2e Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Evan Lezar <[email protected]> * fix(providers): reserve credential placeholder revisions (#2049) * fix(providers): reserve credential placeholder revisions Signed-off-by: John Myers <[email protected]> * fix(providers): share placeholder namespace parser Signed-off-by: John Myers <[email protected]> * test(providers): cover non-revision env key Signed-off-by: John Myers <[email protected]> --------- Signed-off-by: John Myers <[email protected]> Co-authored-by: John Myers <[email protected]> * fix(CONTRIBUTING): update label format for good first issues (#2056) * fix(helm): generate namespace-aware SANs in certgen and cert-manager templates (#2062) The certgen hook and cert-manager Certificate template hardcoded openshell.openshell.svc.cluster.local in server certificate SANs, breaking deployments in any namespace other than openshell. Use .Release.Namespace in the templates so the SANs match the actual service FQDN regardless of the target namespace. Closes #2060 Signed-off-by: Akram <[email protected]> * refactor(core): remove unused extra bind addresses (#2059) Signed-off-by: Evan Lezar <[email protected]> * docs(mcp): fix granular policy lifecycle examples (#2066) Signed-off-by: Shiju <[email protected]> * feat(kubernetes): add combined topology config surface (#2074) * feat(kubernetes): add combined topology config surface Signed-off-by: Taylor Mutch <[email protected]> * docs(kubernetes): clarify topology defaults Signed-off-by: Taylor Mutch <[email protected]> --------- Signed-off-by: Taylor Mutch <[email protected]> * fix(drivers): reject whitespace in mount fields (#2086) Signed-off-by: Evan Lezar <[email protected]> * refactor(api): remove SandboxTemplate.volume_claim_templates (#2088) The field was added during the Kubernetes driver extraction refactor (#817) as a pass-through mechanism, but was never wired up to a CLI flag, Python SDK helper, or any documentation. The only reachable user path was raw gRPC construction. The Kubernetes driver now always injects the default workspace PVC, removing the branching logic that checked for a user-supplied VCT. Field number 9 is reserved in the proto to prevent reuse. Signed-off-by: Evan Lezar <[email protected]> * feat(helm): add TLS termination for Envoy Gateway ingress (#2015) The chart's optional Gateway API ingress only rendered a plaintext HTTP listener, so the gateway could not be exposed over TLS. Add an HTTPS listener option that terminates TLS at the Envoy Gateway and forwards plaintext gRPC to the gateway pod. - gateway.yaml renders an HTTPS listener with `tls.mode: Terminate` and `certificateRefs` when `grpcRoute.gateway.listener.protocol=HTTPS`, keeping the default HTTP listener unchanged. Guards fail the render when `certificateRefs` is empty or `server.disableTls` is not true (the chart does not render a BackendTLSPolicy for re-encryption). - values.yaml adds `grpcRoute.gateway.listener.tls.certificateRefs`. - ci/values-gateway-tls.yaml exercises the HTTPS branch in lint/render. - docs/kubernetes/ingress.mdx documents HTTPS setup and clarifies that Envoy Gateway only terminates TLS (no OIDC SecurityPolicy); client identity uses OIDC bearer tokens, with the client-credentials grant for headless agents. - debug-openshell-cluster skill gains HTTPS-ingress troubleshooting rows. - Regenerated the chart README values table. Signed-off-by: Huabing (Robin) Zhao <[email protected]> * feat(agents): add manifest-driven gator agent (#1826) * chore(gator): add gator gate skill * chore(gator): add sandbox launcher scaffold * chore(gator): add codex image and docs checks * chore(gator): fold approved provider policy rules * chore(gator): add deterministic reviewer runner * chore(gator): clarify ok-to-test comments * chore(gator): structure launcher harnesses * chore(gator): require e2e for dependabot * chore(gator): add codex refresh profile * chore(gator): wip manifest agent launcher * feat(agents): supervise watch cycles in sandbox * fix(agents): preserve gateway refresh state * fix(gator): continue human response threads * fix(agents): keep watch supervisor retrying * fix(agents): use refreshed Codex credential aliases * fix(gator): avoid misleading gh auth checks * docs(agents): remove architecture build update * fix(gator): use REST-backed GitHub writes * fix(agents): bake immutable agent payloads * fix(agents): upload writable agent workspace * fix(agents): surface gator watch progress * fix(agents): prevent codex stdin hang * fix(agents): align codex subagent input * fix(agents): heartbeat during active cycles * fix(agents): clean up heartbeat sleep * fix(agents): disable gh telemetry in codex harness * fix(agents): reconcile closed gator PRs * fix(agents): query closed gator PR labels separately * fix(agents): tolerate rotated credential placeholders * fix(agents): enforce gator same-sha comment guard Signed-off-by: John Myers <[email protected]> * docs(agents): scope gator trusted commentary Signed-off-by: John Myers <[email protected]> * fix(gator): treat reviewer failures as transient Signed-off-by: Evan Lezar <[email protected]> * feat(agents): refine gator supervised workflow Signed-off-by: John Myers <[email protected]> * fix(agents): stream codex prompts via stdin Signed-off-by: John Myers <[email protected]> * docs(agents): clarify trusted gator responses Signed-off-by: John Myers <[email protected]> * refactor(agents): scope gator PR to scripts Signed-off-by: John Myers <[email protected]> --------- Signed-off-by: John Myers <[email protected]> Signed-off-by: Evan Lezar <[email protected]> Co-authored-by: John Myers <[email protected]> Co-authored-by: Evan Lezar <[email protected]> * feat(docker,podman): add SELinux label support for bind mounts (#2092) * feat(docker,podman): add SELinux label support for bind mounts The Docker Engine structured Mount API does not support SELinux relabelling (:z / :Z). Move user-supplied bind mounts from the structured `mounts` field to the legacy string-format `binds` field, which does support these options. Add a shared `SelinuxLabel` enum (shared/private) to openshell-core so both Docker and Podman drivers accept an optional `selinux_label` field on bind mount configs. For Docker, labels are appended to the bind string; for Podman, they are pushed to the mount options vec. Signed-off-by: Florian Bergmann <[email protected]> * fix(docker): reject missing bind source paths on legacy binds Moving user bind mounts from the structured Mount API to the legacy Binds field changed Docker's behavior for missing source directories: the legacy path silently creates them as empty root-owned dirs instead of erroring. Add an explicit Path::exists() check to preserve the fail-fast behavior operators expect. Signed-off-by: Florian Bergmann <[email protected]> --------- Signed-off-by: Florian Bergmann <[email protected]> * test(e2e): run rootless podman on ubuntu host (#2119) * test(e2e): run rootless podman on ubuntu host Signed-off-by: Evan Lezar <[email protected]> * test(e2e): probe rootless capability behavior Signed-off-by: Evan Lezar <[email protected]> * test(e2e): make capability probe observational Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Evan Lezar <[email protected]> * feat(policy): accept numeric UIDs for sandbox process identity (#1973) * feat(policy): accept numeric UIDs in sandbox process identity validation Allow run_as_user and run_as_group to be either the literal 'sandbox' or a numeric UID/GID within [1000, 2_000_000_000]. This removes the hard dependency on a baked-in 'sandbox' user in container images, enabling compute drivers to inject resolved UIDs at sandbox creation. Phase 1 of #1959. Signed-off-by: Seth Jennings <[email protected]> * feat(supervisor): accept numeric UIDs for process identity dropping Allow run_as_user and run_as_group to be numeric UIDs/GIDs, removing the hard dependency on a baked-in 'sandbox' user in container images. Changes: - validate_sandbox_user(): accepts numeric UIDs without passwd lookup (logs OCSF event); keeps passwd check for "sandbox" name; rejects non-numeric non-sandbox strings that fail passwd lookup - prepare_filesystem(): passes numeric UIDs/GIDs directly to chown() instead of requiring a passwd entry - drop_privileges(): resolves numeric UIDs/GIDs directly via UID::from_raw / Gid::from_raw; skips initgroups when target uid matches current euid; uses guard conditions before setgid/setuid calls - session_user_and_home(): falls back to ("{uid}", "/sandbox") for numeric UIDs, avoiding a passwd lookup that will fail Re-exports MIN_SANDBOX_UID and MAX_SANDBOX_UID from openshell-policy so callers have consistent range constants. Phase 2 of #1959. Signed-off-by: Seth Jennings <[email protected]> * feat(driver-kubernetes): resolve sandbox UID/GID from config or OpenShift SCC annotations Phase 3 of the numeric-UID plan: allow operators to specify explicit sandbox_uid/sandbox_gid in Kubernetes driver config, auto-detect from OpenShift SCC namespace annotations, and propagate resolved values to supervisor container env vars and PVC init container securityContext. Changes: - Add sandbox_uid/sandbox_gid fields to KubernetesComputeConfig - Add SANDBOX_UID/SANDBOX_GID env var constants to openshell-core - Implement resolve_sandbox_identity() to fetch namespace annotations and auto-detect OpenShift SCC UID ranges (sa.scc.uid-range) - Pass resolved UID/GID through SandboxPodParams to pod spec builder - Inject SANDBOX_UID/SANDBOX_GID env vars into supervisor container - Update PVC init container securityContext with resolved UID/GID instead of hard-coded root - Add comprehensive unit tests for resolution logic and annotation parsing (resolve_sandbox_uid, resolve_sandbox_gid, OpenShift SCC annotation parsing) Signed-off-by: Seth Jennings <[email protected]> * feat(driver-vm): add configurable sandbox UID/GID and update docs/examples Phase 4 of the numeric-UID plan: replace hardcoded SANDBOX_UID (10001) in VM rootfs preparation with configurable sandbox_uid/sandbox_gid fields. Changes: - Add sandbox_uid/sandbox_gid to VmDriverConfig with serde derives - Pass resolved UID/GID through prepare_sandbox_rootfs_from_image_root to ensure_sandbox_guest_user which writes /etc/passwd/group/gshadow - Update BYOC Dockerfile: remove groupadd/useradd, document runtime UID injection and the ability to skip baked-in sandbox user - Update gateway-config.mdx: document sandbox_uid/sandbox_gid for both Kubernetes (with OpenShift SCC autodetection) and VM drivers - Update sandbox-compute-drivers.mdx: add Sandbox User Identity section explaining numeric UID support across all compute drivers - Update rootfs tests to use non-default UIDs, verify config passthrough Signed-off-by: Seth Jennings <[email protected]> * code review changes * fix(supervisor): harden tests for restricted CI container environments Guard tests against CI-specific constraints: root without CAP_SETPCAP, UIDs with no /etc/passwd entry, and restricted /proc access. Signed-off-by: Seth Jennings <[email protected]> Signed-off-by: Seth Jennings <[email protected]> --------- Signed-off-by: Seth Jennings <[email protected]> Signed-off-by: Seth Jennings <[email protected]> * docs: add Hermes Agent to supported agents table (#2131) * rfc-0006: add driver config passthrough proposal (#1589) * docs(rfc): add driver config passthrough proposal Signed-off-by: Evan Lezar <[email protected]> * docs(rfc): link driver config proposal PR Signed-off-by: Evan Lezar <[email protected]> * docs(rfc): clarify driver config scope Signed-off-by: Evan Lezar <[email protected]> * docs(rfc): clarify driver-local config schemas Signed-off-by: Evan Lezar <[email protected]> * docs(rfc): clarify driver config extension path * docs(rfc): update driver config baseline * docs(drivers): document bind-mount selinux_label and whitespace rules --------- Signed-off-by: Evan Lezar <[email protected]> * chore(deps): bump docker/login-action from 4.2.0 to 4.4.0 (#2146) * docs: fix STYLEGUIDE heading to match filename (#2134) * docs(kubernetes): bump cert-manager to v1.20.3 (#2129) * docs: fix article before OpenShell in sync-files (#2133) * docs: warn to redact credentials from log output before sharing (#2124) Add a reminder to the bug report template's Logs field and a new row in the security best-practices Common Mistakes table advising reporters to redact credentials, API keys, and tokens from stack traces before pasting. Signed-off-by: Evan Lezar <[email protected]> * fix(podman): deliver sandbox JWTs as secrets (#2156) Signed-off-by: Adam Miller <[email protected]> * chore: remove deprecated --keep flag from docs, scripts, and e2e tests (#2126) * docs: remove deprecated --keep flag from tutorials and examples The --keep flag is deprecated, hidden, and a no-op since sandboxes are kept by default. Remove references from tutorial docs and example READMEs that explain it as a real feature. - Remove --keep from sandbox create commands - Remove --keep explanation text - Clarify that sandboxes are kept by default Signed-off-by: Ignas Baranauskas <[email protected]> * chore: remove deprecated --keep usage from scripts and e2e tests The --keep flag is a deprecated no-op since sandboxes are kept by default. Stop passing it in internal scripts, e2e test scripts, and example demo scripts. Signed-off-by: Ignas Baranauskas <[email protected]> --------- Signed-off-by: Ignas Baranauskas <[email protected]> * chore(deps): bump astral-sh/setup-uv from 8.2.0 to 8.3.0 (#2160) Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.2.0 to 8.3.0. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](https://github.com/astral-sh/setup-uv/compare/fac544c07dec837d0ccb6301d7b5580bf5edae39...d31148d669074a8d0a63714ba94f3201e7020bc3) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 8.3.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(driver-podman): gate Linux-only Path import (#2188) Apply the Linux cfg to the Path import so native macOS lint runs do not report it as unused when the only call site is compiled out. This fixes `mise run rust:lint` on macOS. Signed-off-by: Kris Hicks <[email protected]> * docs: fix Docker version format from 28.04 to 28.0 (#2136) * docs: update man page date to 2026 (#2135) * fix(sandbox): acknowledge initial policy revision; expose SDK labels/selectors (#2170) * fix(sandbox): acknowledge initial policy revision The supervisor loaded and enforced a sandbox-scoped policy but never told the gateway which revision it loaded. The policy poll loop seeded itself with the initial revision's hash on its first poll, so `policy_changed` was never true for that revision and `ReportPolicyStatus(LOADED)` — which only ran in the hot-reload branch — was never called. The revision stayed `Pending` and `current_policy_version` stayed 0 even though the sandbox was `Ready` and the policy was effective. This was most visible with sparse policies that get baseline-enriched into a new revision during startup. After the OPA engine is constructed, report the exact sandbox revision the supervisor loaded as LOADED, and seed the poll loop from that revision so it is not re-reported. Report FAILED with the original construction error if engine construction or conversion fails. Only sandbox-sourced revisions (version > 0) whose canonical content matches the loaded policy are acknowledged; global and local-file policies are untouched. Delivery uses the shared bounded retry, is non-fatal on transient failure, and a pending initial acknowledgement is delivered before any newer revision so policy history is never reordered. Signed-off-by: Kyle Zheng <[email protected]> * feat(python): expose sandbox labels and selectors The gateway protobuf and CLI already support request-level sandbox labels (`CreateSandboxRequest.name`/`labels`) and selector-based listing (`ListSandboxesRequest.label_selector`), but the public Python SDK dropped them, so Python-created sandboxes could not be found via `openshell sandbox list --selector ...`. Add optional, source-compatible `name`/`labels` to `SandboxClient.create`, `create_session`, and the high-level `Sandbox`, and `label_selector` to `list`/`list_ids`. `SandboxRef` now carries the gateway labels as an immutable mapping (default empty, so `SandboxRef(id, name, status)` still works). Caller-provided label mappings are copied. Attaching the high-level `Sandbox` to an existing sandbox rejects `name`/`labels` since creation metadata cannot change on attach. Template labels remain a separate concept. No protobuf changes are required. Signed-off-by: Kyle Zheng <[email protected]> * fix(python): keep SandboxRef hashable and copy high-level labels Excluding the new immutable `labels` field from SandboxRef equality/hash (`compare=False`) preserves the original (id, name, status) identity and keeps the frozen dataclass hashable — a MappingProxyType field would otherwise make `hash(SandboxRef(...))` raise. Also defensively copy caller-provided labels in the high-level `Sandbox` so later caller mutation cannot change what is sent. Signed-off-by: Kyle Zheng <[email protected]> * fix(sandbox): bound initial-policy-ack retries The poll loop retried a pending initial acknowledgement before processing any newer revision, but retried unconditionally forever. A permanently undeliverable ack (e.g. the revision was superseded before it could be reported) would then stall all later policy hot-reloads and provider-env refreshes. Cap the retries; after the bound, give up and resume normal polling so the loop cannot livelock on a stuck acknowledgement. Signed-off-by: Kyle Zheng <[email protected]> * test(sandbox): add sparse-policy revision-2 acknowledgement e2e Regression for #2159: create a sandbox with the network-only policy-advisor fixture, which the supervisor enriches with baseline filesystem paths during startup (creating revision 2, superseding revision 1). Assert the effective policy reaches revision 2 and no revision remains Pending once the supervisor acknowledges the load. Adds SandboxGuard::create_keep_with_args to create a kept sandbox with an initial --policy. Signed-off-by: Kyle Zheng <[email protected]> * fix(ci): correct sandbox checks Signed-off-by: Kyle Zheng <[email protected]> * test(cli): serialize mTLS environment access Signed-off-by: Kyle Zheng <[email protected]> * fix(sandbox): address policy review feedback Signed-off-by: Kyle Zheng <[email protected]> * fix(sandbox): preserve exact policy acknowledgements Signed-off-by: Kyle Zheng <[email protected]> * fix(sandbox): preserve local policy overrides Signed-off-by: John Myers <[email protected]> --------- Signed-off-by: Kyle Zheng <[email protected]> Signed-off-by: John Myers <[email protected]> Co-authored-by: John Myers <[email protected]> * chore(deps): bump astral-sh/setup-uv from 8.3.0 to 8.3.1 (#2191) Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.0 to 8.3.1. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](https://github.com/astral-sh/setup-uv/compare/d31148d669074a8d0a63714ba94f3201e7020bc3...f98e06938123ccabd21905ea5d0069192241f9f1) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 8.3.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat(cli): add --secret-material-env to provider refresh configure (#2178) * feat(cli): add --secret-material-env to provider refresh configure * feat(cli): reject duplicate secret material keys Signed-off-by: Hung Le <[email protected]> --------- Signed-off-by: Hung Le <[email protected]> * docs(telemetry): Added first telemetry report for the community (#2190) * docs(telemetry): add community telemetry reports page Add telemetry/README.md to publish aggregate usage trends every two weeks, and link to it from the Telemetry section of the main README. First report covers the July 8, 2026 window. Signed-off-by: Kirit Thadaka <[email protected]> * docs(telemetry): note telemetry start date (June 1, 2026) Clarify that all-time figures are cumulative from #1433, so readers know when the all-time counts begin. Signed-off-by: Kirit Thadaka <[email protected]> --------- Signed-off-by: Kirit Thadaka <[email protected]> * change packit target to new correct copr project (#2185) Signed-off-by: Adam Miller <[email protected]> * test(supervisor-network): add proxy hostname parser regression tests (#2197) Add regression coverage for parser differentials in the egress proxy's CONNECT hostname handling and OPA wildcard policy matching. Signed-off-by: Shane Utt <[email protected]> * fix(tui): route warning logs to status bar instead of stderr (#2210) * fix(tui): route warning logs to status bar instead of stderr tracing::warn/info/debug calls in the TUI crate write to stderr via the global tracing subscriber. In ratatui's alternate-screen/raw-mode, stderr writes corrupt the terminal layout. Error-state sandboxes amplify this as background gRPC polls fail every 2s tick. Replace all 25 tracing calls with app.status_text assignments for direct-access sites, and Vec<String> accumulation for the spawned start_port_forwards task. Add ForwardWarnings event variant to decouple forward warning delivery from the sandbox name in CreateResult, preventing downstream gRPC lookup failures. Closes #2120 Signed-off-by: Ian Miller <[email protected]> * feat(tui): add ForwardWarnings event variant New event type for non-fatal port-forward warnings during sandbox creation. Keeps warning delivery separate from the sandbox name in CreateResult to avoid corrupting downstream gRPC lookups. Signed-off-by: Ian Miller <[email protected]> * chore(tui): remove tracing dependency Compile-time guard against reintroducing stderr-writing tracing calls in the TUI crate. 14 other crates retain the dependency. Signed-off-by: Ian Miller <[email protected]> --------- Signed-off-by: Ian Miller <[email protected]> * fix(mcp): include tool names in policy logs (#2189) Signed-off-by: Kirit93 <[email protected]> * chore(deps): bump astral-sh/setup-uv from 8.3.1 to 8.3.2 (#2206) Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.1 to 8.3.2. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](https://github.com/astral-sh/setup-uv/compare/f98e06938123ccabd21905ea5d0069192241f9f1...11f9893b081a58869d3b5fccaea48c9e9e46f990) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 8.3.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs(openshift): simplify install steps and add Helm README entries for OpenShift overrides (#2125) Signed-off-by: ChristianZaccaria <[email protected]> * docs(issues): require release and duplicate checks (#2214) Signed-off-by: Evan Lezar <[email protected]> * fix(core): pin supervisor image tag to gateway version for all drivers (#2070) * fix(core): pin supervisor image tag to gateway version for all drivers The Podman and Kubernetes drivers defaulted the supervisor image to `:latest` via DEFAULT_SUPERVISOR_IMAGE, while the Docker driver already resolved a version-pinned tag. Extract the tag resolution logic into openshell-core so all three drivers use the same OPENSHELL_IMAGE_TAG > IMAGE_TAG > CARGO_PKG_VERSION priority chain. Closes #2068 Signed-off-by: Florent Benoit <[email protected]> * refactor(core): simplify supervisor image tag resolver to slice-based API Remove the Docker driver's wrapper functions and call openshell_core::config::default_supervisor_image() directly. Simplify resolve_supervisor_image_tag to accept &[&str] instead of three separate parameters. Signed-off-by: Florent Benoit <[email protected]> Signed-off-by: Florent Benoit <[email protected]> --------- Signed-off-by: Florent Benoit <[email protected]> Signed-off-by: Florent Benoit <[email protected]> * fix(helm): propagate supervisor image overrides (#2216) Signed-off-by: Taylor Mutch <[email protected]> * feat(kubernetes): add sidecar supervisor topology (#2076) * feat(kubernetes): add sidecar supervisor topology Add the Kubernetes sidecar supervisor topology, its Helm/Skaffold configuration, topology documentation, and sidecar e2e matrix coverage. Skip root-only sandbox identity rewriting when process enforcement is network-only so the low-permission sidecar process container can start successfully. Signed-off-by: Taylor Mutch <[email protected]> * fix(supervisor): avoid similar process id names Signed-off-by: Taylor Mutch <[email protected]> * fix(supervisor): avoid similar process id names Signed-off-by: Taylor Mutch <[email protected]> * fix(sandbox): avoid similar proxy id names Signed-off-by: Taylor Mutch <[email protected]> * docs(kubernetes): clarify sidecar topology limits Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): keep sidecar process leaf capless Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): refresh sidecar provider env snapshots Signed-off-by: Taylor Mutch <[email protected]> * test(supervisor): align hot-swap identity regression Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): stage sidecar mtls files before proxy chown Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): simplify sidecar supervisor topology Signed-off-by: Taylor Mutch <[email protected]> * chore(helm): reuse sidecar skaffold values Signed-off-by: Taylor Mutch <[email protected]> * fix(supervisor): avoid similar iptables helper names Signed-off-by: Taylor Mutch <[email protected]> * fix(e2e): harden kube gateway wrapper setup Signed-off-by: Taylor Mutch <[email protected]> * fix(supervisor): avoid nft batch rollback on OCP Run nftables setup as individual commands so optional conntrack and log expressions can fail without rolling back required table, chain, and reject rules. Signed-off-by: Seth Jennings <[email protected]> Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): preserve process identity in sidecar topology Render sidecar pods with a shared process namespace, keep binary-aware network policy enabled, and move Kubernetes sidecar settings under the nested sidecar config table. Also apply unprivileged Landlock/seccomp setup in NetworkOnly supervisor mode so sidecar topology keeps sandbox child hardening without privileged process setup. Signed-off-by: Taylor Mutch <[email protected]> * refactor(kubernetes): replace sidecar snapshots with control socket Coordinate sidecar policy and provider bootstrap over a local Unix socket so the process leaf no longer reads policy/provider snapshot files. Report entrypoint startup through the control channel and keep gateway credentials confined to the network sidecar. Signed-off-by: Taylor Mutch <[email protected]> * feat(kubernetes): support relaxed sidecar network identity Signed-off-by: Taylor Mutch <[email protected]> * fix(sandbox): satisfy sidecar clippy lint Signed-off-by: Taylor Mutch <[email protected]> * refactor(kubernetes): standardize topology naming Signed-off-by: Taylor Mutch <[email protected]> * fix(sandbox): satisfy linux clippy timeout import Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): support kata sidecar on ipv4 pods Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): satisfy linux clippy for sidecar fallback Signed-off-by: Taylor Mutch <[email protected]> * chore(kubernetes): remove stale supervisor topology references Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): enable sidecar binary policy inspection Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): harden sidecar control boundary Signed-off-by: Taylor Mutch <[email protected]> * fix(kubernetes): couple sidecar supervisor lifecycles Signed-off-by: Taylor Mutch <[email protected]> --------- Signed-off-by: Taylor Mutch <[email protected]> Signed-off-by: Seth Jennings <[email protected]> Co-authored-by: Seth Jennings <[email protected]> * feat(kubernetes): support PVC subPath driver config (#2034) * feat(kubernetes): support PVC subPath driver config Signed-off-by: mjamiv <[email protected]> * test(kubernetes): cover writable PVC driver config Signed-off-by: mjamiv <[email protected]> * fix(kubernetes): address PVC subPath review feedback Signed-off-by: mjamiv <[email protected]> * fix(kubernetes): address PVC config review follow-up Signed-off-by: mjamiv <[email protected]> * fix(kubernetes): address PVC review follow-ups --------- Signed-off-by: mjamiv <[email protected]> * fix(network): fail closed when credential placeholders cannot be rewritten (#2162) * fix(network): fail closed when credential placeholders cannot be rewritten When the credential rewriter degrades internally, the proxy forwarded the literal `openshell:resolve:env:<NAME>` placeholder (or its provider alias marker) to the upstream instead of the resolved secret, leaking the reserved token on the wire and causing upstream auth failures (#2161). Two fail-open paths are closed: - secrets: `rewrite_http_header_block` returned the header block verbatim when no `SecretResolver` was available, so the fail-closed marker scan (which ran only on the resolved path) never saw the placeholder. It now scans the header region for reserved markers even with no resolver and returns `UnresolvedPlaceholderError` when one is present. Marker-free traffic still passes through unchanged. - proxy: when TLS was detected on a CONNECT but `tls_state` was `None` (ephemeral CA generation or CA file write failed at startup), the handler fell back to a raw `copy_bidirectional` tunnel, bypassing credential rewrite. Inside the proxy handler `tls_state` is `None` only on CA-init failure (`mode != Proxy` never starts the handler, and `tls: skip` is handled earlier), so it now refuses the connection with a 503 and a High-severity denial event instead of tunneling. The two startup CA-failure logs are raised from Medium to High. Tests: resolver=None with a placeholder in the request line, a header value, and the provider-alias form now fail closed; marker-free passthrough is unchanged; the relay integration test asserts the request is rejected before any byte reaches upstream; and the 503 fail-closed response contract is locked. Signed-off-by: Tony Luo <[email protected]> * fix(network): refuse CONNECT before 200 when TLS termination is unavailable The fail-closed refusal for a terminating CONNECT with no TLS termination state (ephemeral CA init failed) was written after the 200 Connection Established response. Because a CONNECT client only sends its TLS ClientHello after reading the 200, the peek-based TLS detection is inherently post-200, so the 503 landed inside the established tunnel and surfaced to the client as a TLS protocol error rather than a readable status. An 'allowed CONNECT' event was also logged first. Move the decision to a pre-200 gate: query_tls_mode resolves purely from the policy decision + host/port (no peeked bytes), so the route's TLS treatment is known before the tunnel is acknowledged. When TLS state is absent and the route is not tls: skip, write the 503 as the first bytes on the socket, emit the High-severity Denied event, and close. tls: skip routes tunnel raw exactly as before, and no allowed-CONNECT event is emitted on the refusal path. The now-unreachable post-200 branch is kept as defense in depth but no longer writes an in-tunnel 503; it fails closed by dropping the connection instead. Add connection-level regression tests over a real loopback socket: the gate refuses with HTTP/1.1 503 as the first bytes for a terminating route, and writes nothing when TLS termination is present or the route is tls: skip. Signed-off-by: Tony Luo <[email protected]> * fix(network): order the CONNECT TLS-unavailable refusal after SSRF Addresses the gator re-check on #2162. Ordering: the pre-200 fail-closed refusal ran before SSRF/allowed_ips validation, so during CA-init failure an internal-address CONNECT got a 503 tls_termination_unavailable instead of the normal 403 ssrf_denied, weakening operator visibility in degraded state. The SSRF branches now return validated addresses; the refusal runs after that validation (an internal address has already been denied with 403) but still before the upstream connect and before 200 Connection Established. effective_tls_skip is still resolved up front since the refusal consumes it. Tests: add connection-level regressions through the real handle_tcp_connection, driving a CONNECT from a child /bin/bash copy so the /proc process-identity binding resolves it against a permissive policy (the hot-swap test's identity pattern). They assert the first bytes are HTTP/1.1 503 for a terminating route with no TLS state, a 403 (not 503) for an internal address, and no refusal for a tls: skip route. These are gated to Linux at runtime (evaluate_opa_tcp needs /proc); a companion test verifies the OPA policy shape (glob allow, tls mode) on every platform so the precondition is locked where /proc is unavailable. rest.rs: tighten the fail-closed relay test to assert the forwarded buffer is_empty() rather than merely lacking the placeholder/secret. Signed-off-by: Tony Luo <[email protected]> * test(network): keep the CONNECT handler test client fork-free The handler regression tests forked cat to read the proxy reply, so the client socket fd was inherited by a second process with a different binary. The identity resolver correctly denies that as ambiguous shared-socket ownership (the same invariant resolve_process_identity_denies_fork_exec_shared_socket_ambiguity pins), so the tests exercised the deny path instead of the allow path — and on busy CI runners the deny-path /proc fallback scan exceeded the test budget and looked like a hang. The client script now uses only bash builtins (exec, printf, read -d '') so exactly one process owns the socket, and the child is left to exit on EOF instead of being killed mid-read. Signed-off-by: Tony Luo <[email protected]> * test(network): drive the CONNECT handler tests with an in-process client The child-process client (even fork-free) made the handler tests environment-sensitive: on CI runners with a busy or restricted /proc, resolving the child's socket ownership degraded into the whole-/proc fallback scan and a deny, which surfaced as a hang. The client is now an in-process TcpStream and the test policy allows current_exe(), so identity resolution binds the socket to the test process itself in the descendant scan — the same in-process pattern the passing resolve_process_identity tests rely on. The tls: skip test additionally asserts that the handler emitted no DenialEvent at any stage, so it can no longer pass vacuously on a policy or identity deny. Refusal budgets widened to 30s as a belt for slow runners; the refusals themselves return in milliseconds. Signed-off-by: Tony Luo <[email protected]> * test(core): pin the percent-encoded marker no-resolver fail-closed path The no-resolver scan already catches the percent-encoded canonical marker through its decoded pass; this regression pins it: a request line carrying openshell%3Aresolve%3Aenv%3AKEY with no resolver must fail closed with UnresolvedPlaceholderError { location: header }. Signed-off-by: Tony Luo <[email protected]> --------- Signed-off-by: Tony Luo <[email protected]> * fix(server): allow newlines in exec command arguments (#1965) * chore(deps): bump actions/stale from 10.3.0 to 10.4.0 (#2234) * fix(tui): redraw after sandbox shell exits (#2230) * fix(tui): redraw after sandbox shell exits Closes #2229 Render the restored terminal before synchronous gateway refreshes and propagate terminal lifecycle failures from both SSH handoff paths. Signed-off-by: John T. Myers <[email protected]> * fix(tui): restore input after sandbox shell Discard stale events accumulated around the suspended TUI and let the normal periodic tick refresh state after resuming. Signed-off-by: John T. Myers <[email protected]> --------- Signed-off-by: John T. Myers <[email protected]> * fix(agents): add confirmation gate to triage-issue batch mode (#2239) Batch triage previously processed all state:triage-needed issues immediately without any confirmation. This led to accidental mass-commenting (29 triage comments on a public repo) when the skill was invoked by mistake. Add a mandatory preview-and-confirm step: the agent must show the issue count and titles, then ask for explicit user confirmation before posting any comments. Single-issue mode is unchanged. Assisted-By: 🤖 Claude Code Signed-off-by: Roland Huß <[email protected]> * fix(gator): retry review after draft blocker clears (#2200) Signed-off-by: John Myers <[email protected]> Co-authored-by: John Myers <[email protected]> * fix(certgen): stage temp dir inside output dir to fix cross-device rename (#2241) generate-certs stages temp files beside --output-dir using dir.with_file_name(). When --output-dir is a container or WSL volume mount, the staging dir lands on a different filesystem and std::fs::rename fails with EXDEV. Move staging inside the output dir so the rename always stays on the same filesystem, preserving atomic replacement. Fixes #2173 Signed-off-by: Grace Smith <[email protected]> * refactor(jsonrpc): carry typed inspection errors (#2244) Signed-off-by: Shiju <[email protected]> * docs(agents): add gator launch skill (#2203) * docs(agents): add gator launch skill Signed-off-by: John Myers <[email protected]> * docs(agents): narrow gator launch trigger Signed-off-by: John Myers <[email protected]> * docs(agents): trim gator launch skill intro Signed-off-by: John Myers <[email protected]> * docs(agents): harden gator launch examples Signed-off-by: John Myers <[email protected]> * docs(agents): avoid assuming gator gateway Signed-off-by: John Myers <[email protected]> * docs(agents): trim gator launch rules Signed-off-by: John Myers <[email protected]> --------- Signed-off-by: John Myers <[email protected]> Co-authored-by: John Myers <[email protected]> * chore(python): lower minimum supported Python to 3.11 (#2247) Debian 12 stable and other LTS environments ship Python 3.11 as the system interpreter. The SDK is pure Python with a bundled native binary, uses no 3.12-only syntax or stdlib APIs, and all dependencies support 3.11. Lower the floor so `pip install openshell` / `uv add openshell` works on 3.11 (security-supported until Oct 2027). Verified with py_compile on CPython 3.11.15 plus ruff and ty. Signed-off-by: Max Dubrinsky <[email protected]> * fix(policy): keep approved chunk when a mechanistic denial resubmits its endpoint (#2242) * fix(policy): keep approved chunk when a mechanistic denial resubmits its endpoint A mechanistic denial flush for an endpoint already covered by an auto-approved mechanistic chunk flipped that chunk approved -> rejected with no human action. The dedup upsert in put_draft_chunk returns the existing row's id, which aliases onto the approved chunk; the self-reject scan then matched the row against itself and rejected it, while the merged rule stayed enforced — the governance ledger disagreed with the live policy. Guard self_reject_mechanistic_if_already_covered to act only on a still pending effective chunk, and exclude the incoming id from the covering scan. Add a regression test and document the dedup/self-reject invariant. Fixes #2165 Refs NVIDIA/NemoClaw#6329 Signed-off-by: Tinson Lai <[email protected]> * fix(policy): reject mechanistic chunk via atomic pending compare-and-set The self-reject-when-covered path read a chunk, confirmed it was pending, then issued an unconditional status update to rejected. An approval that committed between the read and the write flipped an already-approved chunk to rejected while its rule stayed merged, recreating the ledger/enforcement mismatch through a concurrent path. Add conditionally_reject_draft_chunk to the policy store: the pending->rejected transition carries a status = 'pending' predicate on the final write and reports whether a row changed. Zero changed rows is a benign no-op, meaning another operation already decided the chunk. Implemented for both SQLite and PostgreSQL. The pending pre-read and the self-exclusion guard stay as defense-in-depth. Adds persistence-level regression tests proving an approved row cannot be conditionally rejected and that an approval racing the reject wins. Signed-off-by: Tinson Lai <[email protected]> --------- Signed-off-by: Tinson Lai <[email protected]> * fix(tasks): format all Rust workspaces (#2268) Run cargo fmt for both the root workspace and the standalone e2e Rust workspace from the rust format and format-check tasks. Apply rustfmt to the e2e sources so the expanded formatting check passes. Signed-off-by: Kris Hicks <[email protected]> * docs: fix stray bracket in provider create command example (#2275) Key Commands table showed --type [type]] --from-existing with an extra closing bracket. * rfc-0010: gateway interceptors (#1927) * docs(rfc): add gateway interceptors RFC Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): clarify gateway interceptors proposal Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): clarify interceptor source of truth Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): refine gateway interceptor proposal Signed-off-by: Drew Newberry <[email protected]> * wip * docs(rfc): document interceptor order example Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): renumber gateway interceptors RFC Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): clarify gateway interceptor service Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): clarify gateway interceptor limits Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): align gateway interceptor config Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): refine gateway interceptor contract Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): clarify gateway interceptor payload contract Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): add gateway interceptor describe request Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): remove interceptor modifies flag Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): shape interceptor evaluation by phase Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): simplify interceptor mutation phase Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): update interceptor post-commit failure mode Signed-off-by: Drew Newberry <[email protected]> * docs(rfc): accept gateway interceptors Signed-off-by: Drew Newberry <[email protected]> --------- Signed-off-by: Drew Newberry <[email protected]> * fix(release-dev): update azure/setup-helm to v5.0.1 (#2274) Dependabot updated the Helm setup action in release-canary but missed the same reference in the release composite action. Signed-off-by: Kris Hicks <[email protected]> * feat(snap): vendor ssh in openshell snap and remove ssh-keys interface (#2280) Previously, the openshell snap used the ssh-keys interface to get access to the host's ssh binary, which is used for sandbox connect/exec/forward. However, ssh-keys is a privileged interface which also grants access to the public and private ssh keys on the host. As such, it required manual connection in order to be used. This weakened the security sandbox of the snap, and hurt the UX of installing it. This commit changes this by removing the `ssh-keys` interface and instead vendoring the `ssh` binary within the snap. This is safe because OpenShell always invokes the `ssh` binary with `StrictHostKeyChecking=no`, `UserKnownHostsFile=/dev/null`, and `GlobalKnownHostsFile=/dev/null`, and never uses any host credentials or ssh configuration. Openshell only ever access to `~/.ssh/config` to write OpenShell-managed aliases, and this can safely live within the snap sandbox, rather than leaking into the host environment. Signed-off-by: Oliver Calder <[email protected]> * feat(interceptors): initial gateway interceptor implementation and reference example (#2005) * feat(gateway): add descriptor-driven interceptors Signed-off-by: Drew Newberry <[email protected]> * feat(gateway): add service-reflected interceptors Signed-off-by: Drew Newberry <[email protected]> * wip * fix(gateway): harden interceptor evaluation Signed-off-by: Drew Newberry <[email protected]> * feat(interceptors): label metrics and harden governance smoke Signed-off-by: Drew Newberry <[email protected]> * remove on_error: ignore * feat(gateway-interceptors): emit log annotations Signed-off-by: Drew Newberry <[email protected]> * feat(examples): govern provider profiles in interceptor Signed-off-by: Drew Newberry <[email protected]> * fix(gateway): preserve update config annotations Signed-off-by: Drew Newberry <[email protected]> * feat(providers): support interceptor profile catalogs Signed-off-by: Drew Newberry <[email protected]> * wip * feat(governance-interceptor): sign provider profiles Signed-off-by: Drew Newberry <[email protected]> * fix(providers): use configured profile sources for refresh updates Signed-off-by: Drew Newberry <[email protected]> * feat(gateway-interceptors): add phase-specific evaluation payloads Signed-off-by: Drew Newberry <[email protected]> * feat(providers): compose provider profile sources Signed-off-by: Drew Newberry <[email protected]> * fix(gateway-interceptors): preserve committed responses Signed-off-by: Drew Newberry <[email protected]> * fix(gateway-interceptors): reject ambiguous protobuf oneofs Signed-off-by: Drew Newberry <[email protected]> * fix(gateway-interceptors): validate patch candidates per binding Signed-off-by: Drew Newberry <[email protected]> * refactor(gateway-interceptors): use reflected protobuf codec Signed-off-by: Drew Newberry <[email protected]> * fix(governance-example): canonicalize signed protobuf hashes Signed-off-by: Drew Newberry <[email protected]> * fix(gateway): commit policy provenance atomically Signed-off-by: Drew Newberry <[email protected]> * fix(gateway): close signed governance bypasses Signed-off-by: Drew Newberry <[email protected]> * fix(gateway): isolate interceptor secrets and authority Signed-off-by: Drew Newberry <[email protected]> * fix(gateway): snapshot provider profiles per request Signed-off-by: Drew Newberry <[email protected]> * chore(gateway): resolve server clippy warnings Signed-off-by: Drew Newberry <[email protected]> * fix(server): satisfy provider source clippy lint Signed-off-by: Drew Newberry <[email protected]> * fix(server): initialize policy test annotations Signed-off-by: Drew Newberry <[email protected]> * fix(gateway-interceptors): require explicit route allowlist Signed-off-by: Drew Newberry <[email protected]> * docs(proto): clarify update annotation semantics Signed-off-by: Drew Newberry <[email protected]> --------- Signed-off-by: Drew Newberry <[email protected]> * feat(sdk): add openshell-sdk crate (#1862) * feat(sdk): add openshell-sdk crate Additive extraction of the shared async gRPC client core (transport, TLS, OIDC single-flight refresh, edge tunnel, high-level sandbox surface, raw escape hatch) as a new workspace crate. No existing consumers yet; CLI/TUI migration follows in a separate PR. Signed-off-by: Max Dubrinsky <[email protected]> * refactor(sdk,core): reuse shared JWT exp decoder for refresh deadlines Extract the signature-unverified JWT exp decode out of openshell-core/grpc_client.rs into openshell_core::jwt::parse_exp_secs, and have the openshell-sdk refresh path reuse it to derive a proactive refresh deadline from a bearer JWT when the caller does not advertise expires_at. Addresses review feedback to reuse pre-existing logic rather than reimplement JWT expiry handling per client. Signed-off-by: Max Dubrinsky <[email protected]> * fix(sdk): harden refresh single-flight and redact tokens in Debug Addresses review feedback on the openshell-sdk refresh path. - Make the single-flight cleanup cancellation-safe. The in-flight slot is now cleared by the shared refresh computation itself (epoch-guarded) rather than the leader's post-await code. Previously, if the leader future was dropped (e.g. an FFI caller cancelling its promise) after a follower drove the refresh to completion, the completed future was stranded in the slot and later refresh_now() calls re-joined it, pinning the client to a stale or already-rejected token. Adds a regression test that cancels the leader and asserts the next refresh starts a fresh attempt. - Redact bearer secrets from Debug. RefreshedToken and the oidc RefreshTokenInput/RefreshTokenOutput now use manual Debug impls that omit the access/refresh token fields via finish_non_exhaustive, matching the house style (e.g. SecretResolver, SandboxJwtIssuer). Prevents a stray {:?} or a containing struct's derived Debug from writing tokens to logs. Signed-off-by: Max Dubrinsky <[email protected]> * fix(sdk): fail refresh when the new token can't be encoded as metadata store_bearer now returns an error instead of silently keeping the previous bearer value. The TokenSource commits the refreshed token to its state before the client writes it into the interceptor slot, so a silent drop left the interceptor on the old (expiring) token with no path back to a refresh. Surfacing the error fails the call loudly instead. Adds a unit test covering a token that can't be encoded as gRPC metadata. Signed-off-by: Max Dubrinsky <[email protected]> * fix(sdk): refresh OIDC tokens on raw routes and harden rotation Raw gRPC access never triggered OIDC refresh: a client that only used raw_grpc/raw_inference kept sending the initial bearer until it expired, with no proactive or reactive refresh. Add raw_grpc_fresh and raw_inference_fresh accessors that refresh before returning the client, plus force_refresh for reactive recovery after an Unauthenticated raw RPC. Guard the single-flight refresh commit against a concurrent replace(). The in-flight attempt now records the generation it started from and skips its write when an external replace() has advanced it, so timer or callback driven rotation is no longer clobbered by a slower refresh. Remove TokenSource::snapshot(): it returned an empty string under write contention and had no consumer on the CLI/TUI path. Tests read committed state directly instead. Signed-off-by: Max Dubrinsky <[email protected]> * docs(sdk): rewrite crate README for consumers Recast the openshell-sdk README as a usable crate README rather than an RFC excerpt. Drop the Responsibilities/Non-responsibilities/Consumers scope-boundary sections and the mTLS migration rationale, folding the useful facts (explicit token, no disk/name resolution, Refresh trait, SdkError mapping) into the intro, a new Auth and refresh section, and Public surface. Remove the dead relative RFC link and status-label prose so the doc renders cleanly wherever it is published. Signed-off-by: Max Dubrinsky <[email protected]> * fix(sdk): address review feedback on OIDC refresh and transport Apply Drew's review notes on PR #1862: - Drop unused `rustls-pemfile` dependency and move `tokio-stream` to dev-dependencies (only used by tests). - Guard OIDC `expires_at` against u64 overflow with `saturating_add`. - Fix stale `#[non_exhaustive]` rationale in `AuthConfig` (the struct `Oidc` variant it described as future already ships). - Stop double-wrapping refresh errors: store the bare refresh-error text so the single `SdkError::auth` wrap happens once at await. - Strip stale CLI porting breadcrumbs from `build_channel` docs, keeping the branch table. - Treat proactive token refresh as best-effort: a transient failure falls through to the request instead of failing an RPC whose current token is still valid, with a regression test. - Collapse `exec`'s inline auth retry into the shared `unary` helper. Signed-off-by: Max Dubrinsky <[email protected]> * fix(sdk): preserve transient/terminal distinction in refresh errors The refresh single-flight collapsed both `RefreshError::Transient` and `RefreshError::Terminal` into a stringified `SdkError::Auth`, so consumers (CLI, TUI, future language bindings) had no machine-readable way to tell a retryable IdP blip from a dead session that needs re-authentication. Carry the `RefreshError` through the shared outcome (kept `Clone` for `Shared`) instead of its rendered text, and map it at the await site to a new `retryable` flag on `SdkError::Auth`. Add `SdkError::auth_retryable` and a `SdkError::retryable()` accessor; transient refresh failures report `true`, every other error `false`. Add classification tests. Signed-off-by: Max Dubrinsky <[email protected]> --------- Signed-off-by: Max Dubrinsky <[email protected]> * fix(sdk): initialize sandbox annotations (#2296) The SDK merged after CreateSandboxRequest and ObjectMeta gained annotations on main, leaving stale struct initializers that prevented the crate and its tests from compiling. Initialize the curated request and mock metadata with empty annotation maps. Signed-off-by: Drew Newberry <[email protected]> * fix(driver-vm): run sandbox supervisor as guest pid 1 (#2299) Signed-off-by: Drew Newberry <[email protected]> * feat(ci): introduce merge queue (#2024) * feat(ci): introduce merge queue Closes #1946 Signed-off-by: Evan Lezar <[email protected]> * fix(ci): run GPU E2E for merge groups Signed-off-by: Evan Lezar <[email protected]> * docs(ci): clarify merge queue GPU gate Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Evan Lezar <[email protected]> * feat(gateway): add elevated gateway info (#2202) * feat(cli)!: fold gateway metadata into list BREAKING CHANGE: openshell gateway info no longer shows local gateway registration metadata. Use openshell gateway list or openshell gateway list -o json for local registration details. Signed-off-by: Evan Lezar <[email protected]> * feat(gateway): add elevated gateway info Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Evan Lezar <[email protected]> * fix(ci): prune snap assets from dev release (#2302) Signed-off-by: Piotr Mlocek <[email protected]> * feat!(openshell-cli): remove openshell policy prove command and z3 dependency (#2318) Remove Z3 from the openshell CLI. Proving is handled by the gateway, so bundling the solver in the client duplicates functionality and complicates portable CLI builds and packaging. Signed-off-by: Simon Scatton <[email protected]> * fix(server): persist sandbox labels on create (#2306) Signed-off-by: Matthew Grossman <[email protected]> * fix: remove mentions of bundled-z3 in CI and wheel builds (#2322) * fix(gateway): probe Docker socket during driver auto-detection (#2303) Previously, Docker was auto-detected when the CLI was installed or a candidate Unix socket existed. Neither check verified that the Docker API was responsive. A similar check was done when auto-detecting Podman in the past, but was replaced in 1f07bf04 with a probe of candidate Podman sockets instead. This change applies the functional API probing approach introduced for Podman in 1f07bf04 to Docker. It also makes Docker driver initialization use the same socket-selection mechanism as Docker auto-detection instead of Bollard’s local defaults. This means the previously auto-detectable Docker socket paths $HOME/.docker/run/docker.sock and $XDG_RUNTIME_DIR/docker.sock will actually be usable. When no working compute driver can be auto-detected, the gateway exits early with a message saying as much: > configuration error: no compute driver configured and auto-detection found no > suitable driver; set --drivers or OPENSHELL_DRIVERS to kubernetes, podman, > docker, or vm This makes for a better user experience when installing OpenShell without an available supported compute driver. Signed-off-by: Kris Hicks <[email protected]> * chore(deps): bump actions/setup-node from 6.4.0 to 7.0.0 (#2289) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 6.4.0 to 7.0.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e...820762786026740c76f36085b0efc47a31fe5020) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Evan Lezar <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump softprops/action-gh-release from 3.0.1 to 3.0.2 (#2288) Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 3.0.1 to 3.0.2. - [Release notes](https://github.com/softprops/action-gh-release/releases) - [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md) - [Commits](https://github.com/softprops/action-gh-release/compare/718ea10b132b3b2eba29c1007bb80653f286566b...3d0d9888cb7fd7b750713d6e236d1fcb99157228) --- updated-dependencies: - dependency-name: softprops/action-gh-release dependency-version: 3.0.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat(tui): navigate panels via Up/Down arrow overflow at list boundaries (#2287) * feat(tui): navigate panels via Up/Down arrow overflow at list boundaries When at the bottom of a panel's item list, pressing Down/j now moves focus to the next panel instead of being a silent no-op. Likewise, pressing Up/k at the top moves to the previous panel with the cursor on its last item. Empty panels are skipped and the ring wraps around. Closes #2273 Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> * fix(tui): guard Up handlers against stale cursor in empty panels The Down handlers already check whether the list is non-empty before incrementing the cursor, but the Up handlers only checked cursor > 0. When a list becomes empty after a refresh with a nonzero cursor, Up would decrement the stale cursor instead of overflowing to the previous panel. Add the same non-empty guard to all four Up arms. Signed-off-by: Varsha Prasad Narsing <[email protected]> * docs(sandboxes): add dashboard keyboard navigation to manage-sandboxes Describe Tab/Shift+Tab panel cycling, Up/Down and j/k boundary overflow, and middle-pane tab switching in the OpenShell Terminal section. Signed-off-by: Varsha Prasad Narsing <[email protected]> --------- Signed-off-by: Varsha Prasad Narsing <[email protected]> Signed-off-by: Varsha Prasad Narsing <[email protected]> * feat(providers): AWS STS AssumeRole refresh strategy and aws-s3 profile (#1782) Add gateway-managed AWS STS credential refresh (provider-v2, #1576). The gateway calls sts:AssumeRole and writes three short-lived credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN) to the provider record; the proxy re-signs requests with SigV4. Adds the aws and aws-s3 provider profiles and a declarative multi-output refresh model (additional_outputs) so one AssumeRole co-mints all three credentials. Signed-off-by: Russell Bryant <[email protected]> * rfc-0009: supervisor middleware (#1738) Signed-off-by: Piotr Mlocek <[email protected]> * test(e2e): run VM suite in CI (#2305) * test(e2e): run VM suite in CI Signed-off-by: Drew Newberry <[email protected]> * fix(ci): configure KVM permissions directly Signed-off-by: Drew Newberry <[email protected]> * fix(e2e): flush VM overlay before restart Signed-off-by: Drew Newberry <[email protected]> * docs: simplify VM test documentation Signed-off-by: Drew Newberry <[email protected]> * test(e2e): include gateway resume in VM run Signed-off-by: Drew Newberry <[email protected]> --------- Signed-off-by: Drew Newberry <[email protected]> * fix(vm-driver): fixes BYOC sandbox creation failing with ext4-fs write access unavailable (#2150) * feat(supervisor-middleware): add network egress middleware (#2027) Signed-off-by: Piotr Mlocek <[email protected]> * docs(gator): require inline review comments (#2346) Signed-off-by: John Myers <[email protected]> Co-authored-by: John Myers <[email protected]> * fix(cli): preserve symlinks in sandbox upload (#2319) Signed-off-by: lr90 <[email protected]> * fix(kubernetes): validate sandbox names against RFC 1123 requirements (#2295) Signed-off-by: Krzysztof Malczuk <[email protected]> * docs: bump stated Rust MSRV from 1.88 to 1.90 (#2276) * docs: bump stated Rust MSRV from 1.88 to 1.90 Cargo.toml sets rust-version = "1.90" (rust-toolchain.toml pins 1.95.0), so building with the previously documented 1.88 fails Cargo's MSRV check. * docs: bump e2e/rust MSRV to 1.90 * fix: align remaining Rust version fields to 1.90 examples/governance-interceptor/Cargo.toml still had rust-version 1.88. Also bump e2e/rust's prost dependency to 0.14 to match the workspace, since it was on 0.13 in an otherwise standalone crate. * ci: pin docker actions to commit SHA (#2328) login-action and setup-buildx-action used a mutable version tag while every other action in the repo is pinned to a commit SHA. Pin both, and align login-action to the same v4 SHA already used in ci-image.yml. Use the full resolved version in the trailing comment (v3.12.0) to match the more common convention used elsewhere in .github/. * ci(e2e): reuse prebuilt CLI and gateway artifacts (#2311) * ci(e2e): reuse prebuilt CLI artifacts Signed-off-by: Evan Lezar <[email protected]> * ci(e2e): reuse prebuilt gateway artifacts Signed-off-by: Evan Lezar <[email protected]> * ci(e2e): reuse prebuilt VM driver artifact Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Evan Lezar <[email protected]> * docs: fix broken links and small inconsistencies (#2329) - README: fix github-sandbox tutorial link missing get-started segment - README: replace dead community-sandboxes doc link with the actual repo - README: match supported host list to support-matrix.mdx - architecture/README: list the missing google-vertex-ai-provider doc - SECU…
Summary
Add lightweight, opt-out telemetry for anonymous OpenShell usage metrics. This gives the project aggregate visibility into sandbox lifecycle activity, policy updates, provider operations, sandbox creation shape, and sandbox network-denial summaries.
Telemetry does not track individual users and does not collect personal data, sandbox IDs, sandbox names, hosts, paths, model names, provider names, credentials, prompts, request payloads, or user content.
The telemetry events captured are:
Sandbox creation: records whether a sandbox create request succeeded or failed. It also records whether GPU was requested as true/false, the number of providers attached as a count only, whether a custom policy was provided as true/false with no policy details collected, whether the template source was default or image-based without collecting the image name, and which compute driver was used (docker, kubernetes, podman, vm, or unknown).
Sandbox deletion: records whether a sandbox delete request succeeded or failed.
Sandbox policy updates: records whether a gateway-level sandbox policy update succeeded or failed. Policy contents and rule details are not collected.
Provider lifecycle: records provider create, update, and delete outcomes. Provider type is mapped to a broad profile bucket such as anthropic, claude, codex, github, gitlab, nvidia, openai, outlook, or custom. Provider names, credentials, endpoints, and configuration details are not collected.
Policy draft decisions: records whether approve, reject, approve-all, or undo actions succeeded or failed. It also records the number of rules affected as a count only. Rule contents are not collected.
Sandbox network activity summaries: records aggregate network activity counts, denied action counts, denial rate percentage, and denial counts grouped into coarse categories such as connect policy, forward policy, L7 policy, SSRF, bypass, stale policy, or unknown. Destinations, hostnames, URLs, request paths, payloads, binaries, and raw deny messages are not collected.
Telemetry can be disabled at deployment time by setting OPENSHELL_TELEMETRY_ENABLED=false on the gateway.
Related Issue
Fixes #1054
Testing
mise run pre-commitpassesChecklist