Skip to content

Commit 30ef312

Browse files
committed
chore: merge main into UI policy branch
Signed-off-by: Shailendra Singh <[email protected]>
2 parents 00382fa + b799fcc commit 30ef312

100 files changed

Lines changed: 10831 additions & 2167 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/helm-dev-environment/SKILL.md‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -256,15 +256,19 @@ Key Helm values:
256256

257257
### Keycloak OIDC
258258

259-
One-time setup — only needed once per cluster lifetime:
259+
Initial setup — rerun it whenever you want to rotate the development CA:
260260

261261
```bash
262262
mise run keycloak:k8s:setup
263263
```
264264

265265
This deploys Keycloak (`quay.io/keycloak/keycloak:24.0`) into the `keycloak` namespace,
266-
imports the openshell realm from `scripts/keycloak-realm.json`, and prints a port-forward
267-
command for acquiring tokens from the CLI.
266+
imports the openshell realm from `scripts/keycloak-realm.json`, generates a short-lived
267+
development TLS certificate, and publishes its trust anchor as the
268+
`openshell-keycloak-ca` ConfigMap in the OpenShell namespace. The command prints a
269+
port-forward command for acquiring tokens from the CLI. Rerunning setup rotates the
270+
development certificate and trust anchor; redeploy the gateway afterward so it reloads
271+
the mounted CA bundle.
268272

269273
Then activate OIDC in the OpenShell Helm chart:
270274
1. Uncomment `#- ci/values-keycloak.yaml` in `skaffold.yaml`
@@ -355,4 +359,4 @@ for dependencies still declared in `Chart.yaml`.
355359
| `deploy/helm/openshell/ci/values-tls-disabled.yaml` | Lint-only: TLS + auth disabled (reverse-proxy edge termination) |
356360
| `deploy/kube/manifests/envoy-gateway-openshell.yaml` | GatewayClass for Envoy Gateway (`mise run helm:gateway:apply`) |
357361
| `tasks/scripts/helm-k3s-local.sh` | k3d cluster create/delete/start/stop/status |
358-
| `tasks/scripts/keycloak-k8s-setup.sh` | Keycloak deploy + realm import |
362+
| `tasks/scripts/keycloak-k8s-setup.sh` | Keycloak deploy, realm import, and development TLS trust anchor |

‎.github/workflows/conformance.yml‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ jobs:
150150
name: openshell-conformance-x86_64-unknown-linux-musl
151151
path: conformance-input
152152

153-
- name: Run RPM gateway continuity conformance
153+
- name: Run RPM smoke conformance
154154
shell: bash
155155
run: |
156156
set -euo pipefail
@@ -169,11 +169,9 @@ jobs:
169169
--distro fedora \
170170
--with podman-rootless \
171171
--with selinux \
172-
--copy "${candidate_cli_package[0]}:/var/lib/openshell-conformance/candidate/openshell.rpm" \
173-
--copy "${candidate_gateway_package[0]}:/var/lib/openshell-conformance/candidate/openshell-gateway.rpm" \
172+
--install "${candidate_cli_package[0]}" \
173+
--install "${candidate_gateway_package[0]}" \
174174
--copy conformance-input/openshell-conformance:/tmp/openshell-conformance \
175-
--copy nix/test-guest/conformance-plans/gateway-upgrade-restart.toml:/tmp/conformance-plan.toml \
176-
--provision openshell-rpm-latest-release \
175+
--provision openshell-rpm \
177176
--provision gateway-podman \
178-
--provision openshell-rpm-gateway-upgrade \
179-
-- /tmp/openshell-conformance run --plan /tmp/conformance-plan.toml
177+
-- /tmp/openshell-conformance run smoke

‎Cargo.lock‎

Lines changed: 2 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎architecture/build.md‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -292,12 +292,9 @@ pulls and explicit publication. OCI pulls require a trusted manifest digest
292292
and retain that provenance with the local entry; mutable tags are used only
293293
for explicit publication.
294294

295-
CLI conformance runs after target provisioning. Action-free scenarios operate
296-
only through the configured OpenShell CLI. A versioned conformance plan may add
297-
an ordered sequence of target-supplied host-side actions, such as a gateway
298-
restart, while the scenario remains responsible for black-box sandbox
299-
continuity checks. The plan exposes opaque executable paths and timeouts rather
300-
than driver or package-manager configuration; target setup owns those details.
295+
CLI conformance runs after target provisioning and operates only through the
296+
configured OpenShell CLI. The smoke scenario verifies the black-box sandbox
297+
lifecycle by creating, inspecting, executing in, and deleting a sandbox.
301298

302299
## Python Wheel Packaging
303300

‎architecture/gateway.md‎

Lines changed: 6 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -215,7 +215,7 @@ Supported auth modes:
215215
| Plaintext | Local development or a trusted reverse proxy boundary. |
216216
| Unauthenticated local users | Trusted Kubernetes dev or fully trusted proxy deployments only. |
217217
| Cloudflare JWT | Edge-authenticated deployments where Cloudflare Access supplies identity. |
218-
| OIDC | Bearer-token auth for users, with browser or device-code PKCE and client credentials login. JWKS validation accepts RS256, RS384, RS512, PS256, PS384, PS512, ES256, ES384, and EdDSA (Ed25519) signing keys. |
218+
| OIDC | Bearer-token auth for users, with browser or device-code PKCE and client credentials login. Discovery and JWKS retrieval require HTTPS, reject redirects, and pin JWKS to the issuer origin or an explicit origin allowlist. JWKS validation accepts RS256, RS384, RS512, PS256, PS384, PS512, ES256, ES384, and EdDSA (Ed25519) signing keys. |
219219

220220
The CLI persists the scopes requested during OIDC login in gateway metadata and
221221
reuses them when refreshing an access token. This preserves the intended API
@@ -334,16 +334,10 @@ The storage schema is intentionally narrow:
334334

335335
### Protobuf API and storage boundaries
336336

337-
Public RPC contracts and durable protobuf formats have separate ownership. The
338-
`openshell.v1.OpenShell` service currently has 74 RPCs. Their request and
339-
response roots, streaming flags, and transitive message closure come from the
340-
public descriptor set generated by `openshell-core`; a fingerprint test in
341-
`openshell-server` requires this inventory to be reviewed whenever it changes.
342-
Compute-driver, credential-driver, gateway-interceptor, and
343-
supervisor-middleware services are compiled contracts for internal extension
344-
boundaries, not public gateway RPCs. The current public inventory has 74
345-
methods, 279 messages, and 13 enums
346-
(`eec8e3e8c45cb7e17944777887b3c6e1d0ae897263b6994082acd19e839fc3bf`).
337+
Public RPC contracts and durable protobuf formats have separate ownership. The `openshell.v1.OpenShell` service's request and response roots, streaming flags, and transitive message closure come from the public descriptor set generated by `openshell-core`. The `public_and_durable_schema_inventories_are_complete` test in `openshell-server` owns the counts and fingerprints and requires this inventory to be reviewed whenever it changes. Compute-driver, credential-driver, gateway-interceptor, and supervisor-middleware services are compiled contracts for internal extension boundaries.
338+
339+
`ReportEndpointStatus` is a sandbox-authenticated public gateway RPC. Its request, response, and `EndpointObservation` messages belong only to the public closure. `EndpointStatus` and `EndpointResult` also belong to the durable closure because `Sandbox.status.endpoint_statuses` persists them. The repeated status field uses a new wire tag; stored sandboxes without it decode with an empty endpoint list and retain their lifecycle fields. A fixed payload encoded with the earlier sandbox schema verifies that no database rewrite is required.
340+
347341
The removed `NetworkBinary.harness` field remains reserved by number and name,
348342
so protobuf implementations cannot reuse its wire slot or source identifier.
349343
The durable-policy compatibility decoder reads the former boolean before Prost
@@ -363,12 +357,7 @@ Go, Python, and TypeScript client generation inputs do not advertise them.
363357
| Public messages used directly as encoded storage roots | `Sandbox`, `SandboxWorkloadTemplate`, `Provider`, `Workspace`, `WorkspaceMember`, `SshSession`, `ServiceEndpoint` | The generated public type is also the persisted payload. `SshSession` is not in the current public RPC message closure. |
364358
| Embedded encoded root | `SandboxPolicy` | Stored in policy rows and inside the JSON settings envelope. |
365359

366-
The 12 encoded durable roots above have a closure of 82 messages and nine
367-
enums (`69fdc72cc033f37704e8aa4b089dc799990b52cb9cdd5c13decef67fb662177d`).
368-
Its intersection with the public RPC closure contains 72 messages and nine
369-
enums (`9d5c40ec9820139c2fe0c9a9ce2098289c0b5c63c3e3ad6d16de558d504cdecc`).
370-
The descriptor-derived test owns these full inventories; the tables here record
371-
the reviewed roots and classifications.
360+
The descriptor-derived test inventories the complete message and enum closure of the encoded durable roots and its intersection with the public RPC closure. The tables here record the reviewed roots and classifications.
372361

373362
| Dual-purpose encoded root | Current decision |
374363
|---|---|

‎architecture/sandbox.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -361,6 +361,26 @@ auth file without the acknowledgement, and the acknowledgement without an
361361
auth file, so credentials are never sent in cleartext without an explicit
362362
operator decision.
363363

364+
## Tool server connection status
365+
366+
A sandbox can be `Ready` while a call to an external tool server fails. For configured endpoints that use MCP over HTTP, OpenShell records the last observed network result beside the endpoint's address. Users can identify the server and distinguish policy denial, unavailable credentials, TLS or network failure, and an upstream HTTP rejection without combining client and supervisor logs. These observations do not affect sandbox lifecycle readiness.
367+
368+
```mermaid
369+
flowchart LR
370+
Traffic[Calls to configured tool servers] --> Observer[Observe network result]
371+
Observer --> Reporter[Background reporter]
372+
Reporter --> Gateway[Validate and store results]
373+
Gateway --> Status[Endpoint address, last result, report time]
374+
```
375+
376+
The gateway exposes one record per configured endpoint in `Sandbox.status.endpoint_statuses`. Each record contains its address, an opaque identifier, a typed result, and the time the gateway accepted that result. The address remains available when evidence resets to `NoObservedExchange`. Ordinary sandbox conditions continue to describe lifecycle and platform state.
377+
378+
Network observers send only the endpoint identifier and a fixed result classification, without credentials, payloads, or raw upstream errors. Requests capture observation authority before selecting policy or credentials, then bind the selected policy hash and provider revision to that capture. Observation handles also identify the installed endpoint inventory and supervisor authority, so a concurrent update cannot attribute an old request to a new configuration, and reinstalling the same configuration cannot revive an obsolete request. The sandbox reporter coalesces observations and retries an immutable snapshot through `ReportEndpointStatus`. Bounded delivery can drop observations, so endpoint status is not a complete request history.
379+
380+
The gateway validates the reporting supervisor's session, configuration revisions, and report sequence before atomically storing results. Global policy writes share the report's synchronization boundary. An effective policy change resets all endpoint evidence; repeated acknowledgements and metadata-only policy revisions preserve it. A provider environment change resets evidence for endpoints that depend on those credentials. Supervisor disconnection or replacement and gateway restart also invalidate observation authority. Identical report retries leave timestamps unchanged. After an inventory reset, still-valid pending evidence can be accepted again if an acknowledgement was lost, advancing the report time without another exchange.
381+
382+
These are passive observations with no expiry. `HttpResponseReceived` means the server returned a final HTTP status below 400, including a protocol upgrade; that response can still contain an MCP error. Informational responses alone do not establish success. MCP protocol-version and request-body policy rejections produce `PolicyDenied`. A failure before the HTTP path is known updates status only when the host and port identify one distinct endpoint. Ambiguous failures remain in structured events and logs. Results combine callers and effective ports for an endpoint. Consumers that need current tool availability must verify an actual operation. The sandbox management guide explains the public fields and results.
383+
364384
## Credentials
365385

366386
Provider credentials are stored at the gateway and fetched by the supervisor at

0 commit comments

Comments
 (0)