You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 6c2cafa
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: .agents/skills/debug-openshell-cluster/SKILL.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -182,7 +182,7 @@ Expected port: `30051/tcp` (mapped to configurable host port, default 8080; set
182
182
183
183
Component images (server, sandbox) can reach kubelet via two paths:
184
184
185
-
**Local/external pull mode** (default local via `mise run cluster`): Local images are tagged to the configured local registry base (default `127.0.0.1:5000/openshell/*`), pushed to that registry, and pulled by k3s via `registries.yaml` mirror endpoint (typically `host.docker.internal:5000`). The `cluster` task pushes prebuilt local tags (`openshell/*:dev`, falling back to `localhost:5000/openshell/*:dev` or `127.0.0.1:5000/openshell/*:dev`).
185
+
**Local/external pull mode** (default local via `mise run cluster`): Local images are tagged to the configured local registry base (default `127.0.0.1:5000/openshell/*`), pushed to that registry, and pulled by k3s via `registries.yaml` mirror endpoint (typically `host.docker.internal:5000`). The `cluster` task rebuilds the local gateway image before tagging and pushing it, so a fresh bootstrap should not reuse stale `openshell/gateway:dev` bits from a previous source revision.
186
186
187
187
Gateway image builds now stage a partial Rust workspace from `deploy/docker/Dockerfile.images`. If cargo fails with a missing manifest under `/build/crates/...`, or an imported symbol exists locally but is missing in the image build, verify that every current gateway dependency crate (including `openshell-driver-kubernetes` and `openshell-ocsf`) is copied into the staged workspace there.
@@ -129,17 +129,17 @@ The first command installs the CLI. The second command bootstraps the cluster on
129
129
130
130
For more detail, see [Cluster Bootstrap Architecture](cluster-single-node.md).
131
131
132
-
### Sandbox Connect (SSH Tunneling)
132
+
### Sandbox Connect (SSH Forwarding)
133
133
134
134
Users can open interactive terminal sessions into running sandboxes. SSH traffic is tunneled through the gateway rather than exposing sandbox pods directly on the network.
135
135
136
136
The connection flow works as follows:
137
137
138
138
1. The CLI requests a session token from the gateway.
139
-
2. The CLI opens an HTTP CONNECT tunnel to the gateway's SSH tunnel endpoint, passing the token and sandbox identifier.
140
-
3. The gateway validates the token, confirms the sandbox is running, resolves the pod's network address, and establishes a TCP connection to the sandbox's embedded SSH server.
141
-
4.A cryptographic handshake (HMAC-verified) confirms the gateway's identity to the sandbox.
142
-
5. The CLI and sandbox exchange SSH traffic bidirectionally through the tunnel.
139
+
2. The CLI opens a bidirectional gRPC `ForwardTcp` stream with `target.ssh`, passing the token and sandbox identifier.
140
+
3. The gateway validates the token, confirms the sandbox is ready, and asks the already-connected supervisor to open an SSH-targeted relay.
141
+
4.The supervisor connects the relay to the sandbox's embedded SSH server over the local Unix socket.
142
+
5. The CLI and sandbox exchange SSH traffic bidirectionally through the gRPC stream and supervisor relay.
Copy file name to clipboardExpand all lines: architecture/gateway-security.md
+11-10Lines changed: 11 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -234,16 +234,17 @@ The sandbox calls two RPCs over this authenticated channel:
234
234
-`GetSandboxSettings` -- fetches the YAML policy that governs the sandbox's behavior.
235
235
-`GetSandboxProviderEnvironment` -- fetches provider credentials as environment variables.
236
236
237
-
## SSH Tunnel Authentication
237
+
## SSH Forward Authentication
238
238
239
-
SSH connections into sandboxes pass through the gateway's HTTP CONNECT tunnel at `/connect/ssh`. This adds a second authentication layer on top of mTLS.
239
+
SSH connections into sandboxes pass through the gateway's bidirectional gRPC `ForwardTcp` stream with `target.ssh`. This adds a second authorization layer on top of gateway mTLS.
240
240
241
-
### Request Headers
241
+
### Forward Initialization
242
242
243
-
|Header| Purpose |
243
+
|Field| Purpose |
244
244
|---|---|
245
-
|`x-sandbox-id`| Identifies the target sandbox |
246
-
|`x-sandbox-token`| Session token (created via `CreateSshSession` RPC) |
245
+
|`sandbox_id`| Identifies the target sandbox |
246
+
|`target.ssh`| Requests the built-in SSH Unix-socket target |
247
+
|`authorization_token`| Session token created via `CreateSshSession`|
247
248
248
249
The gateway validates the token against the stored `SshSession` record and checks:
249
250
@@ -269,16 +270,16 @@ The gateway enforces two concurrent connection limits to bound the impact of cre
269
270
| Per-token | 10 concurrent tunnels | Limits damage from a single leaked token |
270
271
| Per-sandbox | 20 concurrent tunnels | Prevents bypass via creating many tokens for one sandbox |
271
272
272
-
These limits are tracked in-memory and decremented when tunnels close. Exceeding either limit returns HTTP 429 (Too Many Requests).
273
+
These limits are tracked in-memory and decremented when streams close. Exceeding either limit returns gRPC `ResourceExhausted`.
273
274
274
275
### Supervisor-Initiated Relay Model
275
276
276
-
The gateway never dials the sandbox. Instead, the sandbox supervisor opens an outbound `ConnectSupervisor` bidirectional gRPC stream to the gateway on startup and keeps it alive for the sandbox lifetime. SSH traffic for `/connect/ssh` (and exec traffic for `ExecSandbox`) rides this same TCP+TLS+HTTP/2 connection as separate multiplexed HTTP/2 streams. The gateway-side registry and `RelayStream` handler live in `crates/openshell-server/src/supervisor_session.rs`; the supervisor-side bridge lives in `crates/openshell-sandbox/src/supervisor_session.rs`.
277
+
The gateway never dials the sandbox. Instead, the sandbox supervisor opens an outbound `ConnectSupervisor` bidirectional gRPC stream to the gateway on startup and keeps it alive for the sandbox lifetime. SSH traffic for `ForwardTcp(target.ssh)`and exec traffic for `ExecSandbox` ride this same TCP+TLS+HTTP/2 connection as separate multiplexed HTTP/2 streams. The gateway-side registry and `RelayStream` handler live in `crates/openshell-server/src/supervisor_session.rs`; the supervisor-side bridge lives in `crates/openshell-sandbox/src/supervisor_session.rs`.
277
278
278
279
Per-connection flow:
279
280
280
-
1. CLI presents `x-sandbox-id` + `x-sandbox-token` at `/connect/ssh` and passes gateway token validation.
281
-
2. Gateway calls `SupervisorSessionRegistry::open_relay(sandbox_id, ...)`, which allocates a `channel_id` (UUID) and sends a `RelayOpen` message to the supervisor over the already-established `ConnectSupervisor` stream. If no session is registered yet, it polls with exponential backoff up to a bounded timeout (30 s for `/connect/ssh`, 15 s for `ExecSandbox`).
281
+
1. CLI opens `ForwardTcp` with `TcpForwardInit { sandbox_id, target.ssh, authorization_token }` and passes gateway token validation.
282
+
2. Gateway calls `SupervisorSessionRegistry::open_relay_with_target(sandbox_id, SshRelayTarget, ...)`, which allocates a `channel_id` (UUID) and sends a `RelayOpen` message to the supervisor over the already-established `ConnectSupervisor` stream. If no session is registered yet, it polls with exponential backoff up to a bounded timeout.
282
283
3. The supervisor opens a new `RelayStream` RPC on the same `Channel` — a new HTTP/2 stream, no new TCP connection and no new TLS handshake. The first `RelayFrame` is a `RelayInit { channel_id }` that claims the pending slot on the gateway.
283
284
4.`claim_relay` pairs the gateway-side waiter with the supervisor-side RPC via a `tokio::io::duplex(64 KiB)` pair. Subsequent `RelayFrame::data` frames carry raw SSH bytes in both directions. The supervisor is a dumb byte bridge: it has no protocol awareness of the SSH bytes flowing through.
284
285
5. Inside the sandbox pod, the supervisor connects the relay to sshd over a Unix domain socket at `/run/openshell/ssh.sock` (see `crates/openshell-driver-kubernetes/src/main.rs`).
0 commit comments