Skip to content

Commit d25036a

Browse files
committed
docs(container-gateway): fix Docker driver setup for containerized gateway
The existing docs omitted or misstated several requirements when running the gateway as a container with the Docker compute driver: 1. OPENSHELL_GRPC_ENDPOINT is required. The Docker driver rejects startup if this env var is missing, but it was not mentioned. 2. The supervisor binary must be extracted to a host path before starting the gateway. The gateway validates the path at startup from inside the container, and the host Docker daemon uses the same path as a bind-mount source when creating sandbox containers. Extracting to a path inside the gateway container alone is insufficient. 3. Docker socket access requires adding the docker group. The gateway image runs as nvs:nvs (UID 1000) which does not have access to the Docker socket by default. 4. Port binding should remain 127.0.0.1. The Docker driver automatically binds the gateway to the bridge network interface (gateway_bind_addresses in the driver) so sandbox containers can reach it without exposing the port on 0.0.0.0. 5. The mTLS setup section was missing --server-san host.openshell.internal on generate-certs. Sandbox containers resolve host.openshell.internal to reach the gateway, so this SAN must be present in the server cert. The mTLS docker run was also missing --group-add docker, the supervisor binary mount, OPENSHELL_GRPC_ENDPOINT, and OPENSHELL_DOCKER_SUPERVISOR_BIN. Validated by deploying OpenShell on a Fedora Kinoite (bootc) system using the updated compose.yml.
1 parent 7873f61 commit d25036a

5 files changed

Lines changed: 470 additions & 6 deletions

File tree

‎deploy/docker/docker-compose.yml‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
# OpenShell gateway — docker-compose setup (Docker compute driver)
5+
#
6+
# Prerequisites:
7+
# - Docker Desktop (Windows / macOS) or Docker Engine + Compose plugin (Linux)
8+
# - The openshell CLI installed on your workstation
9+
#
10+
# Quick start:
11+
#
12+
# 1. Start the gateway:
13+
# docker compose up -d
14+
#
15+
# 2. Register the gateway with the CLI (one-time):
16+
# openshell gateway add http://localhost:8080 --name openshell-docker
17+
#
18+
# 3. Configure an AI provider (example: Anthropic):
19+
# ANTHROPIC_API_KEY=sk-ant-... \
20+
# openshell provider create --type anthropic --from-existing
21+
#
22+
# 4. Create a sandboxed agent — Claude Code or OpenClaw:
23+
# openshell sandbox create -- claude
24+
# openshell sandbox create --from openclaw
25+
#
26+
# Sandbox containers are managed by the gateway, not by this Compose file.
27+
# Each `openshell sandbox create` call launches a fresh container; the gateway
28+
# tracks their lifecycle.
29+
#
30+
# Configuration:
31+
# All gateway and driver settings live in gateway.toml in this directory.
32+
# Three values cannot be expressed in the TOML file and remain as env vars:
33+
# - OPENSHELL_DB_URL (explicitly blocked from the config file to prevent
34+
# secrets from being committed to VCS)
35+
# - XDG_DATA_HOME / HOME (OS-level path-resolution vars outside the
36+
# gateway config schema)
37+
#
38+
# command: [] note:
39+
# The gateway image's default CMD is ["--bind-address", "0.0.0.0", "--port",
40+
# "8080"]. CLI flags beat TOML in the merge order, so without clearing the
41+
# CMD the TOML's bind_address = "127.0.0.1:8080" is silently ignored and the
42+
# gateway binds 0.0.0.0. Setting command: [] lets the TOML file own all
43+
# gateway settings.
44+
#
45+
# Data directory note:
46+
# /var/lib/openshell is bind-mounted at the SAME absolute path in both the
47+
# host and the container. This is required so that the supervisor binary
48+
# extracted from the supervisor image can be passed to Docker as a host-side
49+
# bind-mount source when sandbox containers are created. Named volumes
50+
# cannot be used here because Docker resolves bind-mount sources against the
51+
# host filesystem, not the container filesystem.
52+
#
53+
# Linux note:
54+
# host.docker.internal and host.openshell.internal are not automatically
55+
# added on Linux Docker. Add the following under the gateway service:
56+
# extra_hosts:
57+
# - "host.docker.internal:host-gateway"
58+
# - "host.openshell.internal:host-gateway"
59+
60+
services:
61+
gateway:
62+
image: ghcr.io/nvidia/openshell/gateway:${IMAGE_TAG:-latest}
63+
restart: unless-stopped
64+
65+
# Clear the default CMD so gateway.toml owns all settings (see note above).
66+
command: []
67+
68+
# This setup is Docker-outside-of-Docker (DooD), not Docker-in-Docker (DinD).
69+
# The gateway uses the host's Docker socket to create sibling containers on the
70+
# host, rather than running a nested Docker daemon. DooD does NOT require
71+
# --privileged; it only needs read/write access to /var/run/docker.sock.
72+
#
73+
# Run as UID 0 so the gateway can:
74+
# - write the extracted supervisor binary to /var/lib/openshell
75+
# - access /var/run/docker.sock (typically owned by root or the docker group)
76+
# Distroless images have no /etc/passwd, so the numeric UID must be used.
77+
# This is appropriate for local development. Production deployments
78+
# should use a dedicated non-root UID with explicit docker-group membership.
79+
user: "0"
80+
81+
ports:
82+
# gRPC / control-plane API (used by the openshell CLI and sandbox callbacks)
83+
# The Docker driver injects host.openshell.internal:<gateway-port> into sandbox
84+
# containers as the callback endpoint. The gateway's internal port is 8080, so
85+
# host port 8080 must be published at the same number so that
86+
# host.openshell.internal:8080 routes to the gateway container.
87+
# gateway.toml binds to 127.0.0.1 — the Docker driver adds the bridge listener
88+
# automatically so sandbox containers can reach the gateway without 0.0.0.0.
89+
- "127.0.0.1:${OPENSHELL_PORT:-8080}:8080"
90+
# Health endpoint (GET /healthz, GET /readyz)
91+
- "127.0.0.1:${OPENSHELL_HEALTH_PORT:-8081}:8081"
92+
93+
volumes:
94+
# Docker socket — lets the gateway create and manage sandbox containers.
95+
- /var/run/docker.sock:/var/run/docker.sock
96+
97+
# Data directory — must be a bind-mount with source == target so that
98+
# paths written inside the container are resolvable by Docker when it
99+
# creates sandbox containers (see note above).
100+
# /var/lib/openshell is intentionally not namespaced to a sub-path
101+
# (e.g. /var/lib/openshell/gateway): the path must match exactly on
102+
# both the host and inside the container, and a single gateway per host
103+
# is the expected topology.
104+
- type: bind
105+
source: /var/lib/openshell
106+
target: /var/lib/openshell
107+
bind:
108+
create_host_path: true
109+
110+
# TOML config — all gateway and driver settings live here.
111+
- type: bind
112+
source: ./gateway.toml
113+
target: /etc/openshell/gateway.toml
114+
read_only: true
115+
116+
environment:
117+
# Point the gateway at the TOML config file mounted above.
118+
OPENSHELL_GATEWAY_CONFIG: /etc/openshell/gateway.toml
119+
120+
# Database URL cannot be set in the TOML config file — it is explicitly
121+
# blocked there to prevent secrets from being committed to VCS.
122+
OPENSHELL_DB_URL: "sqlite:/var/lib/openshell/gateway.db?mode=rwc"
123+
124+
# XDG path variables are OS-level; they are not part of the gateway config
125+
# schema. Setting them ensures the extracted supervisor binary lands in the
126+
# bind-mounted directory so its path is resolvable by the host Docker daemon.
127+
XDG_DATA_HOME: /var/lib/openshell
128+
HOME: /var/lib/openshell

‎deploy/docker/gateway.toml‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
# OpenShell gateway TOML configuration — Docker compute driver.
5+
#
6+
# This file is the primary configuration source for docker-compose.yml in this
7+
# directory. It is mounted read-only at /etc/openshell/gateway.toml inside the
8+
# gateway container and loaded via OPENSHELL_GATEWAY_CONFIG.
9+
#
10+
# Why docker-compose.yml sets command: []:
11+
# The gateway image's default CMD passes --bind-address 0.0.0.0 --port 8080
12+
# as explicit CLI flags. CLI flags beat the TOML file in the merge order, so
13+
# bind_address = "127.0.0.1:8080" below would be silently ignored without
14+
# clearing the CMD first.
15+
#
16+
# grpc_endpoint note:
17+
# host.docker.internal is automatically resolvable from containers on
18+
# Docker Desktop (Windows / macOS). On Linux, add extra_hosts to the
19+
# gateway service:
20+
# extra_hosts:
21+
# - "host.docker.internal:host-gateway"
22+
# - "host.openshell.internal:host-gateway"
23+
24+
[openshell]
25+
version = 1
26+
27+
[openshell.gateway]
28+
# Bind to loopback only. The Docker driver adds an extra listener on the
29+
# bridge interface automatically so sandbox containers can reach the gateway.
30+
bind_address = "127.0.0.1:8080"
31+
health_bind_address = "127.0.0.1:8081"
32+
log_level = "info"
33+
compute_drivers = ["docker"]
34+
disable_tls = true
35+
36+
[openshell.drivers.docker]
37+
# Default image pulled for `openshell sandbox create` without --from.
38+
default_image = "ghcr.io/nvidia/openshell-community/sandboxes/base:latest"
39+
# Supervisor image from which the openshell-sandbox binary is extracted on
40+
# first start. The binary is cached to XDG_DATA_HOME and reused on restart.
41+
supervisor_image = "ghcr.io/nvidia/openshell/supervisor:latest"
42+
# Only pull images that are not already cached locally.
43+
image_pull_policy = "IfNotPresent"
44+
# Prefix applied to sandbox container names.
45+
sandbox_namespace = "openshell"
46+
# Address sandbox containers use to call back to the gateway.
47+
# The Docker driver replaces the host with host.openshell.internal and the
48+
# port with the gateway's own bind port (8080). Only the scheme survives.
49+
# The gateway must be published on port 8080 on the Docker host so that
50+
# host.openshell.internal:8080 resolves to the gateway container.
51+
grpc_endpoint = "http://host.openshell.internal:8080"

‎docs/about/container-gateway.mdx‎

Lines changed: 82 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,37 +12,79 @@ Use this approach when you want to run the OpenShell gateway as a container inst
1212

1313
The gateway image is published at `ghcr.io/nvidia/openshell/gateway`.
1414

15+
## Prerequisites for the Docker Driver
16+
17+
When the gateway runs as a container and creates Docker-backed sandboxes, the gateway container
18+
communicates with the host Docker daemon via the mounted socket. This requires three things beyond
19+
a basic `docker run`:
20+
21+
1. **Docker socket access.** The gateway process must be able to read and write the Docker socket.
22+
Add the `docker` group (or the GID of `/var/run/docker.sock`) so the socket is accessible
23+
without running as root.
24+
25+
2. **gRPC endpoint.** Sandbox containers call back to the gateway over the `OPENSHELL_GRPC_ENDPOINT`
26+
address. Set this to `http://127.0.0.1:8080` so the driver knows the scheme and port. The
27+
docker driver automatically binds the gateway to the bridge network interface so sandbox
28+
containers can reach it — you do not need to expose the port on `0.0.0.0`.
29+
30+
3. **Supervisor binary on the host.** The gateway bind-mounts the `openshell-sandbox` supervisor
31+
binary into each sandbox container. Because bind-mount paths are resolved by the host Docker
32+
daemon (not inside the gateway container), the binary must exist at a path on the host
33+
filesystem. Extract it before starting the gateway and mount it at the same path.
34+
1535
## Quick Start
1636

17-
This example runs the gateway locally with TLS disabled. It is suitable for development on a single machine. Binding to `127.0.0.1` prevents remote access without authentication.
37+
Extract the supervisor binary to the host once, then start the gateway:
38+
39+
```shell
40+
mkdir -p ~/openshell/supervisor
41+
docker create --name tmp-supervisor ghcr.io/nvidia/openshell/supervisor:latest
42+
docker cp tmp-supervisor:/openshell-sandbox ~/openshell/supervisor/openshell-sandbox
43+
docker rm tmp-supervisor
44+
chmod +x ~/openshell/supervisor/openshell-sandbox
45+
```
46+
47+
Start the gateway:
1848

1949
```shell
2050
docker run -d \
2151
--name openshell-gateway \
2252
--restart unless-stopped \
53+
--group-add docker \
2354
-p 127.0.0.1:8080:8080 \
2455
-v openshell-state:/var/openshell \
2556
-v /var/run/docker.sock:/var/run/docker.sock \
57+
-v ~/openshell/supervisor/openshell-sandbox:~/openshell/supervisor/openshell-sandbox:ro \
2658
-e OPENSHELL_DRIVERS=docker \
59+
-e OPENSHELL_GRPC_ENDPOINT=http://127.0.0.1:8080 \
60+
-e OPENSHELL_DOCKER_SUPERVISOR_BIN=~/openshell/supervisor/openshell-sandbox \
2761
-e OPENSHELL_DB_URL=sqlite:/var/openshell/openshell.db \
2862
-e OPENSHELL_DISABLE_TLS=true \
2963
ghcr.io/nvidia/openshell/gateway:latest
3064
```
3165

32-
Register the gateway with the CLI:
66+
Register the gateway with the CLI. If running on the same machine, use `--local`:
3367

3468
```shell
3569
openshell gateway add http://127.0.0.1:8080 --local --name local
3670
```
3771

72+
If registering from a different machine on the same network, use the host IP and `--remote`:
73+
74+
```shell
75+
openshell gateway add http://HOST_IP:8080 --remote --name remote
76+
```
77+
3878
Confirm the CLI can reach the gateway:
3979

4080
```shell
4181
openshell status
4282
```
4383

4484
<Warning>
45-
Disabling TLS removes authentication. Binding to `127.0.0.1` limits access to the local machine. If you expose the port on `0.0.0.0`, enable TLS and local mTLS user authentication, or put the gateway behind a trusted proxy with its own authentication.
85+
Disabling TLS removes authentication. This example binds to `127.0.0.1` so only local
86+
connections are accepted. To accept remote connections, enable mTLS or restrict access with
87+
a firewall rule.
4688
</Warning>
4789

4890
## Full mTLS Setup
@@ -58,7 +100,9 @@ docker run --rm \
58100
-v "$HOME/.local/state/openshell:/home/openshell/.local/state/openshell" \
59101
-v "$HOME/.config/openshell:/home/openshell/.config/openshell" \
60102
ghcr.io/nvidia/openshell/gateway:latest \
61-
generate-certs --output-dir /home/openshell/.local/state/openshell/tls
103+
generate-certs \
104+
--output-dir /home/openshell/.local/state/openshell/tls \
105+
--server-san host.openshell.internal
62106
```
63107

64108
This writes the server and client certificates under `~/.local/state/openshell/tls/`, writes sandbox JWT signing keys under `~/.local/state/openshell/tls/jwt/`, and copies the client bundle to `~/.config/openshell/gateways/openshell/mtls/` so the CLI picks it up automatically.
@@ -69,10 +113,14 @@ Start the gateway with mTLS enabled:
69113
docker run -d \
70114
--name openshell-gateway \
71115
--restart unless-stopped \
116+
--group-add docker \
72117
-p 127.0.0.1:8080:8080 \
73118
-v "$HOME/.local/state/openshell:/home/openshell/.local/state/openshell" \
74119
-v /var/run/docker.sock:/var/run/docker.sock \
120+
-v ~/openshell/supervisor/openshell-sandbox:~/openshell/supervisor/openshell-sandbox:ro \
75121
-e OPENSHELL_DRIVERS=docker \
122+
-e OPENSHELL_GRPC_ENDPOINT=https://127.0.0.1:8080 \
123+
-e OPENSHELL_DOCKER_SUPERVISOR_BIN=~/openshell/supervisor/openshell-sandbox \
76124
-e OPENSHELL_DB_URL=sqlite:/home/openshell/.local/state/openshell/openshell.db \
77125
-e OPENSHELL_LOCAL_TLS_DIR=/home/openshell/.local/state/openshell/tls \
78126
-e OPENSHELL_TLS_CERT=/home/openshell/.local/state/openshell/tls/server/tls.crt \
@@ -93,20 +141,41 @@ openshell gateway add https://127.0.0.1:8080 --local --name local
93141

94142
## Docker Compose
95143

96-
Save the following as `compose.yml`. This uses the TLS-disabled configuration bound to localhost, suitable for local development.
144+
The following `compose.yml` runs the gateway with the Docker driver on an immutable OS or any
145+
Docker host. It includes all required configuration for sandbox containers to call back to the
146+
gateway.
147+
148+
Before starting, extract the supervisor binary to a host path. The path must be the same on
149+
both the host and inside the gateway container because the host Docker daemon uses it as a
150+
bind-mount source when creating sandbox containers:
151+
152+
```shell
153+
mkdir -p ~/openshell/supervisor
154+
docker create --name tmp-supervisor ghcr.io/nvidia/openshell/supervisor:latest
155+
docker cp tmp-supervisor:/openshell-sandbox ~/openshell/supervisor/openshell-sandbox
156+
docker rm tmp-supervisor
157+
chmod +x ~/openshell/supervisor/openshell-sandbox
158+
```
159+
160+
Save the following as `~/openshell/compose.yml`, substituting your home directory for `HOME`:
97161

98162
```yaml
99163
services:
100164
gateway:
101165
image: ghcr.io/nvidia/openshell/gateway:latest
102166
restart: unless-stopped
167+
group_add:
168+
- docker
103169
ports:
104170
- "127.0.0.1:8080:8080"
105171
volumes:
106172
- openshell-state:/var/openshell
107173
- /var/run/docker.sock:/var/run/docker.sock
174+
- HOME/openshell/supervisor/openshell-sandbox:HOME/openshell/supervisor/openshell-sandbox:ro
108175
environment:
109176
OPENSHELL_DRIVERS: docker
177+
OPENSHELL_GRPC_ENDPOINT: "http://127.0.0.1:8080"
178+
OPENSHELL_DOCKER_SUPERVISOR_BIN: "HOME/openshell/supervisor/openshell-sandbox"
110179
OPENSHELL_DB_URL: "sqlite:/var/openshell/openshell.db"
111180
OPENSHELL_DISABLE_TLS: "true"
112181

@@ -120,12 +189,19 @@ Start the gateway:
120189
docker compose up -d
121190
```
122191

123-
Register the gateway with the CLI:
192+
Register the gateway with the CLI. If registering from the same machine:
124193

125194
```shell
126195
openshell gateway add http://127.0.0.1:8080 --local --name local
127196
```
128197

198+
If registering from a different machine on the same network, replace `HOST_IP` with the
199+
machine's LAN address:
200+
201+
```shell
202+
openshell gateway add http://HOST_IP:8080 --remote --name remote
203+
```
204+
129205
## Using Podman
130206

131207
Replace `docker` with `podman` in the commands above. Mount the Podman socket instead of the Docker socket and set the driver to `podman`:

0 commit comments

Comments
 (0)