Skip to content

Commit a238dea

Browse files
steipetesteipete
authored
fix(docker): package selected external plugins in source-built images (#103629)
* build(docker): package selected external plugins * test(docker): auto-clean plugin fixture * fix(docker): preserve selected image provenance * fix(docker): accept manifest-only plugin selections * fix(docker): resolve selected plugin manifest ids * fix(docker): isolate resolved plugin selection
1 parent 1381999 commit a238dea

21 files changed

Lines changed: 807 additions & 62 deletions

‎Dockerfile‎

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
# Opt-in plugin dependencies at build time (space- or comma-separated directory names).
1+
# Opt-in plugin dependencies and supported runtime builds (space- or comma-separated ids).
2+
# Manifest ids and existing source-directory names are accepted.
23
# Example: docker build --build-arg OPENCLAW_EXTENSIONS="diagnostics-otel,matrix" .
34
#
45
# Multi-stage build produces a minimal runtime image without build tools,
@@ -30,8 +31,10 @@ FROM ${OPENCLAW_NODE_BOOKWORM_IMAGE} AS workspace-deps
3031
ARG OPENCLAW_EXTENSIONS
3132
ARG OPENCLAW_BUNDLED_PLUGIN_DIR
3233
# Copy package.json files for workspace packages used by the install layer.
34+
# Manifest-only bundled plugins remain valid selections but need no workspace metadata.
3335
RUN --mount=type=bind,source=packages,target=/tmp/packages,readonly \
3436
--mount=type=bind,source=${OPENCLAW_BUNDLED_PLUGIN_DIR},target=/tmp/${OPENCLAW_BUNDLED_PLUGIN_DIR},readonly \
37+
--mount=type=bind,source=scripts/lib/docker-plugin-selection.mjs,target=/tmp/docker-plugin-selection.mjs,readonly \
3538
mkdir -p /out/packages "/out/${OPENCLAW_BUNDLED_PLUGIN_DIR}" && \
3639
for manifest in /tmp/packages/*/package.json; do \
3740
[ -f "$manifest" ] || continue; \
@@ -40,18 +43,20 @@ RUN --mount=type=bind,source=packages,target=/tmp/packages,readonly \
4043
mkdir -p "/out/packages/$pkg_name" && \
4144
cp "$manifest" "/out/packages/$pkg_name/package.json"; \
4245
done && \
43-
for ext in $(printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' '); do \
44-
if [ -f "/tmp/${OPENCLAW_BUNDLED_PLUGIN_DIR}/$ext/package.json" ]; then \
46+
node /tmp/docker-plugin-selection.mjs "/tmp/${OPENCLAW_BUNDLED_PLUGIN_DIR}" "$OPENCLAW_EXTENSIONS" \
47+
> /out/openclaw-selected-plugin-dirs && \
48+
while IFS= read -r ext; do \
49+
ext_dir="/tmp/${OPENCLAW_BUNDLED_PLUGIN_DIR}/$ext"; \
50+
if [ -f "$ext_dir/package.json" ]; then \
4551
mkdir -p "/out/${OPENCLAW_BUNDLED_PLUGIN_DIR}/$ext" && \
46-
cp "/tmp/${OPENCLAW_BUNDLED_PLUGIN_DIR}/$ext/package.json" "/out/${OPENCLAW_BUNDLED_PLUGIN_DIR}/$ext/package.json"; \
52+
cp "$ext_dir/package.json" "/out/${OPENCLAW_BUNDLED_PLUGIN_DIR}/$ext/package.json"; \
4753
fi; \
48-
done
54+
done < /out/openclaw-selected-plugin-dirs
4955

5056
# ── Stage 2: Build ──────────────────────────────────────────────
5157
FROM ${OPENCLAW_BUN_IMAGE} AS bun-binary
5258
FROM ${OPENCLAW_NODE_BOOKWORM_IMAGE} AS build
5359
ARG OPENCLAW_BUNDLED_PLUGIN_DIR
54-
ARG OPENCLAW_EXTENSIONS
5560
ARG OPENCLAW_DOCKER_BUILD_NODE_OPTIONS
5661
ARG OPENCLAW_DOCKER_BUILD_TSDOWN_MAX_OLD_SPACE_MB
5762
ARG OPENCLAW_DOCKER_BUILD_SKIP_DTS
@@ -72,6 +77,7 @@ COPY scripts/lib/package-dist-imports.mjs ./scripts/lib/package-dist-imports.mjs
7277

7378
COPY --from=workspace-deps /out/packages/ ./packages/
7479
COPY --from=workspace-deps /out/${OPENCLAW_BUNDLED_PLUGIN_DIR}/ ./${OPENCLAW_BUNDLED_PLUGIN_DIR}/
80+
COPY --from=workspace-deps /out/openclaw-selected-plugin-dirs /tmp/openclaw-selected-plugin-dirs
7581

7682
# Reduce OOM risk on low-memory hosts during dependency installation.
7783
# Docker builds on small VMs may otherwise fail with "Killed" (exit 137).
@@ -86,7 +92,7 @@ RUN --mount=type=cache,id=openclaw-pnpm-store,target=/root/.local/share/pnpm/sto
8692
# still exiting successfully, so retry the package downloader before failing.
8793
# Skip the entire check when matrix is not a bundled extension (e.g. msteams-only builds).
8894
RUN set -eux; \
89-
if ! printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -qx 'matrix'; then \
95+
if ! grep -qx 'matrix' /tmp/openclaw-selected-plugin-dirs; then \
9096
echo "==> matrix not bundled, skipping matrix-sdk-crypto check"; \
9197
exit 0; \
9298
fi; \
@@ -132,16 +138,17 @@ RUN pnpm_config_verify_deps_before_run=false pnpm canvas:a2ui:bundle || \
132138
# Force pnpm for UI build (Bun may fail on ARM/Synology architectures)
133139
ENV OPENCLAW_PREFER_PNPM=1
134140
RUN set -eu; \
141+
selected_plugin_dirs="$(cat /tmp/openclaw-selected-plugin-dirs)"; \
135142
if [ -z "$OPENCLAW_BUILD_TIMESTAMP" ]; then \
136143
OPENCLAW_BUILD_TIMESTAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)"; \
137144
export OPENCLAW_BUILD_TIMESTAMP; \
138145
fi; \
139-
if printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -qx 'qa-lab'; then \
146+
if grep -qx 'qa-lab' /tmp/openclaw-selected-plugin-dirs; then \
140147
export OPENCLAW_BUILD_PRIVATE_QA=1 OPENCLAW_ENABLE_PRIVATE_QA_CLI=1; \
141148
fi; \
142-
OPENCLAW_RUN_NODE_SKIP_DTS_BUILD="$OPENCLAW_DOCKER_BUILD_SKIP_DTS" OPENCLAW_TSDOWN_MAX_OLD_SPACE_MB="$OPENCLAW_DOCKER_BUILD_TSDOWN_MAX_OLD_SPACE_MB" NODE_OPTIONS="$OPENCLAW_DOCKER_BUILD_NODE_OPTIONS" pnpm_config_verify_deps_before_run=false pnpm build:docker; \
149+
OPENCLAW_INTERNAL_DOCKER_BUILD_PLUGIN_IDS="$selected_plugin_dirs" OPENCLAW_RUN_NODE_SKIP_DTS_BUILD="$OPENCLAW_DOCKER_BUILD_SKIP_DTS" OPENCLAW_TSDOWN_MAX_OLD_SPACE_MB="$OPENCLAW_DOCKER_BUILD_TSDOWN_MAX_OLD_SPACE_MB" NODE_OPTIONS="$OPENCLAW_DOCKER_BUILD_NODE_OPTIONS" pnpm_config_verify_deps_before_run=false pnpm build:docker; \
143150
pnpm_config_verify_deps_before_run=false pnpm ui:build
144-
RUN if printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -qx 'qa-lab'; then \
151+
RUN if grep -qx 'qa-lab' /tmp/openclaw-selected-plugin-dirs; then \
145152
pnpm_config_verify_deps_before_run=false pnpm qa:lab:build && \
146153
mkdir -p dist/extensions/qa-lab/web && \
147154
rm -rf dist/extensions/qa-lab/web/dist && \
@@ -151,7 +158,6 @@ RUN if printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -q
151158
# Prune dev dependencies, omitted plugin runtime packages, and build-only
152159
# metadata before copying runtime assets into the final image.
153160
FROM build AS runtime-assets
154-
ARG OPENCLAW_EXTENSIONS
155161
ARG OPENCLAW_BUNDLED_PLUGIN_DIR
156162
# BuildKit cache mounts are not part of cached layers; seed tarballs for the
157163
# installed prod graph in the same step that runs offline prune.
@@ -162,7 +168,7 @@ RUN --mount=type=cache,id=openclaw-pnpm-store,target=/root/.local/share/pnpm/sto
162168
--config.supportedArchitectures.os=linux \
163169
--config.supportedArchitectures.cpu="$(node -p 'process.arch')" \
164170
--config.supportedArchitectures.libc=glibc && \
165-
OPENCLAW_EXTENSIONS="$OPENCLAW_EXTENSIONS" OPENCLAW_BUNDLED_PLUGIN_DIR="$OPENCLAW_BUNDLED_PLUGIN_DIR" node scripts/prune-docker-plugin-dist.mjs && \
171+
OPENCLAW_EXTENSIONS="$(cat /tmp/openclaw-selected-plugin-dirs)" OPENCLAW_BUNDLED_PLUGIN_DIR="$OPENCLAW_BUNDLED_PLUGIN_DIR" node scripts/prune-docker-plugin-dist.mjs && \
166172
node scripts/postinstall-bundled-plugins.mjs && \
167173
find dist -type f \( -name '*.d.ts' -o -name '*.d.mts' -o -name '*.d.cts' -o -name '*.map' \) -delete && \
168174
if [ -L /app/node_modules/@openclaw/ai ]; then \

‎docs/docs_map.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4216,6 +4216,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
42164216
- H3: Manual flow
42174217
- H3: Upgrading container images
42184218
- H3: Environment variables
4219+
- H3: Source-built images with selected plugins
42194220
- H3: Observability
42204221
- H3: Health checks
42214222
- H3: LAN vs loopback

‎docs/install/docker.md‎

Lines changed: 87 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -151,29 +151,29 @@ same PVC, then restart the Deployment or StatefulSet.
151151

152152
Optional variables accepted by `scripts/docker/setup.sh` (and, for the gateway container, by `docker-compose.yml` directly):
153153

154-
| Variable | Purpose |
155-
| ----------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
156-
| `OPENCLAW_IMAGE` | Use a remote image instead of building locally |
157-
| `OPENCLAW_IMAGE_APT_PACKAGES` | Install extra apt packages during build (space-separated). Legacy alias: `OPENCLAW_DOCKER_APT_PACKAGES` |
158-
| `OPENCLAW_IMAGE_PIP_PACKAGES` | Install extra Python packages during build (space-separated) |
159-
| `OPENCLAW_EXTENSIONS` | Pre-install plugin dependencies at build time (comma- or space-separated ids) |
160-
| `OPENCLAW_DOCKER_BUILD_NODE_OPTIONS` | Override the local source-build Node options (default `--max-old-space-size=8192`) |
161-
| `OPENCLAW_DOCKER_BUILD_TSDOWN_MAX_OLD_SPACE_MB` | Override the local source-build tsdown heap in MB |
162-
| `OPENCLAW_DOCKER_BUILD_SKIP_DTS` | Skip declaration output during runtime-only local image builds (default `1`) |
163-
| `OPENCLAW_INSTALL_BROWSER` | Bake Chromium + Xvfb into the image at build time |
164-
| `OPENCLAW_EXTRA_MOUNTS` | Extra host bind mounts (comma-separated `source:target[:opts]`) |
165-
| `OPENCLAW_HOME_VOLUME` | Persist `/home/node` in a named Docker volume |
166-
| `OPENCLAW_SANDBOX` | Opt in to sandbox bootstrap (`1`, `true`, `yes`, `on`) |
167-
| `OPENCLAW_SKIP_ONBOARDING` | Skip the interactive onboarding step (`1`, `true`, `yes`, `on`) |
168-
| `OPENCLAW_DOCKER_SOCKET` | Override the Docker socket path |
169-
| `OPENCLAW_DISABLE_BONJOUR` | Force Bonjour/mDNS advertising on (`0`) or off (`1`); see [Bonjour / mDNS](#bonjour--mdns) |
170-
| `OPENCLAW_DISABLE_BUNDLED_SOURCE_OVERLAYS` | Disable bundled plugin source bind-mount overlays |
171-
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Shared OTLP/HTTP collector endpoint for OpenTelemetry export |
172-
| `OTEL_EXPORTER_OTLP_*_ENDPOINT` | Signal-specific OTLP endpoints for traces, metrics, or logs |
173-
| `OTEL_EXPORTER_OTLP_PROTOCOL` | OTLP protocol override. Only `http/protobuf` is supported today |
174-
| `OTEL_SERVICE_NAME` | Service name used for OpenTelemetry resources |
175-
| `OTEL_SEMCONV_STABILITY_OPT_IN` | Opt in to latest experimental GenAI semantic attributes |
176-
| `OPENCLAW_OTEL_PRELOADED` | Skip starting a second OpenTelemetry SDK when one is preloaded |
154+
| Variable | Purpose |
155+
| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
156+
| `OPENCLAW_IMAGE` | Use a remote image instead of building locally |
157+
| `OPENCLAW_IMAGE_APT_PACKAGES` | Install extra apt packages during build (space-separated). Legacy alias: `OPENCLAW_DOCKER_APT_PACKAGES` |
158+
| `OPENCLAW_IMAGE_PIP_PACKAGES` | Install extra Python packages during build (space-separated) |
159+
| `OPENCLAW_EXTENSIONS` | Compile/package supported selected plugins and install their runtime dependencies (comma- or space-separated ids) |
160+
| `OPENCLAW_DOCKER_BUILD_NODE_OPTIONS` | Override the local source-build Node options (default `--max-old-space-size=8192`) |
161+
| `OPENCLAW_DOCKER_BUILD_TSDOWN_MAX_OLD_SPACE_MB` | Override the local source-build tsdown heap in MB |
162+
| `OPENCLAW_DOCKER_BUILD_SKIP_DTS` | Skip declaration output during runtime-only local image builds (default `1`) |
163+
| `OPENCLAW_INSTALL_BROWSER` | Bake Chromium + Xvfb into the image at build time |
164+
| `OPENCLAW_EXTRA_MOUNTS` | Extra host bind mounts (comma-separated `source:target[:opts]`) |
165+
| `OPENCLAW_HOME_VOLUME` | Persist `/home/node` in a named Docker volume |
166+
| `OPENCLAW_SANDBOX` | Opt in to sandbox bootstrap (`1`, `true`, `yes`, `on`) |
167+
| `OPENCLAW_SKIP_ONBOARDING` | Skip the interactive onboarding step (`1`, `true`, `yes`, `on`) |
168+
| `OPENCLAW_DOCKER_SOCKET` | Override the Docker socket path |
169+
| `OPENCLAW_DISABLE_BONJOUR` | Force Bonjour/mDNS advertising on (`0`) or off (`1`); see [Bonjour / mDNS](#bonjour--mdns) |
170+
| `OPENCLAW_DISABLE_BUNDLED_SOURCE_OVERLAYS` | Disable bundled plugin source bind-mount overlays |
171+
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Shared OTLP/HTTP collector endpoint for OpenTelemetry export |
172+
| `OTEL_EXPORTER_OTLP_*_ENDPOINT` | Signal-specific OTLP endpoints for traces, metrics, or logs |
173+
| `OTEL_EXPORTER_OTLP_PROTOCOL` | OTLP protocol override. Only `http/protobuf` is supported today |
174+
| `OTEL_SERVICE_NAME` | Service name used for OpenTelemetry resources |
175+
| `OTEL_SEMCONV_STABILITY_OPT_IN` | Opt in to latest experimental GenAI semantic attributes |
176+
| `OPENCLAW_OTEL_PRELOADED` | Skip starting a second OpenTelemetry SDK when one is preloaded |
177177

178178
The official image ships no Homebrew. During onboarding, OpenClaw hides brew-only skill dependency installers in a Linux container without `brew`; provide those dependencies through a custom image or install manually. Use `OPENCLAW_IMAGE_APT_PACKAGES` for Debian-packaged dependencies and `OPENCLAW_IMAGE_PIP_PACKAGES` for Python dependencies (runs `python3 -m pip install --break-system-packages` at build time, so pin versions and use only indexes you trust).
179179

@@ -183,6 +183,70 @@ If Docker reports `ResourceExhausted`, `cannot allocate memory`, or aborts durin
183183
OPENCLAW_DOCKER_BUILD_NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_DOCKER_BUILD_TSDOWN_MAX_OLD_SPACE_MB=4096
184184
```
185185

186+
### Source-built images with selected plugins
187+
188+
`OPENCLAW_EXTENSIONS` selects plugin manifest ids from the source checkout;
189+
existing source-directory names are also accepted when they differ. The Docker
190+
build resolves the selection to source directories once, installs production
191+
dependencies, and, when a selected plugin is published separately with
192+
`openclaw.build.bundledDist: false`, compiles its runtime into the root bundled
193+
dist. This Docker-only packaging does not change the plugin's npm or ClawHub
194+
artifact contract. Unknown, invalid, or ambiguous ids fail the image build.
195+
Known dependency/source-only ids keep their existing source and dependency
196+
staging without gaining a compiled root dist entry. A selected plugin with
197+
unified build entries must compile successfully; unselected external plugin
198+
source and runtime output are pruned.
199+
200+
For example, these commands build separate, multi-architecture standalone
201+
FakeCo gateway images for ClickClack, Slack, and Microsoft Teams. ClawRouter is
202+
already part of the root OpenClaw runtime, so the ClickClack image selects only
203+
`clickclack`. The explicit empty browser argument keeps the default image free
204+
of Chromium:
205+
206+
```bash
207+
SOURCE_SHA="$(git rev-parse HEAD)"
208+
BUILD_TIMESTAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
209+
REGISTRY="registry.example.com/fakeco"
210+
211+
build_gateway_image() {
212+
gateway="$1"
213+
selected_plugin="$2"
214+
docker buildx build \
215+
--platform linux/amd64,linux/arm64 \
216+
--build-arg "GIT_COMMIT=${SOURCE_SHA}" \
217+
--build-arg "OPENCLAW_BUILD_TIMESTAMP=${BUILD_TIMESTAMP}" \
218+
--build-arg "OPENCLAW_EXTENSIONS=${selected_plugin}" \
219+
--build-arg OPENCLAW_INSTALL_BROWSER= \
220+
--provenance=mode=max \
221+
--sbom=true \
222+
--tag "${REGISTRY}/openclaw-${gateway}:${SOURCE_SHA}" \
223+
--push \
224+
.
225+
}
226+
227+
build_gateway_image clickclack clickclack
228+
build_gateway_image slack slack
229+
build_gateway_image teams msteams
230+
```
231+
232+
Use `--platform linux/arm64 --load` or `--platform linux/amd64 --load` for a
233+
single native local build. Multi-platform output and attached SBOM/provenance
234+
require a registry or another Buildx output that preserves attestations. After
235+
pushing, inspect the manifest and deploy the immutable digest rather than the
236+
mutable source-SHA tag:
237+
238+
```bash
239+
docker buildx imagetools inspect \
240+
"${REGISTRY}/openclaw-clickclack:${SOURCE_SHA}"
241+
# Deploy: registry.example.com/fakeco/openclaw-clickclack@sha256:<manifest-digest>
242+
```
243+
244+
These images are for standalone OCI-based gateways and generic Docker users.
245+
Crabhelm-managed gateways do not consume them: that delivery path builds a
246+
separate x86_64 appliance archive containing an OpenClaw npm tarball and pins
247+
the Node, archive, and manifest digests. Build that appliance independently
248+
from the same landed OpenClaw source.
249+
186250
To test bundled plugin source against a packaged image, mount one plugin source directory over its packaged source path, e.g. `OPENCLAW_EXTRA_MOUNTS=/path/to/fork/extensions/synology-chat:/app/extensions/synology-chat:ro`. That overrides the matching compiled `/app/dist/extensions/synology-chat` bundle for the same plugin id.
187251

188252
### Observability

‎docs/install/podman.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ The model:
3636
| `OPENCLAW_IMAGE` / `OPENCLAW_PODMAN_IMAGE` | Use an existing/pulled image instead of building `openclaw:local` |
3737
| `OPENCLAW_IMAGE_APT_PACKAGES` | Install extra apt packages during image build (also accepts legacy `OPENCLAW_DOCKER_APT_PACKAGES`) |
3838
| `OPENCLAW_IMAGE_PIP_PACKAGES` | Install extra Python packages during image build; pin versions and use only package indexes you trust |
39-
| `OPENCLAW_EXTENSIONS` | Pre-install plugin dependencies at build time |
39+
| `OPENCLAW_EXTENSIONS` | Compile/package supported selected plugins and install their runtime dependencies |
4040
| `OPENCLAW_INSTALL_BROWSER` | Pre-install Chromium and Xvfb for browser automation (set to `1`) |
4141

4242
For Quadlet-managed setup instead (Linux + systemd user services only):

‎docs/providers/clawrouter.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,13 @@ credential, update the external Secret that supplies `CLAWROUTER_API_KEY` and
125125
restart the gateway workload so the new process environment is loaded. The
126126
config file and model reference do not change.
127127

128+
For a source-built standalone Docker gateway, ClawRouter is already included in
129+
the root runtime. Select only the channel plugin that needs separate packaging,
130+
such as `OPENCLAW_EXTENSIONS=clickclack`, `slack`, or `msteams`; see
131+
[source-built images with selected plugins](/install/docker#source-built-images-with-selected-plugins).
132+
Archive/appliance deployments must package the same landed source through their
133+
own artifact pipeline rather than consuming the OCI image.
134+
128135
## Readiness and live proof
129136

130137
These checks prove different boundaries; do not substitute one for another:

0 commit comments

Comments
 (0)