Skip to content

Commit c8afb73

Browse files
committed
feat(agents): support Kubernetes image delivery and reusable tooling
Signed-off-by: John Myers <[email protected]>
1 parent 161b43a commit c8afb73

8 files changed

Lines changed: 445 additions & 41 deletions

File tree

‎.agents/skills/launch-openshell-gator/SKILL.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,20 @@ The launcher passes the non-secret host account ID as literal `CODEX_ACCOUNT_ID`
8484

8585
The image contains task-independent tooling and a prompt template, not the run's PR assignment or test authorization. The launcher supplies those instructions at runtime through `OPENSHELL_AGENT_TASK` in the canonical process command. The supervisor captures the assembled prompt in memory and supplies it on stdin for every cycle. Task text is non-secret and may appear in runtime command metadata; never put credentials in the task. Task-only changes reuse the same tooling build inputs, although the launcher still builds/tags an image per invocation.
8686

87+
For Kubernetes gateways, select image delivery explicitly: `--k3d-cluster NAME`
88+
builds with Docker and imports into a running local cluster with the matching
89+
current kube context; `--publish-image REGISTRY/REPOSITORY:TAG` builds and pushes,
90+
then launches by resolved digest; `--image REGISTRY/REPOSITORY@sha256:DIGEST`
91+
reuses trusted compatible tooling without local image tools. Do not infer
92+
locality from a loopback endpoint. The k3d selection asserts which cluster
93+
serves the gateway; imported images require `if_not_present` or `never` pull
94+
policy. Registry builds use the builder's platform and existing registry auth;
95+
nodes need their own pull access. `--build-only` builds/delivers without
96+
gateway mutations, provider credential reads, or a sandbox launch. Reused
97+
images must match the local manifest's agent ID and payload version and support
98+
the runtime task contract; legacy baked-task images fail startup. See
99+
`scripts/agents/gator/README.md` for examples and option compatibility.
100+
87101
### Step 5: Verify Gateway Is Registered And Alive
88102

89103
Use the target gateway from the operator request or current session context. Do not assume a gateway name. If the operator did not specify one, list registered gateways and ask before launching when the correct target is ambiguous.
@@ -165,7 +179,7 @@ sandbox_name="gtr-pr-${pr_number}-watch"
165179
"Review and monitor PR #${pr_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number}."
166180
```
167181

168-
The launcher queries the gateway's selected compute driver, builds the gator image in the matching Docker or Podman image store, stages the immutable payload, imports provider profiles, configures provider credentials and refresh, and starts the agent supervisor as the sandbox's canonical main process. The detached main process survives loss of the host CLI connection and reconnects to a restarted gateway. Unless `--keep` is set, the sandbox is marked ephemeral so the gateway deletes it after the canonical main process exits and its terminal result is finalized. `CONTAINER_ENGINE`, when set, must match the gateway driver.
182+
The launcher queries the gateway's selected compute driver and prepares the image through the selected build/import/publish or prebuilt path before importing provider profiles, configuring provider credentials and refresh, and starting the agent supervisor as the sandbox's canonical main process. The detached main process survives loss of the host CLI connection and reconnects to a restarted gateway. Unless `--keep` is set, the sandbox is marked ephemeral so the gateway deletes it after the canonical main process exits and its terminal result is finalized. `CONTAINER_ENGINE`, when set, must match Docker/Podman gateways or Docker for k3d imports; registry builds for Kubernetes may use either local engine.
169183

170184
The launcher streams image-build and provisioning output until the detached workload is ready, then exits. Use `openshell logs <sandbox-name>` or the TUI for runtime output.
171185

‎scripts/agents/gator/README.md‎

Lines changed: 72 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ Launch a headless sandbox agent that runs the `gator-gate` skill against OpenShe
77
- `gh` is authenticated on the host and has access to `NVIDIA/OpenShell`.
88
- For `--harness codex`, `codex login` has created `$HOME/.codex/auth.json`.
99
- For `--harness codex`, local Codex auth must include an access token, refresh token, and account ID.
10-
- A local gateway and either Docker or Podman are available to build the
11-
default sandbox image.
10+
- A Docker, Podman, or Kubernetes gateway is available. Local builds require
11+
Docker or Podman; a prebuilt image does not require a local container engine.
1212

1313
## Usage
1414

@@ -24,11 +24,75 @@ By default the launcher uses `scripts/agents/gator/Dockerfile` as the sandbox im
2424

2525
The PR assignment, operator test authorization, harness selection, watch mode, and polling interval are runtime inputs, not rendered image contents. The launcher supplies the non-secret assignment as `OPENSHELL_AGENT_TASK` in the canonical process command. The supervisor assembles the prompt once in memory and sends it to each harness cycle on stdin; cycles do not read task instructions from an agent-writable file. Assignment text is literal data, so shell syntax and template tokens inside it are not evaluated. Do not put credentials in assignments; use providers. Runtime command metadata can contain the assignment. This separates reusable tooling from per-run instructions; it does not make prompt instructions a security boundary.
2626

27-
The launcher still builds and tags an image for each invocation, but task-only or supervision-setting changes reuse identical tooling build inputs. Publishing and selecting reusable images is a separate launcher capability.
27+
Without `--image`, the launcher builds and tags an image for each invocation,
28+
but task-only or supervision-setting changes reuse identical tooling build
29+
inputs. Docker and Podman builds use the gateway's engine; an explicit
30+
`CONTAINER_ENGINE` must match it.
2831

29-
The launcher queries the selected gateway and builds with its Docker or Podman
30-
compute driver. If `CONTAINER_ENGINE` is set, it must match that driver. Other
31-
gateway drivers cannot run this local-image launcher.
32+
## Kubernetes image delivery
33+
34+
The launcher discovers the compute driver from the selected gateway. It does
35+
not infer a local cluster from a loopback gateway endpoint: that endpoint could
36+
be a port-forward to a remote deployment. Select one of these image paths for
37+
a Kubernetes gateway.
38+
39+
For local k3d, explicitly identify the cluster serving the gateway. The launcher
40+
requires the matching current kube context and a running local cluster, builds
41+
with Docker, and imports the image into that cluster before creating a sandbox:
42+
43+
```shell
44+
OPENSHELL_WORKSPACE=gator mise exec [email protected] -- ./scripts/agents/run.sh \
45+
--agent gator --gateway openshell-dev-main \
46+
--k3d-cluster openshell-dev-main --name gtr-pr-1234 --watch \
47+
"Review and monitor PR #1234 through the gator-gate workflow."
48+
```
49+
50+
The cluster selection is an operator assertion that the gateway uses that
51+
cluster; it is not inferred from the gateway name. Its workload image pull
52+
policy must allow imported images (`if_not_present` or `never`). The local
53+
Skaffold configuration uses `if_not_present`.
54+
55+
For a registry-backed deployment, build and publish tooling once without
56+
changing gateway settings, importing providers, reading provider credentials,
57+
or launching a sandbox:
58+
59+
```shell
60+
./scripts/agents/run.sh --agent gator --gateway remote-kube \
61+
--publish-image registry.example.com/team/gator:payload-13 --build-only
62+
```
63+
64+
The builder uses existing Docker/Podman registry authentication. Both local and
65+
remote Kubernetes nodes must be able to pull from the chosen registry; private
66+
registries require the deployment's image-pull credential configuration. Builds
67+
target the local builder's platform, so build on a matching platform for the
68+
target nodes. Multi-platform builds are not provided by this launcher.
69+
70+
The launcher prints `Tooling image: REGISTRY/REPOSITORY@sha256:DIGEST` after a
71+
successful push. Reuse that full reference for subsequent tasks:
72+
73+
```shell
74+
OPENSHELL_WORKSPACE=gator ./scripts/agents/run.sh \
75+
--agent gator --gateway remote-kube --name gtr-pr-1234 --watch \
76+
--image "$GATOR_IMAGE" \
77+
"Review and monitor PR #1234 through the gator-gate workflow."
78+
```
79+
80+
Set `GATOR_IMAGE` to the printed digest-pinned reference, not a mutable tag.
81+
`--image` skips local builds and cluster imports and works with Docker/Podman
82+
gateways too. It cannot be combined with `--from`, `--codex-bin`,
83+
`--k3d-cluster`, `--publish-image`, or `--build-only`. The image must contain
84+
tooling built by this launcher with the same agent ID and payload version as
85+
the local manifest. Startup validates that contract before running the agent;
86+
legacy images with baked assignments are incompatible. Only reuse trusted
87+
tooling images: the contract check does not authenticate their contents.
88+
89+
Omit `--build-only` from `--publish-image` to build, push, and launch in one
90+
invocation. A failed push, missing repository digest, or failed k3d import
91+
stops the launcher before gateway settings, provider configuration, or sandbox
92+
creation. Kubernetes launches without an explicit image-delivery choice fail
93+
with guidance rather than guessing a cluster or registry.
94+
95+
## Runtime and providers
3296

3397
Use `--harness codex` to select Codex explicitly. Other harness names are rejected until their support is added to `agent.yaml` and `scripts/agents/runtime/harnesses/<name>/`. Agent directories do not carry their own harness implementations; they provide prompt templates and optional skills or subagents for the shared runtime to inject.
3498

@@ -80,5 +144,7 @@ bash scripts/agents/gator/bin/review_feedback_ledger_test.sh
80144
bash scripts/agents/gator/bin/resolve_gator_review_threads_test.sh
81145
bash scripts/agents/runtime/harnesses/codex/exec_test.sh
82146
bash scripts/agents/run_test.sh
147+
bash scripts/agents/image_test.sh
148+
bash scripts/agents/runtime/launch_test.sh
83149
bash scripts/agents/runtime/supervisor_test.sh
84150
```

‎scripts/agents/image.sh‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
# Image transport for the host launcher. The caller supplies fail/log/require_cmd
7+
# and selects the gateway; no locality inference is made from its endpoint.
8+
prepare_agent_image_transport() {
9+
local gateway_info
10+
gateway_info="$("$OPENSHELL_BIN" --gateway "$GATEWAY" gateway info --output json)" || fail "failed to inspect gateway '$GATEWAY'"
11+
GATEWAY_DRIVER="$(printf '%s' "$gateway_info" | ruby -rjson -e '
12+
drivers = JSON.parse(STDIN.read).fetch("compute_drivers", []).map { |d| d.fetch("name").downcase }
13+
abort "gateway must report exactly one compute driver" unless drivers.length == 1
14+
puts drivers.first
15+
')" || fail "gateway '$GATEWAY' did not report exactly one compute driver"
16+
case "$GATEWAY_DRIVER" in
17+
docker|podman|kubernetes) ;;
18+
*) fail "unsupported agent compute driver: $GATEWAY_DRIVER" ;;
19+
esac
20+
21+
if [[ -n "$AGENT_IMAGE" ]]; then
22+
[[ "$AGENT_IMAGE" =~ ^[a-zA-Z0-9][a-zA-Z0-9._:/-]*@sha256:[a-f0-9]{64}$ ]] || fail "--image requires a digest-pinned image reference"
23+
[[ -z "$K3D_CLUSTER$PUBLISH_IMAGE" ]] || fail "--image cannot be combined with --k3d-cluster or --publish-image"
24+
IMAGE_MODE=prebuilt
25+
return
26+
fi
27+
28+
if [[ -n "$K3D_CLUSTER" ]]; then
29+
[[ -z "$PUBLISH_IMAGE" ]] || fail "--k3d-cluster and --publish-image are mutually exclusive"
30+
[[ "$GATEWAY_DRIVER" == kubernetes ]] || fail "--k3d-cluster requires a Kubernetes gateway"
31+
[[ "$K3D_CLUSTER" =~ ^[a-z0-9]([a-z0-9-]{0,30}[a-z0-9])?$ ]] || fail "invalid k3d cluster name"
32+
require_cmd k3d
33+
require_cmd kubectl
34+
local context
35+
context="$(kubectl config current-context)" || fail "cannot determine Kubernetes context"
36+
[[ "$context" == "k3d-$K3D_CLUSTER" ]] || fail "selected kube context '$context' does not match k3d cluster '$K3D_CLUSTER'"
37+
k3d cluster list -o json | ruby -rjson -e '
38+
cluster = JSON.parse(STDIN.read).find { |c| c.fetch("name") == ARGV.fetch(0) }
39+
abort "selected local k3d cluster is not running" unless cluster && cluster.fetch("serversRunning", 0) > 0
40+
' "$K3D_CLUSTER" || fail "cannot use local k3d cluster '$K3D_CLUSTER'"
41+
IMAGE_MODE=k3d
42+
set_agent_build_engine docker
43+
elif [[ -n "$PUBLISH_IMAGE" ]]; then
44+
# Require an explicit registry and tag. No implicit Docker Hub uploads.
45+
[[ "$PUBLISH_IMAGE" =~ ^[a-z0-9][a-z0-9.:-]*/[a-z0-9][a-z0-9._/-]*:[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$ ]] || fail "--publish-image requires REGISTRY/REPOSITORY:TAG"
46+
local registry="${PUBLISH_IMAGE%%/*}"
47+
[[ "$registry" == localhost || "$registry" == *.* || "$registry" == *:* ]] || fail "--publish-image requires an explicit registry hostname (use docker.io for Docker Hub)"
48+
IMAGE_MODE=registry
49+
if [[ "$GATEWAY_DRIVER" == kubernetes ]]; then
50+
# Registry builds may use either local engine, independently of the
51+
# engine used to host the remote Kubernetes cluster.
52+
source "$ROOT_DIR/tasks/scripts/container-engine.sh"
53+
else
54+
set_agent_build_engine "$GATEWAY_DRIVER"
55+
fi
56+
elif [[ "$GATEWAY_DRIVER" == kubernetes ]]; then
57+
fail "Kubernetes needs --image REGISTRY/REPOSITORY@sha256:DIGEST, --k3d-cluster NAME, or --publish-image REGISTRY/REPOSITORY:TAG"
58+
else
59+
IMAGE_MODE=engine
60+
set_agent_build_engine "$GATEWAY_DRIVER"
61+
fi
62+
}
63+
64+
set_agent_build_engine() {
65+
local engine="$1"
66+
if [[ -n "${CONTAINER_ENGINE:-}" && "$(printf '%s' "$CONTAINER_ENGINE" | tr '[:upper:]' '[:lower:]')" != "$engine" ]]; then
67+
fail "CONTAINER_ENGINE=$CONTAINER_ENGINE conflicts with required image build engine '$engine'"
68+
fi
69+
CONTAINER_ENGINE="$engine"
70+
export CONTAINER_ENGINE
71+
source "$ROOT_DIR/tasks/scripts/container-engine.sh"
72+
}
73+
74+
deliver_agent_image() {
75+
local image_tag="$1"
76+
case "$IMAGE_MODE" in
77+
k3d)
78+
log "Importing '$image_tag' into k3d cluster '$K3D_CLUSTER'."
79+
k3d image import --cluster "$K3D_CLUSTER" "$image_tag" || fail "k3d image import failed"
80+
SANDBOX_FROM="$image_tag"
81+
;;
82+
registry)
83+
log "Publishing '$image_tag'."
84+
ce push "$image_tag" || fail "image push failed"
85+
local repository="${image_tag%:*}"
86+
SANDBOX_FROM="$(ce image inspect "$image_tag" --format '{{json .RepoDigests}}' | ruby -rjson -e '
87+
digests = JSON.parse(STDIN.read) || []
88+
normalize = lambda do |repository|
89+
# Docker omits its default registry in RepoDigests.
90+
repository = repository.sub(/\A(?:docker\.io|index\.docker\.io)\//, "")
91+
repository.include?("/") ? repository : "library/#{repository}"
92+
end
93+
digest = digests.find { |d| normalize.call(d.split("@", 2).first) == normalize.call(ARGV.fetch(0)) && d.match?(/@sha256:[a-f0-9]{64}\z/) }
94+
abort "push did not produce a matching repository digest" unless digest
95+
puts "#{ARGV.fetch(0)}@#{digest.split("@", 2).last}"
96+
' "$repository")" || fail "cannot resolve published image digest; refusing mutable-tag launch"
97+
;;
98+
engine) SANDBOX_FROM="$image_tag" ;;
99+
*) fail "invalid image delivery mode: $IMAGE_MODE" ;;
100+
esac
101+
}

‎scripts/agents/image_test.sh‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
set -euo pipefail
7+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
8+
ROOT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
9+
tmp="$(mktemp -d)"
10+
trap 'rm -rf "$tmp"' EXIT
11+
mkdir -p "$tmp/bin"
12+
export CALLS="$tmp/calls"
13+
export TEST_DIGEST="$(printf 'a%.0s' {1..64})"
14+
15+
cat > "$tmp/bin/openshell" <<'SH'
16+
#!/usr/bin/env bash
17+
printf '{"compute_drivers":[{"name":"%s"}]}\n' "${TEST_DRIVER:-kubernetes}"
18+
SH
19+
cat > "$tmp/bin/kubectl" <<'SH'
20+
#!/usr/bin/env bash
21+
echo "${TEST_CONTEXT:-k3d-local}"
22+
SH
23+
cat > "$tmp/bin/k3d" <<'SH'
24+
#!/usr/bin/env bash
25+
echo "k3d $*" >> "$CALLS"
26+
if [[ "$1" == cluster ]]; then
27+
echo '[{"name":"local","serversRunning":1}]'
28+
else
29+
exit "${TEST_IMPORT_STATUS:-0}"
30+
fi
31+
SH
32+
cat > "$tmp/bin/docker" <<'SH'
33+
#!/usr/bin/env bash
34+
set -eu
35+
echo "docker $*" >> "$CALLS"
36+
case "$1" in
37+
--version) echo 'Docker test stub' ;;
38+
push) exit "${TEST_PUSH_STATUS:-0}" ;;
39+
image) printf '["%s@sha256:%s"]\n' "${TEST_INSPECT_REPOSITORY:-registry.test/team/gator}" "$TEST_DIGEST" ;;
40+
esac
41+
SH
42+
chmod +x "$tmp/bin/"*
43+
ln -s docker "$tmp/bin/podman"
44+
export PATH="$tmp/bin:$PATH"
45+
46+
fail() { echo "error: $*" >&2; exit 1; }
47+
log() { echo "$*" >&2; }
48+
require_cmd() { command -v "$1" >/dev/null || fail "missing $1"; }
49+
source "$SCRIPT_DIR/image.sh"
50+
51+
run_transport() (
52+
OPENSHELL_BIN=openshell
53+
GATEWAY=test
54+
AGENT_IMAGE="${TEST_IMAGE:-}"
55+
K3D_CLUSTER="${TEST_CLUSTER:-}"
56+
PUBLISH_IMAGE="${TEST_PUBLISH:-}"
57+
CONTAINER_ENGINE="${TEST_ENGINE:-docker}"
58+
unset _CONTAINER_ENGINE_LOADED CONTAINER_ENGINE_TARGET
59+
prepare_agent_image_transport
60+
if [[ "$IMAGE_MODE" == prebuilt ]]; then
61+
echo "$AGENT_IMAGE"
62+
else
63+
deliver_agent_image "${PUBLISH_IMAGE:-openshell/agent-gator:test}"
64+
echo "$SANDBOX_FROM"
65+
fi
66+
)
67+
68+
expect_failure() {
69+
if "$@" > "$tmp/out" 2> "$tmp/error"; then
70+
fail 'expected failure'
71+
fi
72+
}
73+
74+
TEST_CLUSTER=local run_transport > "$tmp/out"
75+
grep -Fq 'k3d image import --cluster local openshell/agent-gator:test' "$CALLS"
76+
grep -Fxq 'openshell/agent-gator:test' "$tmp/out"
77+
echo 'ok - Kubernetes local import targets the explicit k3d cluster'
78+
79+
TEST_PUBLISH=registry.test/team/gator:tooling run_transport > "$tmp/out"
80+
grep -Fq 'docker push registry.test/team/gator:tooling' "$CALLS"
81+
grep -Fxq "registry.test/team/gator@sha256:$TEST_DIGEST" "$tmp/out"
82+
echo 'ok - registry delivery resolves a pinned digest'
83+
TEST_PUBLISH=docker.io/team/gator:tooling TEST_INSPECT_REPOSITORY=team/gator \
84+
run_transport > "$tmp/out"
85+
grep -Fxq "docker.io/team/gator@sha256:$TEST_DIGEST" "$tmp/out"
86+
echo 'ok - Docker Hub digest normalization preserves the explicit registry'
87+
88+
: > "$CALLS"
89+
TEST_IMAGE="registry.test/team/gator@sha256:$TEST_DIGEST" run_transport > "$tmp/out"
90+
[[ ! -s "$CALLS" ]] || fail 'prebuilt path contacted local image/cluster tools'
91+
echo 'ok - prebuilt path needs no local image engine or kube context'
92+
93+
expect_failure run_transport
94+
grep -Fq 'Kubernetes needs' "$tmp/error"
95+
TEST_CLUSTER=local TEST_CONTEXT=remote expect_failure run_transport
96+
TEST_CLUSTER=missing expect_failure run_transport
97+
TEST_CLUSTER=local TEST_IMPORT_STATUS=1 expect_failure run_transport
98+
TEST_CLUSTER=local TEST_ENGINE=podman expect_failure run_transport
99+
TEST_CLUSTER=local TEST_PUBLISH=registry.test/team/gator:tag expect_failure run_transport
100+
TEST_CLUSTER=local TEST_IMAGE="registry.test/team/gator@sha256:$TEST_DIGEST" expect_failure run_transport
101+
TEST_IMAGE=registry.test/team/gator:latest expect_failure run_transport
102+
TEST_PUBLISH=registry.test/team/gator:tag TEST_PUSH_STATUS=1 expect_failure run_transport
103+
TEST_PUBLISH=registry.test/team/gator:tag TEST_INSPECT_REPOSITORY=other.test/gator expect_failure run_transport
104+
TEST_PUBLISH='implicit-tag' expect_failure run_transport
105+
TEST_PUBLISH=implicit/repository:tag expect_failure run_transport
106+
TEST_DRIVER=vm expect_failure run_transport
107+
echo 'ok - ambiguous targets, conflicting options, and failed delivery fail closed'
108+
109+
TEST_DRIVER=docker run_transport > "$tmp/out"
110+
grep -Fxq 'openshell/agent-gator:test' "$tmp/out"
111+
echo 'ok - Docker delivery retains the local-engine path'
112+
TEST_DRIVER=podman TEST_ENGINE=podman run_transport > "$tmp/out"
113+
grep -Fxq 'openshell/agent-gator:test' "$tmp/out"
114+
echo 'ok - Podman delivery retains the local-engine path'

0 commit comments

Comments
 (0)