Skip to content

Commit 07810e6

Browse files
committed
test(e2e): add GPU workload image artifacts
Closes #1476 Define local GPU workload image sources for smoke-pass, smoke-fail, and cuda-basic validation, plus a mise build task that tags images with the source revision and records the latest local image refs. Signed-off-by: Evan Lezar <[email protected]>
1 parent b41e0df commit 07810e6

13 files changed

Lines changed: 538 additions & 0 deletions

File tree

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,7 @@ logs/
168168
tmp/
169169
temp/
170170
*.tmp
171+
e2e/gpu/images/.build/
171172

172173
# Secrets/credentials (should never be committed)
173174
*.pem

‎e2e/gpu/README.md‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
2+
<!-- SPDX-License-Identifier: Apache-2.0 -->
3+
4+
# GPU workload images
5+
6+
This directory defines GPU workload images used by OpenShell GPU e2e tests.
7+
8+
The image definitions live here first so the OpenShell e2e harness can iterate
9+
against a concrete contract. The long-term image ownership should move to
10+
`NVIDIA/OpenShell-Community`; OpenShell should then keep the contract, local
11+
build task, and tests that consume published image refs.
12+
13+
## Contract
14+
15+
Each workload image must:
16+
17+
- Use the OpenShell community base image as its final-stage base.
18+
- Install the workload at `/usr/local/bin/openshell-gpu-workload`.
19+
- Run the same workload as the image default entrypoint for direct
20+
container-engine validation.
21+
- Require no network access after the image is pulled.
22+
- Print `OPENSHELL_GPU_WORKLOAD_SUCCESS` only when validation succeeds.
23+
- Print `OPENSHELL_GPU_WORKLOAD_FAILURE` and exit non-zero when validation
24+
fails.
25+
- Be usable as an OpenShell sandbox image with `openshell sandbox create
26+
--from <image>`.
27+
28+
OpenShell sandbox creation replaces the image entrypoint with the supervisor and
29+
does not run the OCI image `CMD`. E2e tests that use these images through
30+
OpenShell should run `/usr/local/bin/openshell-gpu-workload` explicitly.
31+
32+
## Images
33+
34+
| Source directory | Image name | Purpose |
35+
| --- | --- | --- |
36+
| `smoke-pass` | `gpu-workload-smoke-pass` | Always succeeds and prints the success marker. |
37+
| `smoke-fail` | `gpu-workload-smoke-fail` | Always fails and prints the failure marker. |
38+
| `cuda-basic` | `gpu-workload-cuda-basic` | Runs CUDA `deviceQuery` and `vectorAdd` validation. |
39+
40+
## Build
41+
42+
Build all workload images:
43+
44+
```shell
45+
mise run e2e:gpu:images:build
46+
```
47+
48+
Build a subset by source directory name:
49+
50+
```shell
51+
OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-pass,smoke-fail \
52+
mise run e2e:gpu:images:build
53+
```
54+
55+
The build task uses `tasks/scripts/container-engine.sh`. Set
56+
`CONTAINER_ENGINE=docker` or `CONTAINER_ENGINE=podman` to choose an engine
57+
explicitly. When unset, the helper uses its existing auto-detection behavior.
58+
59+
Local tags use the current commit short SHA. Dirty local trees append `-dirty`.
60+
Set `OPENSHELL_GPU_WORKLOAD_IMAGE_TAG=<tag>` to override the tag.
61+
62+
The task writes the latest build refs to:
63+
64+
```text
65+
e2e/gpu/images/.build/latest.env
66+
```
67+
68+
Use it in later commands:
69+
70+
```shell
71+
source e2e/gpu/images/.build/latest.env
72+
```
73+
74+
## Direct Validation
75+
76+
Validate smoke pass:
77+
78+
```shell
79+
docker run --rm "${OPENSHELL_E2E_GPU_SMOKE_PASS_IMAGE}"
80+
```
81+
82+
Validate smoke fail:
83+
84+
```shell
85+
docker run --rm "${OPENSHELL_E2E_GPU_SMOKE_FAIL_IMAGE}"
86+
```
87+
88+
The smoke fail command should exit non-zero and print
89+
`OPENSHELL_GPU_WORKLOAD_FAILURE`.
90+
91+
Validate CUDA with Docker CDI:
92+
93+
```shell
94+
docker run --rm --device nvidia.com/gpu=all \
95+
"${OPENSHELL_E2E_GPU_CUDA_WORKLOAD_IMAGE}"
96+
```
97+
98+
Use `podman run` with the same `--device nvidia.com/gpu=all` option on hosts
99+
where Podman CDI is configured.
100+
101+
Direct container-engine validation catches image, CDI, CUDA, and host GPU setup
102+
issues before OpenShell sandbox behavior is involved.
103+
104+
## Publish Guidance
105+
106+
Published tests should reference immutable image refs:
107+
108+
```shell
109+
OPENSHELL_E2E_GPU_CUDA_WORKLOAD_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/gpu-workload-cuda-basic@sha256:<digest>
110+
```
111+
112+
Mutable tags are acceptable for local iteration. CI should use a digest or an
113+
immutable release tag once the images are published from OpenShell-Community.
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
ARG CUDA_BUILD_IMAGE=nvcr.io/nvidia/cuda:12.8.1-base-ubuntu22.04
7+
ARG OPENSHELL_SANDBOX_BASE_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/base:latest
8+
9+
FROM ${CUDA_BUILD_IMAGE} AS builder
10+
11+
ARG DEBIAN_FRONTEND=noninteractive
12+
ARG CUDA_SAMPLES_REF=v12.8
13+
ARG CUDA_SAMPLES_REPO=https://github.com/NVIDIA/cuda-samples
14+
15+
RUN apt-get update && apt-get install -y --no-install-recommends \
16+
build-essential \
17+
ca-certificates \
18+
cmake \
19+
cuda-nvcc-12-8 \
20+
curl \
21+
g++ \
22+
&& rm -rf /var/lib/apt/lists/*
23+
24+
WORKDIR /build/cuda-samples
25+
26+
RUN set -eux; \
27+
curl -fsSL "${CUDA_SAMPLES_REPO}/archive/refs/tags/${CUDA_SAMPLES_REF}.tar.gz" \
28+
-o /tmp/cuda-samples.tar.gz; \
29+
tar -xzf /tmp/cuda-samples.tar.gz \
30+
--strip-components=1 \
31+
--wildcards \
32+
'*/Common/*' \
33+
'*/cmake/*' \
34+
'*/Samples/0_Introduction/vectorAdd/*' \
35+
'*/Samples/1_Utilities/deviceQuery/*' \
36+
'*/LICENSE'; \
37+
sed -i 's/CUDA::cudart/CUDA::cudart_static/g' \
38+
Samples/1_Utilities/deviceQuery/CMakeLists.txt; \
39+
cmake -S Samples/1_Utilities/deviceQuery -B /tmp/build-device-query \
40+
-DCMAKE_BUILD_TYPE=Release \
41+
-DCMAKE_CUDA_RUNTIME_LIBRARY=Static; \
42+
cmake --build /tmp/build-device-query --parallel; \
43+
cmake -S Samples/0_Introduction/vectorAdd -B /tmp/build-vector-add \
44+
-DCMAKE_BUILD_TYPE=Release \
45+
-DCMAKE_CUDA_RUNTIME_LIBRARY=Static; \
46+
cmake --build /tmp/build-vector-add --parallel; \
47+
mkdir -p /opt/openshell-gpu-workload; \
48+
cp /tmp/build-device-query/deviceQuery /opt/openshell-gpu-workload/deviceQuery; \
49+
cp /tmp/build-vector-add/vectorAdd /opt/openshell-gpu-workload/vectorAdd; \
50+
cp LICENSE /opt/openshell-gpu-workload/cuda-samples.LICENSE; \
51+
rm -f /tmp/cuda-samples.tar.gz
52+
53+
FROM ${OPENSHELL_SANDBOX_BASE_IMAGE}
54+
55+
ARG CUDA_SAMPLES_REF=v12.8
56+
57+
LABEL com.nvidia.openshell.gpu-workload.name="cuda-basic" \
58+
com.nvidia.openshell.gpu-workload.cuda-samples-ref="${CUDA_SAMPLES_REF}"
59+
60+
USER root
61+
RUN mkdir -p /usr/local/lib/openshell-gpu-workload \
62+
/usr/local/share/doc/openshell-gpu-workload
63+
COPY --from=builder /opt/openshell-gpu-workload/deviceQuery /usr/local/lib/openshell-gpu-workload/deviceQuery
64+
COPY --from=builder /opt/openshell-gpu-workload/vectorAdd /usr/local/lib/openshell-gpu-workload/vectorAdd
65+
COPY --from=builder /opt/openshell-gpu-workload/cuda-samples.LICENSE /usr/local/share/doc/openshell-gpu-workload/cuda-samples.LICENSE
66+
COPY workload.sh /usr/local/bin/openshell-gpu-workload
67+
RUN chmod 0755 /usr/local/bin/openshell-gpu-workload \
68+
/usr/local/lib/openshell-gpu-workload/deviceQuery \
69+
/usr/local/lib/openshell-gpu-workload/vectorAdd
70+
71+
USER sandbox
72+
ENTRYPOINT ["/usr/local/bin/openshell-gpu-workload"]
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
2+
<!-- SPDX-License-Identifier: Apache-2.0 -->
3+
4+
# GPU workload CUDA basic
5+
6+
`cuda-basic` validates that a GPU-enabled environment can run a basic CUDA
7+
runtime workload. It is a single image that runs two validation steps:
8+
9+
1. `deviceQuery` checks CUDA runtime, driver, and device discovery.
10+
2. `vectorAdd` checks kernel launch, device memory allocation, host/device
11+
copies, synchronization, and result validation.
12+
13+
The image builds the samples from `NVIDIA/cuda-samples` tag `v12.8` with a CUDA
14+
12.8 builder image, then copies only the compiled binaries into the OpenShell
15+
community base final image.
16+
17+
The workload prints `OPENSHELL_GPU_WORKLOAD_SUCCESS` only after both samples
18+
pass. On failure it prints `OPENSHELL_GPU_WORKLOAD_FAILURE` and exits non-zero.
19+
20+
Build it with:
21+
22+
```shell
23+
OPENSHELL_GPU_WORKLOAD_IMAGES=cuda-basic mise run e2e:gpu:images:build
24+
```
25+
26+
Run it directly with Docker CDI:
27+
28+
```shell
29+
source e2e/gpu/images/.build/latest.env
30+
docker run --rm --device nvidia.com/gpu=all \
31+
"${OPENSHELL_E2E_GPU_CUDA_WORKLOAD_IMAGE}"
32+
```
33+
34+
Use `podman run` with the same `--device nvidia.com/gpu=all` option when Podman
35+
CDI is configured.
36+
37+
The image does not vendor GPU driver libraries such as `libcuda.so.1`. Those
38+
libraries must be provided by the host GPU runtime or CDI injection.
39+
40+
The CUDA samples are redistributed under the NVIDIA CUDA samples license. The
41+
license text is copied into the image at
42+
`/usr/local/share/doc/openshell-gpu-workload/cuda-samples.LICENSE`.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
set -euo pipefail
7+
8+
readonly SUCCESS_MARKER="OPENSHELL_GPU_WORKLOAD_SUCCESS"
9+
readonly FAILURE_MARKER="OPENSHELL_GPU_WORKLOAD_FAILURE"
10+
readonly WORKLOAD_DIR="/usr/local/lib/openshell-gpu-workload"
11+
12+
run_sample() {
13+
local name=$1
14+
local expected=$2
15+
local binary="${WORKLOAD_DIR}/${name}"
16+
local output
17+
18+
output="$(mktemp)"
19+
echo "running CUDA sample: ${name}"
20+
if ! "${binary}" >"${output}" 2>&1; then
21+
cat "${output}"
22+
echo "${FAILURE_MARKER} ${name} exited non-zero" >&2
23+
rm -f "${output}"
24+
exit 1
25+
fi
26+
27+
cat "${output}"
28+
if ! grep -Fq "${expected}" "${output}"; then
29+
echo "${FAILURE_MARKER} ${name} did not print expected output: ${expected}" >&2
30+
rm -f "${output}"
31+
exit 1
32+
fi
33+
34+
rm -f "${output}"
35+
}
36+
37+
run_sample "deviceQuery" "Result = PASS"
38+
run_sample "vectorAdd" "Test PASSED"
39+
40+
echo "${SUCCESS_MARKER} cuda-basic"
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
ARG OPENSHELL_SANDBOX_BASE_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/base:latest
7+
8+
FROM ${OPENSHELL_SANDBOX_BASE_IMAGE}
9+
10+
USER root
11+
COPY workload.sh /usr/local/bin/openshell-gpu-workload
12+
RUN chmod 0755 /usr/local/bin/openshell-gpu-workload
13+
14+
USER sandbox
15+
ENTRYPOINT ["/usr/local/bin/openshell-gpu-workload"]
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
2+
<!-- SPDX-License-Identifier: Apache-2.0 -->
3+
4+
# GPU workload smoke fail
5+
6+
`smoke-fail` validates negative-path diagnostics in e2e test plumbing.
7+
8+
The workload does not perform GPU-specific work. It prints
9+
`OPENSHELL_GPU_WORKLOAD_FAILURE`, emits a stable diagnostic, and exits non-zero.
10+
11+
Build it with:
12+
13+
```shell
14+
OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-fail mise run e2e:gpu:images:build
15+
```
16+
17+
Run it directly:
18+
19+
```shell
20+
source e2e/gpu/images/.build/latest.env
21+
docker run --rm "${OPENSHELL_E2E_GPU_SMOKE_FAIL_IMAGE}"
22+
```
23+
24+
The direct run should fail.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
set -euo pipefail
7+
8+
echo "OPENSHELL_GPU_WORKLOAD_FAILURE smoke-fail intentional failure" >&2
9+
exit 42
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
ARG OPENSHELL_SANDBOX_BASE_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/base:latest
7+
8+
FROM ${OPENSHELL_SANDBOX_BASE_IMAGE}
9+
10+
USER root
11+
COPY workload.sh /usr/local/bin/openshell-gpu-workload
12+
RUN chmod 0755 /usr/local/bin/openshell-gpu-workload
13+
14+
USER sandbox
15+
ENTRYPOINT ["/usr/local/bin/openshell-gpu-workload"]
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
2+
<!-- SPDX-License-Identifier: Apache-2.0 -->
3+
4+
# GPU workload smoke pass
5+
6+
`smoke-pass` validates image publishing, sandbox image compatibility, default
7+
entrypoint execution, and success-marker assertion plumbing.
8+
9+
The workload does not perform GPU-specific work. It prints
10+
`OPENSHELL_GPU_WORKLOAD_SUCCESS` and exits `0`.
11+
12+
Build it with:
13+
14+
```shell
15+
OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-pass mise run e2e:gpu:images:build
16+
```
17+
18+
Run it directly:
19+
20+
```shell
21+
source e2e/gpu/images/.build/latest.env
22+
docker run --rm "${OPENSHELL_E2E_GPU_SMOKE_PASS_IMAGE}"
23+
```

0 commit comments

Comments
 (0)