Skip to content

Commit 8c01534

Browse files
authored
test(e2e): add GPU workload image artifacts (#1484)
* test(e2e): add GPU workload image artifacts Closes #1476 Define local GPU workload image sources for smoke-pass, smoke-fail, and cuda-basic validation, plus a mise build task that tags images with the source revision and records the latest local image refs. Signed-off-by: Evan Lezar <[email protected]> * test(e2e): add workload manifest build flow Signed-off-by: Evan Lezar <[email protected]> * test(e2e): fix gpu workload README lint Signed-off-by: Evan Lezar <[email protected]> * test(e2e): address gpu workload review feedback Signed-off-by: Evan Lezar <[email protected]> --------- Signed-off-by: Evan Lezar <[email protected]>
1 parent 6c8cf38 commit 8c01534

13 files changed

Lines changed: 661 additions & 0 deletions

File tree

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,7 @@ logs/
168168
tmp/
169169
temp/
170170
*.tmp
171+
e2e/gpu/images/.build/
171172

172173
# Secrets/credentials (should never be committed)
173174
*.pem

‎e2e/gpu/README.md‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
2+
<!-- SPDX-License-Identifier: Apache-2.0 -->
3+
4+
# GPU workload images
5+
6+
This directory defines workload test images for OpenShell GPU validation.
7+
8+
## Contract
9+
10+
Each workload image must:
11+
12+
- Use the standard OpenShell sandbox base image as its final-stage base or
13+
ensure that the requirements for a sandbox image are met.
14+
- Provide a manifest command that runs the workload inside the sandbox image.
15+
- Run the same workload as the image default entrypoint for direct
16+
container-engine validation.
17+
- Require no network access after the image is pulled.
18+
- Print `OPENSHELL_GPU_WORKLOAD_SUCCESS` only when validation succeeds.
19+
- Print `OPENSHELL_GPU_WORKLOAD_FAILURE` and exit non-zero when validation
20+
fails.
21+
- Be usable as an OpenShell sandbox image when OpenShell invokes the manifest
22+
command explicitly.
23+
24+
OpenShell sandbox creation replaces the image entrypoint with the supervisor and
25+
does not run the OCI image `CMD`. When these images are used through OpenShell,
26+
the workload command from each manifest entry must be passed explicitly.
27+
28+
The image build task writes a local workload manifest. Each workload entry
29+
carries:
30+
31+
- `name`
32+
- `image`
33+
- `command`
34+
- `expect`
35+
- `requirements`
36+
37+
## Images
38+
39+
| Source directory | Image name | Purpose |
40+
| --- | --- | --- |
41+
| `smoke-pass` | `gpu-workload-smoke-pass` | Always succeeds and prints the success marker. |
42+
| `smoke-fail` | `gpu-workload-smoke-fail` | Always fails and prints the failure marker. |
43+
| `cuda-basic` | `gpu-workload-cuda-basic` | Runs CUDA `deviceQuery` and `vectorAdd` validation. |
44+
45+
## Build
46+
47+
Build all workload images:
48+
49+
```shell
50+
mise run e2e:workloads:build
51+
```
52+
53+
Build a subset by source directory name:
54+
55+
```shell
56+
OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-pass,smoke-fail \
57+
mise run e2e:workloads:build
58+
```
59+
60+
The build task uses `tasks/scripts/container-engine.sh`. Set
61+
`CONTAINER_ENGINE=docker` or `CONTAINER_ENGINE=podman` to choose an engine
62+
explicitly. When unset, the helper uses its existing auto-detection behavior.
63+
64+
Local tags use the current commit short SHA plus a short fingerprint of the
65+
external build inputs. Dirty local trees append `-dirty`. Set
66+
`OPENSHELL_GPU_WORKLOAD_IMAGE_TAG=<tag>` to override the tag.
67+
68+
The task writes the latest build refs to:
69+
70+
```text
71+
e2e/gpu/images/.build/latest.env
72+
```
73+
74+
The task also writes a local workload manifest for downstream tooling and
75+
future workload-runner integration:
76+
77+
```text
78+
e2e/gpu/images/.build/workloads.yaml
79+
```
80+
81+
That local manifest is created by `mise run e2e:workloads:build`. It contains
82+
the full image reference, command, expected outcome, and requirements for each
83+
selected workload. It also records the external build inputs used to produce
84+
the workload images.
85+
86+
Use the env file in later commands:
87+
88+
```shell
89+
source e2e/gpu/images/.build/latest.env
90+
```
91+
92+
That env file exports `OPENSHELL_E2E_WORKLOAD_MANIFEST` pointing at the local
93+
manifest. The current checked-in Rust GPU e2e target does not consume this
94+
manifest yet. The per-image refs remain available as a convenience for direct
95+
container-engine validation.
96+
97+
## Direct Validation
98+
99+
Validate smoke pass:
100+
101+
```shell
102+
docker run --rm "${OPENSHELL_E2E_GPU_SMOKE_PASS_IMAGE}"
103+
```
104+
105+
Validate smoke fail:
106+
107+
```shell
108+
docker run --rm "${OPENSHELL_E2E_GPU_SMOKE_FAIL_IMAGE}"
109+
```
110+
111+
The smoke fail command should exit non-zero and print
112+
`OPENSHELL_GPU_WORKLOAD_FAILURE`.
113+
114+
Validate CUDA with Docker CDI:
115+
116+
```shell
117+
docker run --rm --device nvidia.com/gpu=all \
118+
"${OPENSHELL_E2E_GPU_CUDA_WORKLOAD_IMAGE}"
119+
```
120+
121+
Use `podman run` with the same `--device nvidia.com/gpu=all` option on hosts
122+
where Podman CDI is configured.
123+
124+
Direct container-engine validation catches image, CDI, CUDA, and host GPU setup
125+
issues before OpenShell sandbox behavior is involved.
126+
127+
## OpenShell GPU E2E
128+
129+
The current Rust GPU validation target is:
130+
131+
```shell
132+
mise run e2e:gpu
133+
```
134+
135+
That target runs `gpu_device_selection`. It validates GPU request and device
136+
selection behavior against a Docker-backed gateway. It does not run the
137+
workload manifest generated by `mise run e2e:workloads:build`.
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
ARG CUDA_BUILD_IMAGE=nvcr.io/nvidia/cuda:12.8.1-base-ubuntu22.04
7+
ARG OPENSHELL_SANDBOX_BASE_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/base:latest
8+
9+
FROM ${CUDA_BUILD_IMAGE} AS builder
10+
11+
ARG DEBIAN_FRONTEND=noninteractive
12+
ARG CUDA_SAMPLES_REF=v12.8
13+
ARG CUDA_SAMPLES_REPO=https://github.com/NVIDIA/cuda-samples
14+
15+
RUN apt-get update && apt-get install -y --no-install-recommends \
16+
build-essential \
17+
ca-certificates \
18+
cmake \
19+
cuda-nvcc-12-8 \
20+
curl \
21+
g++ \
22+
&& rm -rf /var/lib/apt/lists/*
23+
24+
WORKDIR /build/cuda-samples
25+
26+
RUN set -eux; \
27+
curl -fsSL "${CUDA_SAMPLES_REPO}/archive/refs/tags/${CUDA_SAMPLES_REF}.tar.gz" \
28+
-o /tmp/cuda-samples.tar.gz; \
29+
tar -xzf /tmp/cuda-samples.tar.gz \
30+
--strip-components=1 \
31+
--wildcards \
32+
'*/Common/*' \
33+
'*/cmake/*' \
34+
'*/Samples/0_Introduction/vectorAdd/*' \
35+
'*/Samples/1_Utilities/deviceQuery/*' \
36+
'*/LICENSE'; \
37+
sed -i 's/CUDA::cudart/CUDA::cudart_static/g' \
38+
Samples/1_Utilities/deviceQuery/CMakeLists.txt; \
39+
cmake -S Samples/1_Utilities/deviceQuery -B /tmp/build-device-query \
40+
-DCMAKE_BUILD_TYPE=Release \
41+
-DCMAKE_CUDA_RUNTIME_LIBRARY=Static; \
42+
cmake --build /tmp/build-device-query --parallel; \
43+
cmake -S Samples/0_Introduction/vectorAdd -B /tmp/build-vector-add \
44+
-DCMAKE_BUILD_TYPE=Release \
45+
-DCMAKE_CUDA_RUNTIME_LIBRARY=Static; \
46+
cmake --build /tmp/build-vector-add --parallel; \
47+
mkdir -p /opt/openshell-gpu-workload; \
48+
cp /tmp/build-device-query/deviceQuery /opt/openshell-gpu-workload/deviceQuery; \
49+
cp /tmp/build-vector-add/vectorAdd /opt/openshell-gpu-workload/vectorAdd; \
50+
cp LICENSE /opt/openshell-gpu-workload/cuda-samples.LICENSE; \
51+
rm -f /tmp/cuda-samples.tar.gz
52+
53+
FROM ${OPENSHELL_SANDBOX_BASE_IMAGE}
54+
55+
ARG CUDA_SAMPLES_REF=v12.8
56+
57+
LABEL com.nvidia.openshell.gpu-workload.name="cuda-basic" \
58+
com.nvidia.openshell.gpu-workload.cuda-samples-ref="${CUDA_SAMPLES_REF}"
59+
60+
USER root
61+
RUN mkdir -p /usr/local/lib/openshell-gpu-workload \
62+
/usr/local/share/doc/openshell-gpu-workload
63+
COPY --from=builder /opt/openshell-gpu-workload/deviceQuery /usr/local/lib/openshell-gpu-workload/deviceQuery
64+
COPY --from=builder /opt/openshell-gpu-workload/vectorAdd /usr/local/lib/openshell-gpu-workload/vectorAdd
65+
COPY --from=builder /opt/openshell-gpu-workload/cuda-samples.LICENSE /usr/local/share/doc/openshell-gpu-workload/cuda-samples.LICENSE
66+
COPY workload.sh /usr/local/bin/openshell-gpu-workload
67+
RUN chmod 0755 /usr/local/bin/openshell-gpu-workload \
68+
/usr/local/lib/openshell-gpu-workload/deviceQuery \
69+
/usr/local/lib/openshell-gpu-workload/vectorAdd
70+
71+
USER sandbox
72+
ENTRYPOINT ["/usr/local/bin/openshell-gpu-workload"]
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
2+
<!-- SPDX-License-Identifier: Apache-2.0 -->
3+
4+
# GPU workload CUDA basic
5+
6+
`cuda-basic` validates that a GPU-enabled environment can run a basic CUDA
7+
runtime workload. It is a single image that runs two validation steps:
8+
9+
1. `deviceQuery` checks CUDA runtime, driver, and device discovery.
10+
2. `vectorAdd` checks kernel launch, device memory allocation, host/device
11+
copies, synchronization, and result validation.
12+
13+
The image builds the samples from `NVIDIA/cuda-samples` tag `v12.8` with a CUDA
14+
12.8 builder image, then copies only the compiled binaries into the OpenShell
15+
community base final image.
16+
17+
The workload prints `OPENSHELL_GPU_WORKLOAD_SUCCESS` only after both samples
18+
pass. On failure it prints `OPENSHELL_GPU_WORKLOAD_FAILURE` and exits non-zero.
19+
20+
Build it with:
21+
22+
```shell
23+
mise run e2e:workloads:build
24+
```
25+
26+
That command also refreshes the local workload manifest at
27+
`e2e/gpu/images/.build/workloads.yaml`.
28+
29+
To build only this workload locally, set:
30+
31+
```shell
32+
OPENSHELL_GPU_WORKLOAD_IMAGES=cuda-basic mise run e2e:workloads:build
33+
```
34+
35+
Run it directly with Docker CDI:
36+
37+
```shell
38+
source e2e/gpu/images/.build/latest.env
39+
docker run --rm --device nvidia.com/gpu=all \
40+
"${OPENSHELL_E2E_GPU_CUDA_WORKLOAD_IMAGE}"
41+
```
42+
43+
Use `podman run` with the same `--device nvidia.com/gpu=all` option when Podman
44+
CDI is configured.
45+
46+
The image does not vendor GPU driver libraries such as `libcuda.so.1`. Those
47+
libraries must be provided by the host GPU runtime or CDI injection.
48+
49+
The CUDA samples are redistributed under the NVIDIA CUDA samples license. The
50+
license text is copied into the image at
51+
`/usr/local/share/doc/openshell-gpu-workload/cuda-samples.LICENSE`.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
set -euo pipefail
7+
8+
readonly SUCCESS_MARKER="OPENSHELL_GPU_WORKLOAD_SUCCESS"
9+
readonly FAILURE_MARKER="OPENSHELL_GPU_WORKLOAD_FAILURE"
10+
readonly WORKLOAD_DIR="/usr/local/lib/openshell-gpu-workload"
11+
12+
run_sample() {
13+
local name=$1
14+
local expected=$2
15+
local binary="${WORKLOAD_DIR}/${name}"
16+
local output
17+
18+
output="$(mktemp)"
19+
echo "running CUDA sample: ${name}"
20+
if ! "${binary}" >"${output}" 2>&1; then
21+
cat "${output}"
22+
echo "${FAILURE_MARKER} ${name} exited non-zero" >&2
23+
rm -f "${output}"
24+
exit 1
25+
fi
26+
27+
cat "${output}"
28+
if ! grep -Fq "${expected}" "${output}"; then
29+
echo "${FAILURE_MARKER} ${name} did not print expected output: ${expected}" >&2
30+
rm -f "${output}"
31+
exit 1
32+
fi
33+
34+
rm -f "${output}"
35+
}
36+
37+
run_sample "deviceQuery" "Result = PASS"
38+
run_sample "vectorAdd" "Test PASSED"
39+
40+
echo "${SUCCESS_MARKER} cuda-basic"
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
ARG OPENSHELL_SANDBOX_BASE_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/base:latest
7+
8+
FROM ${OPENSHELL_SANDBOX_BASE_IMAGE}
9+
10+
USER root
11+
COPY workload.sh /usr/local/bin/openshell-gpu-workload
12+
RUN chmod 0755 /usr/local/bin/openshell-gpu-workload
13+
14+
USER sandbox
15+
ENTRYPOINT ["/usr/local/bin/openshell-gpu-workload"]
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
2+
<!-- SPDX-License-Identifier: Apache-2.0 -->
3+
4+
# GPU workload smoke fail
5+
6+
`smoke-fail` validates negative-path diagnostics in e2e test plumbing.
7+
8+
The workload does not perform GPU-specific work. It prints
9+
`OPENSHELL_GPU_WORKLOAD_FAILURE`, emits a stable diagnostic, and exits non-zero.
10+
11+
Build it with:
12+
13+
```shell
14+
mise run e2e:workloads:build
15+
```
16+
17+
That command also refreshes the local workload manifest at
18+
`e2e/gpu/images/.build/workloads.yaml`.
19+
20+
To build only this workload locally, set:
21+
22+
```shell
23+
OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-fail mise run e2e:workloads:build
24+
```
25+
26+
Run it directly:
27+
28+
```shell
29+
source e2e/gpu/images/.build/latest.env
30+
docker run --rm "${OPENSHELL_E2E_GPU_SMOKE_FAIL_IMAGE}"
31+
```
32+
33+
The direct run should fail.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env bash
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
set -euo pipefail
7+
8+
echo "OPENSHELL_GPU_WORKLOAD_FAILURE smoke-fail intentional failure" >&2
9+
exit 42
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
ARG OPENSHELL_SANDBOX_BASE_IMAGE=ghcr.io/nvidia/openshell-community/sandboxes/base:latest
7+
8+
FROM ${OPENSHELL_SANDBOX_BASE_IMAGE}
9+
10+
USER root
11+
COPY workload.sh /usr/local/bin/openshell-gpu-workload
12+
RUN chmod 0755 /usr/local/bin/openshell-gpu-workload
13+
14+
USER sandbox
15+
ENTRYPOINT ["/usr/local/bin/openshell-gpu-workload"]

0 commit comments

Comments
 (0)