Skip to content

Commit 6751633

Browse files
authored
fix(platform): accept the container gpu-operator variant in iaas (#4101)
## What this PR does `bundles.iaas.gpuOperatorVariant: container` failed the whole platform render. The `cozystack.gpu-operator` PackageSource ships that variant, `values-container.yaml` configures it deliberately, and `hack/check-gpu-operator-variants.bats` covers it, but the `iaas` bundle guard only accepted `default` and `vgpu`. The exclusion made sense when it was written and stopped making sense a few minutes later. The variant and the guard came from parallel branches, neither an ancestor of the other, variant merged first. Before the guard the bundle hardcoded `default`, so `container` was selected by hand-writing a Package CR, and leaving it out of a bundle-side list cost nothing. The guard made the bundle the variant owner and closed that route without noticing it was one. Its comment calls itself typo protection and names `passthrough` as the example typo. Allowing the value is not the whole fix. The same file puts `extraFeatureGates: [HostDevices]` on the KubeVirt CR whenever gpu-operator is enabled, and builds a `permittedHostDevices` table under it. Both are VM-facing. The container variant keeps the host driver bound and pins `vfioManager` off, so no GPU reaches a VM: the gate would advertise something the node cannot serve, and the table would name resources no sandbox device plugin registers there. Both now render for `default` and `vgpu` only, and `container` gets the gpu-operator Package with the kubevirt component untouched. The helm-unittest case that pinned the rejection is gone. Two replace it: the Package renders at the container variant with no component overrides, and the KubeVirt wiring stays absent even with `gpu.permittedHostDevices` and `gpu.mediatedDevicesConfiguration` set. Comments and docs that called the mirroring unconditional on variant, or the container variant unmerged, are fixed in the same pass. No issue for this one, so the history is in the body. ### Screenshots No UI change. ### Downstream repositories Walked the trigger map against the diff. The only file it touches that appears there is `packages/core/platform/values.yaml`, and the website hit is not the hand-written key table (that page has no GPU section): it is the documented workaround this change makes obsolete. `content/en/docs/next/operations/gpu-container-workloads.md` currently tells readers the bundle rejects `container` and routes them to a hand-crafted Package CR, and `virtualization/gpu.md` / `vgpu.md` describe the KubeVirt mirroring as unconditional on variant. Filed as an issue rather than a PR because the docs must not land ahead of this change, and the `v1.6` copies describe released behaviour and stay correct as they are. Nothing else matches: no package added, renamed or removed, no `values.schema.json` or version enum touched, no `ApplicationDefinition` semantics, no installer values key, no `hack/` layout change, no platform variant or bundle recomposed. `gpuOperatorVariant` appears in none of `ccp`, `ansible-cozystack` or `terraform-provider-cozystack`. - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: cozystack/website#685 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(platform): accept `container` as a value for `bundles.iaas.gpuOperatorVariant`, so the gpu-operator variant the platform already ships can be selected from platform values instead of a hand-crafted Package CR. The KubeVirt `HostDevices` feature gate and `permittedHostDevices` table are now emitted for the `default` and `vgpu` variants only; `container` serves GPUs to pods and gets neither. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added the `container` GPU operator variant for workloads that do not require VM passthrough. * Container workloads no longer receive KubeVirt host-device or mediated-device GPU wiring. * Clarified GPU configuration behavior for passthrough, vGPU, SR-IOV, and container workloads. * **Documentation** * Updated passthrough guidance, configuration references, examples, and KubeVirt wiring instructions for all supported variants. * Added details for manually configured package resources and platform-specific container setup. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents b7e6085 + 6e8566a commit 6751633

5 files changed

Lines changed: 164 additions & 43 deletions

File tree

‎docs/gpu-vgpu.md‎

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,9 @@ The `default` (passthrough) variant assumes the GPU is **owned by the host kerne
3939

4040
**`FORCE_REINSTALL` does not bypass this.** `k8s-driver-manager` v0.10.0 exposes a `FORCE_REINSTALL` / `--force-reinstall` env+flag pair (visible in the binary's strings table), but it gates a later "same-config already loaded" branch inside `uninstallDriver`, not the `isHostDriver` short-circuit at the top — operators who set it and expect a bypass will report a false bug. There is currently no opt-out for the `isHostDriver` guard itself.
4141

42-
**Current mitigation.** The only mitigation available on `main` today is to remove the host NVIDIA stack before enabling the variant (the "Clean-host workaround" section below).
42+
**Current mitigation.** The only mitigation that keeps VM passthrough working is to remove the host NVIDIA stack before enabling the variant (the "Clean-host workaround" section below).
4343

44-
**Future alternatives and scope.** An upcoming `container` variant of `cozystack.gpu-operator` is tracked in [#2766](https://github.com/cozystack/cozystack/pull/2766) — once merged it will give operators a no-purge path that keeps the host driver and exposes GPUs to pods rather than VMs — but it is not part of the current package and is **not** a usable workaround until it lands. Talos is unaffected because the Talos image ships only the `vfio-pci` extension; this section applies to Linux distributions where you installed the host driver yourself (typically Ubuntu / Debian / RHEL with `apt install nvidia-driver-*` or equivalent). See also [`packages/system/gpu-operator/examples/README.md`](../packages/system/gpu-operator/examples/README.md) for the native-pod-workload reference flow that predates the `container` variant.
44+
**Alternatives and scope.** The `container` variant of `cozystack.gpu-operator` is the no-purge path: it keeps the host driver and exposes GPUs to pods rather than VMs, so it does not replace passthrough for anyone who actually needs a GPU inside a VM. Talos is unaffected because the Talos image ships only the `vfio-pci` extension; this section applies to Linux distributions where you installed the host driver yourself (typically Ubuntu / Debian / RHEL with `apt install nvidia-driver-*` or equivalent). See also [`packages/system/gpu-operator/examples/README.md`](../packages/system/gpu-operator/examples/README.md) for the native-pod-workload reference flow that predates the `container` variant.
4545

4646
### Symptom
4747

@@ -147,10 +147,10 @@ kubectl label node <NODE> nvidia.com/gpu.deploy.driver-
147147

148148
### Known limitation
149149

150-
The skip-on-pre-installed behavior lives in the upstream [`NVIDIA/k8s-driver-manager`](https://github.com/NVIDIA/k8s-driver-manager) Go binary at `cmd/driver-manager/main.go`: the method `(*DriverManager).isHostDriver` is called from `(*DriverManager).uninstallDriver` and has no in-band opt-out in `:v0.10.0`. `FORCE_REINSTALL` exists as an env / CLI flag in the same binary but gates the later "same-config already loaded" branch inside `uninstallDriver`, not the `isHostDriver` short-circuit. Hosts that need to keep the NVIDIA host driver installed for non-Kubernetes workloads cannot currently share the same GPU with the passthrough variant. Two future mitigation paths are in flight; **neither is available today**:
150+
The skip-on-pre-installed behavior lives in the upstream [`NVIDIA/k8s-driver-manager`](https://github.com/NVIDIA/k8s-driver-manager) Go binary at `cmd/driver-manager/main.go`: the method `(*DriverManager).isHostDriver` is called from `(*DriverManager).uninstallDriver` and has no in-band opt-out in `:v0.10.0`. `FORCE_REINSTALL` exists as an env / CLI flag in the same binary but gates the later "same-config already loaded" branch inside `uninstallDriver`, not the `isHostDriver` short-circuit. Hosts that need to keep the NVIDIA host driver installed for non-Kubernetes workloads cannot currently share the same GPU with the passthrough variant. Two mitigation paths:
151151

152-
- **`container` variant — not yet merged** — [#2766](https://github.com/cozystack/cozystack/pull/2766) adds a third variant of `cozystack.gpu-operator` that targets the apt-installed-driver host shape and exposes GPUs to pods (not VMs) without unbinding the host driver. Once it lands, operators on hosts where the purge workaround is unacceptable will have a path that does not require touching the host driver.
153-
- **Upstream override** — an env-var override of `isHostDriver` is the only structural fix that would let the passthrough variant coexist with a host driver. Requested in [NVIDIA/k8s-driver-manager#191](https://github.com/NVIDIA/k8s-driver-manager/issues/191).
152+
- **`container` variant** — a third variant of `cozystack.gpu-operator` that targets the apt-installed-driver host shape and exposes GPUs to pods (not VMs) without unbinding the host driver. Available now, and selectable from platform values with `bundles.iaas.gpuOperatorVariant: container`. It solves the host-driver conflict by giving up VM passthrough, so it is a mitigation only for workloads that can run in a pod.
153+
- **Upstream override** — an env-var override of `isHostDriver` is the only structural fix that would let the passthrough variant coexist with a host driver. Not available today; requested in [NVIDIA/k8s-driver-manager#191](https://github.com/NVIDIA/k8s-driver-manager/issues/191).
154154

155155
## Building the vGPU Manager image
156156

@@ -218,25 +218,27 @@ For Pascal–Ampere GPUs (V100, T4, A100, A30) the mdev model still applies. Fli
218218

219219
## KubeVirt configuration
220220

221-
When `cozystack.gpu-operator` is in `bundles.enabledPackages` (and not also in `bundles.disabledPackages`), the platform mirrors the chosen GPU variant into the `KubeVirt` CR automatically. There is no manual `kubectl patch` step.
221+
When `cozystack.gpu-operator` is in `bundles.enabledPackages` (and not also in `bundles.disabledPackages`), the platform mirrors the chosen GPU variant into the `KubeVirt` CR automatically. There is no manual `kubectl patch` step. This applies to the two VM variants only: `container` serves GPUs to pods and never to VMs, so selecting it emits the gpu-operator Package with no host-device wiring on the KubeVirt side.
222222

223223
If you opt out of bundle management and hand-craft a `cozystack.gpu-operator` Package CR directly — typically to apply overrides the bundle does not expose (driver settings, custom node selectors, validator / dcgmExporter tweaks, etc.) — the platform does NOT auto-wire `HostDevices` or `permittedHostDevices` into the KubeVirt CR. In that flow you also hand-craft a `cozystack.kubevirt` Package CR with `components.kubevirt.values.extraFeatureGates: [HostDevices]` and the appropriate `permittedHostDevices` block. The escape-hatch values shape under `.gpu` (below) is intentionally documented in the bundle-managed flow only. Both hand-crafted Package CRs need their names in `bundles.disabledPackages`: the iaas bundle emits `cozystack.kubevirt` on every render and `cozystack.gpu-operator` on every render that has it in `bundles.enabledPackages`, and that render replaces the hand-written spec rather than merging with it.
224224

225225
- `developerConfiguration.featureGates` gets `HostDevices` appended (current KubeVirt splits this from the `GPU` gate; the admission webhook rejects `spec.template.spec.domain.devices.hostDevices` without it).
226-
- `permittedHostDevices.pciHostDevices` is filled from `packages/core/platform/files/gpu-passthrough-defaults.yaml` when `bundles.iaas.gpuOperatorVariant: default` (the package default). The table covers Hopper (H100/H200), Ada Lovelace (L4/L40/L40S), Ampere (A100 PCIe/SXM, A40, A30, A10), Turing (T4), Volta (V100/V100S). All entries carry `externalResourceProvider: true` because the resource names come from `nvidia-sandbox-device-plugin`, not from KubeVirt's in-tree device plugin.
227-
- `permittedHostDevices.mediatedDevices` is filled from `packages/core/platform/files/gpu-vgpu-defaults.yaml` when `bundles.iaas.gpuOperatorVariant: vgpu`. This list only EXPOSES, by profile name (`mdevNameSelector`), mdevs that the GPU Operator's vGPU Device Manager CREATES on the node; the platform does not ship a numeric `mediatedDevicesConfiguration` default (those `nvidia-NNN` type ids are per-SKU/driver sysfs indices with no portable value — set `.gpu.mediatedDevicesConfiguration` yourself, with host-verified ids, only if you want KubeVirt rather than the Device Manager to create mdevs). The starter set covers Pascal–Ampere mdev profiles (A100-40C/80C, A40-24Q/48Q, A30-24C, A10-24Q, V100D-32C, T4-16Q) — the same family range as the upstream `vgpu-device-manager` walks `/sys/class/mdev_bus/` for. Ada Lovelace / Blackwell SR-IOV vGPU is out of scope for the chart's default list; advertise those VFs via the user-override hook below.
226+
- `permittedHostDevices.pciHostDevices` is filled from `packages/core/platform/files/gpu-passthrough-defaults.yaml` when `bundles.iaas.gpuOperatorVariant: default` (the package default) and `gpu.replaceDefaults` is left at `false`. The table covers Hopper (H100/H200), Ada Lovelace (L4/L40/L40S), Ampere (A100 PCIe/SXM, A40, A30, A10), Turing (T4), Volta (V100/V100S). All entries carry `externalResourceProvider: true` because the resource names come from `nvidia-sandbox-device-plugin`, not from KubeVirt's in-tree device plugin.
227+
- `permittedHostDevices.mediatedDevices` is filled from `packages/core/platform/files/gpu-vgpu-defaults.yaml` when `bundles.iaas.gpuOperatorVariant: vgpu` **and** `gpu.vgpuDeviceManager.enabled: true` — that knob defaults to `false`, so a stock vgpu cluster ships no mdev table at all (see the Ada Lovelace / Blackwell note below for why). `gpu.replaceDefaults: true` also suppresses it. This list only EXPOSES, by profile name (`mdevNameSelector`), mdevs that the GPU Operator's vGPU Device Manager CREATES on the node; the platform does not ship a numeric `mediatedDevicesConfiguration` default (those `nvidia-NNN` type ids are per-SKU/driver sysfs indices with no portable value — set `.gpu.mediatedDevicesConfiguration` yourself, with host-verified ids, only if you want KubeVirt rather than the Device Manager to create mdevs). The starter set covers Pascal–Ampere mdev profiles (A100-40C/80C, A40-24Q/48Q, A30-24C, A10-24Q, V100D-32C, T4-16Q) — the same family range as the upstream `vgpu-device-manager` walks `/sys/class/mdev_bus/` for. Ada Lovelace / Blackwell SR-IOV vGPU is out of scope for the chart's default list; advertise those VFs via the user-override hook below.
228228

229229
### Extending or replacing the default table
230230

231231
The platform exposes three knobs under `.gpu`:
232232

233233
```yaml
234234
gpu:
235-
# Extend the platform defaults with cluster-specific entries. Both list
236-
# keys are read in both variants: pciHostDevices feeds the passthrough
235+
# Extend the platform defaults with cluster-specific entries. Read by
236+
# the two VM variants only; the container variant reads neither key.
237+
# pciHostDevices is read in both of them — it feeds the passthrough
237238
# (vfio-pci) path AND the post-kubevirt#16890 SR-IOV vGPU VF path on
238-
# Ada Lovelace / Blackwell; mediatedDevices feeds the pre-#16890 mdev
239-
# path on Pascal–Ampere. Both render into the same KubeVirt CR.
239+
# Ada Lovelace / Blackwell. mediatedDevices is read in vgpu alone, for
240+
# the pre-#16890 mdev path on Pascal–Ampere; the passthrough variant
241+
# ignores it. Both render into the same KubeVirt CR.
240242
permittedHostDevices:
241243
pciHostDevices:
242244
- pciVendorSelector: "10DE:26B9" # L40S, advertised as a VF for SR-IOV vGPU

‎packages/core/platform/templates/bundles/iaas.yaml‎

Lines changed: 32 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,25 @@
2828
{{- end -}}
2929
{{- /*
3030
When cozystack.gpu-operator is enabled the platform mirrors the variant
31-
into the KubeVirt CR: the HostDevices feature gate plus a starter
32-
permittedHostDevices block. Without this the admission webhook rejects
33-
any VM that references a host-device GPU by resourceName. See
34-
docs/gpu-vgpu.md for the user-facing knobs.
31+
into the KubeVirt CR: the HostDevices feature gate, without which the
32+
admission webhook rejects any VM that references a host-device GPU by
33+
resourceName, plus a permittedHostDevices block assembled per variant.
34+
The platform's own tables are the passthrough one in the default
35+
variant and the mdev one in vgpu behind gpu.vgpuDeviceManager.enabled;
36+
gpu.replaceDefaults drops whichever would apply, and the operator's own
37+
.gpu entries are appended on top (see the per-key comment below for
38+
which key each variant reads). A vgpu cluster at stock values therefore
39+
gets the gate and no device table. See docs/gpu-vgpu.md for the
40+
user-facing knobs.
41+
42+
Both pieces are VM-facing and are emitted only for the two variants
43+
that hand a GPU resource to a VM: default (the whole card, rebound to
44+
vfio-pci) and vgpu (mdev profiles or SR-IOV VFs carved out by the host
45+
vGPU driver). The container variant serves GPUs to ordinary pods
46+
through the device plugin and runs no VM-facing GPU stack at all —
47+
sandboxWorkloads and vfioManager are both off — so the gate would
48+
advertise a capability the node cannot serve and the device table
49+
would list resources no plugin ever registers there.
3550
*/ -}}
3651
{{- /*
3752
Mirror the cozystack.platform.package.optional helper's enable/disable
@@ -44,16 +59,21 @@
4459
{{- $gpuEnabled := and (has $gpuName (default (list) .Values.bundles.enabledPackages)) (not (has $gpuName (default (list) .Values.bundles.disabledPackages))) -}}
4560
{{- $gpuVariant := .Values.bundles.iaas.gpuOperatorVariant | default "default" -}}
4661
{{- /*
47-
Validate the variant up front, not behind $gpuEnabled — a typo like
62+
The accepted set is the variant list of the cozystack.gpu-operator
63+
PackageSource in sources/gpu-operator.yaml — default, vgpu, container.
64+
A variant added there needs adding here too, or the bundle rejects a
65+
value the package ships.
66+
67+
Validate up front, not behind $gpuEnabled — a typo like
4868
gpuOperatorVariant: passthrough (instead of "default") would otherwise
4969
render fine until someone flips gpu-operator into enabledPackages, at
5070
which point the bundle breaks. Failing fast here surfaces the typo on
5171
the first render after the values file changes.
5272
*/ -}}
53-
{{- if not (or (eq $gpuVariant "default") (eq $gpuVariant "vgpu")) -}}
54-
{{- fail (printf "bundles.iaas.gpuOperatorVariant must be \"default\" or \"vgpu\", got %q" $gpuVariant) -}}
73+
{{- if not (or (eq $gpuVariant "default") (eq $gpuVariant "vgpu") (eq $gpuVariant "container")) -}}
74+
{{- fail (printf "bundles.iaas.gpuOperatorVariant must be \"default\", \"vgpu\" or \"container\", got %q" $gpuVariant) -}}
5575
{{- end -}}
56-
{{- if $gpuEnabled }}
76+
{{- if and $gpuEnabled (ne $gpuVariant "container") }}
5777
{{- $_ := set $kubevirtValues "extraFeatureGates" (list "HostDevices") -}}
5878
{{- /*
5979
Each .Values.gpu access is fully parenthesized — (((.Values.gpu).x).y) —
@@ -72,9 +92,10 @@
7292
{{- $userMdevConfig := default (dict) (.Values.gpu).mediatedDevicesConfiguration -}}
7393
{{- $replace := default false (.Values.gpu).replaceDefaults -}}
7494
{{- /*
75-
pciHostDevices applies in both variants — passthrough (vfio-pci, whole
76-
GPU) and post-kubevirt#16890 SR-IOV vGPU VFs (Ada Lovelace / Blackwell
77-
with the vGPU 17/20 driver branch) both advertise via pciHostDevices.
95+
pciHostDevices applies in both VM variants — passthrough (vfio-pci,
96+
whole GPU) and post-kubevirt#16890 SR-IOV vGPU VFs (Ada Lovelace /
97+
Blackwell with the vGPU 17/20 driver branch) both advertise via
98+
pciHostDevices. The container variant never reaches this block.
7899
mediatedDevices is vgpu-only: it EXPOSES, by stable profile name
79100
(mdevNameSelector), the mdev devices the GPU Operator's vGPU Device
80101
Manager creates on the node. Emitting it in the passthrough variant

‎packages/core/platform/tests/bundles_gpu_kubevirt_wiring_test.yaml‎

Lines changed: 88 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -220,12 +220,12 @@ tests:
220220
variant: isp-full
221221
iaas:
222222
enabled: true
223-
gpuOperatorVariant: container
223+
gpuOperatorVariant: sriov
224224
enabledPackages:
225225
- cozystack.gpu-operator
226226
asserts:
227227
- failedTemplate:
228-
errorMessage: 'bundles.iaas.gpuOperatorVariant must be "default" or "vgpu", got "container"'
228+
errorMessage: 'bundles.iaas.gpuOperatorVariant must be "default", "vgpu" or "container", got "sriov"'
229229

230230
- it: rejects an invalid gpuOperatorVariant even when gpu-operator is not enabled so the typo is caught at edit time, not at gpu-enable time
231231
set:
@@ -239,7 +239,92 @@ tests:
239239
enabledPackages: []
240240
asserts:
241241
- failedTemplate:
242-
errorMessage: 'bundles.iaas.gpuOperatorVariant must be "default" or "vgpu", got "passthrough"'
242+
errorMessage: 'bundles.iaas.gpuOperatorVariant must be "default", "vgpu" or "container", got "passthrough"'
243+
244+
# The container variant is the third variant the cozystack.gpu-operator
245+
# PackageSource ships (values.yaml + values-container.yaml). The bundle
246+
# guard rejected it for its first release, which made a shipped and tested
247+
# variant unselectable from platform values.
248+
- it: emits the gpu-operator Package at the container variant so the third shipped variant is selectable from platform values
249+
set:
250+
bundles:
251+
system:
252+
enabled: true
253+
variant: isp-full
254+
iaas:
255+
enabled: true
256+
gpuOperatorVariant: container
257+
enabledPackages:
258+
- cozystack.gpu-operator
259+
# Both legs of the vgpu forwarding block are armed on purpose. It is
260+
# the only writer of spec.components on this Package and container
261+
# newly reaches its gate; with these unset the block yields an empty
262+
# dict either way, so the notExists below would hold even if the gate
263+
# admitted container.
264+
gpu:
265+
vgpuManager:
266+
imagePullSecrets:
267+
- nvidia-registry-secret
268+
vgpuDeviceManager:
269+
enabled: true
270+
documentSelector:
271+
path: metadata.name
272+
value: cozystack.gpu-operator
273+
asserts:
274+
- equal:
275+
path: kind
276+
value: Package
277+
- equal:
278+
path: spec.variant
279+
value: container
280+
- notExists:
281+
path: spec.components
282+
283+
# The container variant keeps the host NVIDIA driver bound and pins
284+
# vfioManager off, so no GPU can be detached for a VM. Emitting the
285+
# HostDevices gate would advertise a KubeVirt capability the node cannot
286+
# serve, and the pciHostDevices table would name resources that no
287+
# sandbox device plugin ever registers.
288+
- it: leaves the KubeVirt host-device wiring out entirely in the container variant, even with gpu.* overrides set
289+
set:
290+
bundles:
291+
system:
292+
enabled: true
293+
variant: isp-full
294+
iaas:
295+
enabled: true
296+
gpuOperatorVariant: container
297+
enabledPackages:
298+
- cozystack.gpu-operator
299+
gpu:
300+
permittedHostDevices:
301+
pciHostDevices:
302+
- pciVendorSelector: "1002:744C"
303+
resourceName: amd.com/MI300X
304+
externalResourceProvider: true
305+
mediatedDevices:
306+
- mdevNameSelector: NVIDIA A100-40C
307+
resourceName: nvidia.com/A100-40C
308+
externalResourceProvider: true
309+
mediatedDevicesConfiguration:
310+
mediatedDeviceTypes:
311+
- nvidia-559
312+
documentSelector:
313+
path: metadata.name
314+
value: cozystack.kubevirt
315+
asserts:
316+
- notExists:
317+
path: spec.components.kubevirt.values.extraFeatureGates
318+
- notExists:
319+
path: spec.components.kubevirt.values.permittedHostDevices
320+
- notExists:
321+
path: spec.components.kubevirt.values.mediatedDevicesConfiguration
322+
# The rest of the kubevirt component values are untouched by the
323+
# variant — this pins that the container branch skips only the GPU
324+
# wiring, not the whole component block.
325+
- equal:
326+
path: spec.components.kubevirt.values.cpuAllocationRatio
327+
value: 10
243328

244329
- it: emits the gpu-operator Package without component overrides for the passthrough variant so vfio-pci defaults from the chart apply unchanged
245330
set:

0 commit comments

Comments
 (0)