Skip to content

Commit 5ba523c

Browse files
committed
docs(gpu): add pre-upgrade migration steps for hand-edited permittedHostDevices
The bundle now owns spec.configuration.permittedHostDevices, so the first reconcile after upgrade overwrites manual kubectl-edit entries with the NVIDIA default table. Tell operators to move custom entries into .gpu.permittedHostDevices and verify each resourceName against node-advertised names before upgrading, since the default slugs (e.g. TU104GL_T4) differ from legacy names (e.g. TU104GL_TESLA_T4) and a mismatch silently rejects GPU VMs. Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent d4ab140 commit 5ba523c

1 file changed

Lines changed: 23 additions & 0 deletions

File tree

  • content/en/docs/next/virtualization

‎content/en/docs/next/virtualization/gpu.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,29 @@ gpu:
136136
externalResourceProvider: true
137137
```
138138
139+
### Upgrading from a hand-edited KubeVirt CR
140+
141+
Earlier Cozystack releases left `spec.configuration.permittedHostDevices` for operators to hand-edit (`kubectl edit kubevirt`). The bundle now **owns** that field: the first reconcile after the upgrade replaces your manual entries with the rendered NVIDIA default table.
142+
143+
Before upgrading:
144+
145+
1. Dump your current entries:
146+
147+
```bash
148+
kubectl get kubevirt -n cozy-kubevirt -o yaml \
149+
| yq '.items[0].spec.configuration.permittedHostDevices'
150+
```
151+
152+
2. Move any custom entries into the Platform Package values under `.gpu.permittedHostDevices` (set `.gpu.replaceDefaults: true` if you want only your own list instead of appending to the NVIDIA defaults).
153+
154+
3. Verify every `resourceName` against what your nodes actually advertise — the default table uses `nvidia-sandbox-device-plugin` slugs (e.g. `nvidia.com/TU104GL_T4`) that differ from legacy driver names (e.g. `TU104GL_TESLA_T4`):
155+
156+
```bash
157+
kubectl describe node <node> | grep nvidia.com/
158+
```
159+
160+
A `resourceName` mismatch is silent until a GPU VM restarts or migrates, at which point the admission webhook rejects it.
161+
139162
### Manual Package-CR override path
140163
141164
If you opt out of bundle management and hand-craft a `cozystack.gpu-operator` Package CR directly (to apply overrides the bundle does not expose — driver settings, custom node selectors, validator / dcgmExporter tweaks), the platform does NOT auto-wire `HostDevices` or `permittedHostDevices` into the KubeVirt CR. In that flow, mirror the bundle behaviour by also creating a `cozystack.kubevirt` Package CR with `components.kubevirt.values.extraFeatureGates: [HostDevices]` and the appropriate `permittedHostDevices` block. The manual Package-CR override path takes precedence over the bundle render whenever both exist.

0 commit comments

Comments
 (0)