Skip to content

Commit 8dbc7bc

Browse files
committed
feat(kubevirt)!: drop HotplugVolumes for declarative volume hotplug
KubeVirt 1.9 deprecates HotplugVolumes in favour of DeclarativeHotplugVolumes, which is Beta and therefore on by default, but virt-controller skips declarative reconciliation for as long as the old gate is on. Keeping it was only needed because kubevirt-csi detached through the VMI-level removevolume, which virt-api accepts on a VM-owned VMI only with the old gate. kubevirt-csi can now skip that call, so its fallback goes off together with the gate. The two live in different HelmReleases and roll out one after the other; in between the driver may issue a VMI-level call virt-api rejects, and the unpublish fails and is retried until both sides agree. The chart now renders --enable-vmi-hotplug-fallback=false, which a kubevirt-csi image built before the flag existed rejects at startup. The driver image is stamped at release preparation, so this needs the rebuilt image that ships in the same release. DeclarativeHotplugVolumes is now listed explicitly. With HotplugVolumes gone it is the only hotplug path, and disabling it through disabledFeatureGates would make virt-api refuse every addvolume, so the chart refuses that render instead. This changes volume hotplug platform-wide. A VM-level addvolume or removevolume now edits VM.spec directly and virt-controller reconciles the VMI hotplug volumes against it, instead of going through VM.status.volumeRequests. On the first reconcile it also removes every hotpluggable VMI volume missing from VM.spec; platform migration 59 copies those into VM.spec beforehand on the variants that run migrations. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent fd8ccb2 commit 8dbc7bc

5 files changed

Lines changed: 49 additions & 21 deletions

File tree

‎packages/apps/kubernetes/templates/csi/deploy.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,9 @@ spec:
4343
- "--infra-cluster-namespace=$(INFRACLUSTER_NAMESPACE)"
4444
- "--infra-cluster-labels=$(INFRACLUSTER_LABELS)"
4545
- "--run-node-service=false"
46+
# false while packages/system/kubevirt leaves HotplugVolumes off:
47+
# without that gate virt-api rejects removevolume on a VM-owned VMI.
48+
- "--enable-vmi-hotplug-fallback=false"
4649
- "--v=5"
4750
ports:
4851
- name: healthz

‎packages/apps/kubernetes/tests/csi_vmi_hotplug_fallback_test.yaml‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,17 +11,20 @@ release:
1111
namespace: tenant-test
1212

1313
tests:
14-
# The driver defaults the flag to true, which is what the KubeVirt CR needs
15-
# while it enables HotplugVolumes. Rendering it anyway would crash a driver
16-
# image built before the flag existed: Go's flag package exits on an
17-
# unknown flag. This test does not read that CR; the two change together.
18-
- it: leaves the VMI hotplug fallback at the driver default while the KubeVirt CR enables HotplugVolumes
14+
# Pinned while packages/system/kubevirt leaves HotplugVolumes out of the
15+
# static featureGates, so virt-api rejects the VMI-level removevolume on a
16+
# VM-owned VMI. This test does not read that CR; the two change together.
17+
- it: turns the VMI hotplug fallback off while the KubeVirt CR leaves HotplugVolumes off
1918
documentSelector:
2019
path: kind
2120
value: Deployment
2221
asserts:
2322
- equal:
2423
path: spec.template.spec.containers[0].name
2524
value: csi-driver
26-
- notExists:
27-
path: spec.template.spec.containers[0].args[?(@ =~ /^--enable-vmi-hotplug-fallback/)]
25+
- contains:
26+
path: spec.template.spec.containers[0].args
27+
content: "--enable-vmi-hotplug-fallback=false"
28+
- notContains:
29+
path: spec.template.spec.containers[0].args
30+
content: "--enable-vmi-hotplug-fallback=true"

‎packages/system/kubevirt/templates/kubevirt-cr.yaml‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@
4747
the self-heal contract; tests/kubevirt_cr_test.yaml carries one case per gate,
4848
which is what makes a name dropped from a copy visible.
4949
*/ -}}
50-
{{- $staticFeatureGates := list "HotplugVolumes" "ExpandDisks" "LiveMigration" "AutoResourceLimitsGate" "CPUManager" "GPU" "VMExport" -}}
50+
{{- $staticFeatureGates := list "DeclarativeHotplugVolumes" "ExpandDisks" "LiveMigration" "AutoResourceLimitsGate" "CPUManager" "GPU" "VMExport" -}}
5151
{{- $staticDisabledFeatureGates := list "Template" "ExternalNetResourceInjection" -}}
5252
{{- if semverCompare "<1.35-0" .Capabilities.KubeVersion.Version -}}
5353
{{- $staticDisabledFeatureGates = append $staticDisabledFeatureGates "ImageVolume" -}}
@@ -98,12 +98,13 @@ spec:
9898
{{- end }}
9999
{{- end }}
100100
featureGates:
101-
# HotplugVolumes: deprecated in 1.9, replaced by DeclarativeHotplugVolumes.
102-
# Kept because virt-api accepts kubevirt-csi's VMI-level removevolume on a
103-
# VM-owned VMI only while it is on — which also keeps the replacement inert.
104-
# Dropping it means setting --enable-vmi-hotplug-fallback=false in
101+
# HotplugVolumes is left off: it is deprecated since 1.9, and while on it
102+
# keeps DeclarativeHotplugVolumes inert. Turning it back on means setting
103+
# --enable-vmi-hotplug-fallback=true in
105104
# packages/apps/kubernetes/templates/csi/deploy.yaml in the same change.
106-
- HotplugVolumes
105+
# DeclarativeHotplugVolumes is Beta and on by default; it is listed so
106+
# disabledFeatureGates cannot take away the only hotplug path left.
107+
- DeclarativeHotplugVolumes
107108
- ExpandDisks
108109
- LiveMigration
109110
- AutoResourceLimitsGate

‎packages/system/kubevirt/tests/kubevirt_cr_test.yaml‎

Lines changed: 28 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ tests:
1717
- equal:
1818
path: spec.configuration.developerConfiguration.featureGates
1919
value:
20-
- HotplugVolumes
20+
- DeclarativeHotplugVolumes
2121
- ExpandDisks
2222
- LiveMigration
2323
- AutoResourceLimitsGate
@@ -109,7 +109,7 @@ tests:
109109
- equal:
110110
path: spec.configuration.developerConfiguration.featureGates
111111
value:
112-
- HotplugVolumes
112+
- DeclarativeHotplugVolumes
113113
- ExpandDisks
114114
- LiveMigration
115115
- AutoResourceLimitsGate
@@ -119,6 +119,23 @@ tests:
119119
- HostDevices
120120
- WorkloadEncryptionSEV
121121

122+
# The chart does not enable HotplugVolumes, so naming it off is a plain
123+
# opt-out and must not trip the both-lists guard.
124+
- it: accepts HotplugVolumes in disabledFeatureGates
125+
set:
126+
disabledFeatureGates:
127+
- HotplugVolumes
128+
asserts:
129+
- notContains:
130+
path: spec.configuration.developerConfiguration.featureGates
131+
content: HotplugVolumes
132+
- equal:
133+
path: spec.configuration.developerConfiguration.disabledFeatureGates
134+
value:
135+
- Template
136+
- ExternalNetResourceInjection
137+
- HotplugVolumes
138+
122139
- it: appends disabledFeatureGates after the static opt-outs
123140
set:
124141
disabledFeatureGates:
@@ -147,13 +164,17 @@ tests:
147164
# entry cannot be disabled from values at all. The three cases further down
148165
# that reach featureGates through extraFeatureGates get the message naming
149166
# both sides instead, which is the distinction the copy also decides.
150-
- it: refuses to render when disabledFeatureGates repeats HotplugVolumes
167+
#
168+
# DeclarativeHotplugVolumes is Beta and on by default, so listing it only
169+
# matters for this guard: with HotplugVolumes off, disabling it would leave
170+
# virt-api refusing every volume hotplug, kubevirt-csi attach included.
171+
- it: refuses to render when disabledFeatureGates repeats DeclarativeHotplugVolumes
151172
set:
152173
disabledFeatureGates:
153-
- HotplugVolumes
174+
- DeclarativeHotplugVolumes
154175
asserts:
155176
- failedTemplate:
156-
errorMessage: 'feature gate "HotplugVolumes" is one this chart enables and cannot be switched off from values: KubeVirt refuses a CR that names the same gate in both lists. Remove it from disabledFeatureGates.'
177+
errorMessage: 'feature gate "DeclarativeHotplugVolumes" is one this chart enables and cannot be switched off from values: KubeVirt refuses a CR that names the same gate in both lists. Remove it from disabledFeatureGates.'
157178

158179
- it: refuses to render when disabledFeatureGates repeats ExpandDisks
159180
set:
@@ -234,7 +255,7 @@ tests:
234255
- equal:
235256
path: spec.configuration.developerConfiguration.featureGates
236257
value:
237-
- HotplugVolumes
258+
- DeclarativeHotplugVolumes
238259
- ExpandDisks
239260
- LiveMigration
240261
- AutoResourceLimitsGate
@@ -344,7 +365,7 @@ tests:
344365
- equal:
345366
path: spec.configuration.developerConfiguration.featureGates
346367
value:
347-
- HotplugVolumes
368+
- DeclarativeHotplugVolumes
348369
- ExpandDisks
349370
- LiveMigration
350371
- AutoResourceLimitsGate

‎packages/system/kubevirt/values.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ _cluster: {}
99
# cpuAllocationRatio: 10
1010

1111
# Feature gates appended after the static defaults
12-
# (HotplugVolumes/ExpandDisks/LiveMigration/AutoResourceLimitsGate/CPUManager/GPU/VMExport).
12+
# (DeclarativeHotplugVolumes/ExpandDisks/LiveMigration/AutoResourceLimitsGate/CPUManager/GPU/VMExport).
1313
# Empty by default. Filled by the platform with "HostDevices" when the
1414
# gpu-operator package is enabled with the passthrough variant — the
1515
# admission webhook rejects spec.template.spec.domain.devices.hostDevices

0 commit comments

Comments
 (0)