Skip to content

Commit 25d4a8c

Browse files
committed
fix(kubevirt)!: evict non-migratable VMs instead of blocking the drain
The cluster-wide evictionStrategy was LiveMigrate. For a VMI that is not live-migratable (a ReadWriteOnce disk, a passthrough GPU, a host device), virt-api's eviction webhook answers every eviction with 429, so a node drain retries forever and never completes. LiveMigrateIfPossible keeps live migration for every VMI that can migrate and lets the eviction through for the rest. The VM is shut down and started again according to its runStrategy. A VM whose disk is bound to the drained node stays down until the node returns. KubeVirt writes the cluster strategy into a VMI's spec when the VMI is created, so a VMI started before the upgrade keeps LiveMigrate and still blocks the drain until it is restarted once. A drain and a launcher update after a KubeVirt upgrade go through the same eviction, so the workload updater's Evict fallback shuts down non-migratable VMs started under the new strategy on every later KubeVirt upgrade, where it used to get the same 429. Evict stays anyway: for kubernetes-nodes workers, which use evictionStrategy External, that eviction marks the VMI for evacuation, CAPK drains the tenant node and deletes the VMI, and the VM starts it again on the new launcher. The test pins the method list so the fallback is not dropped to avoid the shutdowns. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 8d37e0b commit 25d4a8c

2 files changed

Lines changed: 20 additions & 1 deletion

File tree

‎packages/system/kubevirt/templates/kubevirt-cr.yaml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,11 +123,16 @@ spec:
123123
migrations:
124124
{{- toYaml . | nindent 6 }}
125125
{{- end }}
126-
evictionStrategy: LiveMigrate
126+
evictionStrategy: LiveMigrateIfPossible # a VMI that cannot live-migrate is evicted instead of blocking the node drain
127127
virtualMachineOptions:
128128
disableSerialConsoleLog: {}
129129
vmRolloutStrategy: LiveUpdate
130130
workloadUpdateStrategy:
131+
# Evict shuts down VMs that cannot migrate on a KubeVirt upgrade, but it is
132+
# also how such a kubernetes-nodes worker (evictionStrategy External) gets
133+
# the new launcher without a rollout: the eviction marks the VMI for
134+
# evacuation, CAPK drains the tenant node and deletes the VMI, and the VM
135+
# starts it again.
131136
workloadUpdateMethods:
132137
- LiveMigrate
133138
- Evict

‎packages/system/kubevirt/tests/kubevirt_cr_test.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,20 @@ tests:
3838
- notExists:
3939
path: spec.configuration.migrations
4040

41+
- it: sets the cluster evictionStrategy to LiveMigrateIfPossible
42+
asserts:
43+
- equal:
44+
path: spec.configuration.evictionStrategy
45+
value: LiveMigrateIfPossible
46+
47+
- it: keeps Evict as the workload update fallback for VMs that cannot migrate
48+
asserts:
49+
- equal:
50+
path: spec.workloadUpdateStrategy.workloadUpdateMethods
51+
value:
52+
- LiveMigrate
53+
- Evict
54+
4155
- it: omits migrations when explicitly empty
4256
set:
4357
migrations: {}

0 commit comments

Comments
 (0)