Skip to content

Commit 6918234

Browse files
committed
fix(kubevirt)!: evict non-migratable VMs instead of blocking the drain
The cluster-wide evictionStrategy was LiveMigrate. For a VMI that is not live-migratable (a ReadWriteOnce disk, a passthrough GPU, a host device), virt-api's eviction webhook answers every eviction with 429, so a node drain retries forever and never completes. LiveMigrateIfPossible keeps live migration for every VMI that can migrate and lets the eviction through for the rest. The VM is shut down and started again according to its runStrategy. A VM whose disk is bound to the drained node stays down until the node returns. The strategy is read from the cluster config at eviction time, so it applies to running VMs without a restart. Evict goes from workloadUpdateMethods. The workload updater evicts through the same webhook, so with the new strategy its Evict fallback would restart every non-migratable VM on each KubeVirt upgrade, and on a CPU hotplug of such a VM. Under LiveMigrate those evictions got 429 and did nothing; without Evict the updater skips these VMIs, which keeps its behaviour as it was. They take the new launcher on their next restart. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 8d37e0b commit 6918234

2 files changed

Lines changed: 17 additions & 2 deletions

File tree

‎packages/system/kubevirt/templates/kubevirt-cr.yaml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,14 +123,16 @@ spec:
123123
migrations:
124124
{{- toYaml . | nindent 6 }}
125125
{{- end }}
126-
evictionStrategy: LiveMigrate
126+
evictionStrategy: LiveMigrateIfPossible # a VMI that cannot live-migrate is evicted instead of blocking the node drain
127127
virtualMachineOptions:
128128
disableSerialConsoleLog: {}
129129
vmRolloutStrategy: LiveUpdate
130130
workloadUpdateStrategy:
131+
# No Evict: with LiveMigrateIfPossible it would restart every VM that
132+
# cannot migrate on each KubeVirt upgrade. Such a VM takes the new
133+
# launcher on its next restart instead.
131134
workloadUpdateMethods:
132135
- LiveMigrate
133-
- Evict
134136
batchEvictionInterval: 1m
135137
batchEvictionSize: 10
136138
# virt-handler is a node DaemonSet and ran BestEffort. The Talos userspace OOM

‎packages/system/kubevirt/tests/kubevirt_cr_test.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,19 @@ tests:
3838
- notExists:
3939
path: spec.configuration.migrations
4040

41+
- it: sets the cluster evictionStrategy to LiveMigrateIfPossible
42+
asserts:
43+
- equal:
44+
path: spec.configuration.evictionStrategy
45+
value: LiveMigrateIfPossible
46+
47+
- it: updates outdated workloads by live migration only
48+
asserts:
49+
- equal:
50+
path: spec.workloadUpdateStrategy.workloadUpdateMethods
51+
value:
52+
- LiveMigrate
53+
4154
- it: omits migrations when explicitly empty
4255
set:
4356
migrations: {}

0 commit comments

Comments
 (0)