Skip to content

Commit 4b1a75a

Browse files
authored
test(vminstance): dry-run evict the VM launcher pod (#4585)
## What this PR does Part of #4422. The e2e suite now asks KubeVirt whether it would let a VM launcher pod be evicted, without evicting it. Until now nothing in e2e evicted a launcher pod, so the drain behaviour set up in #4403 was known only from reading KubeVirt. Once the `vminstance` VM is running, the suite sends a dry-run Eviction (`dryRun=All`) for its virt-launcher pod and checks the answer. Nothing gets deleted. A dry run still goes through the KubeVirt eviction webhook, which is registered with `sideEffects: NoneOnDryRun` in v1.9.0. With `LiveMigrateIfPossible` its answer follows the VMI's `LiveMigratable` condition. That condition depends on storage, so the test reads it instead of assuming it: - `LiveMigratable=True`: the eviction must be refused with the webhook's `Eviction triggered evacuation of VMI "<ns>/<name>"` message (HTTP 429). A drain retries on that while the VM migrates away. - `LiveMigratable=False`: the eviction must be allowed. The old `LiveMigrate` value refused this case with "is not live-migratable". KubeVirt v1.9.0 creates no PodDisruptionBudget for a VMI, so no PDB can refuse it either. - Anything else, including a missing condition, fails the test. The allowed answer alone proves little, because the webhook also lets an eviction through under `None`. So the test also asserts `evictionStrategy: LiveMigrateIfPossible` in the VMI spec, where virt-api copies the cluster value at creation. After the dry run it checks that the launcher pod is the same object and not terminating, and that the VMI has no `evacuationNodeName`. On the PR lane only the `False` branch runs. Post-install prep pins `StorageProfile/local` to RWO because DRBD is off there, and an RWO disk makes the VMI non-migratable. Nightly and tag runs use `replicated`, where I expect the `True` branch, but I have not seen this suite run on that lane yet. A real drain and a workload update after a KubeVirt bump are still open in #4422. I ran the script against a stub `kubectl`. Both expected answers pass, and it fails on every wrong answer I tried, such as a refusal for a non-migratable VMI, a PDB-style 429, a missing condition, two launcher pods, or a replaced pod after the dry run. `chainsaw lint` (v0.2.15) and shellcheck pass. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches only `hack/e2e-chainsaw/vminstance/` and the suite README, and nothing in the trigger map covers either. ### Release note ```release-note test(vminstance): the e2e suite now dry-run evicts the VM launcher pod and checks the KubeVirt eviction decision against the VMI's LiveMigratable condition. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added end-to-end coverage for dry-run eviction of a running virtual machine’s launcher pod. The test checks the VM’s live-migration policy and `LiveMigratable` condition, expecting eviction to be refused when the VM is live-migratable and accepted otherwise. * Verified that the dry run leaves the launcher pod unchanged and does not record an evacuation node on the VM. * **Documentation** * Documented the test’s behavior and the local `jq` requirement for running it. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 6a98acb + 966fccf commit 4b1a75a

2 files changed

Lines changed: 93 additions & 1 deletion

File tree

‎hack/e2e-chainsaw/README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ Declarative suites assert on `status.conditions` and concrete fields. Inherently
3636

3737
- **openbao** — `bao operator init`/`unseal` via `kubectl exec`.
3838
- **kuberture** — `jq` on Service annotations + external-dns split-horizon log assertions.
39+
- **vminstance** — a server-side dry-run eviction of the VM's launcher pod, checked against the VMI's `LiveMigratable` condition.
3940
- **gateway** — admission-rejection cases that need `kubectl --as` impersonation or error-message greps; the tenant apex is read from the namespace `namespace.cozystack.io/host` label at runtime so they pass on any host.
4041
- **kubernetes-latest/previous** — wrap `_lib/run-kubernetes.sh` (Kamaji bringup, port-forward, LB/NFS/ouroboros). The script `cd`s to the repo root (the test dir is `hack/e2e-chainsaw/<suite>`, so repo root is `../../..`).
4142

@@ -54,7 +55,7 @@ Declarative suites assert on `status.conditions` and concrete fields. Inherently
5455

5556
## Running locally
5657

57-
Requires a cluster with Cozystack installed and a `tenant-test` namespace (the environment `hack/e2e-install-cozystack.bats` produces). `bucket` and `seaweedfs` additionally need `mc`, `nc`, and `jq` on the host; `kuberture`/`openbao`/`harbor` need `jq`.
58+
Requires a cluster with Cozystack installed and a `tenant-test` namespace (the environment `hack/e2e-install-cozystack.bats` produces). `bucket` and `seaweedfs` additionally need `mc`, `nc`, and `jq` on the host; `kuberture`/`openbao`/`harbor`/`vminstance` need `jq`.
5859

5960
```bash
6061
# install chainsaw

‎hack/e2e-chainsaw/vminstance/chainsaw-test.yaml‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,3 +172,94 @@ spec:
172172
status:
173173
(conditions[?type == 'Ready']):
174174
- status: "True"
175+
- name: dry-run-evict-launcher
176+
try:
177+
# virt-api stamps the cluster evictionStrategy into the VMI spec at
178+
# creation, and the eviction webhook reads it from there. virt-handler
179+
# publishes LiveMigratable once it has synced the domain; under
180+
# LiveMigrateIfPossible that condition is what splits the webhook's answer.
181+
- assert:
182+
resource:
183+
apiVersion: kubevirt.io/v1
184+
kind: VirtualMachineInstance
185+
metadata:
186+
name: vm-instance-test
187+
spec:
188+
evictionStrategy: LiveMigrateIfPossible
189+
status:
190+
(conditions[?type == 'LiveMigratable']):
191+
- type: LiveMigratable
192+
# dryRun=All still runs the eviction webhook (sideEffects NoneOnDryRun) and
193+
# deletes nothing. Under LiveMigrateIfPossible a migratable VMI is refused
194+
# with 429 and the webhook's evacuation message, which a drain retries while
195+
# the VM migrates away; a non-migratable one is let through. On the PR
196+
# lane post-install prep pins StorageProfile/local to RWO, so there the
197+
# VMI is not migratable.
198+
- script:
199+
timeout: 1m
200+
content: |
201+
set -eu
202+
vmi=vm-instance-test
203+
vmi_json=$(kubectl -n "$NAMESPACE" get vmi "$vmi" -o json)
204+
uid=$(printf '%s\n' "$vmi_json" | jq -r '.metadata.uid')
205+
migratable=$(printf '%s\n' "$vmi_json" | jq -r '[.status.conditions[] | select(.type == "LiveMigratable") | .status] | if length == 1 then .[0] else "ambiguous" end')
206+
pods=$(kubectl -n "$NAMESPACE" get pods -l "kubevirt.io=virt-launcher,kubevirt.io/created-by=$uid" --field-selector status.phase=Running -o json)
207+
if [ "$(printf '%s\n' "$pods" | jq '.items | length')" != 1 ]; then
208+
echo "FAIL: expected exactly one Running launcher pod for VMI $vmi" >&2
209+
printf '%s\n' "$pods" | jq -r '.items[] | .metadata.name + " " + .status.phase' >&2
210+
exit 1
211+
fi
212+
pod=$(printf '%s\n' "$pods" | jq -r '.items[0].metadata.name')
213+
pod_uid=$(printf '%s\n' "$pods" | jq -r '.items[0].metadata.uid')
214+
215+
eviction=$(jq -cn --arg name "$pod" --arg ns "$NAMESPACE" \
216+
'{apiVersion: "policy/v1", kind: "Eviction", metadata: {name: $name, namespace: $ns}}')
217+
err_file=$(mktemp)
218+
trap 'rm -f "$err_file"' EXIT
219+
if out=$(printf '%s\n' "$eviction" | kubectl create --raw "/api/v1/namespaces/$NAMESPACE/pods/$pod/eviction?dryRun=All" -f - 2>"$err_file"); then
220+
allowed=true
221+
else
222+
allowed=false
223+
fi
224+
err=$(cat "$err_file")
225+
echo "LiveMigratable=$migratable eviction-allowed=$allowed"
226+
printf '%s\n' "response: $out"
227+
printf '%s\n' "stderr: $err"
228+
229+
case "$migratable" in
230+
True)
231+
if [ "$allowed" = true ]; then
232+
echo "FAIL: dry-run eviction of migratable VMI $vmi was allowed; the webhook should refuse it and start an evacuation" >&2
233+
exit 1
234+
fi
235+
if ! printf '%s\n' "$err" | grep -Fq "Eviction triggered evacuation of VMI \"$NAMESPACE/$vmi\""; then
236+
echo "FAIL: the refusal did not come from the KubeVirt evacuation path" >&2
237+
exit 1
238+
fi
239+
;;
240+
False)
241+
if [ "$allowed" != true ]; then
242+
echo "FAIL: dry-run eviction of non-migratable VMI $vmi was refused; LiveMigrateIfPossible should let it through" >&2
243+
exit 1
244+
fi
245+
if ! printf '%s\n' "$out" | jq -e '.kind == "Status" and .status == "Success"' >/dev/null 2>&1; then
246+
echo "FAIL: the eviction returned without a Success status" >&2
247+
exit 1
248+
fi
249+
;;
250+
*)
251+
echo "FAIL: VMI $vmi LiveMigratable condition is '$migratable', expected True or False" >&2
252+
exit 1
253+
;;
254+
esac
255+
256+
if ! kubectl -n "$NAMESPACE" get pod "$pod" -o json |
257+
jq -e --arg uid "$pod_uid" '.metadata.uid == $uid and .metadata.deletionTimestamp == null' >/dev/null; then
258+
echo "FAIL: launcher pod $pod was deleted or replaced by a dry-run eviction" >&2
259+
exit 1
260+
fi
261+
if ! kubectl -n "$NAMESPACE" get vmi "$vmi" -o json |
262+
jq -e '(.status.evacuationNodeName // "") == ""' >/dev/null; then
263+
echo "FAIL: a dry-run eviction recorded an evacuation on VMI $vmi" >&2
264+
exit 1
265+
fi

0 commit comments

Comments
 (0)