Skip to content

Commit 334e2d8

Browse files
test(e2e): record the runner's SVM feature flags in the identity capture (#3945)
Adds one line to the runner identity record, taken from the `lscpu` read it already does: ``` svm features: svm npt nrip_save vgif ``` `vgif` decides whether a guest that is itself running a hypervisor pays a VM exit on every `CLGI`/`STGI`. Here that is the sandbox Talos nodes hosting the tenant KubeVirt VMs, so it only bites at double nesting, and the sandbox nodes' own boot is unaffected. Across the archived reports no run missing the flag has passed a tenant-Kubernetes suite: 0 of 62, against 11 of 12 with it. Exposure varies between hosts of the same shape, so it is not a property of the lane and cannot be recovered once the runner is recycled. Recording it on every run, red and green, is what makes the correlation countable rather than reconstructed by hand. `apply` runs `hack/e2e-runner-identity.sh` before `docker run`, and every workflow that runs the suite reaches it through `prepare-env`, so this is captured once per run on both paths. `lscpu` is already answering there: the `cpu model` line in today's records is parsed out of the same table. No new bounded read, no compiler on the runner, no sysfs. Gated on `svm` so an Intel host's `vnmi` is not reported as an SVM feature. Five tests for the reader. Identity suite 62 assertions, `hack/` sweep 732. No behaviour change and no gating on the flag.
2 parents 3c780ac + a94abe0 commit 334e2d8

2 files changed

Lines changed: 146 additions & 0 deletions

File tree

‎hack/e2e-runner-identity.sh‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -461,6 +461,57 @@ runner_cpu_model() {
461461
sed -n 's/^Model name:[[:space:]]*//p' | sed -n '1p'
462462
}
463463

464+
# runner_svm_features: the AMD SVM sub-features this VM's virtual CPU was given.
465+
#
466+
# Same shape as the model reader above and for the same reason: it transforms an
467+
# `lscpu` table handed to it on stdin, so the one bounded read stays at the one
468+
# call site that owns the ceiling.
469+
#
470+
# WHY THIS LINE IS WORTH A PLACE IN THE RECORD
471+
#
472+
# `svm` says a guest may run a hypervisor. The sub-features say what that costs.
473+
# The one that has mattered here is `vgif`: without it, a guest that is itself
474+
# running a hypervisor traps to the layer below on every CLGI and STGI, and those
475+
# bracket every world switch it performs. A guest that runs no hypervisor never
476+
# executes either instruction, so this is invisible until something nests two
477+
# deep -- which is exactly this suite, where the sandbox Talos nodes host the
478+
# tenant KubeVirt VMs. It explains a shape of failure no resource counter can:
479+
# the cycles are spent handling exits rather than running the guest, so the
480+
# runner reads mostly idle with near-zero steal while its nested guests freeze.
481+
#
482+
# Measured across the archived reports, no run whose kernel did not report
483+
# Virtual GIF has ever passed a tenant-Kubernetes suite, and the two hosts
484+
# compared directly differed in this one flag and nothing else -- same shape,
485+
# same processor model, same kernel. On a flex shape filled from shared capacity
486+
# that is not a property of the lane, so it belongs beside the shape and the
487+
# processor rather than in a reader's memory of a run that has since been
488+
# recycled.
489+
#
490+
# The whole SVM list is kept rather than that one flag, because which features a
491+
# host exposes has turned out to vary between hosts of the same shape, and a
492+
# record that answered only the question already asked would have to be extended
493+
# by whoever asks the next one.
494+
#
495+
# Names are the kernel's own from this leaf, matched whole so `svm` cannot be
496+
# taken from `svm_lock`; `-E` and `-x` are both POSIX. `grep` finding nothing is
497+
# a status this reads as an answer, so no branch here -- an Intel host, or a
498+
# guest given no sub-features, prints an empty list and the caller says so.
499+
runner_svm_features() {
500+
_rsf_flags=$(sed -n 's/^Flags:[[:space:]]*//p' | sed -n '1p')
501+
# Gated on `svm` itself, and not only for tidiness: `vnmi` is a flag name on
502+
# Intel too, where it belongs to VMX, so an ungated list prints one SVM
503+
# sub-feature on a machine that has no SVM at all. Sub-features of an absent
504+
# feature are not a shorter answer, they are a wrong one. Padded case rather
505+
# than a grep so `svm` cannot be taken from `svm_lock`.
506+
case " $_rsf_flags " in
507+
*' svm '*) ;;
508+
*) return 0 ;;
509+
esac
510+
printf '%s\n' "$_rsf_flags" | tr ' ' '\n' |
511+
grep -xE 'svm|svm_lock|npt|lbrv|nrip_save|tsc_scale|vmcb_clean|flushbyasid|decodeassists|pausefilter|pfthreshold|avic|x2avic|v_vmsave_vmload|vgif|vnmi|v_spec_ctrl|svme_addr_chk' |
512+
tr '\n' ' ' | sed 's/[[:space:]]*$//'
513+
}
514+
464515
if [ -n "${E2E_RUNNER_IDENTITY_LIB:-}" ]; then
465516
return 0 2>/dev/null
466517
fi
@@ -646,8 +697,20 @@ if runner_read lscpu; then
646697
else
647698
add 'cpu model: unavailable (lscpu answered, and no Model name line was taken from its table)'
648699
fi
700+
# Out of the same read as the model, so the two lines describe one reading of
701+
# one machine rather than two that a slow first read could have separated.
702+
svm_features=$(printf '%s\n' "$RUNNER_READ_OUT" | runner_svm_features)
703+
if [ -n "$svm_features" ]; then
704+
add "svm features: $svm_features${RUNNER_READ_WHY:+ ($RUNNER_READ_WHY)}"
705+
else
706+
# Distinguished from a missing line on purpose: an empty list is a fact about
707+
# the machine -- no SVM sub-feature was exposed to it -- and a reader who
708+
# cannot tell that from a read that never happened learns nothing from either.
709+
add 'svm features: none (lscpu answered, and its flag list named no SVM sub-feature; an Intel host, or a guest given none)'
710+
fi
649711
else
650712
add "cpu model: unavailable ($RUNNER_READ_WHY)"
713+
add "svm features: unavailable ($RUNNER_READ_WHY)"
651714
fi
652715

653716
# The file first, the log second, and neither conditional on the other. The file

‎hack/runner-identity.bats‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,11 +251,94 @@ stage() {
251251
{
252252
printf 'Architecture: x86_64\n'
253253
printf 'Model name: AMD EPYC 9J14 96-Core Processor\n'
254+
printf 'Flags: fpu vme de pse tsc msr svm npt nrip_save vgif umip pku\n'
254255
} >"$1/lscpu.out"
255256
printf '32\n' >"$1/nproc.out"
256257
printf '6.8.0-1021-oracle\n' >"$1/uname.out"
257258
}
258259

260+
# ---------------------------------------------------------------------------
261+
# The SVM sub-feature reader, driven directly.
262+
#
263+
# These exist because the flag this reader was added for, vgif, is the one field
264+
# in the record that decides whether a nested guest runs at hardware speed or
265+
# traps to the layer below on every world switch, and it is read out of a flag
266+
# list where a whole-word match is the only thing standing between `svm` and
267+
# `svm_lock`.
268+
# ---------------------------------------------------------------------------
269+
270+
@test "the svm reader keeps the sub-features an AMD guest was given" {
271+
tmp=$(mktemp -d)
272+
printf 'Model name: AMD EPYC 7J13 64-Core Processor\n' >"$tmp/lscpu"
273+
printf 'Flags: fpu de tsc svm cr8_legacy npt nrip_save vgif umip pku\n' >>"$tmp/lscpu"
274+
275+
out=$(runner_svm_features <"$tmp/lscpu")
276+
277+
# vgif is the whole reason for this line: present here, and its absence on an
278+
# otherwise identical host is the only difference measured between a runner
279+
# whose nested guests boot and one whose do not.
280+
[ "$out" = "svm npt nrip_save vgif" ] || {
281+
echo "expected 'svm npt nrip_save vgif', got '$out'" >&2
282+
return 1
283+
}
284+
}
285+
286+
@test "the svm reader reports the same host without vgif as lacking it" {
287+
tmp=$(mktemp -d)
288+
printf 'Flags: fpu de tsc svm cr8_legacy npt nrip_save umip pku\n' >"$tmp/lscpu"
289+
290+
out=$(runner_svm_features <"$tmp/lscpu")
291+
292+
[ "$out" = "svm npt nrip_save" ] || {
293+
echo "expected 'svm npt nrip_save', got '$out'" >&2
294+
return 1
295+
}
296+
}
297+
298+
@test "the svm reader says nothing about an Intel host, whose vnmi is not SVM" {
299+
tmp=$(mktemp -d)
300+
# vnmi is a flag name under VMX as well as SVM. Ungated, this reader would
301+
# print one SVM sub-feature for a machine that has no SVM, which is not a
302+
# shorter answer but a wrong one.
303+
printf 'Flags: fpu vme de pse tsc msr vmx ept vpid vnmi\n' >"$tmp/lscpu"
304+
305+
out=$(runner_svm_features <"$tmp/lscpu")
306+
307+
[ -z "$out" ] || {
308+
echo "expected no output for an Intel flag list, got '$out'" >&2
309+
return 1
310+
}
311+
}
312+
313+
@test "the svm reader does not take svm from svm_lock alone" {
314+
tmp=$(mktemp -d)
315+
# A whole-word match is what keeps this honest: svm_lock is a sub-feature name
316+
# in its own right and its presence does not mean SVM is exposed.
317+
printf 'Flags: fpu de tsc svm_lock npt umip\n' >"$tmp/lscpu"
318+
319+
out=$(runner_svm_features <"$tmp/lscpu")
320+
321+
[ -z "$out" ] || {
322+
echo "expected no output when only svm_lock is present, got '$out'" >&2
323+
return 1
324+
}
325+
}
326+
327+
@test "the svm reader reads the first Flags line only" {
328+
tmp=$(mktemp -d)
329+
# lscpu prints one Flags line, but a caller handing this a concatenated table
330+
# would otherwise get two machines' features merged into one answer.
331+
printf 'Flags: fpu svm npt vgif\n' >"$tmp/lscpu"
332+
printf 'Flags: fpu svm npt avic\n' >>"$tmp/lscpu"
333+
334+
out=$(runner_svm_features <"$tmp/lscpu")
335+
336+
[ "$out" = "svm npt vgif" ] || {
337+
echo "expected only the first Flags line, got '$out'" >&2
338+
return 1
339+
}
340+
}
341+
259342
# ---------------------------------------------------------------------------
260343
# The number reader and the scalar guard, driven directly.
261344
# ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)