Skip to content

Commit df2e1c3

Browse files
lexfreilexfrei
authored
feat(kubernetes-nodes): build the Proxmox worker template from the pool's Talos values (#4742)
## What this PR does A `substrate: proxmox` pool clones its workers from a VM template the chart never sees: capmox selects it by `proxmox.templateTags`, and the pool's `talos.*` values reach only the installer written into the worker machineconfig. Nothing in the repository says which image that template has to carry, and the image an operator reaches for is the wrong one: - The release asset `nocloud-amd64.raw.xz` is built from `packages/core/talos/images/talos/profiles/nocloud.yaml`, the bare-metal node image. Its `zfs` extension starts `ext-zfs-service`, which waits for `/dev/zfs`. A VM never has it, so `startAllServices` never finishes, and Talos fails the boot sequence and reboots the guest each time the boot timeout runs out. On our v1.13.6 stand that was every ~70 minutes, visible in the tenant only as `SandboxChanged` on every Pod (218 cilium restarts in 12 days); from the Proxmox side the VM never restarts. The same profile also follows the host nodes' Talos (v1.14.2 since e002f8d), not the pool's `talos.version` (v1.13.6), so the boot image and the installer the machineconfig names drift apart. - The chart's default schematic `ce4c9805…` is not vanilla: it carries `siderolabs/qemu-guest-agent`, whose service `depends` on `/dev/virtio-ports/org.qemu.guest_agent.0`. Proxmox adds that port only to a VM with the agent enabled, so a template built from the right image with the agent off waits the same way. `hack/proxmox-talos-template.sh` builds the template from the chart's own `talos.version`, `talos.schematicID` and `talos.imageFactoryURL`, in the nocloud flavour, with the agent enabled, and prints the `talos.*` and `proxmox.templateTags` a pool selects it with. A pool that overrides `talos.*` passes the same values through `COZY_TALOS_*`. Before downloading anything it checks that the VMID is free cluster-wide and that no other template carries the same tag set: capmox compares the selector with each template's tags as a set and needs exactly one match across the cluster, so a duplicate stops every pool that selects it with `found 2 VM templates`. `print` writes the program that `create` runs as root on the hypervisor, so it can be read first. The layout (ovmf, `x86-64-v2-AES`, `virtio-scsi-pci`, serial console) is the one the Proxmox substrate has been running on, minus the NIC: capmox writes `net0` from `proxmox.network` on every clone. The disk defaults to 20G because capmox clones it at the template's size; `diskSize` does not reach a Proxmox worker. Out of scope on purpose: building a worker image in `packages/core/talos` for air-gapped installs. The template follows `talos.imageFactoryURL`, so a self-hosted Image Factory already covers that case. Refs #3481. ### Also in this diff - `nodegroup.yaml`: the comment on the template selector says which image the template carries, and the comment on `checks` no longer says Talos never ships qemu-guest-agent. - The `proxmox.templateTags` description points at the script and warns off the release asset; README, `values.schema.json`, the CRD and `types.go` are regenerated. ### Testing - `hack/cozytest.sh hack/proxmox-talos-template_test.bats`: 12/12, against stubs of `ssh`, `pvesh`, `qm`, `curl` and `xz`. It goes red on each of `agent enabled=0`, a disabled duplicate-tag check, and `openstack-amd64` in place of `nocloud-amd64`. - `make -C packages/apps/kubernetes-nodes test`: 206/206, snapshots unchanged. - `make generate` at the root and in the package: no further diff. - Not yet run against a live Proxmox. It will build the replacement for our stand's template, which was made from the release asset. The qemu-guest-agent half is derived from the extension's `depends` and the zfs case observed there, not observed on its own. ### Screenshots Not applicable, no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change `kubernetes-nodes` has no generated page on the website (it is in none of the website `Makefile` app lists, and no page mentions the Proxmox substrate), the only schema change is a field description, and `hack/` gains a file without moving or renaming one. ### Release note ```release-note feat(kubernetes-nodes): add `hack/proxmox-talos-template.sh`, which builds the Proxmox VM template a `substrate: proxmox` pool clones from the pool's own `talos.version` and `talos.schematicID` in the nocloud flavour, with the guest agent the default schematic needs. Do not build that template from the release's `nocloud-amd64.raw.xz`: it is the bare-metal node image, and its zfs extension keeps a VM worker from finishing its boot, so Talos reboots it at every boot timeout. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a utility to generate or create Proxmox Talos VM templates using Kubernetes-nodes chart settings. It validates template options, supports local or SSH-based creation, and displays pool configuration after successful creation. * Kubernetes node pool configuration now accepts kernel module names and optional parameters. * **Documentation** * Clarified that Proxmox templates must match the pool’s Talos version and schematic, use the nocloud flavor, and have the guest agent enabled. The bare-metal image is unsuitable and may prevent VMs from completing boot. * Added guidance on NVIDIA module selection and pool rollouts. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 1d813c9 + 6852f72 commit df2e1c3

8 files changed

Lines changed: 387 additions & 20 deletions

File tree

‎api/apps/v1alpha1/kubernetesnodes/types.go‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎hack/proxmox-talos-template.sh‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
#!/bin/sh
2+
# Builds the Proxmox VM template a `substrate: proxmox` kubernetes-nodes pool
3+
# clones its workers from.
4+
#
5+
# The chart takes no image on that substrate: capmox clones whichever template
6+
# carries the pool's proxmox.templateTags, and the pool's talos.* values reach
7+
# only the installer written into the worker machineconfig. Nothing compares the
8+
# two, so the template is built here from the same talos.version and
9+
# talos.schematicID the chart renders, read from its values.yaml.
10+
#
11+
# Two images that look right are not:
12+
# - the platform release's nocloud-amd64.raw.xz is the bare-metal node image.
13+
# Its zfs extension starts a service that waits for /dev/zfs, which a VM never
14+
# has, so the Talos boot sequence never completes and the guest is rebooted
15+
# when the boot timeout runs out (about 70 minutes on v1.13), over and over.
16+
# - the openstack image the kubevirt substrate imports ignores the nocloud ISO
17+
# capmox hands the bootstrap data over on, and the worker never joins.
18+
#
19+
# The chart's default schematic carries qemu-guest-agent, whose service waits
20+
# the same way for the agent's virtio port. Proxmox adds that port only to a VM
21+
# with the agent enabled, so the template is created with it on.
22+
#
23+
# Usage:
24+
# hack/proxmox-talos-template.sh print <vmid> <storage>
25+
# hack/proxmox-talos-template.sh create <vmid> <storage>
26+
#
27+
# print writes the program that create runs, for review. create runs it over
28+
# ssh on COZY_PVE_SSH (user@host), or on this machine when that is unset, which
29+
# then has to be a Proxmox VE node. The template is created on that node.
30+
#
31+
# Optional overrides:
32+
# COZY_TALOS_VERSION, COZY_TALOS_SCHEMATIC_ID, COZY_TALOS_FACTORY_URL
33+
# default to talos.version, talos.schematicID and talos.imageFactoryURL of
34+
# packages/apps/kubernetes-nodes/values.yaml. A pool that overrides those
35+
# needs a template built with the same values.
36+
# COZY_TEMPLATE_TAGS comma-separated, default
37+
# cozystack,talos,talos-<version>,schematic-<8 chars>
38+
# COZY_TEMPLATE_DISK_SIZE default 20G. capmox clones the disk at the size the
39+
# template has, so this is every worker's system disk.
40+
# COZY_TEMPLATE_CPU default x86-64-v2-AES. Talos needs x86-64-v2, which
41+
# the Proxmox default kvm64 does not provide.
42+
# COZY_PVE_SSH_KEY default ~/.ssh/id_ed25519
43+
set -eu
44+
45+
VALUES="$(cd "$(dirname "$0")/.." && pwd)/packages/apps/kubernetes-nodes/values.yaml"
46+
47+
die() { printf 'proxmox-talos-template: %s\n' "$*" >&2; exit 1; }
48+
49+
usage() {
50+
die "usage: $0 print|create <vmid> <storage>"
51+
}
52+
53+
# The talos block of values.yaml is flat, two-space indented and quoted, so awk
54+
# reads it without making yq a requirement of an operator's workstation.
55+
chart_talos() {
56+
awk -v key="$1" '
57+
/^talos:/ { in_block = 1; next }
58+
in_block && /^[^ #]/ { exit }
59+
in_block && $1 == key ":" { v = $2; gsub(/"/, "", v); print v; exit }
60+
' "$VALUES"
61+
}
62+
63+
# grep -x matches line by line, so a value with a newline in it would pass on
64+
# any one of its lines and carry the rest into the program as commands of their
65+
# own. Such a value is refused before grep sees it.
66+
matches() {
67+
case $1 in *'
68+
'*) return 1 ;; esac
69+
printf '%s' "$1" | grep -Eqx "$2"
70+
}
71+
72+
[ $# -eq 3 ] || usage
73+
MODE=$1
74+
VMID=$2
75+
STORAGE=$3
76+
case "$MODE" in print|create) ;; *) usage ;; esac
77+
78+
VERSION=${COZY_TALOS_VERSION:-$(chart_talos version)}
79+
SCHEMATIC=${COZY_TALOS_SCHEMATIC_ID:-$(chart_talos schematicID)}
80+
FACTORY=${COZY_TALOS_FACTORY_URL:-$(chart_talos imageFactoryURL)}
81+
FACTORY=${FACTORY%/}
82+
DISK_SIZE=${COZY_TEMPLATE_DISK_SIZE:-20G}
83+
CPU=${COZY_TEMPLATE_CPU:-x86-64-v2-AES}
84+
TAGS=${COZY_TEMPLATE_TAGS:-cozystack,talos,talos-$VERSION,schematic-$(printf '%s' "$SCHEMATIC" | cut -c1-8)}
85+
86+
# Every value below is pasted into a program that runs as root on the
87+
# hypervisor, so each is held to the shape it is expected to have.
88+
matches "$VMID" '[1-9][0-9]{2,8}' || die "vmid $VMID is not a Proxmox VM id (100-999999999)"
89+
matches "$STORAGE" '[A-Za-z][A-Za-z0-9_.-]*' || die "storage $STORAGE is not a Proxmox storage id"
90+
# The shape the chart's render accepts for talos.version, pre-releases included.
91+
matches "$VERSION" 'v[0-9]+\.[0-9]+\.[0-9]+(-[0-9a-z.]+)?' || die "Talos version '$VERSION' is not vMAJOR.MINOR.PATCH with an optional pre-release suffix"
92+
matches "$SCHEMATIC" '[0-9a-f]{64}' || die "schematic '$SCHEMATIC' is not an Image Factory schematic id"
93+
matches "$FACTORY" 'https?://[A-Za-z0-9.-]+(:[0-9]+)?(/[A-Za-z0-9._~/-]*)?' || die "factory URL '$FACTORY' is not a plain http(s) URL"
94+
matches "$DISK_SIZE" '[1-9][0-9]*G' || die "disk size $DISK_SIZE is not a whole number of gigabytes such as 20G"
95+
matches "$CPU" '[A-Za-z0-9_.+-]+' || die "cpu type $CPU is not a Proxmox CPU model name"
96+
97+
# capmox lowercases the selector and compares it with the template's tags as a
98+
# set, so the tags are normalised the same way before they are checked and
99+
# written, and the selector printed at the end is exactly what will match. The
100+
# split is on commas only: a newline would split a tag the operator meant whole.
101+
case $TAGS in *'
102+
'*) die "template tags are comma-separated and may not contain a newline" ;; esac
103+
TAG_SET=$(printf '%s\n' "$TAGS" | tr ',' '\n' | tr '[:upper:]' '[:lower:]' | sed '/^$/d' | LC_ALL=C sort -u)
104+
[ -n "$TAG_SET" ] || die "the template needs at least one tag"
105+
bad=$(printf '%s\n' "$TAG_SET" | grep -Evx '[a-z0-9_][a-z0-9_+.-]*' || true)
106+
[ -z "$bad" ] || die "not a valid Proxmox tag: $(printf '%s' "$bad" | paste -sd ' ' -)"
107+
PVE_TAGS=$(printf '%s\n' "$TAG_SET" | paste -sd ';' -)
108+
109+
URL="$FACTORY/image/$SCHEMATIC/$VERSION/nocloud-amd64.raw.xz"
110+
NAME="talos-$VERSION-$(printf '%s' "$SCHEMATIC" | cut -c1-8)"
111+
112+
program() {
113+
cat <<EOF
114+
set -eu
115+
VMID=$VMID
116+
STORAGE=$STORAGE
117+
URL='$URL'
118+
NAME='$NAME'
119+
TAGS='$PVE_TAGS'
120+
DISK_SIZE=$DISK_SIZE
121+
CPU='$CPU'
122+
EOF
123+
cat <<'EOF'
124+
if ! command -v qm >/dev/null || ! command -v pvesh >/dev/null; then
125+
echo "this is not a Proxmox VE node: qm and pvesh are missing" >&2
126+
exit 1
127+
fi
128+
129+
# nextid answers with the id only when no VM in the cluster holds it.
130+
pvesh get /cluster/nextid --vmid "$VMID" >/dev/null
131+
132+
# capmox requires exactly one template with the selector's tag set across the
133+
# whole cluster and reports "found 2 VM templates" otherwise, so a second
134+
# template with the same set would stop every pool that selects it.
135+
same=$(pvesh get /cluster/resources --type vm --output-format json | perl -MJSON::PP -e '
136+
my $want = shift;
137+
local $/;
138+
for my $vm (@{ JSON::PP->new->decode(<STDIN>) }) {
139+
next unless $vm->{template} && defined $vm->{tags};
140+
my %t = map { lc($_) => 1 } grep { length } split /;/, $vm->{tags};
141+
print "$vm->{vmid} ($vm->{name}) on $vm->{node}\n" if join(";", sort keys %t) eq $want;
142+
}' "$TAGS")
143+
if [ -n "$same" ]; then
144+
echo "a template already carries the tags $TAGS: $same" >&2
145+
exit 1
146+
fi
147+
148+
work=$(mktemp -d "${TMPDIR:-/var/tmp}/talos-template.XXXXXX")
149+
trap 'rm -rf "$work"' EXIT
150+
curl -fL --retry 3 -o "$work/disk.raw.xz" "$URL"
151+
xz -d "$work/disk.raw.xz"
152+
153+
# The layout of the templates the Proxmox substrate was verified on. No NIC:
154+
# capmox writes net0 from the pool's proxmox.network on every clone.
155+
qm create "$VMID" --name "$NAME" --tags "$TAGS" --ostype l26 \
156+
--cpu "$CPU" --sockets 1 --cores 2 --memory 4096 \
157+
--bios ovmf --efidisk0 "$STORAGE:1,efitype=4m,pre-enrolled-keys=0" \
158+
--scsihw virtio-scsi-pci \
159+
--scsi0 "$STORAGE:0,import-from=$work/disk.raw,cache=none,discard=on,ssd=1" \
160+
--boot order=scsi0 --serial0 socket --vga std \
161+
--agent enabled=1
162+
# A failure from here on leaves a VM holding the id, and the nextid check above
163+
# stops the next run until it is gone.
164+
qm disk resize "$VMID" scsi0 "$DISK_SIZE" && qm template "$VMID" || {
165+
echo "VM $VMID was created but is not a template; remove it with: qm destroy $VMID" >&2
166+
exit 1
167+
}
168+
qm config "$VMID"
169+
EOF
170+
}
171+
172+
if [ "$MODE" = print ]; then
173+
program
174+
exit 0
175+
fi
176+
177+
if [ -n "${COZY_PVE_SSH:-}" ]; then
178+
program | ssh -o BatchMode=yes -o StrictHostKeyChecking=accept-new \
179+
-i "${COZY_PVE_SSH_KEY:-$HOME/.ssh/id_ed25519}" "$COZY_PVE_SSH" sh -s
180+
else
181+
program | sh -s
182+
fi
183+
184+
cat <<EOF
185+
186+
Template $VMID ($NAME) is ready. A pool clones it with:
187+
188+
talos:
189+
version: "$VERSION"
190+
schematicID: "$SCHEMATIC"
191+
proxmox:
192+
templateTags:
193+
EOF
194+
for tag in $TAG_SET; do
195+
printf ' - "%s"\n' "$tag"
196+
done

‎hack/proxmox-talos-template_test.bats‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
#!/usr/bin/env bats
2+
# hack/proxmox-talos-template.sh builds the VM template a proxmox kubernetes-nodes
3+
# pool clones. Nothing at render time can see what the template carries, so what
4+
# the script writes is pinned here: the image is the chart's own talos.* in its
5+
# nocloud flavour, the agent the default schematic waits for is enabled, and the
6+
# tag set capmox selects on is checked before anything is downloaded.
7+
8+
load test_helper
9+
10+
REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
11+
SCRIPT="$REPO_ROOT/hack/proxmox-talos-template.sh"
12+
VALUES="$REPO_ROOT/packages/apps/kubernetes-nodes/values.yaml"
13+
14+
chart() { sed -n "/^talos:/,/^[^ #]/s/^ $1: \"\(.*\)\"$/\1/p" "$VALUES"; }
15+
16+
# Stubs for the hypervisor side of the program. Each call to qm and curl is
17+
# logged, so a test can tell what ran and, as importantly, what did not.
18+
stub_pve() {
19+
stub="$1"
20+
printf '#!/bin/sh\necho "qm $*" >> "%s/calls"\n' "$stub" > "$stub/qm"
21+
cat > "$stub/pvesh" <<EOF
22+
#!/bin/sh
23+
case "\$2" in
24+
/cluster/nextid) echo "\$4" ;;
25+
/cluster/resources) cat "$stub/resources.json" ;;
26+
esac
27+
EOF
28+
cat > "$stub/curl" <<EOF
29+
#!/bin/sh
30+
echo "curl \$*" >> "$stub/calls"
31+
while [ \$# -gt 0 ]; do [ "\$1" = -o ] && : > "\$2"; shift; done
32+
EOF
33+
printf '#!/bin/sh\nmv "$2" "${2%%.xz}"\n' > "$stub/xz"
34+
chmod +x "$stub/qm" "$stub/pvesh" "$stub/curl" "$stub/xz"
35+
: > "$stub/calls"
36+
}
37+
38+
@test "the template image is the chart's talos schematic and version, in the nocloud flavour" {
39+
out="$("$SCRIPT" print 9002 main-pool)"
40+
url="$(chart imageFactoryURL)/image/$(chart schematicID)/$(chart version)/nocloud-amd64.raw.xz"
41+
printf '%s' "$out" | grep -qxF "URL='$url'"
42+
if printf '%s' "$out" | grep -q openstack; then echo "FAIL: the template must not be the openstack image"; false; fi
43+
}
44+
45+
@test "the template enables the guest agent whose port the default schematic waits for" {
46+
"$SCRIPT" print 9002 main-pool | grep -qE -- '--agent enabled=1$'
47+
}
48+
49+
@test "the default tags name the version and the schematic, in the order Proxmox stores them" {
50+
out="$("$SCRIPT" print 9002 main-pool)"
51+
want="TAGS='cozystack;schematic-$(chart schematicID | cut -c1-8);talos;talos-$(chart version)'"
52+
printf '%s' "$out" | grep -qxF "$want"
53+
}
54+
55+
@test "given tags are lowercased, deduplicated and sorted the way capmox compares them" {
56+
out="$(COZY_TEMPLATE_TAGS='Talos,cozystack,,talos,Pool-A' "$SCRIPT" print 9002 main-pool)"
57+
printf '%s' "$out" | grep -qxF "TAGS='cozystack;pool-a;talos'"
58+
}
59+
60+
@test "an override of the talos values reaches the image URL" {
61+
sid=376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba
62+
out="$(COZY_TALOS_VERSION=v1.14.2 COZY_TALOS_SCHEMATIC_ID=$sid COZY_TALOS_FACTORY_URL=https://factory.example:8080/ "$SCRIPT" print 9002 main-pool)"
63+
printf '%s' "$out" | grep -qxF "URL='https://factory.example:8080/image/$sid/v1.14.2/nocloud-amd64.raw.xz'"
64+
}
65+
66+
@test "values that would land in the root program unchecked are refused" {
67+
nl='
68+
'
69+
for pair in "9002;id|main-pool" "99|main-pool" "9002|main pool" "9002|main'pool" \
70+
"9002${nl}id|main-pool" "9002|main-pool${nl}id"; do
71+
vmid="${pair%%|*}"
72+
storage="${pair#*|}"
73+
rc=0
74+
out="$("$SCRIPT" print "$vmid" "$storage" 2>&1)" || rc=$?
75+
[ "$rc" -ne 0 ] || { echo "accepted: $pair" >&2; exit 1; }
76+
case "$out" in *"set -eu"*) echo "printed a program for: $pair" >&2; exit 1 ;; esac
77+
done
78+
rc=0; COZY_TALOS_VERSION='v1.13.6;id' "$SCRIPT" print 9002 main-pool >/dev/null 2>&1 || rc=$?
79+
[ "$rc" -ne 0 ]
80+
rc=0; COZY_TALOS_SCHEMATIC_ID='ce4c9805' "$SCRIPT" print 9002 main-pool >/dev/null 2>&1 || rc=$?
81+
[ "$rc" -ne 0 ]
82+
rc=0; COZY_TEMPLATE_DISK_SIZE="20G${nl}id" "$SCRIPT" print 9002 main-pool >/dev/null 2>&1 || rc=$?
83+
[ "$rc" -ne 0 ]
84+
rc=0; COZY_TEMPLATE_TAGS="talos,it's" "$SCRIPT" print 9002 main-pool >/dev/null 2>&1 || rc=$?
85+
[ "$rc" -ne 0 ]
86+
rc=0; COZY_TEMPLATE_TAGS="talos-v1.${nl}13,cozystack" "$SCRIPT" print 9002 main-pool >/dev/null 2>&1 || rc=$?
87+
[ "$rc" -ne 0 ]
88+
}
89+
90+
@test "a pre-release Talos version the chart accepts gets a template" {
91+
out="$(COZY_TALOS_VERSION=v1.13.6-beta.1 "$SCRIPT" print 9002 main-pool)"
92+
printf '%s' "$out" | grep -qF "/v1.13.6-beta.1/nocloud-amd64.raw.xz'"
93+
printf '%s' "$out" | grep -qxF "NAME='talos-v1.13.6-beta.1-$(chart schematicID | cut -c1-8)'"
94+
}
95+
96+
@test "a template already carrying the same tag set stops the program before the download" {
97+
stub="$(mktemp -d)"
98+
stub_pve "$stub"
99+
printf '[{"vmid":9000,"name":"old","node":"pve1","template":1,"tags":"talos-v1.13.6;Talos;schematic-%s;cozystack"}]' \
100+
"$(chart schematicID | cut -c1-8)" > "$stub/resources.json"
101+
rc=0
102+
out="$("$SCRIPT" print 9002 main-pool | PATH="$stub:$PATH" TMPDIR="$stub" sh 2>&1)" || rc=$?
103+
calls="$(cat "$stub/calls")"
104+
[ "$rc" -ne 0 ]
105+
case "$out" in *"already carries"*"9000 (old) on pve1"*) ;; *) echo "$out" >&2; exit 1 ;; esac
106+
[ -z "$calls" ] || { echo "$calls" >&2; exit 1; }
107+
rm -rf "$stub"
108+
}
109+
110+
@test "a template whose tags are a superset does not count, and the template is built" {
111+
stub="$(mktemp -d)"
112+
stub_pve "$stub"
113+
printf '[{"vmid":9000,"name":"old","node":"pve1","template":1,"tags":"cozystack;schematic-%s;talos;talos-v1.13.6;zfs"},{"vmid":101,"name":"vm","node":"pve1","template":0}]' \
114+
"$(chart schematicID | cut -c1-8)" > "$stub/resources.json"
115+
"$SCRIPT" print 9002 main-pool | PATH="$stub:$PATH" TMPDIR="$stub" sh >/dev/null
116+
calls="$(cat "$stub/calls")"
117+
printf '%s\n' "$calls" | grep -q '^curl .*/nocloud-amd64.raw.xz$'
118+
printf '%s\n' "$calls" | grep -qE '^qm create 9002 .*--scsi0 main-pool:0,import-from=[^ ]*/disk.raw,'
119+
printf '%s\n' "$calls" | grep -qx 'qm disk resize 9002 scsi0 20G'
120+
printf '%s\n' "$calls" | grep -qx 'qm template 9002'
121+
rm -rf "$stub"
122+
}
123+
124+
@test "a VM left behind by a failed conversion is named with the command that removes it" {
125+
stub="$(mktemp -d)"
126+
stub_pve "$stub"
127+
printf '#!/bin/sh\necho "qm $*" >> "%s/calls"\n[ "$1" != template ]\n' "$stub" > "$stub/qm"
128+
echo '[]' > "$stub/resources.json"
129+
rc=0
130+
out="$("$SCRIPT" print 9002 main-pool | PATH="$stub:$PATH" TMPDIR="$stub" sh 2>&1)" || rc=$?
131+
calls="$(cat "$stub/calls")"
132+
[ "$rc" -ne 0 ]
133+
printf '%s\n' "$calls" | grep -qx 'qm template 9002'
134+
case "$out" in *"remove it with: qm destroy 9002"*) ;; *) echo "$out" >&2; exit 1 ;; esac
135+
rm -rf "$stub"
136+
}
137+
138+
@test "create runs the printed program on the hypervisor over ssh and prints the pool's selector" {
139+
stub="$(mktemp -d)"
140+
printf '#!/bin/sh\necho "$*" > "%s/args"\ncat > "%s/stdin"\n' "$stub" "$stub" > "$stub/ssh"
141+
chmod +x "$stub/ssh"
142+
out="$(PATH="$stub:$PATH" COZY_PVE_SSH=root@pve "$SCRIPT" create 9002 main-pool)"
143+
args="$(cat "$stub/args")"
144+
sent="$(cat "$stub/stdin")"
145+
case "$args" in *"root@pve sh -s") ;; *) echo "$args" >&2; exit 1 ;; esac
146+
[ "$sent" = "$("$SCRIPT" print 9002 main-pool)" ]
147+
printf '%s' "$out" | grep -qxF ' - "talos-'"$(chart version)"'"'
148+
printf '%s' "$out" | grep -qxF ' schematicID: "'"$(chart schematicID)"'"'
149+
rm -rf "$stub"
150+
}
151+
152+
@test "create fails when the hypervisor program fails, and prints no selector" {
153+
stub="$(mktemp -d)"
154+
printf '#!/bin/sh\ncat >/dev/null\necho "ssh: connect to host pve: Connection refused" >&2\nexit 255\n' > "$stub/ssh"
155+
chmod +x "$stub/ssh"
156+
rc=0
157+
out="$(PATH="$stub:$PATH" COZY_PVE_SSH=root@pve "$SCRIPT" create 9002 main-pool 2>&1)" || rc=$?
158+
[ "$rc" -ne 0 ]
159+
case "$out" in *templateTags*) echo "$out" >&2; exit 1 ;; esac
160+
rm -rf "$stub"
161+
}

0 commit comments

Comments
 (0)