Skip to content

Commit 8722462

Browse files
committed
fix(kubernetes-nodes): refuse osImage on proxmox instead of honouring half of it
#4171 gave a worker pool an osImage, choosing where its system disk comes from: a CDI clone of a shared golden image, or an HTTP import from an Image Factory. A Proxmox worker's disk comes from neither. capmox clones a VM template that already carries Talos, and nothing in this chart can change what that template holds. Half of the field would still appear to work, which is why it is refused rather than ignored. osImage.factory.version reaches the machineconfig's installer image and changes the reconcile Job, so a pool could set it, watch the Job roll, and end up running one Talos while booting another: v1.12.6 moves the installer to .../installer/<schematic>:v1.12.6 and leaves the ProxmoxMachineTemplate untouched. osImage.builtin has no meaning here at all. Nothing is lost by refusing it: the installer target on this substrate is settable through talos.version and talos.schematicID, which do reach it. The branches suite gains the refusal case. The substrate parameter's description said changing the value "rolls the pool". The chart refuses the change on a live pool, and the description says so; it reaches the README, the values schema and the CRD. Signed-off-by: Marian Koreniuk <[email protected]> Assisted-by: LLM
1 parent 4bc6621 commit 8722462

7 files changed

Lines changed: 48 additions & 20 deletions

File tree

‎api/apps/v1alpha1/kubernetesnodes/types.go‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/apps/kubernetes-nodes/README.md‎

Lines changed: 12 additions & 12 deletions
Large diffs are not rendered by default.

‎packages/apps/kubernetes-nodes/templates/nodegroup.yaml‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,19 @@
6060
{{- if not $.Values.proxmox.dnsServers }}
6161
{{- fail "substrate proxmox requires proxmox.dnsServers: an off-cluster worker cannot resolve through the management cluster's CoreDNS" }}
6262
{{- end }}
63+
{{- /* osImage decides where a KubeVirt worker's system disk comes from, and a
64+
Proxmox worker's disk does not come from there at all: capmox clones a
65+
VM template that already carries Talos. Half of the field would still
66+
appear to work, which is the reason to refuse it rather than ignore it
67+
— osImage.factory.version reaches the machineconfig's installer image,
68+
so a pool could set it, see the Job change, and be running a different
69+
Talos than it asked for, with the boot image still whatever the
70+
template holds. osImage.builtin, a CDI clone of a golden image, has no
71+
meaning here at all. The installer target is settable on this substrate
72+
through talos.version, which does reach it. */}}
73+
{{- if $.Values.osImage }}
74+
{{- fail "substrate proxmox does not take osImage: the worker disk is cloned from the Proxmox VM template, not imported from an image source. Set the Talos the template carries, and use talos.version / talos.schematicID for the in-guest installer target." }}
75+
{{- end }}
6376
{{- /* A target storage is a full-clone parameter. A linked clone always lives on
6477
the template's storage, and both Proxmox and capmox's CRD refuse the pair;
6578
refusing it here names the two values instead of an object the operator
@@ -853,10 +866,9 @@ spec:
853866
{{- $_ := set $context "group" $group }}
854867
{{- $_ := set $context "groupName" $groupName }}
855868
{{- $_ := set $context "clusterName" $clusterName }}
856-
{{- /* The $kubevirtmachinetemplate* variables now hold either kind; the names
857-
are historical. Renaming them is safe — the content hash below is taken
858-
over the rendered template, not over the variable name — and is left out
859-
of this change only to keep the diff to the substrate. */}}
869+
{{- /* These variables hold either kind despite their names. The content hash
870+
below is taken over the rendered template, so the names reach nothing
871+
outside this block. */}}
860872
{{- $machineTemplateKind := ternary "ProxmoxMachineTemplate" "KubevirtMachineTemplate" $isProxmox }}
861873
{{- $kubevirtmachinetemplate := include (ternary "proxmoxmachinetemplate" "kubevirtmachinetemplate" $isProxmox) $context }}
862874
{{- $kubevirtmachinetemplateHash := $kubevirtmachinetemplate | sha256sum | trunc 6 }}

‎packages/apps/kubernetes-nodes/tests/substrate_branches_test.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,3 +156,19 @@ tests:
156156
path: spec.minReplicas
157157
- exists:
158158
path: spec.selector
159+
160+
- it: refuses osImage on proxmox, where the disk comes from the template
161+
set:
162+
substrate: proxmox
163+
proxmox:
164+
templateTags: [talos]
165+
network:
166+
bridge: vmbr0
167+
dnsServers: ["10.0.0.1"]
168+
osImage:
169+
factory:
170+
version: v1.12.6
171+
template: templates/nodegroup.yaml
172+
asserts:
173+
- failedTemplate:
174+
errorPattern: "substrate proxmox does not take osImage"

‎packages/apps/kubernetes-nodes/values.schema.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -301,7 +301,7 @@
301301
}
302302
},
303303
"substrate": {
304-
"description": "Which infrastructure provider backs this pool's worker VMs. `kubevirt` runs them inside this cluster on KubeVirt, sized by `instanceType` and booted from a Talos disk image CDI streams from the image factory. `proxmox` runs them on an external Proxmox VE cluster through capmox, sized by `resources`, and cloned from a Talos VM template that already exists on the hypervisor. The two substrates take different Talos platform images and different sizing inputs, so switching an existing pool is not a supported in-place change: create a new pool instead. Changing it rolls the pool, because the worker machine template is named by a hash of its content.",
304+
"description": "Which infrastructure provider backs this pool's worker VMs. `kubevirt` runs them inside this cluster on KubeVirt, sized by `instanceType` and booted from a Talos disk image CDI streams from the image factory. `proxmox` runs them on an external Proxmox VE cluster through capmox, sized by `resources`, and cloned from a Talos VM template that already exists on the hypervisor. The two substrates take different Talos platform images and different sizing inputs, so switching an existing pool is refused: the chart reads the live MachineDeployment's template kind and fails the render rather than rolling every worker to the other hypervisor and deleting the template the draining MachineSet still references. Create a new pool on the other substrate and scale this one down.",
305305
"type": "string",
306306
"default": "kubevirt",
307307
"x-kubernetes-validations": [

‎packages/apps/kubernetes-nodes/values.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ talos:
147147
## @section Substrate
148148
##
149149

150-
## @param {string} substrate="kubevirt" - Which infrastructure provider backs this pool's worker VMs. `kubevirt` runs them inside this cluster on KubeVirt, sized by `instanceType` and booted from a Talos disk image CDI streams from the image factory. `proxmox` runs them on an external Proxmox VE cluster through capmox, sized by `resources`, and cloned from a Talos VM template that already exists on the hypervisor. The two substrates take different Talos platform images and different sizing inputs, so switching an existing pool is not a supported in-place change: create a new pool instead. Changing it rolls the pool, because the worker machine template is named by a hash of its content.
150+
## @param {string} substrate="kubevirt" - Which infrastructure provider backs this pool's worker VMs. `kubevirt` runs them inside this cluster on KubeVirt, sized by `instanceType` and booted from a Talos disk image CDI streams from the image factory. `proxmox` runs them on an external Proxmox VE cluster through capmox, sized by `resources`, and cloned from a Talos VM template that already exists on the hypervisor. The two substrates take different Talos platform images and different sizing inputs, so switching an existing pool is refused: the chart reads the live MachineDeployment's template kind and fails the render rather than rolling every worker to the other hypervisor and deleting the template the draining MachineSet still references. Create a new pool on the other substrate and scale this one down.
151151
## @immutable
152152
substrate: kubevirt
153153

‎packages/system/kubernetes-nodes-rd/cozyrds/kubernetes-nodes.yaml‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)