Skip to content

Commit 3abdff5

Browse files
authored
fix(kubernetes-nodes): refuse a partial resources spec on every pool (#4973)
The `resources.cpu`/`resources.memory` pair check in `nodegroup.yaml` ran only when an `instanceType` was set, while the field description (values.yaml, README, schema, the KubernetesNodes OpenAPI schema) promised a render-time error for a partial spec on any pool. Without an instancetype a cpu-only pool renders a VM with no memory, which KubeVirt refuses when the Machine is created, and a memory-only pool silently sizes the guest at KubeVirt's default of one vCPU. - The check now runs on every kubevirt pool. The proxmox branch already required both, so this is the kubevirt half of one rule, and the field descriptions are true as written (no doc change). On a pool with no `instanceType` the message ends in "or neither and set instanceType", so it does not send the user straight into the next refusal. - A kubevirt pool with neither an `instanceType` nor both resources fails at render with a message naming both, the way `vm-instance` does. No such pool could have passed admission (no memory), so this refusal breaks nothing that runs today. This is the decision the issue asked for; if the gate should stay narrower, this commit can be dropped on its own. - **Upgrade impact.** One shape that renders on main stops rendering: `instanceType: ""` with only `resources.memory`. On main it gets `domain.memory.guest` and no `domain.cpu`, so KubeVirt runs the worker with one vCPU; with this PR it fails with the partial-spec message, and the pool's HelmRelease sits in a failed upgrade until the values are edited. To keep the current size such a pool needs `resources.cpu: "1"`. Adding that line renames the machine template (for the test values, `kubernetes-myk8s-md0-28d812` becomes `kubernetes-myk8s-md0-1ce103`, the same name on main and on this branch), so the pool's workers are replaced once. cpu-only with `instanceType: ""` is not in this class: it renders a VM with no memory, which KubeVirt refuses, so it never ran. - `cores:` printed an unquoted (float64) core count of one million or more in exponent form (`1e+06`). It is printed as an integer now, with the same bytes for every smaller whole number. Tests: `tests/resources_partial_test.yaml` covers both halves with and without an instancetype, the no-sizing case, both fields set, and the float64 formatting. The existing unquoted-zero case in `resources_cpu_test.yaml` now reads the field as unset on an instancetype-sized pool (with a `kubernetesProvider` stub for the lookup), and zero beside memory is pinned as a partial spec. Byte stability, checked by snapshotting the `KubevirtMachineTemplate` on main for nine spellings of `resources.cpu` and replaying the snapshots on this branch: `"2"`, `2`, `2.0`, `"2.0"`, `"+2"`, `"1e1"`, `"010"`, `999999` render identical bytes and template names. Only unquoted `4294967295` changes, from `cores: 4.294967295e+09` to `cores: 4294967295`. Verified with `helm unittest .` in `packages/apps/kubernetes-nodes` (38 suites, 233 tests, 12 snapshots), and on a trial merge with current main (39 suites, 237 tests). Closes #4546. The leading-zero inconsistency from the issue's comment is addressed separately, together with #4554. ### Release note ```release-note fix(kubernetes-nodes): a partial `resources` spec (only `cpu` or only `memory`) is now refused at render time on every kubevirt worker pool, not only on a pool with an `instanceType`, and a pool with neither an `instanceType` nor both resources is refused as well. Action required: a pool with `instanceType: ""` and only `resources.memory` rendered before (KubeVirt ran it with one vCPU) and now fails to upgrade; add `resources.cpu: "1"` to keep its current size. Adding that line renames the pool's machine template, so its workers are replaced once. Whole-number core counts of one million or more are now printed as integers instead of in exponent form. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Bug Fixes** * KubeVirt worker pools now require either an instance type or both CPU and memory values. Configurations with only one resource or no sizing information are rejected. * Whole-number CPU values render without exponent notation, including values written with a decimal fraction. * An unquoted zero CPU value is treated as unset when an instance type is provided; without an instance type, pairing zero CPU with memory fails validation. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 60d1f53 + 0e77e2b commit 3abdff5

3 files changed

Lines changed: 229 additions & 13 deletions

File tree

‎packages/apps/kubernetes-nodes/templates/nodegroup.yaml‎

Lines changed: 31 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -580,7 +580,13 @@ spec:
580580
{{- if and .group.resources .group.resources.cpu }}
581581
cpu:
582582
threads: 1
583-
cores: {{ .group.resources.cpu }}
583+
{{- /* An unquoted value arrives as float64, which Go prints in
584+
exponent form from 1e6 upward (1000000 -> 1e+06), and the
585+
CRD field is an integer. %.0f prints the same bytes as the
586+
default formatting for every whole number below that, so
587+
no existing template name moves; a string is emitted as
588+
written, as it always was. */}}
589+
cores: {{ if kindIs "float64" .group.resources.cpu }}{{ printf "%.0f" .group.resources.cpu }}{{ else }}{{ .group.resources.cpu }}{{ end }}
584590
sockets: 1
585591
{{- end }}
586592
devices:
@@ -663,11 +669,32 @@ spec:
663669
valid, resource-sized group. Explicit resources are all-or-nothing: a
664670
partial spec (only one of cpu/memory) would emit one domain override
665671
alongside the instancetype, which KubeVirt rejects, so it fails here with
666-
a clear render-time error instead of a late admission rejection. */}}
672+
a clear render-time error instead of a late admission rejection. The
673+
same rule holds without an instancetype, which is what the field
674+
description promises: cpu alone renders a VM with no memory, which
675+
KubeVirt refuses when the Machine is created, and memory alone sizes the
676+
guest at KubeVirt's default of one vCPU with nothing saying so. The
677+
proxmox branch requires both up front, so this is the kubevirt half of
678+
one rule. Zero is falsy here as everywhere in this chart (see the
679+
resources.cpu check above), so an unquoted zero counts as unset. */}}
667680
{{- $hasCpu := and $group.resources $group.resources.cpu }}
668681
{{- $hasMem := and $group.resources $group.resources.memory }}
669-
{{- if and $group.instanceType (or (and $hasCpu (not $hasMem)) (and $hasMem (not $hasCpu))) }}
670-
{{- fail (printf "nodeGroup %s: set both resources.cpu and resources.memory, or neither" $groupName) }}
682+
{{- if or (and $hasCpu (not $hasMem)) (and $hasMem (not $hasCpu)) }}
683+
{{- $orElse := "neither" }}
684+
{{- /* Without an instancetype "neither" alone leads straight into the
685+
refusal below, so name the other sizing source too. */}}
686+
{{- if not $group.instanceType }}
687+
{{- $orElse = "neither and set instanceType" }}
688+
{{- end }}
689+
{{- fail (printf "nodeGroup %s: set both resources.cpu and resources.memory, or %s" $groupName $orElse) }}
690+
{{- end }}
691+
{{- /* With neither sizing source the VM has no memory at all, and KubeVirt
692+
refuses it at admission, long after helm has reported success. No pool
693+
could ever have run in this state, so refusing it here costs nothing
694+
and names the choice. Proxmox has no instancetype and checks the same
695+
thing at the top of this file. */}}
696+
{{- if and (not $isProxmox) (not $group.instanceType) (not (and $hasCpu $hasMem)) }}
697+
{{- fail (printf "nodeGroup %s: set instanceType, or both resources.cpu and resources.memory: with neither the worker VM has no size, and KubeVirt refuses a VirtualMachine that declares no memory" $groupName) }}
671698
{{- end }}
672699
{{- /* Presence, not truth, for both fields below: zero is a quantity the
673700
schema accepts, and a guard keyed on truthiness would wave through

‎packages/apps/kubernetes-nodes/tests/resources_cpu_test.yaml‎

Lines changed: 77 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -158,23 +158,61 @@ tests:
158158
- failedTemplate:
159159
errorPattern: 'nodeGroup md0: resources.cpu \(1.5\) must be a whole number of vCPU cores between 1 and 4294967295'
160160

161-
# An unquoted zero is falsy, so it has always rendered as if the field were
162-
# unset: no domain.cpu, sizing left to the instancetype. The template name
163-
# below is what the chart rendered for it before this guard existed.
164-
- it: keeps rendering an unquoted zero as an unset field
161+
# An unquoted zero is falsy, so it has always counted as an unset field.
162+
# Beside a memory value it is therefore a partial spec, which the chart
163+
# refuses on every pool, and alone it leaves the instancetype to size the
164+
# guest: no domain.cpu, and the matcher stays on the VM.
165+
- it: reads an unquoted zero as an unset field
165166
set:
167+
instanceType: u1.medium
166168
resources:
167169
cpu: 0
168-
memory: 4Gi
170+
# The instancetype the lookup resolves, plus the LIST kinds the
171+
# old-template-preservation loop registers with the fake client, as in
172+
# worker_image_source_test.yaml.
173+
kubernetesProvider:
174+
scheme:
175+
"instancetype.kubevirt.io/v1beta1/VirtualMachineClusterInstancetype":
176+
gvr: {group: instancetype.kubevirt.io, version: v1beta1, resource: virtualmachineclusterinstancetypes}
177+
namespaced: false
178+
"cluster.x-k8s.io/v1beta1/MachineSet":
179+
gvr: {group: cluster.x-k8s.io, version: v1beta1, resource: machinesets}
180+
namespaced: true
181+
"infrastructure.cluster.x-k8s.io/v1alpha1/KubevirtMachineTemplate":
182+
gvr: {group: infrastructure.cluster.x-k8s.io, version: v1alpha1, resource: kubevirtmachinetemplates}
183+
namespaced: true
184+
"cdi.kubevirt.io/v1beta1/DataVolume":
185+
gvr: {group: cdi.kubevirt.io, version: v1beta1, resource: datavolumes}
186+
namespaced: true
187+
objects:
188+
- kind: VirtualMachineClusterInstancetype
189+
apiVersion: instancetype.kubevirt.io/v1beta1
190+
metadata: {name: u1.medium}
191+
- kind: MachineSet
192+
apiVersion: cluster.x-k8s.io/v1beta1
193+
metadata: {name: ignored-ms, namespace: tenant-test}
194+
spec: {template: {spec: {infrastructureRef: {kind: Other, name: ignored}}}}
195+
- kind: KubevirtMachineTemplate
196+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
197+
metadata: {name: ignored-kmt, namespace: tenant-test}
169198
documentSelector:
170199
path: kind
171200
value: KubevirtMachineTemplate
172201
asserts:
173202
- notExists:
174203
path: spec.template.spec.virtualMachineTemplate.spec.template.spec.domain.cpu
175204
- equal:
176-
path: metadata.name
177-
value: kubernetes-myk8s-md0-28d812
205+
path: spec.template.spec.virtualMachineTemplate.spec.instancetype.name
206+
value: u1.medium
207+
208+
- it: refuses an unquoted zero beside memory on a pool with no instancetype
209+
set:
210+
resources:
211+
cpu: 0
212+
memory: 4Gi
213+
asserts:
214+
- failedTemplate:
215+
errorPattern: 'nodeGroup md0: set both resources.cpu and resources.memory, or neither and set instanceType$'
178216

179217
# A quoted zero is truthy, so it never took the unset path. It was refused
180218
# before this guard existed as well, by the kubelet CPU reservation check,
@@ -298,10 +336,40 @@ tests:
298336
- failedTemplate:
299337
errorPattern: 'nodeGroup md0: resources.cpu \(1e10\) must be a whole number of vCPU cores between 1 and 4294967295'
300338

339+
# Memory alone is a partial spec on every pool now (resources_partial_test.yaml),
340+
# so the unset field is read on a pool its instancetype sizes.
301341
- it: leaves domain.cpu out when resources.cpu is unset
302342
set:
303-
resources:
304-
memory: 4Gi
343+
instanceType: u1.medium
344+
resources: {}
345+
# The instancetype the lookup resolves, plus the LIST kinds the
346+
# old-template-preservation loop registers with the fake client, as in
347+
# worker_image_source_test.yaml.
348+
kubernetesProvider:
349+
scheme:
350+
"instancetype.kubevirt.io/v1beta1/VirtualMachineClusterInstancetype":
351+
gvr: {group: instancetype.kubevirt.io, version: v1beta1, resource: virtualmachineclusterinstancetypes}
352+
namespaced: false
353+
"cluster.x-k8s.io/v1beta1/MachineSet":
354+
gvr: {group: cluster.x-k8s.io, version: v1beta1, resource: machinesets}
355+
namespaced: true
356+
"infrastructure.cluster.x-k8s.io/v1alpha1/KubevirtMachineTemplate":
357+
gvr: {group: infrastructure.cluster.x-k8s.io, version: v1alpha1, resource: kubevirtmachinetemplates}
358+
namespaced: true
359+
"cdi.kubevirt.io/v1beta1/DataVolume":
360+
gvr: {group: cdi.kubevirt.io, version: v1beta1, resource: datavolumes}
361+
namespaced: true
362+
objects:
363+
- kind: VirtualMachineClusterInstancetype
364+
apiVersion: instancetype.kubevirt.io/v1beta1
365+
metadata: {name: u1.medium}
366+
- kind: MachineSet
367+
apiVersion: cluster.x-k8s.io/v1beta1
368+
metadata: {name: ignored-ms, namespace: tenant-test}
369+
spec: {template: {spec: {infrastructureRef: {kind: Other, name: ignored}}}}
370+
- kind: KubevirtMachineTemplate
371+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
372+
metadata: {name: ignored-kmt, namespace: tenant-test}
305373
documentSelector:
306374
path: kind
307375
value: KubevirtMachineTemplate
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
suite: worker pool resources are all-or-nothing
2+
3+
# resources.cpu and resources.memory size the worker VM together or not at
4+
# all. With an instancetype a lone override is refused by KubeVirt; without one
5+
# cpu alone renders a VM with no memory, which KubeVirt refuses at admission,
6+
# and memory alone sizes the guest at one vCPU with nothing saying so. The
7+
# field description promises a render-time error for a partial spec, and this
8+
# file holds the chart to it on every pool. A pool with neither sizing source
9+
# is refused too, the way vm-instance refuses it: no such pool ever passed
10+
# admission, so nothing that runs today is affected.
11+
12+
templates:
13+
- templates/nodegroup.yaml
14+
release:
15+
name: kubernetes-nodes-myk8s-md0
16+
namespace: tenant-test
17+
set:
18+
cluster: myk8s
19+
_cluster:
20+
cluster-domain: cozy.local
21+
version: "v1.35"
22+
minReplicas: 0
23+
maxReplicas: 3
24+
instanceType: ""
25+
diskSize: 20Gi
26+
storageClass: replicated
27+
tests:
28+
- it: refuses cpu without memory on a pool with no instancetype
29+
set:
30+
resources:
31+
cpu: "2"
32+
asserts:
33+
- failedTemplate:
34+
errorPattern: 'nodeGroup md0: set both resources.cpu and resources.memory, or neither and set instanceType$'
35+
36+
- it: refuses memory without cpu on a pool with no instancetype
37+
set:
38+
resources:
39+
memory: 4Gi
40+
asserts:
41+
- failedTemplate:
42+
errorPattern: 'nodeGroup md0: set both resources.cpu and resources.memory, or neither and set instanceType$'
43+
44+
# The case the check always caught. Pinned so the two halves cannot drift:
45+
# the partial check runs before the instancetype lookup, so no stub is needed.
46+
- it: refuses cpu without memory on a pool sized by an instancetype
47+
set:
48+
instanceType: u1.medium
49+
resources:
50+
cpu: "2"
51+
asserts:
52+
- failedTemplate:
53+
errorPattern: 'nodeGroup md0: set both resources.cpu and resources.memory, or neither$'
54+
55+
- it: refuses memory without cpu on a pool sized by an instancetype
56+
set:
57+
instanceType: u1.medium
58+
resources:
59+
memory: 4Gi
60+
asserts:
61+
- failedTemplate:
62+
errorPattern: 'nodeGroup md0: set both resources.cpu and resources.memory, or neither$'
63+
64+
- it: refuses a pool with neither an instancetype nor explicit resources
65+
asserts:
66+
- failedTemplate:
67+
errorPattern: 'nodeGroup md0: set instanceType, or both resources.cpu and resources.memory'
68+
69+
- it: refuses an empty resources block beside an empty instancetype
70+
set:
71+
resources: {}
72+
asserts:
73+
- failedTemplate:
74+
errorPattern: 'nodeGroup md0: set instanceType, or both resources.cpu and resources.memory'
75+
76+
- it: sizes the guest from both fields when both are set
77+
set:
78+
resources:
79+
cpu: "2"
80+
memory: 4Gi
81+
documentSelector:
82+
path: kind
83+
value: KubevirtMachineTemplate
84+
asserts:
85+
- equal:
86+
path: spec.template.spec.virtualMachineTemplate.spec.template.spec.domain.cpu.cores
87+
value: 2
88+
- equal:
89+
path: spec.template.spec.virtualMachineTemplate.spec.template.spec.domain.memory.guest
90+
value: 4Gi
91+
- notExists:
92+
path: spec.template.spec.virtualMachineTemplate.spec.template.spec.instancetype
93+
94+
# An unquoted value is decoded as float64, which Go prints in exponent form
95+
# from one million upward. The CRD field is an integer, so the chart prints
96+
# the whole number; every smaller value prints the same bytes as before.
97+
- it: prints a float64 core count of a million as an integer
98+
set:
99+
resources:
100+
cpu: 1000000
101+
memory: 4Gi
102+
documentSelector:
103+
path: kind
104+
value: KubevirtMachineTemplate
105+
asserts:
106+
- equal:
107+
path: spec.template.spec.virtualMachineTemplate.spec.template.spec.domain.cpu.cores
108+
value: 1000000
109+
110+
- it: prints a whole float64 below a million the way it always did
111+
set:
112+
resources:
113+
cpu: 4.0
114+
memory: 4Gi
115+
documentSelector:
116+
path: kind
117+
value: KubevirtMachineTemplate
118+
asserts:
119+
- equal:
120+
path: spec.template.spec.virtualMachineTemplate.spec.template.spec.domain.cpu.cores
121+
value: 4

0 commit comments

Comments
 (0)