Skip to content

Commit 4e9c5ce

Browse files
fix(kubernetes): render the talos-reconcile Job for the default md0 group
The Job in templates/talos/talos-reconcile-job.yaml is the only producer of the TalosConfigTemplate that each worker MachineDeployment names in spec.template.spec.bootstrap.configRef; cluster.yaml renders the reference but deliberately not the object. Its loop read .Values.nodeGroups directly, while the MachineDeployment loop reads the effective set through the kubernetes.nodeGroups helper. For a cluster that supplies no nodeGroups the two disagree: the helper's else-branch emits the built-in md0 group, so the md0 MachineDeployment renders, but the raw map is empty and no Job renders at all. Every Machine the cluster-autoscaler adds to md0 then blocks indefinitely on a TalosConfigTemplate that nothing will ever create, and the KamajiControlPlane spec.network.certSANs patch the same Job performs is skipped too. Range over the helper so the Job set tracks the MachineDeployment set exactly. No change for a cluster that declares its own groups, and md0 stays removable: the helper's if-branch keeps a user-supplied map authoritative. The gap survived because every helm-unittest fixture and e2e suite declares md0 explicitly. tests/nodegroups_default_test.yaml does render the empty-nodeGroups case but lists only templates/cluster.yaml, so it never looked at the Job. The new suite pins both halves of the helper contract and fails without this fix. Fixes #3504 Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 3f967ec commit 4e9c5ce

2 files changed

Lines changed: 139 additions & 1 deletion

File tree

‎packages/apps/kubernetes/templates/talos/talos-reconcile-job.yaml‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -485,7 +485,14 @@ spec:
485485
*management* cluster-domain, sourced from .Values._cluster
486486
(populated by cozystack-controller from the platform config). */}}
487487
{{- $mgmtClusterDomain := (index .Values._cluster "cluster-domain") | default "cozy.local" }}
488-
{{- range $groupName, $group := .Values.nodeGroups }}
488+
{{- /* Iterate the EFFECTIVE node groups (the kubernetes.nodeGroups helper),
489+
never the raw .Values.nodeGroups map. The built-in md0 group lives in
490+
the helper's else-branch, so a cluster that supplies no nodeGroups has
491+
an empty raw map: ranging over it renders zero Jobs, leaving the md0
492+
MachineDeployment below pointing at a TalosConfigTemplate that only
493+
this Job ever creates. CAPI then blocks every Machine the autoscaler
494+
adds. Must stay in lockstep with the loop in cluster.yaml. */}}
495+
{{- range $groupName, $group := (include "kubernetes.nodeGroups" $ | fromYaml) }}
489496
{{- /* Mirror the kubelet-reservation computation from cluster.yaml so the
490497
worker machineconfig the Job applies carries the same numbers the
491498
MachineDeployment validator was happy with. Auto-computed
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
suite: talos-reconcile Job covers every effective node group
2+
3+
# Regression guard for cozystack/cozystack#3504. The Job in
4+
# templates/talos/talos-reconcile-job.yaml is the only producer of the
5+
# TalosConfigTemplate that each worker MachineDeployment names in
6+
# `spec.template.spec.bootstrap.configRef` (cluster.yaml renders the reference
7+
# but not the object). Its loop must therefore iterate the same effective group
8+
# set as the MachineDeployment loop — the kubernetes.nodeGroups helper — not the
9+
# raw .Values.nodeGroups map. When the raw map was read, a cluster that supplied
10+
# no nodeGroups got the helper's built-in md0 MachineDeployment and zero Jobs, so
11+
# every Machine the autoscaler added to md0 blocked forever on a
12+
# TalosConfigTemplate nothing would ever create, and the KamajiControlPlane
13+
# certSANs patch the same Job performs never ran either.
14+
#
15+
# The two cases below pin both halves of the helper contract: the default group
16+
# gets a Job (the bug), and a user-supplied group set stays authoritative — md0
17+
# must NOT reappear alongside it, which is the removability #2936 introduced.
18+
#
19+
# Suite-level `templates` is load-bearing: it limits the render to this one
20+
# template. Declaring `templates` per test does not filter the render, and the
21+
# unrelated `lookup` in templates/helmreleases/csi.yaml then fails the suite.
22+
23+
templates:
24+
- templates/talos/talos-reconcile-job.yaml
25+
26+
tests:
27+
- it: renders a Job for the built-in md0 group when nodeGroups is empty
28+
release:
29+
name: test-k8s
30+
namespace: tenant-test
31+
set:
32+
_namespace:
33+
etcd: etcd
34+
ingress: nginx
35+
host: example.com
36+
_cluster:
37+
cluster-domain: cozy.local
38+
# nodeGroups intentionally left at the chart default ({}) so the helper
39+
# emits the built-in md0 — the case that rendered no Job at all.
40+
asserts:
41+
# 6 scaffolding documents (ServiceAccount, Role, CiliumNetworkPolicy,
42+
# RoleBinding, ClusterRole, ClusterRoleBinding) plus one Job for md0.
43+
# Before the fix this was 6: the Job was missing entirely.
44+
- hasDocuments:
45+
count: 7
46+
- isKind:
47+
of: Job
48+
documentIndex: 6
49+
- matchRegex:
50+
path: metadata.name
51+
pattern: ^test-k8s-talos-reconcile-md0-[0-9a-f]{6}$
52+
documentIndex: 6
53+
# The Job applies TalosConfigTemplate ${RELEASE}-${GROUP_NAME}, so
54+
# GROUP_NAME is what ties it to the MachineDeployment's configRef name
55+
# (test-k8s-md0). A Job for the wrong group name would satisfy the
56+
# document count above while leaving the MachineDeployment stuck.
57+
- contains:
58+
path: spec.template.spec.containers[0].env
59+
content:
60+
name: GROUP_NAME
61+
value: md0
62+
documentIndex: 6
63+
- matchRegex:
64+
path: spec.template.spec.containers[0].command[2]
65+
pattern: 'kind: TalosConfigTemplate'
66+
documentIndex: 6
67+
# Secondary effect of the missing Job: the KamajiControlPlane certSANs
68+
# patch that adds the live apiserver Service ClusterIP never ran.
69+
- matchRegex:
70+
path: spec.template.spec.containers[0].command[2]
71+
pattern: 'patch kamajicontrolplane'
72+
documentIndex: 6
73+
74+
- it: user-supplied node groups stay authoritative — one Job each, no md0
75+
release:
76+
name: test-k8s
77+
namespace: tenant-test
78+
set:
79+
_namespace:
80+
etcd: etcd
81+
ingress: nginx
82+
host: example.com
83+
_cluster:
84+
cluster-domain: cozy.local
85+
nodeGroups:
86+
worker0:
87+
minReplicas: 1
88+
maxReplicas: 3
89+
instanceType: u1.medium
90+
diskSize: 20Gi
91+
storageClass: ""
92+
roles:
93+
- ingress-nginx
94+
resources: {}
95+
gpus: []
96+
kubelet: {}
97+
worker1:
98+
minReplicas: 0
99+
maxReplicas: 5
100+
instanceType: u1.medium
101+
diskSize: 20Gi
102+
storageClass: ""
103+
roles: []
104+
resources: {}
105+
gpus: []
106+
kubelet: {}
107+
asserts:
108+
# 6 scaffolding documents + exactly two Jobs. A fix that unconditionally
109+
# merged md0 into the loop would render three and fail here.
110+
- hasDocuments:
111+
count: 8
112+
- matchRegex:
113+
path: metadata.name
114+
pattern: ^test-k8s-talos-reconcile-worker0-[0-9a-f]{6}$
115+
documentIndex: 6
116+
- matchRegex:
117+
path: metadata.name
118+
pattern: ^test-k8s-talos-reconcile-worker1-[0-9a-f]{6}$
119+
documentIndex: 7
120+
- contains:
121+
path: spec.template.spec.containers[0].env
122+
content:
123+
name: GROUP_NAME
124+
value: worker0
125+
documentIndex: 6
126+
- contains:
127+
path: spec.template.spec.containers[0].env
128+
content:
129+
name: GROUP_NAME
130+
value: worker1
131+
documentIndex: 7

0 commit comments

Comments
 (0)