Skip to content

Commit eda99e7

Browse files
myasnikovdaniilmattia-eleuteri
myasnikovdaniil
and
mattia-eleuteri
committed
feat(kubernetes): per-node-group nodeHealthCheck overrides
Add optional nodeGroups[name].maxUnhealthy and nodeGroups[name].nodeStartupTimeout fields that override the cluster-wide nodeHealthCheck defaults per worker node group. When unset, a group inherits nodeHealthCheck.{maxUnhealthy,nodeStartupTimeout}, so existing values are unaffected. The MachineHealthCheck render resolves the per-group value first, reuses the existing int-or-string validation, and names the offending group in the fail() message when an override is invalid. Delivers the per-group granularity of the standalone maxUnhealthy proposal on top of the cluster-wide nodeHealthCheck from the Talos worker bootstrap PR. Co-authored-by: mattia-eleuteri <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 4404c5c commit eda99e7

7 files changed

Lines changed: 116 additions & 5 deletions

File tree

‎api/apps/v1alpha1/kubernetes/types.go‎

Lines changed: 4 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/apps/kubernetes/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,8 @@ See the reference for components utilized in this service:
129129
| `nodeGroups[name].kubelet.kubeReservedCpu` | CPU reserved for kubelet and container runtime. Auto-computed from instanceType if empty. | `string` | `""` |
130130
| `nodeGroups[name].kubelet.evictionHardMemory` | Hard eviction threshold for memory (absolute like 200Mi or percentage like 7%). | `string` | `7%` |
131131
| `nodeGroups[name].kubelet.evictionSoftMemory` | Soft eviction threshold for memory (absolute like 1Gi or percentage like 10%). | `string` | `10%` |
132+
| `nodeGroups[name].maxUnhealthy` | Per-group override for `nodeHealthCheck.maxUnhealthy`. When unset, the cluster-wide `nodeHealthCheck.maxUnhealthy` applies. Accepts a bare integer ("0", "1", ...) or an integer percentage ("0%", "50%"). | `string` | `""` |
133+
| `nodeGroups[name].nodeStartupTimeout` | Per-group override for `nodeHealthCheck.nodeStartupTimeout`. When unset, the cluster-wide `nodeHealthCheck.nodeStartupTimeout` applies. | `string` | `""` |
132134
| `version` | Kubernetes major.minor version to deploy | `string` | `v1.35` |
133135
| `host` | External hostname for Kubernetes cluster. Defaults to `<cluster-name>.<tenant-host>` if empty. | `string` | `""` |
134136

‎packages/apps/kubernetes/templates/cluster.yaml‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -674,19 +674,28 @@ spec:
674674
bare integer (0, 1, ...) or a percentage string ("0%", "50%"), but
675675
rejects a quoted plain integer ("0"). Quote only when the operator
676676
supplied a percentage; otherwise render the value as a bare int so
677-
the admission webhook is happy with the chart-default "0". */}}
677+
the admission webhook is happy with the chart-default "0". A node
678+
group may override the cluster-wide nodeHealthCheck via
679+
nodeGroups.<name>.maxUnhealthy / .nodeStartupTimeout. */}}
678680
{{- $maxUnhealthy := toString $.Values.nodeHealthCheck.maxUnhealthy }}
681+
{{- $muSource := "nodeHealthCheck.maxUnhealthy" }}
682+
{{- if hasKey $group "maxUnhealthy" }}
683+
{{- $maxUnhealthy = toString $group.maxUnhealthy }}
684+
{{- $muSource = printf "nodeGroups.%s.maxUnhealthy" $groupName }}
685+
{{- end }}
679686
{{- if hasSuffix "%" $maxUnhealthy }}
680687
{{- if not (regexMatch "^[0-9]+%$" $maxUnhealthy) }}
681-
{{- fail (printf "nodeHealthCheck.maxUnhealthy must be a bare integer (e.g. 0, 1) or an integer percentage (e.g. 50%%), got %q" $maxUnhealthy) }}
688+
{{- fail (printf "%s must be a bare integer (e.g. 0, 1) or an integer percentage (e.g. 50%%), got %q" $muSource $maxUnhealthy) }}
682689
{{- end }}
683690
maxUnhealthy: {{ $maxUnhealthy | quote }}
684691
{{- else if regexMatch "^[0-9]+$" $maxUnhealthy }}
685692
maxUnhealthy: {{ $maxUnhealthy | int }}
686693
{{- else }}
687-
{{- fail (printf "nodeHealthCheck.maxUnhealthy must be a bare integer (e.g. 0, 1) or an integer percentage (e.g. 50%%), got %q" $maxUnhealthy) }}
694+
{{- fail (printf "%s must be a bare integer (e.g. 0, 1) or an integer percentage (e.g. 50%%), got %q" $muSource $maxUnhealthy) }}
688695
{{- end }}
689-
nodeStartupTimeout: {{ $.Values.nodeHealthCheck.nodeStartupTimeout }}
696+
{{- $nodeStartupTimeout := $.Values.nodeHealthCheck.nodeStartupTimeout }}
697+
{{- if hasKey $group "nodeStartupTimeout" }}{{ $nodeStartupTimeout = $group.nodeStartupTimeout }}{{ end }}
698+
nodeStartupTimeout: {{ $nodeStartupTimeout }}
690699
selector:
691700
matchLabels:
692701
cluster.x-k8s.io/cluster-name: {{ $.Release.Name }}

‎packages/apps/kubernetes/tests/mhc_rendering_test.yaml‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,3 +73,89 @@ tests:
7373
documentSelector:
7474
path: kind
7575
value: MachineHealthCheck
76+
77+
- it: per-group maxUnhealthy overrides the cluster-wide nodeHealthCheck value
78+
release:
79+
name: test-k8s
80+
namespace: tenant-test
81+
set:
82+
nodeHealthCheck:
83+
maxUnhealthy: "50%"
84+
nodeStartupTimeout: "10m"
85+
nodeGroups.md0.maxUnhealthy: "10%"
86+
asserts:
87+
- equal:
88+
path: spec.maxUnhealthy
89+
value: "10%"
90+
documentSelector:
91+
path: kind
92+
value: MachineHealthCheck
93+
94+
- it: per-group bare-integer maxUnhealthy override renders as an unquoted int
95+
release:
96+
name: test-k8s
97+
namespace: tenant-test
98+
set:
99+
nodeHealthCheck:
100+
maxUnhealthy: "50%"
101+
nodeStartupTimeout: "10m"
102+
nodeGroups.md0.maxUnhealthy: "3"
103+
asserts:
104+
- equal:
105+
path: spec.maxUnhealthy
106+
value: 3
107+
documentSelector:
108+
path: kind
109+
value: MachineHealthCheck
110+
111+
- it: per-group nodeStartupTimeout overrides the cluster-wide value
112+
release:
113+
name: test-k8s
114+
namespace: tenant-test
115+
set:
116+
nodeHealthCheck:
117+
maxUnhealthy: "50%"
118+
nodeStartupTimeout: "10m"
119+
nodeGroups.md0.nodeStartupTimeout: "30m"
120+
asserts:
121+
- equal:
122+
path: spec.nodeStartupTimeout
123+
value: "30m"
124+
documentSelector:
125+
path: kind
126+
value: MachineHealthCheck
127+
128+
- it: falls back to the cluster-wide nodeHealthCheck when no per-group override is set
129+
release:
130+
name: test-k8s
131+
namespace: tenant-test
132+
set:
133+
nodeHealthCheck:
134+
maxUnhealthy: "40%"
135+
nodeStartupTimeout: "15m"
136+
asserts:
137+
- equal:
138+
path: spec.maxUnhealthy
139+
value: "40%"
140+
documentSelector:
141+
path: kind
142+
value: MachineHealthCheck
143+
- equal:
144+
path: spec.nodeStartupTimeout
145+
value: "15m"
146+
documentSelector:
147+
path: kind
148+
value: MachineHealthCheck
149+
150+
- it: rejects an invalid per-group maxUnhealthy and names the offending group
151+
release:
152+
name: test-k8s
153+
namespace: tenant-test
154+
set:
155+
nodeHealthCheck:
156+
maxUnhealthy: "50%"
157+
nodeStartupTimeout: "10m"
158+
nodeGroups.md0.maxUnhealthy: "abc"
159+
asserts:
160+
- failedTemplate:
161+
errorMessage: 'nodeGroups.md0.maxUnhealthy must be a bare integer (e.g. 0, 1) or an integer percentage (e.g. 50%), got "abc"'

‎packages/apps/kubernetes/values.schema.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,11 +122,19 @@
122122
"type": "integer",
123123
"default": 10
124124
},
125+
"maxUnhealthy": {
126+
"description": "Per-group override for `nodeHealthCheck.maxUnhealthy`. When unset, the cluster-wide `nodeHealthCheck.maxUnhealthy` applies. Accepts a bare integer (\"0\", \"1\", ...) or an integer percentage (\"0%\", \"50%\").",
127+
"type": "string"
128+
},
125129
"minReplicas": {
126130
"description": "Minimum number of replicas.",
127131
"type": "integer",
128132
"default": 0
129133
},
134+
"nodeStartupTimeout": {
135+
"description": "Per-group override for `nodeHealthCheck.nodeStartupTimeout`. When unset, the cluster-wide `nodeHealthCheck.nodeStartupTimeout` applies.",
136+
"type": "string"
137+
},
130138
"resources": {
131139
"description": "CPU and memory resources for each worker node.",
132140
"type": "object",

‎packages/apps/kubernetes/values.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,8 @@ storageClass: replicated
8888
## @field {Resources} resources - CPU and memory resources for each worker node.
8989
## @field {[]GPU} gpus - List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM).
9090
## @field {Kubelet} [kubelet] - Kubelet resource reservations for this node group.
91+
## @field {string} [maxUnhealthy] - Per-group override for `nodeHealthCheck.maxUnhealthy`. When unset, the cluster-wide `nodeHealthCheck.maxUnhealthy` applies. Accepts a bare integer ("0", "1", ...) or an integer percentage ("0%", "50%").
92+
## @field {string} [nodeStartupTimeout] - Per-group override for `nodeHealthCheck.nodeStartupTimeout`. When unset, the cluster-wide `nodeHealthCheck.nodeStartupTimeout` applies.
9193

9294
## @param {map[string]NodeGroup} nodeGroups - Worker nodes configuration map.
9395
nodeGroups:

‎packages/system/kubernetes-rd/cozyrds/kubernetes.yaml‎

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)