Skip to content

Commit 1ee9e4c

Browse files
committed
fix(computeplane): validate nodeGroups and require an ingress-nginx role
nodeGroups was a bare object: typos surfaced as a stuck release deep in the wrapped chart instead of an admission error, and groups without the ingress-nginx role silently left the pinned ingress-nginx controller (and with it the ComputePlane ingress) Pending forever, since supplied groups are not merged with the wrapped chart's default md0. Reuse the kubernetes app's nodeGroups typedefs for the schema and refuse rendering when no group carries the role. Assisted-By: Claude <[email protected]> Signed-off-by: Andrei Kvapil <[email protected]>
1 parent b573540 commit 1ee9e4c

6 files changed

Lines changed: 249 additions & 7 deletions

File tree

‎packages/extra/computeplane/README.md‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,25 @@ See the full design in
2525

2626
### Cluster shape
2727

28-
| Name | Description | Type | Value |
29-
| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ----- |
30-
| `nodeGroups` | Worker node groups of the ComputePlane cluster (cluster shape only — the security posture is fixed by the module and cannot be overridden). Uses the same structure as the `kubernetes` application's `nodeGroups`. When left empty, the wrapped `kubernetes` chart emits its default scale-from-zero `md0` group, so the cluster sits idle until a pending Pod (e.g. the ingress-nginx controller) triggers the cluster-autoscaler. | `object` | `{}` |
28+
| Name | Description | Type | Value |
29+
| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | ----------- |
30+
| `nodeGroups` | Worker node groups of the ComputePlane cluster (cluster shape only — the security posture is fixed by the module and cannot be overridden). Same structure as the `kubernetes` application's `nodeGroups`. When left empty, the wrapped `kubernetes` chart emits its default scale-from-zero `md0` group with `roles: [ingress-nginx]`, so the cluster sits idle until a pending Pod (e.g. the ingress-nginx controller) triggers the cluster-autoscaler. Supplied groups are authoritative (not merged with the default), so at least one of them must carry `roles: [ingress-nginx]` — enforced at render time. | `map[string]object` | `{}` |
31+
| `nodeGroups[name].minReplicas` | Minimum number of replicas. | `int` | `0` |
32+
| `nodeGroups[name].maxReplicas` | Maximum number of replicas. | `int` | `10` |
33+
| `nodeGroups[name].instanceType` | Virtual machine instance type. | `string` | `u1.medium` |
34+
| `nodeGroups[name].diskSize` | System disk size for the worker VM. Carries the Talos OS image, kubelet state, containerd image cache, and any local-path PVCs. | `quantity` | `20Gi` |
35+
| `nodeGroups[name].storageClass` | StorageClass for worker node persistent disks. When empty, falls back to the application-level storageClass of the wrapped kubernetes app. | `string` | `""` |
36+
| `nodeGroups[name].roles` | List of node roles. At least one group must carry `ingress-nginx` — the module pins the ingress-nginx addon on, and its controller only schedules onto nodes with that role (enforced at render time). | `[]string` | `[]` |
37+
| `nodeGroups[name].resources` | Explicit CPU and memory for each worker node, as an alternative to `instanceType` sizing. Set both `cpu` and `memory` together or neither. | `object` | `{}` |
38+
| `nodeGroups[name].resources.cpu` | CPU available. | `quantity` | `""` |
39+
| `nodeGroups[name].resources.memory` | Memory (RAM) available. | `quantity` | `""` |
40+
| `nodeGroups[name].gpus` | List of GPUs to attach (NVIDIA driver requires at least 4 GiB RAM). | `[]object` | `[]` |
41+
| `nodeGroups[name].gpus[i].name` | Name of GPU, such as "nvidia.com/AD102GL_L40S". | `string` | `""` |
42+
| `nodeGroups[name].kubelet` | Kubelet resource reservations for this node group. | `object` | `{}` |
43+
| `nodeGroups[name].kubelet.systemReservedMemory` | Memory reserved for host OS. Auto-computed from instanceType if empty. | `string` | `""` |
44+
| `nodeGroups[name].kubelet.kubeReservedMemory` | Memory reserved for kubelet and container runtime. Auto-computed from instanceType if empty. | `string` | `""` |
45+
| `nodeGroups[name].kubelet.systemReservedCpu` | CPU reserved for host OS. Auto-computed from instanceType if empty. | `string` | `""` |
46+
| `nodeGroups[name].kubelet.kubeReservedCpu` | CPU reserved for kubelet and container runtime. Auto-computed from instanceType if empty. | `string` | `""` |
47+
| `nodeGroups[name].kubelet.evictionHardMemory` | Hard eviction threshold for memory (absolute like 200Mi or percentage like 7%). | `string` | `7%` |
48+
| `nodeGroups[name].kubelet.evictionSoftMemory` | Soft eviction threshold for memory (absolute like 1Gi or percentage like 10%). | `string` | `10%` |
3149

‎packages/extra/computeplane/templates/cluster.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,25 @@
2626
a talos-reconcile Job that cannot become Ready until worker nodes exist, so
2727
the parent must not block on them.
2828
*/}}
29+
{{- /*
30+
Supplied nodeGroups are authoritative — the wrapped chart does not merge
31+
them with its default md0 (which carries roles: [ingress-nginx]) — while the
32+
module pins the ingress-nginx addon on and its controller only schedules
33+
onto nodes with that role. Without this guard, any tenant-supplied
34+
nodeGroups lacking the role would leave the controller Pending forever and
35+
the ComputePlane ingress would never come up.
36+
*/}}
37+
{{- if .Values.nodeGroups }}
38+
{{- $hasIngressRole := false }}
39+
{{- range $name, $group := .Values.nodeGroups }}
40+
{{- if has "ingress-nginx" (default (list) $group.roles) }}
41+
{{- $hasIngressRole = true }}
42+
{{- end }}
43+
{{- end }}
44+
{{- if not $hasIngressRole }}
45+
{{- fail "computeplane: at least one nodeGroup must carry roles: [ingress-nginx] — the module pins the ingress-nginx addon on, and its controller only schedules onto ingress-nginx nodes" }}
46+
{{- end }}
47+
{{- end }}
2948
apiVersion: helm.toolkit.fluxcd.io/v2
3049
kind: HelmRelease
3150
metadata:

‎packages/extra/computeplane/tests/cluster_test.yaml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,10 +67,47 @@ tests:
6767
maxReplicas: 3
6868
instanceType: u1.large
6969
diskSize: 32Gi
70+
roles: ["ingress-nginx"]
7071
asserts:
7172
- equal:
7273
path: spec.values.nodeGroups.md0.maxReplicas
7374
value: 3
7475
- equal:
7576
path: spec.values.nodeGroups.md0.instanceType
7677
value: u1.large
78+
79+
# Supplied nodeGroups are authoritative (not merged with the wrapped chart's
80+
# default md0), while the module pins ingress-nginx on — groups without the
81+
# role would leave the controller Pending forever, so the render must refuse.
82+
- it: rejects nodeGroups without an ingress-nginx role
83+
set:
84+
nodeGroups:
85+
gpu0:
86+
minReplicas: 1
87+
maxReplicas: 2
88+
instanceType: u1.large
89+
diskSize: 32Gi
90+
roles: []
91+
asserts:
92+
- failedTemplate:
93+
errorPattern: "at least one nodeGroup must carry roles"
94+
95+
- it: accepts nodeGroups when any group carries the ingress-nginx role
96+
set:
97+
nodeGroups:
98+
gpu0:
99+
minReplicas: 1
100+
maxReplicas: 2
101+
instanceType: u1.large
102+
diskSize: 32Gi
103+
roles: []
104+
edge:
105+
minReplicas: 1
106+
maxReplicas: 1
107+
instanceType: u1.medium
108+
diskSize: 20Gi
109+
roles: ["ingress-nginx"]
110+
asserts:
111+
- equal:
112+
path: spec.values.nodeGroups.edge.roles[0]
113+
value: ingress-nginx

0 commit comments

Comments
 (0)