Skip to content

Commit b3aac80

Browse files
committed
fix(apps/kubernetes): omit chart-managed spec.replicas so cluster-autoscaler owns the field alone
Structural follow-up to the previous commit's `replicas: {{ $group.minReplicas }}`. Emitting the field from the chart at all — even with the semantically- correct value — keeps the helm-controller ↔ cluster-autoscaler field- ownership fight alive: helm-controller applies with SSA + force-conflicts (https://fluxcd.io/flux/components/helm/helmreleases/), so every routine `helm upgrade` action re-asserts `replicas: {{ $group.minReplicas }}` and drains whatever the autoscaler had patched in the meantime, with a real disruption window on running clusters. Drop the field from the manifest entirely. CAPI's MachineDeployment defaulting webhook seeds `spec.replicas` from the `cluster-api-autoscaler-node-group-min-size` annotation on Create (see https://cluster-api.sigs.k8s.io/tasks/automated-machine-management/autoscaling), so fresh installs still start with the intended minimum. From that point on the autoscaler owns the field via SSA field-manager tracking and chart apply never touches it again. Behavioural summary: * Fresh install with `minReplicas: 0` → CAPI defaults `replicas: 0` on Create → no worker DataVolumes provisioned until scale-out. * Fresh install with `minReplicas: 2` → CAPI defaults `replicas: 2` on Create → byte-identical to pre-#3231 behaviour. * Existing customer (any minReplicas) → next chart upgrade does not touch spec.replicas — the autoscaler-held value is preserved unconditionally. Regression tests updated to assert: * `notExists: spec.replicas` on the emitted MachineDeployment (the exact contract this commit establishes). * Autoscaler `min-size` / `max-size` annotations continue to carry the nodeGroup's declared bounds (so a future refactor cannot accidentally drop the wiring the defaulting webhook depends on). helm-unittest suite stays at 173/173. Ref: #3231. Signed-off-by: IvanHunters <[email protected]>
1 parent d30a2cf commit b3aac80

2 files changed

Lines changed: 55 additions & 21 deletions

File tree

‎packages/apps/kubernetes/templates/cluster.yaml‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -743,14 +743,18 @@ metadata:
743743
{{- end }}
744744
spec:
745745
clusterName: {{ $.Release.Name }}
746-
# Honour the nodeGroup's minReplicas so `minReplicas: 0` scales from
747-
# zero and does not silently provision two DRBD DataVolumes per
748-
# render (see cozystack/cozystack#3231). If the cluster-autoscaler
749-
# is enabled it will scale up on demand via the annotation-driven
750-
# min/max range above; existing installs with `minReplicas >= 2`
751-
# render byte-identical to before, so the previous default is
752-
# preserved for the common case.
753-
replicas: {{ $group.minReplicas }}
746+
# `spec.replicas` is intentionally omitted so the field is owned by
747+
# the cluster-autoscaler alone. CAPI's MachineDeployment defaulting
748+
# webhook seeds `spec.replicas` from the `cluster-api-autoscaler-node-
749+
# group-min-size` annotation on Create (see
750+
# https://cluster-api.sigs.k8s.io/tasks/automated-machine-management/autoscaling),
751+
# and Server-Side-Apply field-manager ownership then keeps chart
752+
# upgrades from overwriting whatever value the autoscaler patched in
753+
# the meantime. Emitting a chart-managed `replicas: N` here (as this
754+
# chart did before cozystack/cozystack#3231) drains workers on every
755+
# `helm upgrade` action until the autoscaler notices unschedulable
756+
# Pods and scales back up — a real disruption window on every routine
757+
# platform bump.
754758
strategy:
755759
rollingUpdate:
756760
maxSurge: {{ $group.maxReplicas }}

‎packages/apps/kubernetes/tests/nodegroups_default_test.yaml‎

Lines changed: 43 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -127,14 +127,22 @@ tests:
127127
value: MachineDeployment
128128
matchMany: true
129129

130-
# Regression guard for cozystack/cozystack#3231 — spec.replicas MUST reflect
131-
# the nodeGroup's minReplicas, not a hardcoded 2. Two matched cases so a
132-
# future refactor to a literal cannot silently regress both sides:
133-
# * minReplicas: 0 renders spec.replicas: 0 (the render-side OIDC test
134-
# path — no worker DataVolumes provisioned).
135-
# * minReplicas: 2 renders spec.replicas: 2 (the kubernetes-latest /
136-
# kubernetes-previous bats path — byte-identical to pre-PR behaviour).
137-
- it: MachineDeployment.spec.replicas equals nodeGroup.minReplicas — zero case
130+
# Regression guard for cozystack/cozystack#3231. The chart intentionally
131+
# OMITS `MachineDeployment.spec.replicas` so the cluster-autoscaler owns
132+
# the field alone: CAPI's defaulting webhook seeds it from the
133+
# `cluster-api-autoscaler-node-group-min-size` annotation on Create, and
134+
# SSA field-manager ownership keeps chart upgrades from overwriting
135+
# whatever value the autoscaler patched in the meantime. Two matched
136+
# cases so a future refactor cannot silently re-hardcode the field nor
137+
# break the annotation-driven min/max wiring:
138+
# * minReplicas: 0 (the render-side OIDC bats path — MachineDeployment
139+
# emitted with the min-size annotation "0", no chart-owned replicas
140+
# field, no worker DataVolumes provisioned until scale-out).
141+
# * minReplicas: 2 (the kubernetes-latest / kubernetes-previous bats
142+
# path — MachineDeployment emitted with the min-size annotation "2",
143+
# no chart-owned replicas field, defaulting webhook fills it to 2 on
144+
# Create).
145+
- it: MachineDeployment omits chart-managed spec.replicas — zero-minReplicas case
138146
release:
139147
name: test-k8s
140148
namespace: tenant-test
@@ -158,15 +166,31 @@ tests:
158166
gpus: []
159167
kubelet: {}
160168
asserts:
161-
- equal:
169+
# spec.replicas must not appear in the rendered manifest — CAPI's
170+
# defaulting webhook will seed it from the annotation on Create.
171+
- notExists:
162172
path: spec.replicas
163-
value: 0
173+
documentSelector:
174+
path: kind
175+
value: MachineDeployment
176+
matchMany: true
177+
# Autoscaler annotation wiring stays intact.
178+
- equal:
179+
path: metadata.annotations["cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size"]
180+
value: "0"
181+
documentSelector:
182+
path: kind
183+
value: MachineDeployment
184+
matchMany: true
185+
- equal:
186+
path: metadata.annotations["cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size"]
187+
value: "10"
164188
documentSelector:
165189
path: kind
166190
value: MachineDeployment
167191
matchMany: true
168192

169-
- it: MachineDeployment.spec.replicas equals nodeGroup.minReplicas — nonzero case
193+
- it: MachineDeployment omits chart-managed spec.replicas — nonzero-minReplicas case
170194
release:
171195
name: test-k8s
172196
namespace: tenant-test
@@ -190,9 +214,15 @@ tests:
190214
gpus: []
191215
kubelet: {}
192216
asserts:
193-
- equal:
217+
- notExists:
194218
path: spec.replicas
195-
value: 2
219+
documentSelector:
220+
path: kind
221+
value: MachineDeployment
222+
matchMany: true
223+
- equal:
224+
path: metadata.annotations["cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size"]
225+
value: "2"
196226
documentSelector:
197227
path: kind
198228
value: MachineDeployment

0 commit comments

Comments
 (0)