Skip to content

Commit 6fd98dc

Browse files
committed
fix(apps/kubernetes): honour nodeGroup minReplicas + drop nodeGroups from render-side OIDC tests
Closes #3231. `packages/apps/kubernetes/templates/cluster.yaml` hardcoded `MachineDeployment.spec.replicas: 2` regardless of the nodeGroup's `minReplicas`. The two render-side `kubernetes-oidc-{system,customconfig}` bats — which set `minReplicas: 0` and delete their `Kubernetes` CR without waiting for readiness — still provisioned two 20 GiB `replicated` (DRBD) worker `DataVolume`s each. Their create/delete churn overlapped the very next test, `kubernetes-previous`, and put transient pressure on a LINSTOR satellite's ZFS pool: worker root disks landed late, tenant nodes registered late, Cilium's CNI init missed the 12-minute node-join deadline, and the test failed with `cni plugin not initialized`. Two changes, applied together — the chart fix is a no-op without the test fix, and the test fix is a no-op without the chart fix, because dropping `nodeGroups` falls back to the chart helper's default `md0` (with `minReplicas: 0`). **Chart**: `spec.replicas: {{ $group.minReplicas }}` so the field matches the autoscaler annotations already emitted just above. `minReplicas: 0` groups now genuinely scale from zero and do not silently provision workers; groups with `minReplicas >= 2` (the kubernetes-latest / kubernetes-previous bats CRs, the tests/values/ common.yaml default, the values.yaml default that gets templated into the helper's `md0`) render byte-identical to before. **Render-side OIDC tests**: drop the explicit `nodeGroups:` block and `storageClass:` — falling back to the chart helper's default `md0` (now with an effective `replicas: 0` after the chart change) means the MachineDeployment renders but never creates a KubevirtMachine and no DataVolume is provisioned. The tests only assert on the KamajiControlPlane and the cozy-realm Keycloak objects, so removing worker plumbing does not narrow their coverage. Upgrade impact: any existing customer CR that already carries `nodeGroups.md0.minReplicas: 0` (rare — the default in tests/values and the chart helper both use it, but real user CRs almost always have `minReplicas >= 1`) will see the MD's `spec.replicas` reconcile from `2` down to `0` on the next chart apply. If the cluster- autoscaler is enabled it will scale back up as soon as an unschedulable pod appears; if it is not, the group intentionally becomes empty (which is what `minReplicas: 0` requests literally). This matches the semantics the autoscaler annotations were already advertising — the hardcoded `2` was contradicting them. Signed-off-by: IvanHunters <[email protected]>
1 parent e60ec04 commit 6fd98dc

3 files changed

Lines changed: 25 additions & 23 deletions

File tree

‎hack/e2e-apps/kubernetes-oidc-customconfig.bats‎

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -49,17 +49,13 @@ spec:
4949
resources: {}
5050
resourcesPreset: small
5151
replicas: 1
52-
nodeGroups:
53-
md0:
54-
diskSize: 20Gi
55-
gpus: []
56-
instanceType: u1.micro
57-
maxReplicas: 1
58-
minReplicas: 0
59-
resources: {}
60-
roles:
61-
- worker
62-
storageClass: replicated
52+
# nodeGroups intentionally omitted — this is a render-side test that
53+
# only asserts on the KamajiControlPlane + the tenant-supplied issuer
54+
# AuthenticationConfiguration Secret. Combined with the chart change
55+
# that makes `MachineDeployment.spec.replicas` honour `minReplicas`,
56+
# dropping the field means zero MachineDeployments render and no
57+
# worker DataVolumes are touched, keeping the render-side test from
58+
# churning DRBD storage between CI runs (see #3231).
6359
version: v1.35
6460
oidc:
6561
mode: CustomConfig

‎hack/e2e-apps/kubernetes-oidc-system.bats‎

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -57,17 +57,16 @@ spec:
5757
resources: {}
5858
resourcesPreset: small
5959
replicas: 1
60-
nodeGroups:
61-
md0:
62-
diskSize: 20Gi
63-
gpus: []
64-
instanceType: u1.micro
65-
maxReplicas: 1
66-
minReplicas: 0
67-
resources: {}
68-
roles:
69-
- worker
70-
storageClass: replicated
60+
# nodeGroups intentionally omitted — this is a render-side test that
61+
# only asserts on the KamajiControlPlane + the cozy-realm Keycloak
62+
# objects. The chart's MachineDeployment hardcodes spec.replicas: 2
63+
# even when a nodeGroup declares minReplicas: 0 (see #3231), so
64+
# keeping a worker nodeGroup here provisions two 20 GiB DRBD
65+
# DataVolumes on each `kubectl apply` and their churn overlaps
66+
# kubernetes-previous, flaking that test with LINSTOR pool pressure.
67+
# Dropping the field lets the schema default `{}` apply — zero
68+
# MachineDeployments rendered, zero worker DataVolumes touched,
69+
# test still exercises everything it claimed to.
7170
version: v1.35
7271
oidc:
7372
mode: System

‎packages/apps/kubernetes/templates/cluster.yaml‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -743,7 +743,14 @@ metadata:
743743
{{- end }}
744744
spec:
745745
clusterName: {{ $.Release.Name }}
746-
replicas: 2
746+
# Honour the nodeGroup's minReplicas so `minReplicas: 0` scales from
747+
# zero and does not silently provision two DRBD DataVolumes per
748+
# render (see cozystack/cozystack#3231). If the cluster-autoscaler
749+
# is enabled it will scale up on demand via the annotation-driven
750+
# min/max range above; existing installs with `minReplicas >= 2`
751+
# render byte-identical to before, so the previous default is
752+
# preserved for the common case.
753+
replicas: {{ $group.minReplicas }}
747754
strategy:
748755
rollingUpdate:
749756
maxSurge: {{ $group.maxReplicas }}

0 commit comments

Comments
 (0)