Skip to content

Commit 1245b32

Browse files
committed
fix(platform)!: refuse a networking variant change on a live cluster
Changing the variant of the cozystack.cozystack-platform Package on a running cluster changes bundles.system.variant and can re-render the cozystack.networking Package with another variant. The operator then deletes every HelmRelease of that Package the new variant does not install, so isp-full -> isp-hosted removes Cilium and Kube-OVN and the cluster loses its CNI; isp-full -> isp-slim removes Kube-OVN. The platform render now looks up the live Package and fails when its variant differs from the one about to be rendered, unless networking.acknowledgeVariantChange names the new variant. The acknowledgement is a variant name rather than a boolean so that a value left behind after one switch does not cover a later one. The check is inert where lookup returns nothing (helm template, client-side dry-run, first install) and skipped when cozystack.networking is in bundles.disabledPackages, since the platform then does not render it. A live variant set by hand, such as cilium-kilo, is pointed at bundles.disabledPackages rather than at the acknowledgement: the acknowledged render would write the bundle's variant over it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 6addef4 commit 1245b32

3 files changed

Lines changed: 267 additions & 0 deletions

File tree

‎packages/core/platform/templates/_helpers.tpl‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,9 @@
88
{{- end -}}
99
{{- $disabled := default (list) $root.Values.bundles.disabledPackages -}}
1010
{{- if not (has $name $disabled) -}}
11+
{{- if eq $name "cozystack.networking" -}}
12+
{{- include "cozystack.platform.networking.variantGuard" (list $root $variant) -}}
13+
{{- end }}
1114
---
1215
apiVersion: cozystack.io/v1alpha1
1316
kind: Package
@@ -24,6 +27,29 @@ spec:
2427
{{- end }}
2528
{{ end }}
2629

30+
{{/*
31+
Refuses to change the variant of a live cozystack.networking Package. The
32+
operator deletes every HelmRelease of a Package that its new variant does not
33+
install, so isp-full -> isp-hosted removes cilium and kubeovn and leaves the
34+
cluster without a CNI. The acknowledgement names the target variant rather than
35+
being a boolean, so a value left set after one switch does not cover the next.
36+
Inert where lookup returns nothing: helm template, client-side dry-run, first
37+
install.
38+
Call as (list $ <variant about to be rendered>).
39+
*/}}
40+
{{- define "cozystack.platform.networking.variantGuard" -}}
41+
{{- $root := index . 0 -}}
42+
{{- $variant := index . 1 -}}
43+
{{- $live := lookup "cozystack.io/v1alpha1" "Package" "" "cozystack.networking" -}}
44+
{{- if $live -}}
45+
{{- $liveVariant := default "default" (dig "spec" "variant" "" $live) -}}
46+
{{- $ack := dig "acknowledgeVariantChange" "" ($root.Values.networking | default dict) | toString -}}
47+
{{- if and (ne $liveVariant $variant) (ne $ack $variant) -}}
48+
{{- fail (printf "the live cozystack.networking Package runs variant %q and this render would switch it to %q (bundles.system.variant: %s). The operator then deletes any HelmRelease of that Package the new variant does not install (none if it installs everything the current one does): isp-full to isp-hosted removes Cilium and Kube-OVN and leaves every pod without networking, isp-full to isp-slim removes Kube-OVN. Either restore the previous spec.variant of the cozystack.cozystack-platform Package, or, once the cluster has another CNI or no longer needs the removed one, set networking.acknowledgeVariantChange: %q in its spec.components.platform.values to confirm this exact switch. If the live variant was set by hand and has to stay, add cozystack.networking to bundles.disabledPackages: acknowledging lets this render replace it." $liveVariant $variant $root.Values.bundles.system.variant $variant) -}}
49+
{{- end -}}
50+
{{- end -}}
51+
{{- end }}
52+
2753
{{- define "cozystack.platform.package.default" -}}
2854
{{- $name := index . 0 -}}
2955
{{- $root := index . 1 -}}
Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
1+
suite: networking variant change guard (lookup of the live cozystack.networking Package)
2+
3+
# A new networking variant is not a migration. The operator reconciles the
4+
# Package against the new variant and deletes every HelmRelease of the Package
5+
# that the new variant does not install: isp-full -> isp-hosted (noop) removes
6+
# cilium and kubeovn, isp-full -> isp-slim (cilium) removes kubeovn. The guard
7+
# reads the live Package and is inert where lookup returns nothing (helm
8+
# template, client-side dry-run, fresh install).
9+
templates:
10+
- templates/bundles/system.yaml
11+
release:
12+
name: cozystack
13+
namespace: cozy-system
14+
set:
15+
bundles:
16+
system:
17+
enabled: true
18+
variant: isp-full
19+
tests:
20+
- it: refuses isp-full -> isp-hosted on a live kubeovn-cilium cluster and names the acknowledge value
21+
set:
22+
bundles:
23+
system:
24+
variant: isp-hosted
25+
kubernetesProvider:
26+
scheme: &scheme
27+
"cozystack.io/v1alpha1/Package":
28+
gvr: {group: "cozystack.io", version: "v1alpha1", resource: "packages"}
29+
namespaced: false
30+
objects: &liveKubeovnCilium
31+
- kind: Package
32+
apiVersion: cozystack.io/v1alpha1
33+
metadata: {name: cozystack.networking}
34+
spec: {variant: kubeovn-cilium}
35+
asserts:
36+
- failedTemplate:
37+
errorPattern: 'cozystack.networking Package runs variant "kubeovn-cilium".*render would switch it to "noop".*restore the previous spec.variant of the cozystack.cozystack-platform Package.*networking.acknowledgeVariantChange: "noop" in its spec.components.platform.values'
38+
39+
- it: refuses isp-full -> isp-slim on a live kubeovn-cilium cluster
40+
set:
41+
bundles:
42+
system:
43+
variant: isp-slim
44+
kubernetesProvider:
45+
scheme: *scheme
46+
objects: *liveKubeovnCilium
47+
asserts:
48+
- failedTemplate:
49+
errorPattern: 'runs variant "kubeovn-cilium".*switch it to "cilium".*networking.acknowledgeVariantChange: "cilium"'
50+
51+
# Nothing is deleted on this switch, so the error must not say otherwise.
52+
- it: refuses isp-hosted -> isp-full on a live noop cluster
53+
kubernetesProvider:
54+
scheme: *scheme
55+
objects:
56+
- kind: Package
57+
apiVersion: cozystack.io/v1alpha1
58+
metadata: {name: cozystack.networking}
59+
spec: {variant: noop}
60+
asserts:
61+
- failedTemplate:
62+
errorPattern: 'runs variant "noop".*switch it to "kubeovn-cilium".*deletes any HelmRelease of that Package the new variant does not install \(none if it installs everything the current one does\)'
63+
64+
- it: treats a live Package without spec.variant as the default variant
65+
kubernetesProvider:
66+
scheme: *scheme
67+
objects:
68+
- kind: Package
69+
apiVersion: cozystack.io/v1alpha1
70+
metadata: {name: cozystack.networking}
71+
spec: {}
72+
asserts:
73+
- failedTemplate:
74+
errorPattern: 'runs variant "default".*switch it to "kubeovn-cilium"'
75+
76+
# No bundle renders cilium-kilo, so a live one was set by hand. Acknowledging
77+
# lets the render write the bundle's variant over it, so the error has to
78+
# point such a cluster at disabledPackages instead.
79+
- it: points a hand-set live variant at disabledPackages
80+
kubernetesProvider:
81+
scheme: *scheme
82+
objects: &liveCiliumKilo
83+
- kind: Package
84+
apiVersion: cozystack.io/v1alpha1
85+
metadata: {name: cozystack.networking}
86+
spec: {variant: cilium-kilo}
87+
asserts:
88+
- failedTemplate:
89+
errorPattern: 'runs variant "cilium-kilo".*restore the previous spec.variant of the cozystack.cozystack-platform Package.*networking.acknowledgeVariantChange: "kubeovn-cilium".*If the live variant was set by hand and has to stay, add cozystack.networking to bundles.disabledPackages: acknowledging lets this render replace it'
90+
91+
- it: lets an acknowledged switch replace a hand-set live variant
92+
set:
93+
networking:
94+
acknowledgeVariantChange: kubeovn-cilium
95+
kubernetesProvider:
96+
scheme: *scheme
97+
objects: *liveCiliumKilo
98+
documentSelector:
99+
path: metadata.name
100+
value: cozystack.networking
101+
asserts:
102+
- equal:
103+
path: spec.variant
104+
value: kubeovn-cilium
105+
106+
- it: renders when the live variant matches the rendered one
107+
kubernetesProvider:
108+
scheme: *scheme
109+
objects: *liveKubeovnCilium
110+
documentSelector:
111+
path: metadata.name
112+
value: cozystack.networking
113+
asserts:
114+
- equal:
115+
path: spec.variant
116+
value: kubeovn-cilium
117+
118+
- it: renders the change when acknowledgeVariantChange names the new variant
119+
set:
120+
bundles:
121+
system:
122+
variant: isp-hosted
123+
networking:
124+
acknowledgeVariantChange: noop
125+
kubernetesProvider:
126+
scheme: *scheme
127+
objects: *liveKubeovnCilium
128+
documentSelector:
129+
path: metadata.name
130+
value: cozystack.networking
131+
asserts:
132+
- equal:
133+
path: spec.variant
134+
value: noop
135+
136+
# The acknowledgement is bound to one target, so a value left behind after
137+
# one switch does not wave through the next one.
138+
- it: still refuses when acknowledgeVariantChange names a different variant
139+
set:
140+
bundles:
141+
system:
142+
variant: isp-hosted
143+
networking:
144+
acknowledgeVariantChange: cilium
145+
kubernetesProvider:
146+
scheme: *scheme
147+
objects: *liveKubeovnCilium
148+
asserts:
149+
- failedTemplate:
150+
errorPattern: 'networking.acknowledgeVariantChange: "noop"'
151+
152+
- it: still refuses when acknowledgeVariantChange is a bare boolean
153+
set:
154+
bundles:
155+
system:
156+
variant: isp-hosted
157+
networking:
158+
acknowledgeVariantChange: true
159+
kubernetesProvider:
160+
scheme: *scheme
161+
objects: *liveKubeovnCilium
162+
asserts:
163+
- failedTemplate:
164+
errorPattern: 'networking.acknowledgeVariantChange: "noop"'
165+
166+
# A Package left out of the render is not re-rendered, so Helm does not
167+
# touch its variant; helm.sh/resource-policy: keep leaves it as it is.
168+
- it: renders when cozystack.networking is in bundles.disabledPackages
169+
set:
170+
bundles:
171+
system:
172+
variant: isp-hosted
173+
disabledPackages:
174+
- cozystack.networking
175+
kubernetesProvider:
176+
scheme: *scheme
177+
objects: *liveKubeovnCilium
178+
asserts:
179+
- containsDocument:
180+
apiVersion: cozystack.io/v1alpha1
181+
kind: Package
182+
name: cozystack.networking
183+
any: true
184+
not: true
185+
186+
- it: renders offline (no live Package) for isp-hosted
187+
set:
188+
bundles:
189+
system:
190+
variant: isp-hosted
191+
documentSelector:
192+
path: metadata.name
193+
value: cozystack.networking
194+
asserts:
195+
- equal:
196+
path: spec.variant
197+
value: noop
198+
199+
- it: renders offline (no live Package) for isp-slim
200+
set:
201+
bundles:
202+
system:
203+
variant: isp-slim
204+
documentSelector:
205+
path: metadata.name
206+
value: cozystack.networking
207+
asserts:
208+
- equal:
209+
path: spec.variant
210+
value: cilium
211+
212+
- it: renders on a cluster that has other Packages but no cozystack.networking yet
213+
set:
214+
bundles:
215+
system:
216+
variant: isp-full-generic
217+
kubernetesProvider:
218+
scheme: *scheme
219+
objects:
220+
- kind: Package
221+
apiVersion: cozystack.io/v1alpha1
222+
metadata: {name: cozystack.cert-manager}
223+
spec: {variant: default}
224+
documentSelector:
225+
path: metadata.name
226+
value: cozystack.networking
227+
asserts:
228+
- equal:
229+
path: spec.variant
230+
value: kubeovn-cilium-generic

‎packages/core/platform/values.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,17 @@ networking:
9595
# Set this to comma-separated list of master node IPs to override.
9696
kubeovn:
9797
MASTER_NODES: ""
98+
# Confirms a change of the cozystack.networking Package variant on a running
99+
# cluster. Changing the variant of the cozystack.cozystack-platform Package
100+
# can change that variant (isp-full runs kubeovn-cilium, isp-slim runs cilium,
101+
# isp-hosted runs noop), and the operator then deletes the networking
102+
# HelmReleases the new variant does not install: isp-full -> isp-hosted
103+
# removes Cilium and Kube-OVN. The platform render refuses such a change
104+
# while the live Package has another variant, unless this is set to the NEW
105+
# networking variant, as named in the render error. It confirms that one
106+
# switch only; a leftover value does not cover a later switch to a different
107+
# variant.
108+
acknowledgeVariantChange: ""
98109
# In-cluster (east-west) transport encryption. Opt-in, OFF by default.
99110
#
100111
# When enabled, pod-to-pod traffic that crosses nodes is encrypted at the

0 commit comments

Comments
 (0)