Skip to content

Commit 730037c

Browse files
fix(installer): address review feedback on cozy-system bootstrap
- Bump alpine/k8s to 1.33.4 (was 1.32.0, EOL); matches in-tree pin used by cozystack-api, seaweedfs, vm-disk, and vm-instance hooks. - Skip hostNetwork=true and not-ready/CNI tolerations on the hosted variant: managed clusters provide CNI before install, and kube-system may carry strict PSA there that would reject hostNetwork pods. - Add packages/core/installer/templates/cozy-system-namespace.yaml gated on .Values.bareNamespace (default false). The top-level Makefile's manifests target now passes --set bareNamespace=true and renders Namespace + operator into _out/assets/cozystack-operator-*.yaml so the kubectl-apply install path stays self-contained. Helm install/upgrade callers keep the default and bootstrap via --create-namespace + the pre-install labeler hook. - Add a bats assertion in hack/e2e-install-cozystack.bats checking cozy-system carries pod-security.kubernetes.io/enforce=privileged and cozystack.io/system=true after helm install. - Inline comment near hostNetwork=true documenting why dnsPolicy left at Default (cluster DNS not up at install time; ClusterFirstWithHostNet would not help). Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 85ced0d commit 730037c

5 files changed

Lines changed: 74 additions & 11 deletions

File tree

‎Makefile‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,19 +41,30 @@ build: build-deps
4141
manifests:
4242
mkdir -p _out/assets
4343
cat internal/crdinstall/manifests/*.yaml > _out/assets/cozystack-crds.yaml
44+
# kubectl-apply install path: render the bare Namespace resource alongside
45+
# the operator. bareNamespace=true gates a Namespace cozy-system with PSA +
46+
# identity labels (see packages/core/installer/templates/cozy-system-namespace.yaml).
47+
# helm install/upgrade users keep the default (false) and use --create-namespace
48+
# + the pre-install labeler hook.
4449
# Talos variant (default)
4550
helm template installer packages/core/installer -n cozy-system \
51+
--set bareNamespace=true \
52+
--show-only templates/cozy-system-namespace.yaml \
4653
--show-only templates/cozystack-operator.yaml \
4754
> _out/assets/cozystack-operator-talos.yaml
4855
# Generic Kubernetes variant (k3s, kubeadm, RKE2)
4956
helm template installer packages/core/installer -n cozy-system \
57+
--set bareNamespace=true \
5058
--set cozystackOperator.variant=generic \
5159
--set cozystack.apiServerHost=REPLACE_ME \
60+
--show-only templates/cozy-system-namespace.yaml \
5261
--show-only templates/cozystack-operator.yaml \
5362
> _out/assets/cozystack-operator-generic.yaml
5463
# Hosted variant (managed Kubernetes)
5564
helm template installer packages/core/installer -n cozy-system \
65+
--set bareNamespace=true \
5666
--set cozystackOperator.variant=hosted \
67+
--show-only templates/cozy-system-namespace.yaml \
5768
--show-only templates/cozystack-operator.yaml \
5869
> _out/assets/cozystack-operator-hosted.yaml
5970

‎hack/e2e-install-cozystack.bats‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,13 @@
1616
--wait \
1717
--timeout 2m
1818

19+
# The pre-install hook (cozy-system-labeler) must have stamped the PSA and
20+
# cozystack identity labels onto cozy-system. Operator pods need
21+
# enforce=privileged for hostNetwork=true; a silent regression in the hook
22+
# would let helm install succeed but break operator admission downstream.
23+
kubectl get ns cozy-system -o jsonpath='{.metadata.labels.pod-security\.kubernetes\.io/enforce}' | grep -qx privileged
24+
kubectl get ns cozy-system -o jsonpath='{.metadata.labels.cozystack\.io/system}' | grep -qx true
25+
1926
# Verify the operator deployment is available
2027
kubectl wait deployment/cozystack-operator -n cozy-system --timeout=1m --for=condition=Available
2128

‎packages/core/installer/templates/cozy-system-labels.yaml‎

Lines changed: 24 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,16 +11,22 @@
1111
resource; callers run helm with --create-namespace and the namespace
1212
is born bare (no labels).
1313

14-
2. The operator pod needs hostNetwork=true (no CNI yet at install time),
15-
which violates PodSecurity baseline. So cozy-system needs
16-
enforce=privileged before the operator pod is admitted. But a labeler
17-
pod inside cozy-system would itself need hostNetwork=true and hit
18-
the same admission denial. The hook therefore lives in kube-system,
19-
which is PSA-exempt on Talos (defaults.exemptions.namespaces in the
20-
apiserver PodSecurityConfiguration) and unlabeled on vanilla
21-
kubeadm/kind/k3s. The Job's SA is bound to a ClusterRole scoped via
22-
resourceNames to "cozy-system" only, so the cluster permission grant
23-
is minimal.
14+
2. On talos / generic, the operator pod needs hostNetwork=true (no CNI
15+
yet at install time), which violates PodSecurity baseline. So
16+
cozy-system needs enforce=privileged before the operator pod is
17+
admitted. But a labeler pod inside cozy-system would itself need
18+
hostNetwork=true and hit the same admission denial. The hook
19+
therefore lives in kube-system, which is PSA-exempt on Talos
20+
(defaults.exemptions.namespaces in the apiserver
21+
PodSecurityConfiguration) and unlabeled on vanilla kubeadm/kind/k3s.
22+
The Job's SA is bound to a ClusterRole scoped via resourceNames to
23+
"cozy-system" only, so the cluster permission grant is minimal.
24+
25+
Hosted variant: the platform CNI is already up, so the labeler runs
26+
without hostNetwork and without the not-ready/CNI-not-ready tolerations.
27+
This also keeps the pod admissible on managed clusters
28+
(EKS/GKE/AKS/OpenShift) where kube-system may carry strict PSA labels
29+
(enforce=baseline or stricter) that would reject hostNetwork pods.
2430
*/}}
2531
---
2632
apiVersion: v1
@@ -79,10 +85,16 @@ spec:
7985
spec:
8086
serviceAccountName: cozy-system-labeler
8187
restartPolicy: OnFailure
88+
{{- if ne .Values.cozystackOperator.variant "hosted" }}
8289
# The hook runs BEFORE Cilium / kube-ovn / any CNI is installed, so the
8390
# cluster's nodes are NotReady-NoSchedule and have no pod network. Mirror
8491
# the cozystack-operator deployment's scheduling: hostNetwork=true so the
8592
# pod doesn't need CNI, plus the same tolerations the operator uses.
93+
# dnsPolicy left at the default (ClusterFirst). With hostNetwork=true that
94+
# downgrades to "node DNS only", which is fine: kubectl talks to the
95+
# apiserver via KubePrism / explicit IP / kubelet-injected env, no DNS
96+
# lookups happen. ClusterFirstWithHostNet would not help anyway because
97+
# cluster DNS is not yet up at install time.
8698
hostNetwork: true
8799
tolerations:
88100
- key: "node.kubernetes.io/not-ready"
@@ -93,12 +105,13 @@ spec:
93105
operator: "Exists"
94106
- key: "node.cloudprovider.kubernetes.io/uninitialized"
95107
operator: "Exists"
108+
{{- end }}
96109
securityContext:
97110
seccompProfile:
98111
type: RuntimeDefault
99112
containers:
100113
- name: kubectl
101-
image: alpine/k8s:1.32.0@sha256:4c28b0f0c6accfa07fa449493ea936291e4ff1270500571d73f75bb406676c3b
114+
image: docker.io/alpine/k8s:1.33.4@sha256:b0523f0a244ddc4c8e055aa335c040d3d78b3ead5528f4544395f7f9f69c7b68
102115
imagePullPolicy: IfNotPresent
103116
securityContext:
104117
allowPrivilegeEscalation: false
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{{/*
2+
Bare Namespace resource for the kubectl-apply install path only
3+
(the rendered _out/assets/cozystack-operator-*.yaml release artifacts).
4+
5+
Helm install/upgrade callers must NOT render this — it conflicts with
6+
`helm --create-namespace`, which is the chicken-and-egg this PR fixes.
7+
Helm callers leave `bareNamespace` at its default (false) and let the
8+
pre-install labeler hook (cozy-system-labels.yaml) stamp the namespace
9+
after `--create-namespace` has created it bare.
10+
11+
Top-level Makefile passes `--set bareNamespace=true` together with
12+
`--show-only` for the manifest render only.
13+
*/}}
14+
{{- if .Values.bareNamespace }}
15+
---
16+
apiVersion: v1
17+
kind: Namespace
18+
metadata:
19+
name: cozy-system
20+
labels:
21+
cozystack.io/system: "true"
22+
pod-security.kubernetes.io/enforce: privileged
23+
annotations:
24+
helm.sh/resource-policy: keep
25+
{{- end }}

‎packages/core/installer/values.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
# Render a bare `Namespace cozy-system` resource (with PSA + identity labels).
2+
# Off by default: helm install/upgrade callers create the namespace via
3+
# `--create-namespace` and let the pre-install labeler hook stamp the labels.
4+
# The top-level Makefile turns this on for the kubectl-apply release artifacts
5+
# (_out/assets/cozystack-operator-*.yaml).
6+
bareNamespace: false
7+
18
cozystackOperator:
29
# Deployment variant: talos, generic, hosted
310
variant: talos

0 commit comments

Comments
 (0)