Skip to content

Commit 243ce65

Browse files
committed
feat(kubernetes): expose tenant apiserver authentication in app values
Tenant Kubernetes clusters had no way to configure the kube-apiserver authentication through the app values: no knob for OIDC, the structured AuthenticationConfiguration, extra apiserver flags, or extra volumes. The only workaround was patching the underlying KamajiControlPlane out-of-band, which is invisible to GitOps and lost on recreation. Add three opt-in fields under controlPlane.apiServer, all empty by default so existing clusters render unchanged: - extraArgs -> KamajiControlPlane spec.apiServer.extraArgs - extraVolumeMounts -> spec.apiServer.extraVolumeMounts - extraVolumes -> appended to spec.deployment.extraVolumes This unblocks OIDC SSO via the --oidc-* flags and, more importantly, structured authentication: --authentication-config mounted from a ConfigMap/Secret plus --requestheader-uid-headers, which is the only way to map a user UID for aggregated/extension apiservers. Because the control-plane Deployment runs on the management cluster, the passthrough is validated to preserve tenant<->management isolation: - extraVolumes allow only configMap and secret sources; host-reaching sources (hostPath, csi, nfs, persistentVolumeClaim) and token projection (projected) are rejected, so a tenant with self-service access to the Kubernetes CR cannot mount management-node paths or project arbitrary tokens into the apiserver pod. - each extraVolume must have a unique, non-empty name and exactly one source; the reserved names talos-ca and talos-tls-cert are rejected to avoid a duplicate volume name in the rendered podspec. - each extraVolumeMount must reference a user-declared extraVolume; the chart-managed talos secret volumes cannot be mounted. Regenerate values.schema.json, README, the Go types and deepcopy, and the ApplicationDefinition. Add helm unittest coverage for the default (no-op) render, each field's passthrough, the name/source guards, and the mount-reference guard (valid and rejected inputs). Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent f5c408d commit 243ce65

8 files changed

Lines changed: 530 additions & 31 deletions

File tree

‎api/apps/v1alpha1/kubernetes/types.go‎

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/apps/v1alpha1/kubernetes/zz_generated.deepcopy.go‎

Lines changed: 19 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/apps/kubernetes/README.md‎

Lines changed: 32 additions & 29 deletions
Large diffs are not rendered by default.

‎packages/apps/kubernetes/templates/cluster.yaml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,60 @@ data:
185185
to reconcile - this HelmRelease will pick up the DataStore on its next
186186
5m reconcile loop and provision the cluster.
187187
{{- else }}
188+
{{- /* Validate user-supplied apiServer.extraVolumes / extraVolumeMounts before
189+
they are wired into the control-plane Deployment and apiserver container.
190+
191+
The KamajiControlPlane -> TenantControlPlane Deployment runs on the
192+
MANAGEMENT cluster (in the tenant namespace), so volumes declared here
193+
land on a management-cluster pod. To preserve tenant<->management
194+
isolation we deny by default and allow only the ConfigMap/Secret sources
195+
that satisfy the intended use case (mounting an AuthenticationConfiguration
196+
file). Host-reaching sources (hostPath, csi, nfs, persistentVolumeClaim)
197+
and token-projection sources (projected) are rejected — a tenant with
198+
self-service access to the Kubernetes CR must not be able to mount
199+
management-node paths or project arbitrary service-account tokens into
200+
the apiserver pod.
201+
202+
Names must be unique and non-empty: the chart injects two reserved
203+
volumes (talos-ca, talos-tls-cert), and a duplicate or empty volume name
204+
would render an invalid podspec the apiserver rejects with an opaque,
205+
apply-time error invisible to GitOps. Mounts may only reference a volume
206+
the user declared in extraVolumes — the chart-managed talos secret
207+
volumes are deliberately not mountable. */}}
208+
{{- $allowedVolumeSources := list "configMap" "secret" }}
209+
{{- $reservedVolumeNames := list "talos-ca" "talos-tls-cert" }}
210+
{{- $userVolumeNames := list }}
211+
{{- range $i, $vol := .Values.controlPlane.apiServer.extraVolumes }}
212+
{{- $volName := $vol.name | default "" | toString }}
213+
{{- if eq $volName "" }}
214+
{{- fail (printf "controlPlane.apiServer.extraVolumes[%d]: every volume must set a non-empty name" $i) }}
215+
{{- end }}
216+
{{- if has $volName $reservedVolumeNames }}
217+
{{- fail (printf "controlPlane.apiServer.extraVolumes: volume name %q is reserved by the chart for the talos control-plane secrets — rename your volume to avoid a duplicate volume name in the control-plane Deployment" $volName) }}
218+
{{- end }}
219+
{{- if has $volName $userVolumeNames }}
220+
{{- fail (printf "controlPlane.apiServer.extraVolumes: duplicate volume name %q — each extra volume must have a unique name" $volName) }}
221+
{{- end }}
222+
{{- $userVolumeNames = append $userVolumeNames $volName }}
223+
{{- $sources := without (keys $vol) "name" | sortAlpha }}
224+
{{- if ne (len $sources) 1 }}
225+
{{- fail (printf "controlPlane.apiServer.extraVolumes: volume %q must declare exactly one volume source, got %d (%v)" $volName (len $sources) $sources) }}
226+
{{- end }}
227+
{{- range $source := $sources }}
228+
{{- if not (has $source $allowedVolumeSources) }}
229+
{{- fail (printf "controlPlane.apiServer.extraVolumes: volume %q uses a %q source, which is not permitted on a management-cluster control-plane pod — only configMap and secret sources are allowed" $volName $source) }}
230+
{{- end }}
231+
{{- end }}
232+
{{- end }}
233+
{{- range $i, $mount := .Values.controlPlane.apiServer.extraVolumeMounts }}
234+
{{- $mountName := $mount.name | default "" | toString }}
235+
{{- if eq $mountName "" }}
236+
{{- fail (printf "controlPlane.apiServer.extraVolumeMounts[%d]: every mount must set a non-empty name" $i) }}
237+
{{- end }}
238+
{{- if not (has $mountName $userVolumeNames) }}
239+
{{- fail (printf "controlPlane.apiServer.extraVolumeMounts: mount %q references no volume declared in controlPlane.apiServer.extraVolumes — the chart-managed talos volumes cannot be mounted" $mountName) }}
240+
{{- end }}
241+
{{- end }}
188242
---
189243
apiVersion: cluster.x-k8s.io/v1beta1
190244
kind: Cluster
@@ -222,6 +276,14 @@ metadata:
222276
spec:
223277
apiServer:
224278
resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.controlPlane.apiServer.resourcesPreset .Values.controlPlane.apiServer.resources $) | nindent 6 }}
279+
{{- with .Values.controlPlane.apiServer.extraArgs }}
280+
extraArgs:
281+
{{- toYaml . | nindent 6 }}
282+
{{- end }}
283+
{{- with .Values.controlPlane.apiServer.extraVolumeMounts }}
284+
extraVolumeMounts:
285+
{{- toYaml . | nindent 6 }}
286+
{{- end }}
225287
controllerManager:
226288
resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.controlPlane.controllerManager.resourcesPreset .Values.controlPlane.controllerManager.resources $) | nindent 6 }}
227289
scheduler:
@@ -315,6 +377,9 @@ spec:
315377
- name: talos-tls-cert
316378
secret:
317379
secretName: {{ .Release.Name }}-talos-tls-cert
380+
{{- with .Values.controlPlane.apiServer.extraVolumes }}
381+
{{- toYaml . | nindent 6 }}
382+
{{- end }}
318383
replicas: {{ .Values.controlPlane.replicas }}
319384
version: {{ include "kubernetes.versionMap" $ }}
320385
---

0 commit comments

Comments
 (0)