Skip to content

Commit f4aef73

Browse files
committed
fix(linstor-scheduler): re-vendor to upstream 0.3.1 with native admission webhook
Re-vendor the linstor-scheduler chart from upstream 0.3.1 (appVersion v0.3.6), which carries the admission webhook binary that stopped stripping unknown Pod fields — the image VolumeSource and native-sidecar restartPolicy, which broke KubeVirt containerDisk VMs and native sidecars cluster-wide — and now reloads its TLS certificate on rotation instead of serving a stale one until restarted. The admission webhook ships in the upstream chart, so the previous downstream copy is dropped: `make update` pulls it directly and applies only cozystack-scheduler-deployment.patch (pod component label and a distro-suffix-stripped image tag), keeping the vendoring deterministic. Because the webhook now reloads its own certificate, the external cert-reload workaround is dropped. The upstream admission Deployment keeps the same name but changes its immutable pod selector, so an in-place upgrade of an existing install would fail. A migration deletes the old admission Deployment before the upgrade so the new one is recreated cleanly, and bumps the migration target version. Add a helm-unittest suite covering the admission resources, PDB selector isolation, least-privilege RBAC, the scheduler image tag, and backward compatibility. Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 249f7b5 commit f4aef73

23 files changed

Lines changed: 753 additions & 171 deletions
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/bin/sh
2+
# Migration 52 --> 53
3+
# Delete the old linstor-scheduler admission webhook Deployment so the upgrade
4+
# can recreate it.
5+
#
6+
# Cozystack used to ship the linstor-scheduler admission webhook as loose
7+
# downstream templates whose Deployment selected pods by the base scheduler
8+
# selectorLabels plus app.kubernetes.io/component: admission. Upstream chart
9+
# 0.3.0 now ships the admission webhook natively; its Deployment keeps the same
10+
# name (linstor-scheduler-admission) but selects on a distinct
11+
# app.kubernetes.io/name: linstor-scheduler-admission. A Deployment's
12+
# spec.selector is immutable, so an in-place upgrade of an existing install
13+
# would be rejected. Deleting the old Deployment before the component reconciles
14+
# lets the new one be created cleanly (its ReplicaSet and pods follow); the
15+
# webhook's failurePolicy is Ignore, so the brief gap does not block Pod
16+
# creation. Safe when absent: fresh installs and clusters without the linstor
17+
# stack simply skip it via --ignore-not-found.
18+
19+
set -euo pipefail
20+
21+
# shellcheck source=lib/cozystack-version.sh
22+
. "$(dirname "$0")/lib/cozystack-version.sh"
23+
24+
kubectl delete deployment linstor-scheduler-admission --namespace cozy-linstor --ignore-not-found
25+
26+
# Stamp version via the shared helper so the cozystack-version ConfigMap keeps
27+
# the platform.cozystack.io/no-delete label.
28+
stamp_cozystack_version 53

‎packages/core/platform/values.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ migrations:
1414
# refuses to advance past a migration file missing from the image (exit 1), so
1515
# a stale pin fails loudly rather than silently skipping the etcd adoption.
1616
image: ghcr.io/cozystack/cozystack/platform-migrations:v1.5.0@sha256:8bf61f17e99eb4a3365394e3d97ac52a1d68cc84aa104894ecb09de25721dc1d
17-
targetVersion: 52
17+
targetVersion: 53
1818
# Bundle deployment configuration
1919
bundles:
2020
system:

‎packages/system/linstor-scheduler/Makefile‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ update:
1010
rm -rf charts
1111
helm repo add piraeus-charts https://piraeus.io/helm-charts/
1212
helm repo update piraeus-charts
13-
helm pull piraeus-charts/linstor-scheduler --untar --untardir charts
14-
patch --no-backup-if-mismatch -p4 < patches/disable-ca-key-rotation.patch
15-
patch --no-backup-if-mismatch -p4 < patches/add-scheduler-component-label.patch
13+
helm pull piraeus-charts/linstor-scheduler --untar --untardir charts --version 0.3.1
14+
# Drop the vendored upstream helm-unittest suite: it asserts upstream
15+
# defaults and fails under the cozystack umbrella values. The cozystack-side
16+
# tests/ are the source of truth (same policy as packages/system/coredns).
17+
rm -rf charts/linstor-scheduler/tests
18+
patch --no-backup-if-mismatch -p4 < patches/cozystack-scheduler-deployment.patch

‎packages/system/linstor-scheduler/charts/linstor-scheduler/Chart.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: v2
2-
appVersion: v0.3.2
2+
appVersion: v0.3.6
33
deprecated: true
44
description: 'Deploys a new kubernetes scheduler, configured to take advantage of
55
storage system information. If a Pod is using a LINSTOR volume, the scheduler will
@@ -16,4 +16,4 @@ name: linstor-scheduler
1616
sources:
1717
- https://github.com/piraeusdatastore/linstor-scheduler-extender
1818
type: application
19-
version: 0.2.3
19+
version: 0.3.1

‎packages/system/linstor-scheduler/charts/linstor-scheduler/README.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,4 +69,29 @@ The following options are available:
6969
| `serviceAccount.create` | Create the service account resource | `true` |
7070
| `serviceAccount.name` | Sets the name of the service account. If left empty, will use the release name as default | `""` |
7171
| `podDisruptionBudget.enabled` | Enable creation of a pod disruption budget to protect the availability of the scheduler | `true` |
72-
| `autoscaling.enabled` | Enable creation of a horizontal pod autoscaler to ensure availability in case of high usage` | `"false` |
72+
| `autoscaling.enabled` | Enable creation of a horizontal pod autoscaler to ensure availability in case of high usage | `false` |
73+
| `admission.enabled` | Deploy the admission webhook that auto-sets `schedulerName` on Pods using LINSTOR-backed PVCs. | `false` |
74+
| `admission.replicaCount` | Number of webhook replicas to deploy. | `2` |
75+
| `admission.driver` | CSI driver name treated as LINSTOR-backed. Pods whose PVCs resolve to it get the mutation. | `linstor.csi.linbit.com` |
76+
| `admission.failurePolicy` | Webhook `failurePolicy`. `Ignore` keeps Pod creation working if the webhook is unavailable. | `Ignore` |
77+
| `admission.timeoutSeconds` | Webhook call timeout. Kept under the Kubernetes default of 10s to limit Pod-creation latency. | `5` |
78+
| `admission.namespaceSelector` | Namespace selector for the webhook. Empty selects all namespaces (including `kube-system`). | `{}` |
79+
| `admission.resources` | Resources to request and limit on the webhook container. | `{}` |
80+
| `admission.podSecurityContext` | Pod-level security context for the webhook pod. | `{runAsNonRoot: true, runAsUser: 1000, runAsGroup: 1000, fsGroup: 1000}` |
81+
| `admission.securityContext` | Container-level security context for the webhook container. | `{allowPrivilegeEscalation: false, capabilities: {drop: [ALL]}, readOnlyRootFilesystem: true}` |
82+
| `admission.createTLS` | How to provision the webhook TLS: `cert-manager`, `helm`, or `""` (bring your own). See note below. | `cert-manager` |
83+
| `admission.tls.secretName` | Name of the TLS secret. Empty defaults to `<fullname>-admission-tls`. | `""` |
84+
| `admission.tls.caBundle` | CA bundle (base64 PEM) injected into the webhook. Only used when `createTLS` is `""`. | `""` |
85+
| `admission.tls.certManager.rotationPolicy` | `privateKey.rotationPolicy` of the cert-manager CA. `Never` keeps the CA key (and caBundle) stable. | `Never` |
86+
| `admission.affinity` | Affinity for the webhook pods. Empty applies a soft anti-affinity spreading replicas across nodes. | `{}` |
87+
| `admission.reloadOnCertChange` | Add Stakater Reloader annotations so the pods restart when the TLS secret changes (see note below). | `false` |
88+
89+
## Admission webhook (optional)
90+
91+
As noted above, the LINSTOR Scheduler itself is no longer maintained and `volumeBindingMode: WaitForFirstConsumer` is the preferred approach for new deployments. For clusters that still run this scheduler, the optional admission webhook (`admission.enabled`, disabled by default) removes the need to set `schedulerName` on every Pod: it watches Pod creation and, when a Pod uses a PersistentVolumeClaim backed by the LINSTOR CSI driver, sets `schedulerName` to this scheduler automatically. It changes nothing when disabled.
92+
93+
> **Note:** `admission.createTLS: helm` regenerates the CA and certificate on every `helm upgrade`; it suits one-shot installs, not GitOps or continuously-reconciled setups (Flux/Argo) where the certificate would churn on each reconcile. Use `cert-manager` or bring your own secret (`""`) there.
94+
95+
> **Note:** the default `failurePolicy: Ignore` fails open, so an unreachable webhook never blocks Pod creation (affected Pods just fall back to the default scheduler). Setting `failurePolicy: Fail` makes the webhook mandatory for every Pod `CREATE` in the selected namespaces; with `createTLS: cert-manager` this also requires the cert-manager CA injector to be running so the `caBundle` is populated, otherwise all matching Pod creation is blocked until injection completes. Because the empty default `namespaceSelector` also intercepts the webhook's own namespace and `kube-system`, running with `failurePolicy: Fail` can deadlock a restart of the webhook (its own new Pods cannot be admitted while all replicas are down) — exclude the release namespace and `kube-system` via `admission.namespaceSelector` if you switch to `Fail`.
96+
97+
> **Note:** the webhook binary loads its serving certificate once at startup and does not hot-reload it. When the certificate is renewed — cert-manager rotates the leaf roughly yearly, or you rotate an external secret — the running Pods keep serving the old certificate until they restart, after which the webhook silently stops mutating. Set `admission.reloadOnCertChange: true` (requires the Stakater Reloader controller) to roll the Deployment automatically when the secret changes, or restart the webhook Pods manually after each renewal. In `createTLS: helm` mode a `helm upgrade` regenerates the certificate and already rolls the Pods.

‎packages/system/linstor-scheduler/charts/linstor-scheduler/templates/NOTES.txt‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,28 @@ Scheduler {{ include "linstor-scheduler.fullname" . }} deployed!
33
Used LINSTOR URL: {{ include "linstor-scheduler.linstorEndpoint" .}}
44

55
Please run `helm test {{ .Release.Name }}` to ensure it's properly working.
6+
{{ if .Values.admission.enabled -}}
7+
The optional admission webhook is enabled: Pods using LINSTOR-backed PersistentVolumeClaims
8+
are automatically scheduled by {{ include "linstor-scheduler.fullname" . }}, so you
9+
do not need to set schedulerName by hand. To schedule a Pod that does not use a
10+
LINSTOR volume, set its schedulerName explicitly:
611

12+
---
13+
spec:
14+
schedulerName: {{ include "linstor-scheduler.fullname" . }}
15+
---
16+
{{- if eq .Values.admission.createTLS "" }}
17+
{{ if not .Values.admission.tls.caBundle }}
18+
NOTE: admission.createTLS is "" and admission.tls.caBundle is empty. Unless an
19+
external controller injects the webhook's caBundle, the API server cannot reach
20+
the webhook and Pods will be scheduled WITHOUT the mutation (failurePolicy: {{ .Values.admission.failurePolicy }}).
21+
{{- end }}
22+
{{- end }}
23+
{{- else -}}
724
Specify the scheduler on your pods to start smart scheduling based on your Persistent Volumes:
825

926
---
1027
spec:
1128
schedulerName: {{ include "linstor-scheduler.fullname" . }}
1229
---
30+
{{- end }}

‎packages/system/linstor-scheduler/charts/linstor-scheduler/templates/_helpers.tpl‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,65 @@ Create the name of the service account to use
6161
{{- end }}
6262
{{- end }}
6363

64+
{{/*
65+
Fully qualified name for the admission webhook resources. The base fullname is
66+
truncated before the "-admission" suffix (and further sub-suffixes such as
67+
"-selfsigned") are appended, so the longest derived name still fits the 63-char
68+
Kubernetes name limit for long release names.
69+
*/}}
70+
{{- define "linstor-scheduler.admissionFullname" -}}
71+
{{- printf "%s-admission" (include "linstor-scheduler.fullname" . | trunc 42 | trimSuffix "-") }}
72+
{{- end }}
73+
74+
{{/*
75+
Selector labels for the admission webhook. The name carries an "-admission"
76+
suffix so the webhook pods are NOT matched by the base scheduler's selectors
77+
(Deployment, PodDisruptionBudget), whose selector fields are immutable and so
78+
cannot be narrowed after the fact. This keeps the base PDB and HPA scoped to the
79+
scheduler pods only. The base name is truncated BEFORE the suffix is appended so
80+
a long nameOverride can never eat the "-admission" suffix and collapse the name
81+
back onto the base selector.
82+
*/}}
83+
{{- define "linstor-scheduler.admissionSelectorLabels" -}}
84+
app.kubernetes.io/name: {{ printf "%s-admission" (include "linstor-scheduler.name" . | trunc 53 | trimSuffix "-") }}
85+
app.kubernetes.io/instance: {{ .Release.Name }}
86+
{{- end }}
87+
88+
{{/*
89+
Common labels for the admission webhook resources.
90+
*/}}
91+
{{- define "linstor-scheduler.admissionLabels" -}}
92+
helm.sh/chart: {{ include "linstor-scheduler.chart" . }}
93+
{{ include "linstor-scheduler.admissionSelectorLabels" . }}
94+
{{- if .Chart.AppVersion }}
95+
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
96+
{{- end }}
97+
app.kubernetes.io/managed-by: {{ .Release.Service }}
98+
app.kubernetes.io/component: admission
99+
{{- end }}
100+
101+
{{/*
102+
Name of the secret holding the admission webhook's TLS certificate.
103+
Defaults to "<admissionFullname>-tls" unless admission.tls.secretName is set,
104+
so users managing the secret externally can point the Deployment at their own.
105+
*/}}
106+
{{- define "linstor-scheduler.admissionTLSSecretName" -}}
107+
{{- default (printf "%s-tls" (include "linstor-scheduler.admissionFullname" .)) .Values.admission.tls.secretName }}
108+
{{- end }}
109+
110+
{{/*
111+
Name of the service account used by the admission webhook. When the chart
112+
creates service accounts it gets a dedicated one, so the webhook pod does not
113+
inherit the scheduler's system:kube-scheduler / system:volume-scheduler bindings.
114+
*/}}
115+
{{- define "linstor-scheduler.admissionServiceAccountName" -}}
116+
{{- if .Values.serviceAccount.create }}
117+
{{- include "linstor-scheduler.admissionFullname" . }}
118+
{{- else }}
119+
{{- default "default" .Values.serviceAccount.name }}
120+
{{- end }}
121+
{{- end }}
122+
64123
{{/*
65124
Get the kubernetes version we should assume for creating scheduler configs.
66125
Strips distribution suffixes like +k3s1, +rke2r1 from version string.
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
{{- if and .Values.admission.enabled (eq .Values.admission.createTLS "cert-manager") }}
2+
---
3+
apiVersion: cert-manager.io/v1
4+
kind: Issuer
5+
metadata:
6+
name: {{ include "linstor-scheduler.admissionFullname" . }}-selfsigned
7+
namespace: {{ .Release.Namespace }}
8+
labels:
9+
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
10+
spec:
11+
selfSigned: {}
12+
---
13+
apiVersion: cert-manager.io/v1
14+
kind: Certificate
15+
metadata:
16+
name: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
17+
namespace: {{ .Release.Namespace }}
18+
labels:
19+
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
20+
spec:
21+
secretName: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
22+
duration: 43800h # 5 years
23+
commonName: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
24+
privateKey:
25+
rotationPolicy: {{ .Values.admission.tls.certManager.rotationPolicy }}
26+
issuerRef:
27+
name: {{ include "linstor-scheduler.admissionFullname" . }}-selfsigned
28+
isCA: true
29+
---
30+
apiVersion: cert-manager.io/v1
31+
kind: Issuer
32+
metadata:
33+
name: {{ include "linstor-scheduler.admissionFullname" . }}-ca-issuer
34+
namespace: {{ .Release.Namespace }}
35+
labels:
36+
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
37+
spec:
38+
ca:
39+
secretName: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
40+
---
41+
apiVersion: cert-manager.io/v1
42+
kind: Certificate
43+
metadata:
44+
name: {{ include "linstor-scheduler.admissionFullname" . }}-cert
45+
namespace: {{ .Release.Namespace }}
46+
labels:
47+
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
48+
spec:
49+
secretName: {{ include "linstor-scheduler.admissionTLSSecretName" . }}
50+
duration: 8760h # 1 year
51+
renewBefore: 24h
52+
issuerRef:
53+
name: {{ include "linstor-scheduler.admissionFullname" . }}-ca-issuer
54+
commonName: {{ include "linstor-scheduler.admissionFullname" . }}
55+
dnsNames:
56+
- {{ include "linstor-scheduler.admissionFullname" . }}
57+
- {{ include "linstor-scheduler.admissionFullname" . }}.{{ .Release.Namespace }}.svc
58+
{{- end }}

0 commit comments

Comments
 (0)