Skip to content

Commit b5309e1

Browse files
fix(kubernetes): reject a cross-StorageClass builtin worker clone
CDI takes the storage-layer CSI-clone path only when source and target share a StorageClass. Across classes it silently falls back to a host-assisted copy over the pod network — precisely the per-worker network transfer image.builtin exists to eliminate (#3231) — so a node group whose storageClass diverged from the golden's got the old failure mode back with no signal that anything had changed. Fail the render when both StorageClasses are known and differ. When either side is empty the cluster default applies, which is not resolvable at render time, so the guard skips rather than guessing a mismatch. The defaults on both sides are "replicated", so the common path is unaffected. This rules out the guaranteed-host-assisted case, not every one: a matching StorageClass still leaves the clone strategy to the CDI StorageProfile, which the worker DataVolume inherits. Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 0b00b55 commit b5309e1

3 files changed

Lines changed: 106 additions & 10 deletions

File tree

‎packages/apps/kubernetes/templates/cluster.yaml‎

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -132,9 +132,10 @@ spec:
132132
{{- end }}
133133
{{- $httpURL := printf "%s/image/%s/%s/openstack-amd64.raw.xz" ($reqFactoryURL | trimSuffix "/") $reqSchematicID $reqVersion }}
134134
{{- $goldenName := printf "talos-worker-%s-%s" $reqSchematicID $reqVersion }}
135-
{{- /* A builtin group whose golden is absent or not yet populated fails loudly
136-
here rather than degrading silently. And because a CDI clone target must
137-
be at least as large as its source, a worker diskSize below the golden's
135+
{{- /* A builtin group whose golden is absent, not yet populated, or on a
136+
StorageClass the worker disk cannot CSI-clone from fails loudly here
137+
rather than degrading silently. And because a CDI clone target must be
138+
at least as large as its source, a worker diskSize below the golden's
138139
storage would fail the clone at runtime — reject it at render when the
139140
golden's size is known (lookup is empty under `helm template`, so all
140141
of these guards are no-ops there). */}}
@@ -157,6 +158,20 @@ spec:
157158
{{- if ne $goldenPhase "Succeeded" }}
158159
{{- fail (printf "nodeGroup %q: golden DataVolume %q in cozy-public is not ready to clone (phase %s, want Succeeded) — wait for the kubernetes-worker-image import to finish; if it never does, check the importer pod in cozy-public." .groupName $goldenName $goldenPhase) }}
159160
{{- end }}
161+
{{- /* CDI takes the storage-layer CSI-clone path only when source and target
162+
share a StorageClass; across classes it silently falls back to a
163+
host-assisted copy over the pod network — exactly the per-worker
164+
network transfer image.builtin exists to avoid (#3231). Reject the
165+
case we can prove. Either side empty means "cluster default", which is
166+
not resolvable at render time, so skip rather than guess. This rules
167+
out the guaranteed-host-assisted case, not every one: a matching
168+
StorageClass still leaves the strategy to the CDI StorageProfile,
169+
which the worker DataVolume inherits. */}}
170+
{{- $workerSC := .group.storageClass | default $.Values.storageClass | toString }}
171+
{{- $goldenSC := dig "spec" "storage" "storageClassName" "" $goldenDV | toString }}
172+
{{- if and $workerSC $goldenSC (ne $workerSC $goldenSC) }}
173+
{{- fail (printf "nodeGroup %q: storageClass %q differs from golden image %q storageClass %q — CDI cannot CSI-clone across StorageClasses and would fall back to a host-assisted copy over the pod network. Set the kubernetes-worker-image catalog entry's storageClass to %q, or point this group at %q, or switch this group to image.factory." .groupName $workerSC $goldenName $goldenSC $workerSC $goldenSC) }}
174+
{{- end }}
160175
{{- $goldenStorage := dig "spec" "storage" "resources" "requests" "storage" "" $goldenDV }}
161176
{{- if $goldenStorage }}
162177
{{- $workerDisk := .group.diskSize | default "20Gi" | toString }}

‎packages/apps/kubernetes/tests/worker_image_source_test.yaml‎

Lines changed: 83 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,9 @@ kubernetesProvider:
4040
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
4141
metadata: {name: ignored-kmt, namespace: tenant-test}
4242
# The golden for the default talos.* (present and populated, so builtin clones
43-
# succeed). storage 6Gi mirrors the catalog default, so the diskSize>=golden
44-
# guard has a size to compare against; phase Succeeded satisfies the readiness
45-
# gate.
43+
# succeed). storage 6Gi and storageClassName replicated mirror the catalog
44+
# defaults, giving the diskSize>=golden and same-StorageClass guards real values
45+
# to compare against; phase Succeeded satisfies the readiness gate.
4646
- kind: DataVolume
4747
apiVersion: cdi.kubevirt.io/v1beta1
4848
metadata:
@@ -53,6 +53,7 @@ kubernetesProvider:
5353
resources:
5454
requests:
5555
storage: 6Gi
56+
storageClassName: replicated
5657
status:
5758
phase: Succeeded
5859
# A golden mid-import, for the readiness gate.
@@ -66,8 +67,23 @@ kubernetesProvider:
6667
resources:
6768
requests:
6869
storage: 6Gi
70+
storageClassName: replicated
6971
status:
7072
phase: ImportInProgress
73+
# A populated golden with no storageClassName (cluster default), for the
74+
# skip-when-unresolvable arm of the StorageClass guard.
75+
- kind: DataVolume
76+
apiVersion: cdi.kubevirt.io/v1beta1
77+
metadata:
78+
name: talos-worker-nosc-v1.0.0
79+
namespace: cozy-public
80+
spec:
81+
storage:
82+
resources:
83+
requests:
84+
storage: 6Gi
85+
status:
86+
phase: Succeeded
7187

7288
tests:
7389
- it: image omitted imports over HTTP from the cluster talos.* factory (back-compat)
@@ -257,6 +273,70 @@ tests:
257273
- failedTemplate:
258274
errorPattern: 'golden DataVolume "talos-worker-importing-v1\.0\.0" in cozy-public is not ready to clone \(phase ImportInProgress, want Succeeded\)'
259275

276+
# Cross-StorageClass clone: CDI would silently fall back to a host-assisted copy
277+
# over the pod network, the exact transfer image.builtin exists to avoid.
278+
- it: image.builtin fails when the group storageClass differs from the golden's
279+
set:
280+
_namespace: {etcd: true, ingress: true, host: example.com}
281+
_cluster: {cluster-domain: cozy.local}
282+
nodeGroups:
283+
md0:
284+
minReplicas: 1
285+
maxReplicas: 3
286+
instanceType: u1.medium
287+
diskSize: 20Gi
288+
roles: [ingress-nginx]
289+
storageClass: local
290+
image:
291+
builtin: {}
292+
asserts:
293+
- failedTemplate:
294+
errorPattern: 'storageClass "local" differs from golden image .* storageClass "replicated"'
295+
296+
- it: image.builtin clones when the group storageClass matches the golden's
297+
set:
298+
_namespace: {etcd: true, ingress: true, host: example.com}
299+
_cluster: {cluster-domain: cozy.local}
300+
nodeGroups:
301+
md0:
302+
minReplicas: 1
303+
maxReplicas: 3
304+
instanceType: u1.medium
305+
diskSize: 20Gi
306+
roles: [ingress-nginx]
307+
storageClass: replicated
308+
image:
309+
builtin: {}
310+
asserts:
311+
- equal:
312+
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.source.pvc.name
313+
value: talos-worker-ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515-v1.13.6
314+
documentSelector: {path: kind, value: KubevirtMachineTemplate}
315+
316+
# A golden with no storageClassName resolves to the cluster default, which the
317+
# render cannot know — the guard skips rather than guessing a mismatch.
318+
- it: image.builtin clones when the golden has no explicit storageClass
319+
set:
320+
_namespace: {etcd: true, ingress: true, host: example.com}
321+
_cluster: {cluster-domain: cozy.local}
322+
nodeGroups:
323+
md0:
324+
minReplicas: 1
325+
maxReplicas: 3
326+
instanceType: u1.medium
327+
diskSize: 20Gi
328+
roles: [ingress-nginx]
329+
storageClass: local
330+
image:
331+
builtin:
332+
schematicID: nosc
333+
version: v1.0.0
334+
asserts:
335+
- equal:
336+
path: spec.template.spec.virtualMachineTemplate.spec.dataVolumeTemplates[0].spec.source.pvc.name
337+
value: talos-worker-nosc-v1.0.0
338+
documentSelector: {path: kind, value: KubevirtMachineTemplate}
339+
260340
# Regression guard for the no-worker-roll invariant: an image-omitted group must
261341
# render a KubevirtMachineTemplate whose content-hash name is stable. If a future
262342
# change to the image-selection block alters the rendered KMT, this pinned hash

‎packages/system/kubernetes-worker-image/values.yaml‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,16 @@
2020
## a `Kubernetes` resource; the app then clones the golden named deterministically
2121
## `talos-worker-<schematicID>-<version>` from cozy-public. The tenant render fails
2222
## rather than degrade silently when the referenced (schematicID, version) has no
23-
## matching entry in this catalog, or when its golden has not finished importing
24-
## (phase != Succeeded) — so keep the catalog in sync with the (schematicID,
25-
## version) pairs that tenant node groups reference. Node groups that do not set
23+
## matching entry in this catalog, when its golden has not finished importing
24+
## (phase != Succeeded), or when the node group's storageClass differs from the
25+
## golden's — so keep the catalog in sync with the (schematicID, version) pairs and
26+
## StorageClasses that tenant node groups reference. Node groups that do not set
2627
## `image.builtin` import over HTTP and never touch this catalog.
2728

2829
## @param {string} imageFactoryURL - Default base URL of the Talos Image Factory serving the raw worker OS image (`<url>/image/<schematicID>/<version>/openstack-amd64.raw.xz`). Overridable per image. No trailing slash.
2930
imageFactoryURL: "https://factory.talos.dev"
3031

31-
## @param {string} storageClass - Default StorageClass for the golden image(s). Must use the same CSI driver as the tenant worker disks so csi-clone works, and be snapshot-capable (linstor ships the default VolumeSnapshotClass). Overridable per image.
32+
## @param {string} storageClass - Default StorageClass for the golden image(s). Must match the StorageClass of the worker disks that clone it: CDI cannot CSI-clone across StorageClasses and would silently fall back to a host-assisted copy over the pod network, so the tenant render rejects a mismatch outright. Must also be snapshot-capable (linstor ships the default VolumeSnapshotClass). Overridable per image.
3233
storageClass: "replicated"
3334

3435
## @param {quantity} storage - Default storage size allocated for each golden image. Must be >= the imported raw image virtual size and <= the smallest worker diskSize that clones it. The Talos v1.13.x openstack raw is ~4.15 GiB, so 6Gi leaves headroom; bump per image if a future Talos version's raw grows. Overridable per image.

0 commit comments

Comments
 (0)