Skip to content

Commit 631580e

Browse files
committed
fix(vm-disk): fail on storageClass/source edit of existing disk
Editing a VMDisk's storageClass or source after its DataVolume already exists was silently ignored: dv.yaml reused the existing DataVolume.spec verbatim (DataVolume.spec is immutable in CDI), so the edit rendered as a no-op while the HelmRelease reported success and the CR diverged from the cluster. When a DataVolume already exists, fail with a clear message if the requested storageClass or source differs from what the disk was created with. storageClass is read from the bound PVC (authoritative, and present even when the DataVolume omits storageClassName), falling back to the DataVolume spec, and skipped only when neither records a class. source is compared against the operator input recorded in a vm-disk.cozystack.io/source annotation at creation, not the rendered spec.source: the image-clone PVC name carries a chart-version prefix (vm-image- was renamed to vm-default-images-, commit bec35e3) that drifts across upgrades without any edit, so comparing the rendered form would falsely reject long-lived image disks. A disk created before the annotation existed is compared against its stored spec.source directly for every source type except image (all of which reproduce the stored spec exactly); image is the one documented exemption. The annotation is written only at creation and preserved verbatim afterwards, never overwritten with an unapplied edit. An unrecognized source type now fails explicitly instead of rendering an empty source. Storage growth is unaffected and still handled by pvc-resize-hook.yaml. Mark source as immutable in values.yaml so the generated schema carries the same x-kubernetes-validations rule already present on storageClass, and document the vm-disk source contract in docs/storage-immutability.md and the chart README. Aggregated-apiserver CEL enforcement is tracked separately in #2657; the chart-level fail is the substantive guard today. Add helm-unittest coverage for the immutability guard (rejection, no-op reuse with annotation preserved, legacy image-disk drift, legacy non-image fallback, annotation non-poisoning, PVC-backed storageClass on an adopted disk, unknown source) and initial-creation source rendering. Fixes #2985 Signed-off-by: Alexey Artamonov <[email protected]>
1 parent eef95f4 commit 631580e

9 files changed

Lines changed: 535 additions & 26 deletions

File tree

‎api/apps/v1alpha1/vmdisk/types.go‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/storage-immutability.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,12 @@ The following charts annotate at least one `storageClass` field as immutable:
2222

2323
The `kubernetes-nodes` chart annotates both its per-pool `storageClass` (the worker node system-disk class, defaulted `replicated`) and its `cluster` field immutable. `storageClass` binds the same PVC-template contract as every other chart in the table, so the plain `self == oldSelf` rule applies for the reason in "Why" above. `cluster` is immutable for a different reason: it wires the pool's CAPI objects to one parent cluster `kubernetes-<cluster>`, and repointing a live pool at another cluster would orphan its running worker VMs rather than migrate them.
2424

25+
## `vm-disk` also freezes `source`
26+
27+
The `vm-disk` chart annotates `source` immutable in addition to `storageClass`, for a mechanism distinct from the PVC-template reasoning above. A `vm-disk` renders a single `DataVolume`, and `DataVolume.spec` is immutable in CDI: once the DataVolume exists, editing `spec.source` (or `spec.storage.storageClassName`) has no effect. The chart therefore looks the existing DataVolume up and reuses its stored `spec`, so an edit to `.Values.source` would otherwise render as a silent no-op while the HelmRelease still reported success — the CR and the cluster diverge with no diagnostic (cozystack/cozystack#2985).
28+
29+
Because the aggregated apiserver does not yet evaluate the CEL rule (see the table above, cozystack/cozystack#2657), `vm-disk` adds a second, chart-level guard that is the substantive enforcement today: `templates/dv.yaml` fails the render when the requested `storageClass` or `source` differs from the existing disk. The `source` comparison keys off the operator's input recorded in a `vm-disk.cozystack.io/source` annotation at creation, not the rendered `spec.source` — the image-clone PVC target name carries a chart-version prefix (`vm-image-` was renamed to `vm-default-images-`) that drifts across upgrades without any operator edit, and comparing the rendered form would falsely reject long-lived image disks. A disk created before that annotation existed is instead compared against its stored `spec.source` directly for every source type except image clones: `http`, `upload`, `disk` and `blank` all reproduce the stored spec exactly, so an edit to one of them is still caught, while the image case is skipped rather than fired on drift. `storageClass` is compared against the bound PVC's class (falling back to the DataVolume spec) and is not subject to that exemption. To actually change `storageClass` or `source`, delete and recreate the disk; size growth is the one mutable dimension and is handled separately by `templates/pvc-resize-hook.yaml`.
30+
2531
If you add a new stateful chart that exposes `storageClass`, annotate it the same way:
2632

2733
```yaml

‎packages/apps/vm-disk/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
A Virtual Machine Disk
44

5-
> `storageClass` is annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it.
5+
> `storageClass` and `source` are annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it. `DataVolume.spec` is immutable in CDI, so the chart also fails the release on an attempted `storageClass`/`source` edit of an existing disk rather than silently reusing the old spec; delete and recreate the disk to change either field.
66
77
## Parameters
88

‎packages/apps/vm-disk/templates/dv.yaml‎

Lines changed: 77 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,97 @@
11
{{- $existingDV := lookup "cdi.kubevirt.io/v1beta1" "DataVolume" .Release.Namespace .Release.Name }}
2+
{{- $existingPVC := lookup "v1" "PersistentVolumeClaim" .Release.Namespace .Release.Name }}
3+
{{- $sourceAnnotation := "vm-disk.cozystack.io/source" }}
4+
{{- $requestedSource := toJson (.Values.source | default dict) }}
5+
{{- /* Build the desired source spec from .Values so it can be both rendered and compared. */}}
6+
{{- $desiredSource := dict }}
7+
{{- if .Values.source }}
8+
{{- if gt (len .Values.source) 1 }}
9+
{{- fail "Exactly one type of source is expected!" }}
10+
{{- end }}
11+
{{- if hasKey .Values.source "image" }}
12+
{{- $desiredSource = dict "pvc" (dict "name" (printf "vm-default-images-%s" (required "A valid .Values.source.image.name entry required!" .Values.source.image.name)) "namespace" "cozy-public") }}
13+
{{- else if hasKey .Values.source "http" }}
14+
{{- $desiredSource = dict "http" (dict "url" (required "A valid .Values.source.http.url entry required!" .Values.source.http.url)) }}
15+
{{- else if hasKey .Values.source "upload" }}
16+
{{- $desiredSource = dict "upload" dict }}
17+
{{- else if hasKey .Values.source "disk" }}
18+
{{- $desiredSource = dict "pvc" (dict "name" (printf "vm-disk-%s" (required "A valid .Values.source.disk.name entry required!" .Values.source.disk.name)) "namespace" .Release.Namespace) }}
19+
{{- else }}
20+
{{- fail (printf "Unknown source type: %s. Expected one of: image, http, upload, disk." (keys .Values.source | join ", ")) }}
21+
{{- end }}
22+
{{- else }}
23+
{{- $desiredSource = dict "blank" dict }}
24+
{{- end }}
25+
{{- /*
26+
DataVolume.spec is immutable in CDI, so editing storageClass or source on an
27+
existing disk cannot take effect. Reusing the stored spec silently would let
28+
the CR diverge from the cluster while the release reports success (#2985);
29+
fail loudly instead.
30+
31+
storageClass: the effective class is read from the bound PVC (authoritative,
32+
and present even for a DataVolume created via spec.pvc or without an explicit
33+
storageClassName), falling back to the DataVolume spec. When neither records a
34+
class the check is skipped rather than misreading it as "<nil>".
35+
36+
source: the operator's input is recorded in an annotation at creation and
37+
compared back on later renders. Comparing the rendered spec.source instead
38+
would be wrong for image clones — the PVC name carries a chart-version prefix
39+
(vm-image- -> vm-default-images-, commit bec35e3a) that drifts across upgrades
40+
with no operator edit. A disk that predates the annotation is matched against
41+
its stored spec.source directly for every source type except image (all of
42+
which reproduce the stored spec exactly); the image case is the one
43+
documented exemption. The annotation is written only at creation and preserved
44+
verbatim afterwards, never overwritten with an unapplied edit.
45+
46+
Size growth is intentionally excluded and handled by pvc-resize-hook.yaml.
47+
*/}}
48+
{{- if $existingDV }}
49+
{{- $existingSC := "" }}
50+
{{- if $existingPVC }}
51+
{{- $existingSC = ($existingPVC.spec.storageClassName | default "" | toString) }}
52+
{{- end }}
53+
{{- if not $existingSC }}
54+
{{- $existingSC = ((($existingDV.spec.storage | default dict).storageClassName) | default "" | toString) }}
55+
{{- end }}
56+
{{- if and .Values.storageClass $existingSC (ne $existingSC (.Values.storageClass | toString)) }}
57+
{{- fail (printf "VMDisk storageClass is immutable after creation (current: %q, requested: %q). Delete and recreate the disk to change it." $existingSC .Values.storageClass) }}
58+
{{- end }}
59+
{{- $recordedSource := (index ($existingDV.metadata.annotations | default dict) $sourceAnnotation) | default "" }}
60+
{{- if $recordedSource }}
61+
{{- if ne $recordedSource $requestedSource }}
62+
{{- fail (printf "VMDisk source is immutable after creation (current: %s, requested: %s). Delete and recreate the disk to change it." $recordedSource $requestedSource) }}
63+
{{- end }}
64+
{{- else if not (hasKey .Values.source "image") }}
65+
{{- if ne (toYaml $existingDV.spec.source) (toYaml $desiredSource) }}
66+
{{- fail (printf "VMDisk source is immutable after creation (current: %s, requested: %s). Delete and recreate the disk to change it." (toJson $existingDV.spec.source) $requestedSource) }}
67+
{{- end }}
68+
{{- end }}
69+
{{- end }}
270
apiVersion: cdi.kubevirt.io/v1beta1
371
kind: DataVolume
472
metadata:
573
annotations:
674
{{- if hasKey .Values.source "upload" }}
775
cdi.kubevirt.io/storage.bind.immediate.requested: ""
876
{{- end }}
77+
{{- if $existingDV }}
78+
{{- with (index ($existingDV.metadata.annotations | default dict) $sourceAnnotation) }}
79+
{{ $sourceAnnotation }}: {{ . | quote }}
80+
{{- end }}
81+
{{- else }}
82+
{{ $sourceAnnotation }}: {{ $requestedSource | quote }}
83+
{{- end }}
984
vm-disk.cozystack.io/optical: "{{ .Values.optical }}"
1085
labels:
1186
app.kubernetes.io/instance: {{ .Release.Name }}
1287
name: {{ .Release.Name }}
1388
spec:
1489
{{- if $existingDV }}
1590
{{- toYaml $existingDV.spec | nindent 2 }}
16-
{{- else }}
17-
contentType: kubevirt
18-
{{- if .Values.source }}
19-
{{- if gt (len .Values.source) 1 }}
20-
{{- fail "Exactly one type of source is expected!" }}
21-
{{- end }}
22-
source:
23-
{{- if hasKey .Values.source "image" }}
24-
pvc:
25-
name: vm-default-images-{{ required "A valid .Values.source.image.name entry required!" .Values.source.image.name }}
26-
namespace: cozy-public
27-
{{- else if hasKey .Values.source "http" }}
28-
http:
29-
url: {{ required "A valid .Values.source.http.url entry required!" .Values.source.http.url }}
30-
{{- else if hasKey .Values.source "upload" }}
31-
upload: {}
32-
{{- else if hasKey .Values.source "disk" }}
33-
pvc:
34-
name: vm-disk-{{ required "A valid .Values.source.disk.name entry required!" .Values.source.disk.name }}
35-
namespace: {{ $.Release.Namespace }}
36-
{{- end }}
3791
{{- else }}
92+
contentType: kubevirt
3893
source:
39-
blank: {}
40-
{{- end }}
94+
{{- toYaml $desiredSource | nindent 4 }}
4195
storage:
4296
resources:
4397
requests:

0 commit comments

Comments
 (0)