Skip to content

Commit 3a70f91

Browse files
authored
fix(vm-disk): fail on storageClass/source edit of existing disk (#3969)
## What this PR does Fixes #2985. Editing a `VMDisk`'s `storageClass` or `source` after its DataVolume already exists was silently ignored: `packages/apps/vm-disk/templates/dv.yaml` looks up the existing DataVolume and, when found, reused its `spec` verbatim (because `DataVolume.spec` is immutable in CDI). The edit therefore rendered as a no-op — the typed CR accepted it and the HelmRelease reported success, while the DataVolume/PVC kept the original spec and the source of truth silently diverged from the cluster. The template now builds the desired `source` into a shared `$desiredSource` variable (used both for rendering and comparison) and, when a DataVolume already exists, calls `fail()` with an explicit message if the requested `storageClass` or `source` differs from the immutable existing spec — mirroring the explicit size handling already done in `pvc-resize-hook.yaml`. Storage growth is unaffected and still flows through the resize hook. `source` is also marked `@immutable` in `values.yaml`, so the generated schema now carries the same `x-kubernetes-validations` transition rule (`self == oldSelf`) that `storageClass` already had; `types.go`, `values.schema.json` and the generated `vm-disk-rd` CRD are regenerated to match. ### Verification - `helm unittest` covers both the immutability guard (via a mocked existing DataVolume through `kubernetesProvider`, since `lookup` needs a cluster) and initial-creation source rendering for every source type — 12/12 pass. - Verified live on a dev cluster with `helm install --dry-run=server` against a real DataVolume: an unchanged edit renders, a `storageClass` change fails, a `source` change fails, and a storage grow still renders the resize hook without tripping the guard. ### Scope note This PR only makes the drift visible and marks `source` immutable. Two deeper platform findings sit outside its scope: (1) #2657 (pre-existing) — the `apps.cozystack.io` aggregated apiserver does not evaluate the `x-kubernetes-validations` CEL rules on update, so `@immutable` is a no-op at admission for every Application kind, which is exactly why the chart-level `fail()` — not the CEL rule — is the substantive guard here; (2) #3971 — that same apiserver does not honour `--dry-run=server` and persists the change (surfaced while testing this fix). ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff file by file. The diff touches only `packages/apps/vm-disk/` (template, `values.yaml`, regenerated `values.schema.json`, tests), the regenerated `packages/system/vm-disk-rd/cozyrds/vm-disk.yaml`, and `api/apps/v1alpha1/vmdisk/types.go`. No package was added, renamed or removed; no `values.schema.json` field was added, renamed or removed (only a validation rule on the existing `source` field); no default changed; `README.md` is unchanged; the `cozyrds` file was regenerated, not renamed. The terraform provider models fields, defaults and enums but not CEL immutability, so it is unaffected. No downstream repository is reached by this change. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(vm-disk): editing storageClass or source on an existing VMDisk now fails with a clear message instead of being silently ignored; delete and recreate the disk to change these immutable fields. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for blank, HTTP, image, disk, and upload VM disk sources. * VM disk sources and storage classes are now protected from unintended changes, while storage expansion remains supported. * Existing disks preserve their configured source and storage settings. * **Bug Fixes** * Invalid or conflicting source configurations are rejected. * Prevented silent no-op updates and incorrect immutability errors for existing disks. * **Documentation** * Documented source and storage class immutability, including the need to recreate disks when changing either setting. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 86fe8e5 + 06624ba commit 3a70f91

9 files changed

Lines changed: 1151 additions & 27 deletions

File tree

‎api/apps/v1alpha1/vmdisk/types.go‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/storage-immutability.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,18 @@ The following charts annotate at least one `storageClass` field as immutable:
2222

2323
The `kubernetes-nodes` chart annotates both its per-pool `storageClass` (the worker node system-disk class, defaulted `replicated`) and its `cluster` field immutable. `storageClass` binds the same PVC-template contract as every other chart in the table, so the plain `self == oldSelf` rule applies for the reason in "Why" above. `cluster` is immutable for a different reason: it wires the pool's CAPI objects to one parent cluster `kubernetes-<cluster>`, and repointing a live pool at another cluster would orphan its running worker VMs rather than migrate them.
2424

25+
## `vm-disk` also freezes `source`
26+
27+
The `vm-disk` chart annotates `source` immutable in addition to `storageClass`, for a mechanism distinct from the PVC-template reasoning above. A `vm-disk` renders a single `DataVolume`, and `DataVolume.spec` is immutable in CDI: once the DataVolume exists, editing `spec.source` (or `spec.storage.storageClassName`) has no effect. The chart therefore looks the existing DataVolume up and reuses its stored `spec`, so an edit to `.Values.source` would otherwise render as a silent no-op while the HelmRelease still reported success — the CR and the cluster diverge with no diagnostic (cozystack/cozystack#2985).
28+
29+
Because the aggregated apiserver does not yet evaluate the CEL rule (see the table above, cozystack/cozystack#2657), `vm-disk` adds a second, chart-level guard that is the substantive enforcement today: `templates/dv.yaml` fails the render when the requested `storageClass` or `source` differs from what the disk was created with. Both checks compare the operator's input recorded in an annotation at creation — `vm-disk.cozystack.io/storage-class` and `vm-disk.cozystack.io/source` — written only at creation and preserved verbatim afterward, never overwritten with an unapplied edit. The storage-class annotation is written even for the cluster default (`""`), and its key presence — not its value — marks a recorded class, so a disk created with `""` is still caught if later changed to a concrete class. Recording the input rather than reading the live object matters on both axes: the image-clone PVC name carries a chart-version prefix (`vm-image-` was renamed to `vm-default-images-`) that drifts across upgrades with no operator edit, and the live PVC's `storageClassName` on an adopted or foreign DataVolume may legitimately differ from the CR's default without being a change.
30+
31+
Disks created before these annotations existed (or out-of-band) are handled conservatively so a working release never deadlocks. For `source`, such a disk is compared against its stored `spec.source` directly, but only when that stored source is a shape the chart itself produces (`blank`, `http` with just a `url`, `upload`, or a `pvc` with just `name`/`namespace`). An adopted DataVolume using `spec.sourceRef`, or a source type the chart cannot render (`registry`, `snapshot`, `imageio`, `http` with a `secretRef`, a foreign-namespace `pvc`), has no comparable provenance and is left alone rather than failed on a mismatch. The image-clone exemption fires only when the stored source is a `cozy-public` `pvc` **and** the request is itself an image — the one case where the `vm-image-` → `vm-default-images-` prefix drift is indistinguishable from a real image-name change; a change away from an image source to another type is still caught. For `storageClass` there is no safe live fallback (the CR default need not reflect an adopted disk's real class), so an un-annotated disk is simply not guarded here; apiserver CEL (cozystack/cozystack#2657) is the eventual guard for those. To actually change `storageClass` or `source`, delete and recreate the disk; size growth is the one mutable dimension and is handled separately by `templates/pvc-resize-hook.yaml`.
32+
33+
The `storageClass` annotation records the effective value applied at creation, including when it came from the chart's `values.yaml` default (`replicated`) rather than an explicit operator value. Applications created through the aggregated apiserver have the default stamped into the CR, so it stays stable; but a direct HelmRelease consumer that omits `storageClass` records the chart default of the day, so changing that default in a future chart version would make every such untouched release start failing the render until its CR pins the old value or the disk is recreated. Treat the `storageClass` default as a frozen contract for the same reason the `source` serialization above is.
34+
35+
The `source` annotation is a byte-for-byte `toJson` of `.Values.source`, so it is coupled to the exact serialization of the source schema. Adding a subfield with a schema default (which the apiserver would stamp onto every CR) would make the recorded and requested JSON diverge on an untouched disk; such a change needs an annotation-migration story rather than a bare schema bump.
36+
2537
If you add a new stateful chart that exposes `storageClass`, annotate it the same way:
2638

2739
```yaml

‎packages/apps/vm-disk/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
A Virtual Machine Disk
44

5-
> `storageClass` is annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it.
5+
> `storageClass` and `source` are annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it. `DataVolume.spec` is immutable in CDI, so the chart also fails the release on an attempted `storageClass`/`source` edit of an existing disk rather than silently reusing the old spec; delete and recreate the disk to change either field.
66
77
## Parameters
88

‎packages/apps/vm-disk/templates/dv.yaml‎

Lines changed: 101 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,120 @@
11
{{- $existingDV := lookup "cdi.kubevirt.io/v1beta1" "DataVolume" .Release.Namespace .Release.Name }}
2+
{{- $sourceAnnotation := "vm-disk.cozystack.io/source" }}
3+
{{- $storageClassAnnotation := "vm-disk.cozystack.io/storage-class" }}
4+
{{- $requestedSource := toJson (.Values.source | default dict) }}
5+
{{- $requestedSC := (.Values.storageClass | default "") | toString }}
6+
{{- /* Build the desired source spec from .Values so it can be both rendered and compared. */}}
7+
{{- $desiredSource := dict }}
8+
{{- if .Values.source }}
9+
{{- if gt (len .Values.source) 1 }}
10+
{{- fail "Exactly one type of source is expected!" }}
11+
{{- end }}
12+
{{- if hasKey .Values.source "image" }}
13+
{{- $desiredSource = dict "pvc" (dict "name" (printf "vm-default-images-%s" (required "A valid .Values.source.image.name entry required!" .Values.source.image.name)) "namespace" "cozy-public") }}
14+
{{- else if hasKey .Values.source "http" }}
15+
{{- $desiredSource = dict "http" (dict "url" (required "A valid .Values.source.http.url entry required!" .Values.source.http.url)) }}
16+
{{- else if hasKey .Values.source "upload" }}
17+
{{- $desiredSource = dict "upload" dict }}
18+
{{- else if hasKey .Values.source "disk" }}
19+
{{- $desiredSource = dict "pvc" (dict "name" (printf "vm-disk-%s" (required "A valid .Values.source.disk.name entry required!" .Values.source.disk.name)) "namespace" .Release.Namespace) }}
20+
{{- else }}
21+
{{- fail (printf "Unknown source type: %s. Expected one of: image, http, upload, disk." (keys .Values.source | join ", ")) }}
22+
{{- end }}
23+
{{- else }}
24+
{{- $desiredSource = dict "blank" dict }}
25+
{{- end }}
26+
{{- /*
27+
DataVolume.spec is immutable in CDI, so editing storageClass/source on an
28+
existing disk is a silent no-op that diverges the CR from the cluster (#2985);
29+
fail instead. Both are compared against the operator input recorded in an
30+
annotation at creation (legacy disks fall back to stored spec.source). Full
31+
contract + caveats: docs/storage-immutability.md. Size growth: pvc-resize-hook.
32+
*/}}
33+
{{- if $existingDV }}
34+
{{- $existingAnnotations := $existingDV.metadata.annotations | default dict }}
35+
{{- if hasKey $existingAnnotations $storageClassAnnotation }}
36+
{{- $recordedSC := (index $existingAnnotations $storageClassAnnotation) | toString }}
37+
{{- if ne $recordedSC $requestedSC }}
38+
{{- fail (printf "VMDisk storageClass is immutable after creation (current: %q, requested: %q). Delete and recreate the disk to change it." $recordedSC $requestedSC) }}
39+
{{- end }}
40+
{{- end }}
41+
{{- $recordedSource := (index $existingAnnotations $sourceAnnotation) | default "" }}
42+
{{- if $recordedSource }}
43+
{{- if ne $recordedSource $requestedSource }}
44+
{{- fail (printf "VMDisk source is immutable after creation (current: %s, requested: %s). Delete and recreate the disk to change it." $recordedSource $requestedSource) }}
45+
{{- end }}
46+
{{- else }}
47+
{{- $storedSource := $existingDV.spec.source | default dict }}
48+
{{- /* Legacy disk (no annotation): compare only chart-producible stored
49+
shapes; anything else (sourceRef, non-map, foreign source) is left
50+
alone, never crashed or deadlocked. Vars are declared before the map
51+
guards so they stay in scope below. See docs/storage-immutability.md. */}}
52+
{{- $storedType := "" }}
53+
{{- $chartShaped := false }}
54+
{{- $storedIsImageClone := false }}
55+
{{- if kindIs "map" $storedSource }}
56+
{{- if eq (len (keys $storedSource)) 1 }}{{- $storedType = first (keys $storedSource) }}{{- end }}
57+
{{- $storedBody := index $storedSource $storedType }}
58+
{{- if kindIs "map" $storedBody }}
59+
{{- if or (eq $storedType "blank") (eq $storedType "upload") }}
60+
{{- $chartShaped = true }}
61+
{{- else if eq $storedType "http" }}
62+
{{- if eq (keys $storedBody | sortAlpha | join ",") "url" }}{{- $chartShaped = true }}{{- end }}
63+
{{- else if eq $storedType "pvc" }}
64+
{{- /* The chart only ever clones a pvc from cozy-public (image) or the
65+
release namespace (disk); a pvc in any other namespace is a
66+
foreign source with no chart-producible provenance. */}}
67+
{{- if eq (keys $storedBody | sortAlpha | join ",") "name,namespace" }}
68+
{{- if or (eq ($storedBody.namespace | toString) "cozy-public") (eq ($storedBody.namespace | toString) .Release.Namespace) }}
69+
{{- $chartShaped = true }}
70+
{{- end }}
71+
{{- if eq ($storedBody.namespace | toString) "cozy-public" }}{{- $storedIsImageClone = true }}{{- end }}
72+
{{- end }}
73+
{{- end }}
74+
{{- end }}
75+
{{- end }}
76+
{{- /* The image-clone exemption applies only when the request is ALSO an
77+
image: the vm-image- -> vm-default-images- prefix drift is
78+
indistinguishable from a real image-name change only then. A change
79+
away from an image source to another type is a different shape and is
80+
still caught. */}}
81+
{{- $exemptImage := and $storedIsImageClone (hasKey (.Values.source | default dict) "image") }}
82+
{{- if and $chartShaped (not $exemptImage) }}
83+
{{- if ne (toYaml $storedSource) (toYaml $desiredSource) }}
84+
{{- fail (printf "VMDisk source is immutable after creation (current: %s, requested: %s). Delete and recreate the disk to change it." (toJson $storedSource) $requestedSource) }}
85+
{{- end }}
86+
{{- end }}
87+
{{- end }}
88+
{{- end }}
289
apiVersion: cdi.kubevirt.io/v1beta1
390
kind: DataVolume
491
metadata:
592
annotations:
6-
{{- if hasKey .Values.source "upload" }}
93+
{{- if hasKey (.Values.source | default dict) "upload" }}
794
cdi.kubevirt.io/storage.bind.immediate.requested: ""
895
{{- end }}
96+
{{- if $existingDV }}
97+
{{- if hasKey ($existingDV.metadata.annotations | default dict) $sourceAnnotation }}
98+
{{ $sourceAnnotation }}: {{ index ($existingDV.metadata.annotations | default dict) $sourceAnnotation | quote }}
99+
{{- end }}
100+
{{- if hasKey ($existingDV.metadata.annotations | default dict) $storageClassAnnotation }}
101+
{{ $storageClassAnnotation }}: {{ index ($existingDV.metadata.annotations | default dict) $storageClassAnnotation | quote }}
102+
{{- end }}
103+
{{- else }}
104+
{{ $sourceAnnotation }}: {{ $requestedSource | quote }}
105+
{{ $storageClassAnnotation }}: {{ $requestedSC | quote }}
106+
{{- end }}
9107
vm-disk.cozystack.io/optical: "{{ .Values.optical }}"
10108
labels:
11109
app.kubernetes.io/instance: {{ .Release.Name }}
12110
name: {{ .Release.Name }}
13111
spec:
14112
{{- if $existingDV }}
15113
{{- toYaml $existingDV.spec | nindent 2 }}
16-
{{- else }}
17-
contentType: kubevirt
18-
{{- if .Values.source }}
19-
{{- if gt (len .Values.source) 1 }}
20-
{{- fail "Exactly one type of source is expected!" }}
21-
{{- end }}
22-
source:
23-
{{- if hasKey .Values.source "image" }}
24-
pvc:
25-
name: vm-default-images-{{ required "A valid .Values.source.image.name entry required!" .Values.source.image.name }}
26-
namespace: cozy-public
27-
{{- else if hasKey .Values.source "http" }}
28-
http:
29-
url: {{ required "A valid .Values.source.http.url entry required!" .Values.source.http.url }}
30-
{{- else if hasKey .Values.source "upload" }}
31-
upload: {}
32-
{{- else if hasKey .Values.source "disk" }}
33-
pvc:
34-
name: vm-disk-{{ required "A valid .Values.source.disk.name entry required!" .Values.source.disk.name }}
35-
namespace: {{ $.Release.Namespace }}
36-
{{- end }}
37114
{{- else }}
115+
contentType: kubevirt
38116
source:
39-
blank: {}
40-
{{- end }}
117+
{{- toYaml $desiredSource | nindent 4 }}
41118
storage:
42119
resources:
43120
requests:

0 commit comments

Comments
 (0)