Skip to content

Commit 2231fe5

Browse files
fl0pp5fl0pp5
committed
feat(platform): add registries.imageOverrides
The platform values take the image override table next to the registry mirrors it complements and pass it to every release as _cluster.image-overrides. The chart refuses a table the operator and the API would refuse, naming the entry, and requires every field to be a string so that an unquoted tag such as 1.20 cannot reach a release as 1.2. Assisted-by: LLM Signed-off-by: Ivan Pepelyaev <[email protected]>
1 parent 9f62800 commit 2231fe5

4 files changed

Lines changed: 210 additions & 0 deletions

File tree

‎docs/agents/image-refs.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,22 @@ Two rows do not source the shared library, for opposite reasons. `hack/overlay-m
7171

7272
`nightly-mirror.sh` is the one where a missed file is worst. The set it rewrites hosts in must equal the set it mirrored: a file rewritten but not mirrored leaves a dangling reference to an image never pushed, and a file mirrored but not rewritten leaves the published tree pointing at the private build registry.
7373

74+
## Overriding images at install time
75+
76+
Everything above is about the references the tree pins. An installation can replace them without a rebuild through `registries.imageOverrides` in the platform values: a list of kustomize `images` entries (`name`, `newName`, `newTag`, `digest`) that rename, retag or repin any container image by its tag-less name. Registry mirrors on the nodes (`registries.mirrors`) can only serve an image under the name it was pinned with, so the table is what runs an image set rebuilt under other names, such as a distribution's own registry.
77+
78+
The platform chart validates the table and writes it into the `cozystack-values` Secret as `_cluster.image-overrides`. From there it reaches every HelmRelease as a kustomize `images` post-renderer, by three paths that must stay in step:
79+
80+
- `cozystack-operator` sets it on every release it generates for a Package, and re-reconciles every Package when the Secret changes.
81+
- `cozystack-api` sets it on every Application release it writes, and the ApplicationDefinition release reconciler in `cozystack-controller` brings releases written before the table changed in line. Both merge it with the post-renderers a release already has: those of its own stay, in order, and the table replaces the previous one at the end.
82+
- A HelmRelease a chart renders gets it from `cozy-lib.postRenderers`, called in its spec. Nothing else builds such a release, so a template that does not call the helper installs upstream images. `hack/helmrelease-postrenderers_test.bats` fails on any HelmRelease template without the call and on any package rendering one without `charts/cozy-lib`. A release with post-renderers of its own passes them as the helper's first argument so that the override comes last.
83+
84+
The two Go paths read the table through `pkg/imageoverrides`; the chart path reads `_cluster.image-overrides` from the values as they are and does not check the table itself, which is safe only because the platform chart refuses a bad table before it reaches the Secret: its checks in `cozystack.platform.imageOverrides` mirror `imageoverrides.Validate`, and the two must change together. A table the Go side cannot read stops the Package or refuses the Application write rather than installing without it, because on an air-gapped cluster a release without its overrides fails later, at image pull, far from the cause. For the same reason a missing `cozy-system/cozystack-values` is not read as an empty table: before the platform first writes it no release carries overrides, and once it exists a deletion says nothing about the table, so every path keeps the post-renderers a release already has until the Secret is back. Removing overrides means setting an empty `registries.imageOverrides`.
85+
86+
The post-renderer rewrites only `image` fields of `containers` and `initContainers`, in any kind. References in arguments, environment variables, ConfigMaps or custom resource fields (CNPG `imageName`, VictoriaMetrics CRs), and images an operator picks at runtime, are not reached. Flux and `cozystack-operator` are deployed by the installer, not through a HelmRelease, so the table does not reach them: the operator image is set by the installer's `cozystackOperator.image`, and the Flux images are the ones embedded in `internal/fluxinstall/manifests/fluxcd.yaml`.
87+
88+
Two consequences of that are easy to miss. An operator that also passes its own image in an argument or an environment variable, for the pods it starts, is split by an override: the operator runs the overridden image, while the pods it starts use the upstream one (CloudNativePG's `OPERATOR_IMAGE_NAME`, which every PostgreSQL pod runs as its bootstrap init container; kube-ovn `--image`; kamaji `--migrate-image`; Strimzi's entity operator images; mariadb-operator's `MARIADB_OPERATOR_IMAGE`), so on an air-gapped cluster such an override is not enough on its own. And an entry matches an image by name alone, so `newTag` and `digest` apply to every version of an image a release runs at once: for `foundationdb/fdb-kubernetes-monitor`, which fdb-operator runs at 7.3.63 and 7.4.1 in one Deployment, either moves both init containers onto one version. For such images only `newName` is safe, and only if the target registry keeps the upstream tags.
89+
7490
## Known gaps
7591

7692
These predate the shared enumeration. Each entry says whether the tree works around it today; they are recorded here so nobody rediscovers them as surprises.

‎packages/core/platform/templates/_helpers.tpl‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,3 +285,53 @@ sliced into /24s ever allowed.
285285
{{- define "cozystack.platform.ciliumPoolRange" -}}
286286
{{- join ".." (splitList " " (include "cozystack.platform.ciliumPoolBounds" .)) -}}
287287
{{- end }}
288+
289+
{{/*
290+
Validated registries.imageOverrides, as a YAML list. It travels to every
291+
release as _cluster.image-overrides; cozystack-operator and cozystack-api
292+
refuse to build releases from a table they cannot apply, so a broken table
293+
fails this render instead, where the error names the value that is wrong.
294+
The checks match pkg/imageoverrides.Validate.
295+
*/}}
296+
{{- define "cozystack.platform.imageOverrides" -}}
297+
{{- /* Checked before defaulting: default would turn false or 0 into an empty
298+
list and drop the table without a word. */}}
299+
{{- $overrides := (.Values.registries | default dict).imageOverrides }}
300+
{{- if and (not (kindIs "invalid" $overrides)) (not (kindIs "slice" $overrides)) }}
301+
{{- fail (printf "registries.imageOverrides must be a list, got %s" (kindOf $overrides)) }}
302+
{{- end }}
303+
{{- $overrides = $overrides | default list }}
304+
{{- $seen := dict }}
305+
{{- range $i, $o := $overrides }}
306+
{{- if not (kindIs "map" $o) }}
307+
{{- fail (printf "registries.imageOverrides[%d] must be a map" $i) }}
308+
{{- end }}
309+
{{- range $k, $v := $o }}
310+
{{- if not (has $k (list "name" "newName" "newTag" "digest")) }}
311+
{{- fail (printf "registries.imageOverrides[%d]: unknown field %q" $i $k) }}
312+
{{- end }}
313+
{{- /* An unquoted tag such as 1.20 arrives as the number 1.2. */}}
314+
{{- if not (kindIs "string" $v) }}
315+
{{- fail (printf "registries.imageOverrides[%d].%s must be a string, got %s; quote it" $i $k (kindOf $v)) }}
316+
{{- end }}
317+
{{- end }}
318+
{{- $name := $o.name | default "" | toString }}
319+
{{- if not $name }}
320+
{{- fail (printf "registries.imageOverrides[%d]: name is required" $i) }}
321+
{{- end }}
322+
{{- if or (contains "@" $name) (contains ":" (last (splitList "/" $name))) }}
323+
{{- fail (printf "registries.imageOverrides[%d]: name %q must not carry a tag or digest" $i $name) }}
324+
{{- end }}
325+
{{- if not (or $o.newName $o.newTag $o.digest) }}
326+
{{- fail (printf "registries.imageOverrides[%d]: %q sets none of newName, newTag, digest" $i $name) }}
327+
{{- end }}
328+
{{- if and $o.digest (not (regexMatch "^sha256:[0-9a-f]{64}$" (toString $o.digest))) }}
329+
{{- fail (printf "registries.imageOverrides[%d]: digest %q is not sha256:<64 hex>" $i (toString $o.digest)) }}
330+
{{- end }}
331+
{{- if hasKey $seen $name }}
332+
{{- fail (printf "registries.imageOverrides[%d]: %q is listed twice" $i $name) }}
333+
{{- end }}
334+
{{- $_ := set $seen $name true }}
335+
{{- end }}
336+
{{- toYaml $overrides }}
337+
{{- end }}

‎packages/core/platform/templates/apps.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -301,6 +301,10 @@ stringData:
301301
scheduling:
302302
{{- . | toYaml | nindent 8 }}
303303
{{- end }}
304+
{{- with include "cozystack.platform.imageOverrides" . | fromYamlArray }}
305+
image-overrides:
306+
{{- . | toYaml | nindent 8 }}
307+
{{- end }}
304308
{{- with $kubeRootCa.data }}
305309
kube-root-ca: {{ index . "ca.crt" | b64enc | quote }}
306310
{{- end }}

‎packages/core/platform/tests/apps_image_overrides_test.yaml‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
suite: apps.yaml registries.imageOverrides → _cluster.image-overrides wiring
2+
3+
templates:
4+
- templates/apps.yaml
5+
6+
release:
7+
name: cozy-platform
8+
namespace: cozy-system
9+
10+
# The table reaches every release through the cozystack-values Secret, where
11+
# cozystack-operator and cozystack-api read it for the releases they build and
12+
# cozy-lib.postRenderers for the ones charts render. Both Go readers refuse a
13+
# table they cannot apply, so the render must refuse it first, naming the bad
14+
# entry.
15+
16+
tests:
17+
- it: carries no image-overrides key by default
18+
asserts:
19+
- notMatchRegex:
20+
path: stringData["values.yaml"]
21+
pattern: 'image-overrides'
22+
23+
- it: treats a null table as no table
24+
set:
25+
registries:
26+
imageOverrides: null
27+
asserts:
28+
- notMatchRegex:
29+
path: stringData["values.yaml"]
30+
pattern: 'image-overrides'
31+
32+
- it: refuses false rather than reading it as an empty table
33+
set:
34+
registries:
35+
imageOverrides: false
36+
asserts:
37+
- failedTemplate:
38+
errorPattern: 'registries.imageOverrides must be a list, got bool'
39+
40+
- it: refuses 0 rather than reading it as an empty table
41+
set:
42+
registries:
43+
imageOverrides: 0
44+
asserts:
45+
- failedTemplate:
46+
errorPattern: 'registries.imageOverrides must be a list, got (float64|int64)'
47+
48+
- it: passes the table through under _cluster
49+
set:
50+
registries:
51+
imageOverrides:
52+
- name: registry.k8s.io/sig-storage/csi-attacher
53+
newName: registry.example.org/os/external-attacher
54+
newTag: "4.10.0"
55+
digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f
56+
asserts:
57+
- matchRegex:
58+
path: stringData["values.yaml"]
59+
pattern: '(?m)^ image-overrides:\n - digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f\n name: registry.k8s.io/sig-storage/csi-attacher\n newName: registry.example.org/os/external-attacher\n newTag: 4.10.0$'
60+
61+
- it: keeps a registry port in the name
62+
set:
63+
registries:
64+
imageOverrides:
65+
- name: registry.example.org:5000/os/kubeovn
66+
newTag: v2
67+
asserts:
68+
- matchRegex:
69+
path: stringData["values.yaml"]
70+
pattern: 'name: registry.example.org:5000/os/kubeovn'
71+
72+
- it: refuses an entry without a name
73+
set:
74+
registries:
75+
imageOverrides:
76+
- newName: c/d
77+
asserts:
78+
- failedTemplate:
79+
errorPattern: 'registries.imageOverrides\[0\]: name is required'
80+
81+
- it: refuses a name carrying a tag
82+
set:
83+
registries:
84+
imageOverrides:
85+
- name: a/b:v1
86+
newName: c/d
87+
asserts:
88+
- failedTemplate:
89+
errorPattern: 'must not carry a tag or digest'
90+
91+
- it: refuses an entry that changes nothing
92+
set:
93+
registries:
94+
imageOverrides:
95+
- name: a/b
96+
asserts:
97+
- failedTemplate:
98+
errorPattern: 'sets none of newName, newTag, digest'
99+
100+
- it: refuses a malformed digest
101+
set:
102+
registries:
103+
imageOverrides:
104+
- name: a/b
105+
digest: sha256:abc
106+
asserts:
107+
- failedTemplate:
108+
errorPattern: 'is not sha256:<64 hex>'
109+
110+
- it: refuses an unknown field
111+
set:
112+
registries:
113+
imageOverrides:
114+
- name: a/b
115+
newImage: c/d
116+
asserts:
117+
- failedTemplate:
118+
errorPattern: 'unknown field "newImage"'
119+
120+
- it: refuses a name listed twice
121+
set:
122+
registries:
123+
imageOverrides:
124+
- name: a/b
125+
newName: c/d
126+
- name: a/b
127+
newTag: v2
128+
asserts:
129+
- failedTemplate:
130+
errorPattern: '"a/b" is listed twice'
131+
132+
- it: refuses a tag YAML read as a number
133+
set:
134+
registries:
135+
imageOverrides:
136+
- name: a/b
137+
newTag: 1.20
138+
asserts:
139+
- failedTemplate:
140+
errorPattern: 'registries.imageOverrides\[0\].newTag must be a string, got float64; quote it'

0 commit comments

Comments
 (0)