Skip to content

Commit 14ca9c3

Browse files
fl0pp5fl0pp5
committed
feat(platform): add registries.imageOverrides
The platform values take the image override table next to the registry mirrors it complements and pass it to every release as _cluster.image-overrides. The chart refuses a table the operator and the API would refuse, naming the entry, and requires every field to be a string so that an unquoted tag such as 1.20 cannot reach a release as 1.2. Assisted-by: LLM Signed-off-by: Ivan Pepelyaev <[email protected]>
1 parent d80d316 commit 14ca9c3

5 files changed

Lines changed: 226 additions & 0 deletions

File tree

‎docs/agents/image-refs.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,20 @@ Two rows do not source the shared library, for opposite reasons. `hack/overlay-m
7070

7171
`nightly-mirror.sh` is the one where a missed file is worst. The set it rewrites hosts in must equal the set it mirrored: a file rewritten but not mirrored leaves a dangling reference to an image never pushed, and a file mirrored but not rewritten leaves the published tree pointing at the private build registry.
7272

73+
## Overriding images at install time
74+
75+
Everything above is about the references the tree pins. An installation can replace them without a rebuild through `registries.imageOverrides` in the platform values: a list of kustomize `images` entries (`name`, `newName`, `newTag`, `digest`) that rename, retag or repin any container image by its tag-less name. Registry mirrors on the nodes (`registries.mirrors`) can only serve an image under the name it was pinned with, so the table is what runs an image set rebuilt under other names, such as a distribution's own registry.
76+
77+
The platform chart validates the table and writes it into the `cozystack-values` Secret as `_cluster.image-overrides`. From there it reaches every HelmRelease as a kustomize `images` post-renderer, by three paths that must stay in step:
78+
79+
- `cozystack-operator` sets it on every release it generates for a Package, and re-reconciles every Package when the Secret changes.
80+
- `cozystack-api` sets it on every Application release it writes, and the ApplicationDefinition release reconciler in `cozystack-controller` brings releases written before the table changed in line. Both merge it with the post-renderers a release already has: those of its own stay, in order, and the table replaces the previous one at the end.
81+
- A HelmRelease a chart renders gets it from `cozy-lib.postRenderers`, called in its spec. Nothing else builds such a release, so a template that does not call the helper installs upstream images. `hack/helmrelease-postrenderers_test.bats` fails on any HelmRelease template without the call and on any package rendering one without `charts/cozy-lib`. A release with post-renderers of its own passes them as the helper's first argument so that the override comes last.
82+
83+
The two Go paths read the table through `pkg/imageoverrides`; the chart path reads `_cluster.image-overrides` from the values as they are and does not check the table itself, which is safe only because the platform chart refuses a bad table before it reaches the Secret: its checks in `cozystack.platform.imageOverrides` mirror `imageoverrides.Validate`, and the two must change together. A table the Go side cannot read stops the Package or refuses the Application write rather than installing without it, because on an air-gapped cluster a release without its overrides fails later, at image pull, far from the cause. For the same reason a missing `cozy-system/cozystack-values` is not read as an empty table: before the platform first writes it no release carries overrides, and once it exists a deletion says nothing about the table, so every path keeps the post-renderers a release already has until the Secret is back. Removing overrides means setting an empty `registries.imageOverrides`.
84+
85+
The post-renderer rewrites only `image` fields of `containers` and `initContainers`, in any kind. References in arguments, environment variables, ConfigMaps or custom resource fields (CNPG `imageName`, VictoriaMetrics CRs), and images an operator picks at runtime, are not reached. Flux and `cozystack-operator` are deployed by the installer, not through a HelmRelease, so the table does not reach them: the operator image is set by the installer's `cozystackOperator.image`, and the Flux images are the ones embedded in `internal/fluxinstall/manifests/fluxcd.yaml`.
86+
7387
## Known gaps
7488

7589
These predate the shared enumeration. Each entry says whether the tree works around it today; they are recorded here so nobody rediscovers them as surprises.

‎packages/core/platform/templates/_helpers.tpl‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,3 +285,53 @@ sliced into /24s ever allowed.
285285
{{- define "cozystack.platform.ciliumPoolRange" -}}
286286
{{- join ".." (splitList " " (include "cozystack.platform.ciliumPoolBounds" .)) -}}
287287
{{- end }}
288+
289+
{{/*
290+
Validated registries.imageOverrides, as a YAML list. It travels to every
291+
release as _cluster.image-overrides; cozystack-operator and cozystack-api
292+
refuse to build releases from a table they cannot apply, so a broken table
293+
fails this render instead, where the error names the value that is wrong.
294+
The checks match pkg/imageoverrides.Validate.
295+
*/}}
296+
{{- define "cozystack.platform.imageOverrides" -}}
297+
{{- /* Checked before defaulting: default would turn false or 0 into an empty
298+
list and drop the table without a word. */}}
299+
{{- $overrides := (.Values.registries | default dict).imageOverrides }}
300+
{{- if and (not (kindIs "invalid" $overrides)) (not (kindIs "slice" $overrides)) }}
301+
{{- fail (printf "registries.imageOverrides must be a list, got %s" (kindOf $overrides)) }}
302+
{{- end }}
303+
{{- $overrides = $overrides | default list }}
304+
{{- $seen := dict }}
305+
{{- range $i, $o := $overrides }}
306+
{{- if not (kindIs "map" $o) }}
307+
{{- fail (printf "registries.imageOverrides[%d] must be a map" $i) }}
308+
{{- end }}
309+
{{- range $k, $v := $o }}
310+
{{- if not (has $k (list "name" "newName" "newTag" "digest")) }}
311+
{{- fail (printf "registries.imageOverrides[%d]: unknown field %q" $i $k) }}
312+
{{- end }}
313+
{{- /* An unquoted tag such as 1.20 arrives as the number 1.2. */}}
314+
{{- if not (kindIs "string" $v) }}
315+
{{- fail (printf "registries.imageOverrides[%d].%s must be a string, got %s; quote it" $i $k (kindOf $v)) }}
316+
{{- end }}
317+
{{- end }}
318+
{{- $name := $o.name | default "" | toString }}
319+
{{- if not $name }}
320+
{{- fail (printf "registries.imageOverrides[%d]: name is required" $i) }}
321+
{{- end }}
322+
{{- if or (contains "@" $name) (contains ":" (last (splitList "/" $name))) }}
323+
{{- fail (printf "registries.imageOverrides[%d]: name %q must not carry a tag or digest" $i $name) }}
324+
{{- end }}
325+
{{- if not (or $o.newName $o.newTag $o.digest) }}
326+
{{- fail (printf "registries.imageOverrides[%d]: %q sets none of newName, newTag, digest" $i $name) }}
327+
{{- end }}
328+
{{- if and $o.digest (not (regexMatch "^sha256:[0-9a-f]{64}$" (toString $o.digest))) }}
329+
{{- fail (printf "registries.imageOverrides[%d]: digest %q is not sha256:<64 hex>" $i (toString $o.digest)) }}
330+
{{- end }}
331+
{{- if hasKey $seen $name }}
332+
{{- fail (printf "registries.imageOverrides[%d]: %q is listed twice" $i $name) }}
333+
{{- end }}
334+
{{- $_ := set $seen $name true }}
335+
{{- end }}
336+
{{- toYaml $overrides }}
337+
{{- end }}

‎packages/core/platform/templates/apps.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,10 @@ stringData:
293293
scheduling:
294294
{{- . | toYaml | nindent 8 }}
295295
{{- end }}
296+
{{- with include "cozystack.platform.imageOverrides" . | fromYamlArray }}
297+
image-overrides:
298+
{{- . | toYaml | nindent 8 }}
299+
{{- end }}
296300
{{- with $kubeRootCa.data }}
297301
kube-root-ca: {{ index . "ca.crt" | b64enc | quote }}
298302
{{- end }}

‎packages/core/platform/tests/apps_image_overrides_test.yaml‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
suite: apps.yaml registries.imageOverrides → _cluster.image-overrides wiring
2+
3+
templates:
4+
- templates/apps.yaml
5+
6+
release:
7+
name: cozy-platform
8+
namespace: cozy-system
9+
10+
# The table reaches every release through the cozystack-values Secret, where
11+
# cozystack-operator and cozystack-api read it for the releases they build and
12+
# cozy-lib.postRenderers for the ones charts render. Both Go readers refuse a
13+
# table they cannot apply, so the render must refuse it first, naming the bad
14+
# entry.
15+
16+
tests:
17+
- it: carries no image-overrides key by default
18+
asserts:
19+
- notMatchRegex:
20+
path: stringData["values.yaml"]
21+
pattern: 'image-overrides'
22+
23+
- it: treats a null table as no table
24+
set:
25+
registries:
26+
imageOverrides: null
27+
asserts:
28+
- notMatchRegex:
29+
path: stringData["values.yaml"]
30+
pattern: 'image-overrides'
31+
32+
- it: refuses false rather than reading it as an empty table
33+
set:
34+
registries:
35+
imageOverrides: false
36+
asserts:
37+
- failedTemplate:
38+
errorPattern: 'registries.imageOverrides must be a list, got bool'
39+
40+
- it: refuses 0 rather than reading it as an empty table
41+
set:
42+
registries:
43+
imageOverrides: 0
44+
asserts:
45+
- failedTemplate:
46+
errorPattern: 'registries.imageOverrides must be a list, got (float64|int64)'
47+
48+
- it: passes the table through under _cluster
49+
set:
50+
registries:
51+
imageOverrides:
52+
- name: registry.k8s.io/sig-storage/csi-attacher
53+
newName: registry.example.org/os/external-attacher
54+
newTag: "4.10.0"
55+
digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f
56+
asserts:
57+
- matchRegex:
58+
path: stringData["values.yaml"]
59+
pattern: '(?m)^ image-overrides:\n - digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f\n name: registry.k8s.io/sig-storage/csi-attacher\n newName: registry.example.org/os/external-attacher\n newTag: 4.10.0$'
60+
61+
- it: keeps a registry port in the name
62+
set:
63+
registries:
64+
imageOverrides:
65+
- name: registry.example.org:5000/os/kubeovn
66+
newTag: v2
67+
asserts:
68+
- matchRegex:
69+
path: stringData["values.yaml"]
70+
pattern: 'name: registry.example.org:5000/os/kubeovn'
71+
72+
- it: refuses an entry without a name
73+
set:
74+
registries:
75+
imageOverrides:
76+
- newName: c/d
77+
asserts:
78+
- failedTemplate:
79+
errorPattern: 'registries.imageOverrides\[0\]: name is required'
80+
81+
- it: refuses a name carrying a tag
82+
set:
83+
registries:
84+
imageOverrides:
85+
- name: a/b:v1
86+
newName: c/d
87+
asserts:
88+
- failedTemplate:
89+
errorPattern: 'must not carry a tag or digest'
90+
91+
- it: refuses an entry that changes nothing
92+
set:
93+
registries:
94+
imageOverrides:
95+
- name: a/b
96+
asserts:
97+
- failedTemplate:
98+
errorPattern: 'sets none of newName, newTag, digest'
99+
100+
- it: refuses a malformed digest
101+
set:
102+
registries:
103+
imageOverrides:
104+
- name: a/b
105+
digest: sha256:abc
106+
asserts:
107+
- failedTemplate:
108+
errorPattern: 'is not sha256:<64 hex>'
109+
110+
- it: refuses an unknown field
111+
set:
112+
registries:
113+
imageOverrides:
114+
- name: a/b
115+
newImage: c/d
116+
asserts:
117+
- failedTemplate:
118+
errorPattern: 'unknown field "newImage"'
119+
120+
- it: refuses a name listed twice
121+
set:
122+
registries:
123+
imageOverrides:
124+
- name: a/b
125+
newName: c/d
126+
- name: a/b
127+
newTag: v2
128+
asserts:
129+
- failedTemplate:
130+
errorPattern: '"a/b" is listed twice'
131+
132+
- it: refuses a tag YAML read as a number
133+
set:
134+
registries:
135+
imageOverrides:
136+
- name: a/b
137+
newTag: 1.20
138+
asserts:
139+
- failedTemplate:
140+
errorPattern: 'registries.imageOverrides\[0\].newTag must be a string, got float64; quote it'

‎packages/core/platform/values.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -798,6 +798,24 @@ branding: {}
798798
# "10.0.0.1:8082":
799799
# tls:
800800
# insecureSkipVerify: true
801+
#
802+
# Mirrors serve an image only under the name it was pinned with. To run an
803+
# image under another name, tag or digest, for example one rebuilt in a
804+
# distribution's own registry, list it in imageOverrides. Each entry rewrites
805+
# every container image whose name (without tag or digest) equals `name`, in
806+
# every HelmRelease the platform installs, including the addons of tenant
807+
# Kubernetes clusters, but not Flux or cozystack-operator, which the installer
808+
# deploys; the fields are those of a kustomize `images` entry. A
809+
# digest replaces the tag that was pinned; newTag alongside it only labels the
810+
# reference. Image references outside container specs (arguments, environment
811+
# variables, fields of custom resources) and images an operator picks at
812+
# runtime are not rewritten.
813+
#
814+
# imageOverrides:
815+
# - name: registry.k8s.io/sig-storage/csi-attacher
816+
# newName: registry.example.org/os/external-attacher
817+
# newTag: "4.10.0"
818+
# digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f
801819
registries: {}
802820
# Resource allocation ratios
803821
resources:

0 commit comments

Comments
 (0)