Skip to content

Commit 22b33fd

Browse files
fl0pp5fl0pp5
committed
feat(platform): add registries.imageOverrides
The platform values take the image override table next to the registry mirrors it complements and pass it to every release as _cluster.image-overrides. The chart refuses a table the operator and the API would refuse, naming the entry, and requires every field to be a string so that an unquoted tag such as 1.20 cannot reach a release as 1.2. Assisted-by: LLM Signed-off-by: Ivan Pepelyaev <[email protected]>
1 parent 560556d commit 22b33fd

5 files changed

Lines changed: 198 additions & 0 deletions

File tree

‎docs/agents/image-refs.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,20 @@ Two rows do not source the shared library, for opposite reasons. `hack/overlay-m
7070

7171
`nightly-mirror.sh` is the one where a missed file is worst. The set it rewrites hosts in must equal the set it mirrored: a file rewritten but not mirrored leaves a dangling reference to an image never pushed, and a file mirrored but not rewritten leaves the published tree pointing at the private build registry.
7272

73+
## Overriding images at install time
74+
75+
Everything above is about the references the tree pins. An installation can replace them without a rebuild through `registries.imageOverrides` in the platform values: a list of kustomize `images` entries (`name`, `newName`, `newTag`, `digest`) that rename, retag or repin any container image by its tag-less name. Registry mirrors on the nodes (`registries.mirrors`) can only serve an image under the name it was pinned with, so the table is what runs an image set rebuilt under other names, such as a distribution's own registry.
76+
77+
The platform chart validates the table and writes it into the `cozystack-values` Secret as `_cluster.image-overrides`. From there it reaches every HelmRelease as a kustomize `images` post-renderer, by three paths that must stay in step:
78+
79+
- `cozystack-operator` sets it on every release it generates for a Package, and re-reconciles every Package when the Secret changes.
80+
- `cozystack-api` sets it on every Application release it writes, and the ApplicationDefinition release reconciler in `cozystack-controller` brings releases written before the table changed in line. Both merge it with the post-renderers a release already has: those of its own stay, in order, and the table replaces the previous one at the end.
81+
- A HelmRelease a chart renders gets it from `cozy-lib.postRenderers`, called in its spec. Nothing else builds such a release, so a template that does not call the helper installs upstream images. `hack/helmrelease-postrenderers_test.bats` fails on any HelmRelease template without the call and on any package rendering one without `charts/cozy-lib`. A release with post-renderers of its own passes them as the helper's first argument so that the override comes last.
82+
83+
All three read the table through `pkg/imageoverrides`, and the chart's checks mirror its `Validate`. A table the Go side cannot read stops the Package or refuses the Application write rather than installing without it, because on an air-gapped cluster a release without its overrides fails later, at image pull, far from the cause. For the same reason a missing `cozy-system/cozystack-values` is not read as an empty table: before the platform first writes it no release carries overrides, and once it exists a deletion says nothing about the table, so every path keeps the post-renderers a release already has until the Secret is back. Removing overrides means setting an empty `registries.imageOverrides`.
84+
85+
The post-renderer rewrites only `image` fields of `containers` and `initContainers`, in any kind. References in arguments, environment variables, ConfigMaps or custom resource fields (CNPG `imageName`, VictoriaMetrics CRs), and images an operator picks at runtime, are not reached. Flux and `cozystack-operator` are deployed by the installer, not through a HelmRelease, so the table does not reach them: the operator image is set by the installer's `cozystackOperator.image`, and the Flux images are the ones embedded in `internal/fluxinstall/manifests/fluxcd.yaml`.
86+
7387
## Known gaps
7488

7589
These predate the shared enumeration. Each entry says whether the tree works around it today; they are recorded here so nobody rediscovers them as surprises.

‎packages/core/platform/templates/_helpers.tpl‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,3 +285,50 @@ sliced into /24s ever allowed.
285285
{{- define "cozystack.platform.ciliumPoolRange" -}}
286286
{{- join ".." (splitList " " (include "cozystack.platform.ciliumPoolBounds" .)) -}}
287287
{{- end }}
288+
289+
{{/*
290+
Validated registries.imageOverrides, as a YAML list. It travels to every
291+
release as _cluster.image-overrides; cozystack-operator and cozystack-api
292+
refuse to build releases from a table they cannot apply, so a broken table
293+
fails this render instead, where the error names the value that is wrong.
294+
The checks match pkg/imageoverrides.Validate.
295+
*/}}
296+
{{- define "cozystack.platform.imageOverrides" -}}
297+
{{- $overrides := (.Values.registries | default dict).imageOverrides | default list }}
298+
{{- if not (kindIs "slice" $overrides) }}
299+
{{- fail (printf "registries.imageOverrides must be a list, got %s" (kindOf $overrides)) }}
300+
{{- end }}
301+
{{- $seen := dict }}
302+
{{- range $i, $o := $overrides }}
303+
{{- if not (kindIs "map" $o) }}
304+
{{- fail (printf "registries.imageOverrides[%d] must be a map" $i) }}
305+
{{- end }}
306+
{{- range $k, $v := $o }}
307+
{{- if not (has $k (list "name" "newName" "newTag" "digest")) }}
308+
{{- fail (printf "registries.imageOverrides[%d]: unknown field %q" $i $k) }}
309+
{{- end }}
310+
{{- /* An unquoted tag such as 1.20 arrives as the number 1.2. */}}
311+
{{- if not (kindIs "string" $v) }}
312+
{{- fail (printf "registries.imageOverrides[%d].%s must be a string, got %s; quote it" $i $k (kindOf $v)) }}
313+
{{- end }}
314+
{{- end }}
315+
{{- $name := $o.name | default "" | toString }}
316+
{{- if not $name }}
317+
{{- fail (printf "registries.imageOverrides[%d]: name is required" $i) }}
318+
{{- end }}
319+
{{- if or (contains "@" $name) (contains ":" (last (splitList "/" $name))) }}
320+
{{- fail (printf "registries.imageOverrides[%d]: name %q must not carry a tag or digest" $i $name) }}
321+
{{- end }}
322+
{{- if not (or $o.newName $o.newTag $o.digest) }}
323+
{{- fail (printf "registries.imageOverrides[%d]: %q sets none of newName, newTag, digest" $i $name) }}
324+
{{- end }}
325+
{{- if and $o.digest (not (regexMatch "^sha256:[0-9a-f]{64}$" (toString $o.digest))) }}
326+
{{- fail (printf "registries.imageOverrides[%d]: digest %q is not sha256:<64 hex>" $i (toString $o.digest)) }}
327+
{{- end }}
328+
{{- if hasKey $seen $name }}
329+
{{- fail (printf "registries.imageOverrides[%d]: %q is listed twice" $i $name) }}
330+
{{- end }}
331+
{{- $_ := set $seen $name true }}
332+
{{- end }}
333+
{{- toYaml $overrides }}
334+
{{- end }}

‎packages/core/platform/templates/apps.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,10 @@ stringData:
293293
scheduling:
294294
{{- . | toYaml | nindent 8 }}
295295
{{- end }}
296+
{{- with include "cozystack.platform.imageOverrides" . | fromYamlArray }}
297+
image-overrides:
298+
{{- . | toYaml | nindent 8 }}
299+
{{- end }}
296300
{{- with $kubeRootCa.data }}
297301
kube-root-ca: {{ index . "ca.crt" | b64enc | quote }}
298302
{{- end }}

‎packages/core/platform/tests/apps_image_overrides_test.yaml‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
suite: apps.yaml registries.imageOverrides → _cluster.image-overrides wiring
2+
3+
templates:
4+
- templates/apps.yaml
5+
6+
release:
7+
name: cozy-platform
8+
namespace: cozy-system
9+
10+
# The table reaches every release through the cozystack-values Secret, where
11+
# cozystack-operator and cozystack-api read it for the releases they build and
12+
# cozy-lib.postRenderers for the ones charts render. Both Go readers refuse a
13+
# table they cannot apply, so the render must refuse it first, naming the bad
14+
# entry.
15+
16+
tests:
17+
- it: carries no image-overrides key by default
18+
asserts:
19+
- notMatchRegex:
20+
path: stringData["values.yaml"]
21+
pattern: 'image-overrides'
22+
23+
- it: passes the table through under _cluster
24+
set:
25+
registries:
26+
imageOverrides:
27+
- name: registry.k8s.io/sig-storage/csi-attacher
28+
newName: registry.example.org/os/external-attacher
29+
newTag: "4.10.0"
30+
digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f
31+
asserts:
32+
- matchRegex:
33+
path: stringData["values.yaml"]
34+
pattern: '(?m)^ image-overrides:\n - digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f\n name: registry.k8s.io/sig-storage/csi-attacher\n newName: registry.example.org/os/external-attacher\n newTag: 4.10.0$'
35+
36+
- it: keeps a registry port in the name
37+
set:
38+
registries:
39+
imageOverrides:
40+
- name: registry.example.org:5000/os/kubeovn
41+
newTag: v2
42+
asserts:
43+
- matchRegex:
44+
path: stringData["values.yaml"]
45+
pattern: 'name: registry.example.org:5000/os/kubeovn'
46+
47+
- it: refuses an entry without a name
48+
set:
49+
registries:
50+
imageOverrides:
51+
- newName: c/d
52+
asserts:
53+
- failedTemplate:
54+
errorPattern: 'registries.imageOverrides\[0\]: name is required'
55+
56+
- it: refuses a name carrying a tag
57+
set:
58+
registries:
59+
imageOverrides:
60+
- name: a/b:v1
61+
newName: c/d
62+
asserts:
63+
- failedTemplate:
64+
errorPattern: 'must not carry a tag or digest'
65+
66+
- it: refuses an entry that changes nothing
67+
set:
68+
registries:
69+
imageOverrides:
70+
- name: a/b
71+
asserts:
72+
- failedTemplate:
73+
errorPattern: 'sets none of newName, newTag, digest'
74+
75+
- it: refuses a malformed digest
76+
set:
77+
registries:
78+
imageOverrides:
79+
- name: a/b
80+
digest: sha256:abc
81+
asserts:
82+
- failedTemplate:
83+
errorPattern: 'is not sha256:<64 hex>'
84+
85+
- it: refuses an unknown field
86+
set:
87+
registries:
88+
imageOverrides:
89+
- name: a/b
90+
newImage: c/d
91+
asserts:
92+
- failedTemplate:
93+
errorPattern: 'unknown field "newImage"'
94+
95+
- it: refuses a name listed twice
96+
set:
97+
registries:
98+
imageOverrides:
99+
- name: a/b
100+
newName: c/d
101+
- name: a/b
102+
newTag: v2
103+
asserts:
104+
- failedTemplate:
105+
errorPattern: '"a/b" is listed twice'
106+
107+
- it: refuses a tag YAML read as a number
108+
set:
109+
registries:
110+
imageOverrides:
111+
- name: a/b
112+
newTag: 1.20
113+
asserts:
114+
- failedTemplate:
115+
errorPattern: 'registries.imageOverrides\[0\].newTag must be a string, got float64; quote it'

‎packages/core/platform/values.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -798,6 +798,24 @@ branding: {}
798798
# "10.0.0.1:8082":
799799
# tls:
800800
# insecureSkipVerify: true
801+
#
802+
# Mirrors serve an image only under the name it was pinned with. To run an
803+
# image under another name, tag or digest, for example one rebuilt in a
804+
# distribution's own registry, list it in imageOverrides. Each entry rewrites
805+
# every container image whose name (without tag or digest) equals `name`, in
806+
# every HelmRelease the platform installs, including the addons of tenant
807+
# Kubernetes clusters, but not Flux or cozystack-operator, which the installer
808+
# deploys; the fields are those of a kustomize `images` entry. A
809+
# digest replaces the tag that was pinned; newTag alongside it only labels the
810+
# reference. Image references outside container specs (arguments, environment
811+
# variables, fields of custom resources) and images an operator picks at
812+
# runtime are not rewritten.
813+
#
814+
# imageOverrides:
815+
# - name: registry.k8s.io/sig-storage/csi-attacher
816+
# newName: registry.example.org/os/external-attacher
817+
# newTag: "4.10.0"
818+
# digest: sha256:f80b60d4523dcdecb225c3113bcbd0eedc45b4d2b46f87c0ccf4f14d09a4a35f
801819
registries: {}
802820
# Resource allocation ratios
803821
resources:

0 commit comments

Comments
 (0)