Skip to content

Commit 68cd102

Browse files
batonogovlexfrei
authored andcommitted
feat(kubernetes-nodes): let CDI trust a self-hosted Talos image factory
talos.imageFactoryURL is documented for self-hosted factories and mirrors in air-gapped, rate-limited or flaky-egress environments. Such an endpoint is normally served with a certificate from an internal CA, and CDI — which trusts only its system store — refuses the import with "x509: certificate signed by unknown authority". The DataVolume never completes, no worker joins, and the release sits in install until it times out. DataVolume.spec.source.http.certConfigMap exists for exactly this, but the chart rendered source.http.url alone and no value reached the field. Add an optional talos.imageFactoryCA taking the CA in PEM form. When set, the chart renders it into a ConfigMap in the release namespace — CDI requires certConfigMap to sit alongside the DataVolume — and references it from the worker DataVolume on the HTTP import path; a pool cloning a golden image via osImage.builtin builds no source.http and renders neither. When empty, nothing is rendered and the output is byte-for-byte what it was. CDI appends the bundle to the system pool rather than replacing it, so a release pulling from a publicly-trusted endpoint keeps verifying either way. The value is emitted verbatim into a ConfigMap readable by anything that can read the namespace, so the render fails closed on private key material and on anything that is not complete PEM certificate blocks. Those guards live in a new validate-only cozy-lib.tls.validateCACert helper that takes the caller's field name for its messages, and cozy-lib.tls.caCertSecret is refactored onto it, so the check exists in exactly one place. Assisted-by: LLM Signed-off-by: Fedor Batonogov <[email protected]>
1 parent f4bde9e commit 68cd102

13 files changed

Lines changed: 674 additions & 31 deletions

File tree

‎api/apps/v1alpha1/kubernetesnodes/types.go‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/apps/kubernetes-nodes/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,7 @@ the parent cluster `kubernetes-<cluster>` in the same namespace, plus a
8686
| `talos.imageFactoryURL` | Default base URL of the Talos Image Factory that serves the worker OS disk image (the `openstack-amd64.raw.xz` raw artifact streamed in by CDI over HTTP). Applies only when this pool imports over HTTP — the default and `osImage.factory`, which overrides it via `osImage.factory.imageFactoryURL`. A pool on `osImage.builtin` clones a golden image from the worker image catalog and does not use this field at all. Defaults to the public factory. Point at a self-hosted Image Factory, a caching mirror, or an internal HTTP file server for air-gapped, rate-limited, or flaky-egress environments. No trailing slash. It is interpolated into the worker DataVolume source URL, so on those HTTP paths it must be a plain `http(s)` URL and the render refuses anything else; a pool on `osImage.builtin` builds no such URL and is not held to it, so blanking this field there is legal. | `string` | `https://factory.talos.dev` |
8787
| `talos.installerRepository` | OCI repository prefix for the Talos installer image used by the in-guest `talos-reconcile` upgrade Job. Resolved as `<installerRepository>/<schematicID>:<version>` against whatever this pool's `osImage` pins, falling back to `talos.schematicID` and `talos.version` — so the installer matches the OS the pool actually booted. The prefix itself has no `osImage.*` override, so an air-gapped pool needs it pointed at a reachable registry as well. Defaults to the public factory's installer path. Override for air-gapped or mirrored registries. No trailing slash. | `string` | `factory.talos.dev/installer` |
8888
| `talos.registryMirrors` | Talos `machine.registries.mirrors` passthrough for worker nodes: a map of upstream registry host to `{ endpoints: [ ... ] }`. Empty by default, so workers pull container images (the Talos `kubelet` image included) directly from the upstream registry. Point a host such as `ghcr.io` at an in-cluster pull-through mirror for air-gapped, rate-limited, or flaky-egress environments so a worker's boot does not depend on live public egress. Talos still falls back to the upstream registry unless a host also sets `skipFallback: true`, so a mirror alone does not enforce air-gap. This value is part of the worker machine config, whose content hash names the TalosConfigTemplate, so changing it rolls every existing worker in the pool rather than only affecting nodes created later. | `object` | `{}` |
89+
| `talos.imageFactoryCA` | Complete PEM certificate block(s) CDI must trust when importing this pool's worker OS disk image over HTTPS — the no-osImage default and `osImage.factory` path. Empty by default, so the importer uses only its system trust store, which is what the public factory needs. Set this when the factory in use (`imageFactoryURL` or `osImage.factory.imageFactoryURL`) is a self-hosted factory or mirror served by an internal CA, which the importer otherwise rejects with `x509: certificate signed by unknown authority`. The chart renders it into a ConfigMap in the release namespace — CDI requires certConfigMap to sit alongside the DataVolume — and references that from the worker DataVolume's `source.http.certConfigMap`; CDI appends it to the system pool rather than replacing it, so a publicly-trusted endpoint verifies either way. A pool on `osImage.builtin` clones a golden image and builds no HTTP source, so this field has no effect there. | `string` | `""` |
8990

9091

9192
### Substrate

‎packages/apps/kubernetes-nodes/templates/_helpers.tpl‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -402,6 +402,46 @@ dict the result is written into), groupName (named in every error message).
402402
{{- $_ := set .out "clone" $clone -}}
403403
{{- end -}}
404404

405+
{{/*
406+
Name of the ConfigMap carrying the CA bundle CDI verifies the Talos Image
407+
Factory against. Shared by the ConfigMap itself (templates/image-factory-ca.yaml)
408+
and the DataVolume that references it through source.http.certConfigMap
409+
(templates/nodegroup.yaml), so the two cannot drift.
410+
411+
Deliberately free of a content hash: CDI resolves certConfigMap by name at
412+
import time, so a stable name lets a CA rotation land by updating the object
413+
in place. A hashed name would instead require the DataVolume — and therefore
414+
the whole KubevirtMachineTemplate, which is immutable — to be rebuilt.
415+
*/}}
416+
{{- define "kubernetes-nodes.talosImageFactoryCAConfigMapName" -}}
417+
{{- printf "%s-talos-factory-ca" .Release.Name | trunc 63 | trimSuffix "-" }}
418+
{{- end }}
419+
420+
{{/*
421+
Validated .Values.talos.imageFactoryCA: renders the trimmed PEM when the value
422+
is set, and nothing at all when it is empty. Callers therefore gate on it with
423+
`if`, and an unset value leaves the render byte-for-byte as before — required,
424+
because the DataVolume source it joins is hashed into the
425+
KubevirtMachineTemplate name.
426+
427+
The PEM guards live in cozy-lib.tls.validateCACert: the value is emitted
428+
verbatim into a ConfigMap readable by anything that can read the namespace, so
429+
it must be complete certificate blocks and nothing else, and must never carry
430+
a private key.
431+
432+
Note what neither side does: a Helm template has no x509 parser, so a body
433+
spelled in base64 characters passes however meaningless. CDI reports such a
434+
bundle at import time — AppendCertsFromPEM simply adds nothing and verification
435+
still fails.
436+
*/}}
437+
{{- define "kubernetes-nodes.talosImageFactoryCA" -}}
438+
{{- $ca := printf "%v" (default "" .Values.talos.imageFactoryCA) -}}
439+
{{- if ne (trim $ca) "" -}}
440+
{{- include "cozy-lib.tls.validateCACert" (dict "caCert" $ca "field" "talos.imageFactoryCA") -}}
441+
{{- trim $ca -}}
442+
{{- end -}}
443+
{{- end }}
444+
405445
{{- /*
406446
Name of the cluster's default StorageClass, or the empty string when there is
407447
none (and always under `helm template`, which has no cluster to read).
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
{{- /*
2+
CA bundle CDI verifies the Talos Image Factory against while importing a
3+
worker's OS disk image.
4+
5+
Why this object exists: workers on the HTTP path (the no-osImage default and
6+
osImage.factory) boot from a raw disk image CDI streams from the pool's Image
7+
Factory. That endpoint is documented for self-hosted factories and mirrors in
8+
air-gapped or flaky-egress environments, but such an endpoint is typically
9+
served with a certificate from an internal CA — and the importer, trusting only
10+
its system store, refuses it with "x509: certificate signed by unknown
11+
authority". The DataVolume API has carried source.http.certConfigMap for
12+
exactly this since the beginning; this chart just had no way to fill it in.
13+
14+
Rendered only on the HTTP import path with the value set. A pool on
15+
osImage.builtin clones a golden image and builds no source.http, so it gets no
16+
ConfigMap even with the field set, and an unset value renders nothing at all.
17+
18+
Namespace is the release namespace by omission, which is what CDI requires:
19+
"secretRef and certConfigMap must be in the same namespace as the DataVolume".
20+
21+
The key name is arbitrary — CDI mounts the whole ConfigMap into the importer's
22+
cert directory and reads every file in it — so ca.pem follows the upstream
23+
example in kubevirt/containerized-data-importer manifests/example/cert-configmap.yaml.
24+
*/}}
25+
{{- $resolved := dict -}}
26+
{{- include "kubernetes-nodes.resolveOsImage" (dict
27+
"osImage" .Values.osImage
28+
"talos" .Values.talos
29+
"out" $resolved
30+
"groupName" (include "kubernetes-nodes.groupName" .)) -}}
31+
{{- if not $resolved.clone }}
32+
{{- with (include "kubernetes-nodes.talosImageFactoryCA" $) }}
33+
apiVersion: v1
34+
kind: ConfigMap
35+
metadata:
36+
name: {{ include "kubernetes-nodes.talosImageFactoryCAConfigMapName" $ }}
37+
labels:
38+
{{- include "kubernetes.labels" $ | nindent 4 }}
39+
data:
40+
ca.pem: |
41+
{{- . | nindent 4 }}
42+
{{- end }}
43+
{{- end }}

‎packages/apps/kubernetes-nodes/templates/nodegroup.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -485,6 +485,16 @@ spec:
485485
safe here. The clone path's name below is quoted because no
486486
existing worker renders it. */}}
487487
url: {{ $httpURL }}
488+
{{- /* Only when talos.imageFactoryCA is set: a self-hosted factory
489+
served with a certificate from an internal CA. CDI appends the
490+
bundle to the system pool instead of replacing it, so a
491+
publicly-trusted endpoint verifies either way. The ConfigMap
492+
is rendered by templates/image-factory-ca.yaml. certConfigMap
493+
is a field of source.http only, so the clone path above never
494+
carries it. */}}
495+
{{- if (include "kubernetes-nodes.talosImageFactoryCA" $) }}
496+
certConfigMap: {{ include "kubernetes-nodes.talosImageFactoryCAConfigMapName" $ }}
497+
{{- end }}
488498
{{- end }}
489499
storage:
490500
resources:

0 commit comments

Comments
 (0)