Platform-wide defaults for tenant Talos worker settings (imageFactoryURL + registryMirrors) #3950

Activity

1. IvanHunters

   added

   kind/featureCategorizes issue or PR as related to a new featureCategorizes issue or PR as related to a new feature

   area/kubernetesIssues or PRs related to the tenant Kubernetes appIssues or PRs related to the tenant Kubernetes app

   on Aug 24, 2026
2. github-actions

   added

   triage/needs-triageIndicates an issue needs triage by a maintainerIndicates an issue needs triage by a maintainer

   on Aug 24, 2026
3. IvanHunters

   changed the title [-]Platform-wide default for tenant Kubernetes worker image source (imageFactoryURL / registryMirrors)[/-] [+]Platform-wide defaults for tenant Talos worker settings (imageFactoryURL + registryMirrors)[/+] on Aug 24, 2026
4. lexfrei commented on Sep 1, 2026

   

   Aleksei Sviridkin (lexfrei)

   on Sep 1, 2026

   Contributor

   More actions

   Worth fixing the precedence chain before anyone implements this, because the failure mode is silent.

   per-CR > _cluster > built-in default cannot be written as a plain default chain. packages/apps/kubernetes-nodes/values.yaml has imageFactoryURL: "https://factory.talos.dev" on line 126 and installerRepository: "factory.talos.dev/installer" on line 127, both non-empty chart defaults. Something like .Values.talos.imageFactoryURL | default $cluster.imageFactoryURL then resolves to the chart default in every render, because Helm cannot tell "the operator did not set this" from "the operator set it to the same string as the default". The platform default never wins, the render is clean, and nothing reports an error anywhere. registryMirrors on line 128 is {}, so that knob is the one of the three that needs no special handling.

   Two ways out. Empty those two keys in values.yaml and move the public factory literal into the template as the last fallback, or use an explicit sentinel. Either works.

   Second thing, cheap and in the same area: the consumer belongs in kubernetes-nodes, not the parent kubernetes chart. All three knobs are read there, nodegroup.yaml:89 for the factory URL. The parent chart reads only .Values.talos.version, at cluster.yaml:21-25. The file citations in the issue point at the parent and predate the split.
5. myasnikovdaniil

   self-assigned this

   on Sep 2, 2026
6. SerjioTT commented on Sep 7, 2026

   

   SerjioTT

   on Sep 7, 2026

   Contributor

   More actions

   Operator case for the record, since the issue says it came from one and we are in the same position: our clusters cannot reach the public Image Factory at all. The mirror is up and a golden image is imported, and a newly created KubernetesNodes still goes to factory.talos.dev unless whoever created it set the field by hand.

   A third channel has appeared since this was filed. #3294 adds osImage on KubernetesNodes, where osImage.builtin clones a golden from an opt-in catalog. The catalog side got a platform knob in that PR — kubernetesWorkerImage forwarded into the Package component values — so an operator can point the single import at a mirror through platform configuration. The consuming side did not: osImage defaults to {} and nodegroup.yaml reads no _cluster key for the image source. The list of channels here is now three, and the third has the same shape as the first two.

   The proposed direction has a consequence worth settling before anyone implements it, and it is larger than the precedence problem Aleksei Sviridkin (@lexfrei) raised above. imageFactoryURL is not only a value the render reads: packages/apps/kubernetes-nodes/templates/nodegroup.yaml:89 puts it into the DataVolume URL inside the kubevirtmachinetemplate define, and line 477 hashes that whole define into the KubevirtMachineTemplate name. A _cluster fallback consumed at render time therefore changes the resolved URL of every pool that relied on the built-in default, the moment an operator first sets the platform value, which re-hashes the KMT and rolls every worker in the fleet. myasnikovdaniil reached the same conclusion independently in the #3294 thread, and noted that fixing the precedence trap is what makes the hazard reachable rather than what avoids it.

   That is the exact invariant #3294 is built around, and the one its approving review verified by rendering both sides byte-identical. The two efforts are in tension on this point today.

   registryMirrors does not have this property, since it reaches the guest through the machine config rather than through the disk template. So the two channels need different answers even though the issue treats them as one shape.

   The alternative that keeps the no-roll invariant by construction is to apply the default when a resource is created rather than when it is rendered: a platform-level setting stamped into the spec of a new KubernetesNodes, never applied to one that already exists. Stored values of existing releases stay untouched, so nothing re-renders and nothing rolls, while a new pool arrives with the operator's choice already in its spec, as an ordinary field from then on. #3956 is opening the seam for it: it hooks normalizeSpecNulls into Create and Update in rest.go, so a create-only defaulting step has somewhere to sit.

   The tradeoff should be stated rather than discovered, because #3956's own description names it: writing a value into the spec pins the application to it, so a later change to the default never reaches a pool that already has one. For a fleet default that is the intended behaviour rather than a defect — an operator who repoints a mirror wants new pools to follow and existing ones to stay put until they are deliberately re-imaged, which is the same bargain the per-CR field already offers. It does mean the platform value is a default for new resources and not a fleet-wide setting, and the docs should say so in those words.

   One shape that looks like this and is not: a default in the values schema. Defaults are applied on read in cozystack-api (applySpecDefaults, called from ConvertHelmReleaseToApplication) rather than written into the HelmRelease, so a schema default surfaces on every existing pool at once and gets persisted by the first kubectl apply or dashboard edit that follows — the same fleet roll, only delayed and unpredictable in timing.
7. myasnikovdaniil

   added a commit that references this issue on Sep 7, 2026

   feat(kubernetes): boot tenant worker disks from a shared golden Talos…

   ...

   3748352
8. myasnikovdaniil commented on Sep 8, 2026

   

   myasnikovdaniil

   on Sep 8, 2026

   Contributor

   More actions

   Three knobs in this issue are not one class, and the proposed direction is safe for two of them and a fleet roll for the third.

   registryMirrors and installerRepository are read only in kubernetes-nodes/templates/talos-reconcile-job.yaml (:338 and :332), and that Job applies a TalosConfigTemplate with a stable name, kubernetes-<cluster>-<group>, which is what MachineDeployment.bootstrap.configRef points at. An unchanged render re-applies a byte-identical TCT, a CAPI no-op, nothing rolls, and the comment at :33-35 says exactly that. So a platform default is safe for these two. It lands in a worker at bootstrap though, so already running nodes keep the config they were created with until they are replaced.

   imageFactoryURL is the different one. It renders into url at nodegroup.yaml:89, inside the block hashed at :477, and MachineDeployment.infrastructureRef uses that hashed name (:80). The day an operator sets the platform default, every pool that never set the field per-CR gets a new KubevirtMachineTemplate name and rolls its workers.

   This chains with Aleksei Sviridkin (@lexfrei)'s comment above: fixing the precedence trap is what activates the roll. Today a plain default chain would silently never win, so nothing happens at all, and the moment precedence works the first render moves the hash. Order matters here, not only correctness.

   For mirrors this is not a new operator knob. The platform already has registries.mirrors (packages/core/platform/values.yaml:573-600) in the same host to { endpoints: [ ... ] } shape, and its only consumer containerd-registry-secret.yaml is dead, which is #4006. Restoring reach of that value and deleting the dead renderer are one change: doing either alone leaves the operator setting lost or keeps rendering a Secret nobody reads.

   Separate from the missing default, and worse in a way: the parent kubernetes chart still exposes all three fields in values.yaml:255-259, the schema, the README and the dashboard form, while no template there reads them after the split, only talos.version (cluster.yaml:21). An operator sets a mirror on the Kubernetes CR and gets silence, no error anywhere. Probably its own issue.

   For imageFactoryURL the only shape that gives a platform default without rolling is stamping the value into the spec at create, as SerjioTT put it on #3294. Defaults are applied on read only today, the write path stores the spec verbatim, and #3956 already hooks Create and Update in rest.go.

   registries.config (overridePath, auth, #3331) I have not decided: either map it into the Talos shape or carry endpoints only on the first step.
9. Aleksei Sviridkin (lexfrei)

   added

   triage/acceptedIndicates an issue is ready to be actively worked onIndicates an issue is ready to be actively worked on

   priority/important-longtermImportant over the long term, but may not be staffed and/or may need multiple releases to completeImportant over the long term, but may not be staffed and/or may need multiple releases to complete

   and removed

   triage/needs-triageIndicates an issue needs triage by a maintainerIndicates an issue needs triage by a maintainer

   on Sep 23, 2026
10. myasnikovdaniil

    added a commit that references this issue on Sep 24, 2026

    feat(kubernetes): boot tenant worker disks from a shared golden Talos…

    ...

    9bec214