Skip to content

Commit e052727

Browse files
committed
feat(charts): expose Talos operator extension points in cozystack + generic presets
Operators wanting to extend the Talos machine config rendered by the cozystack or generic presets had to fork the chart or hand-edit the generated per-node YAML against the autogen banner asking them to keep template edits in values. Add four "extra*" values keys that plug into the load-bearing sections of machine.* — kernel modules, sysctls, kubelet.extraConfig, machine.files — without overriding the preset's hardcoded defaults. Cozystack preset (append / merge to existing built-ins): extraKernelModules [] append to openvswitch / drbd / zfs / spl / vfio_pci / vfio_iommu_type1 extraKubeletExtraArgs {} merge into cpuManagerPolicy + maxPods; last-key-wins on conflict (escape hatch) extraSysctls {} merge into gc_thresh* + nr_hugepages extraMachineFiles [] append to CRI customization + lvm.conf Generic preset (emit only when set): extraKernelModules [] emits machine.kernel.modules block extraKubeletExtraArgs {} emits kubelet.extraConfig block extraSysctls {} emits machine.sysctls block extraMachineFiles [] emits machine.files block Contract tests under pkg/engine pin append vs merge semantics for every key on cozystack, on / off state on generic, and the default generic render still emits no machine.kernel / sysctls / files / extraConfig blocks (the existing NoCozystackOpinionsOnGeneric guard stays green). Manual-test-plan B7 / B8 cover both presets with forward-looking yq queries against the rendered output. Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 8d81fc5 commit e052727

6 files changed

Lines changed: 600 additions & 0 deletions

File tree

‎charts/cozystack/templates/_helpers.tpl‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,16 +39,41 @@ machine:
3939
- {{ . }}
4040
{{- end }}
4141
{{- end }}
42+
{{- /* extraKubeletExtraArgs MUST NOT collide with the preset's
43+
built-in extraConfig keys — yaml.v3 (used by Talos config
44+
decode and by the upgrade-time body write-back) rejects
45+
duplicate map keys, so a silent merge would emit a config
46+
that cannot decode. Fail at render with a precise hint
47+
naming the offending key; operators wanting a different
48+
default fork the preset. */ -}}
49+
{{- range $k, $_ := .Values.extraKubeletExtraArgs }}
50+
{{- if or (eq $k "cpuManagerPolicy") (eq $k "maxPods") }}
51+
{{- fail (printf "values.yaml: extraKubeletExtraArgs.%s collides with the cozystack preset's built-in kubelet.extraConfig — keys never override (yaml.v3 rejects duplicate map keys on decode). Remove the entry from extraKubeletExtraArgs, or fork the chart preset if you need a different default." $k) }}
52+
{{- end }}
53+
{{- end }}
4254
extraConfig:
4355
cpuManagerPolicy: static
4456
maxPods: 512
57+
{{- with .Values.extraKubeletExtraArgs }}
58+
{{- toYaml . | nindent 6 }}
59+
{{- end }}
60+
{{- /* extraSysctls MUST NOT collide with the preset's built-in
61+
sysctls; same rationale as extraKubeletExtraArgs. */ -}}
62+
{{- range $k, $_ := .Values.extraSysctls }}
63+
{{- if or (eq $k "vm.nr_hugepages") (eq $k "net.ipv4.neigh.default.gc_thresh1") (eq $k "net.ipv4.neigh.default.gc_thresh2") (eq $k "net.ipv4.neigh.default.gc_thresh3") }}
64+
{{- fail (printf "values.yaml: extraSysctls.%s collides with the cozystack preset's built-in machine.sysctls — keys never override (yaml.v3 rejects duplicate map keys on decode). Remove the entry from extraSysctls, or fork the chart preset if you need a different default." $k) }}
65+
{{- end }}
66+
{{- end }}
4567
sysctls:
4668
{{- with $.Values.nr_hugepages }}
4769
vm.nr_hugepages: {{ . | quote }}
4870
{{- end }}
4971
net.ipv4.neigh.default.gc_thresh1: "4096"
5072
net.ipv4.neigh.default.gc_thresh2: "8192"
5173
net.ipv4.neigh.default.gc_thresh3: "16384"
74+
{{- with .Values.extraSysctls }}
75+
{{- toYaml . | nindent 4 }}
76+
{{- end }}
5277
kernel:
5378
modules:
5479
- name: openvswitch
@@ -59,6 +84,9 @@ machine:
5984
- name: spl
6085
- name: vfio_pci
6186
- name: vfio_iommu_type1
87+
{{- with .Values.extraKernelModules }}
88+
{{- toYaml . | nindent 4 }}
89+
{{- end }}
6290
certSANs:
6391
- 127.0.0.1
6492
{{- with .Values.certSANs }}
@@ -84,6 +112,9 @@ machine:
84112
devices {
85113
global_filter = [ "r|^/dev/drbd.*|", "r|^/dev/dm-.*|", "r|^/dev/zd.*|" ]
86114
}
115+
{{- with .Values.extraMachineFiles }}
116+
{{- toYaml . | nindent 2 }}
117+
{{- end }}
87118
install:
88119
{{- with .Values.image }}
89120
image: {{ . }}

‎charts/cozystack/values.yaml‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,3 +102,62 @@ oidcIssuerUrl: ""
102102
certSANs: []
103103
nr_hugepages: 0
104104
allocateNodeCIDRs: true
105+
106+
# Operator-supplied extension points: each `extra*` key ADDS to a
107+
# load-bearing default the cozystack preset ships. For every key
108+
# below: leaving it at the empty default is a no-op; the preset's
109+
# hardcoded set continues to render unchanged. Keys never override
110+
# the preset's built-ins — if a map-shaped `extra*` value contains a
111+
# key that collides with a built-in, the chart fails at render time
112+
# with a guidance message pointing at the offending key. yaml.v3
113+
# rejects duplicate map keys on decode anyway, so a silent override
114+
# would produce a Talos config that cannot round-trip. Operators who
115+
# need different defaults fork the preset.
116+
117+
# Extra kernel modules to load on every node, appended to the preset's
118+
# built-in list (openvswitch, drbd, zfs, spl, vfio_pci, vfio_iommu_type1).
119+
# Lists append unconditionally — duplicates by `name` are tolerated by
120+
# Talos at load time. Each entry is a Talos kernel module spec; see
121+
# Talos docs for the full shape. Examples:
122+
# extraKernelModules:
123+
# - name: nf_conntrack
124+
# - name: foo
125+
# parameters:
126+
# - bar=baz
127+
extraKernelModules: []
128+
129+
# Extra kubelet args added to the preset's kubelet.extraConfig
130+
# (cpuManagerPolicy: static, maxPods: 512). Operator keys must be
131+
# DISJOINT from the built-in set; a collision fails the render with
132+
# a hint pointing at the offending key. Example:
133+
# extraKubeletExtraArgs:
134+
# feature-gates: "NodeSwap=true"
135+
extraKubeletExtraArgs: {}
136+
137+
# Extra kernel sysctls added to machine.sysctls alongside the
138+
# preset's built-ins (net.ipv4.neigh.default.gc_thresh{1,2,3} and
139+
# vm.nr_hugepages — the latter is gated by the dedicated `nr_hugepages`
140+
# values key above; set THAT key, not extraSysctls.vm.nr_hugepages,
141+
# even when the gate is currently inactive — the collision check
142+
# treats vm.nr_hugepages as preset-owned regardless of the gate state).
143+
# Operator keys must be DISJOINT from the built-in set; a collision
144+
# fails the render. Values must be YAML strings (Talos expects
145+
# strings even for numeric sysctls). Example:
146+
# extraSysctls:
147+
# net.core.somaxconn: "65535"
148+
# fs.inotify.max_user_watches: "524288"
149+
extraSysctls: {}
150+
151+
# Extra machine.files entries appended to the preset's built-in
152+
# files (CRI customization and lvm.conf). Lists append; Talos
153+
# rejects duplicate `path:` entries at apply time, so operator
154+
# entries must not name a path the preset already writes. Each
155+
# entry is a Talos machine.files spec (content / path / permissions
156+
# / op). Example:
157+
# extraMachineFiles:
158+
# - path: /etc/cri/conf.d/30-extra.part
159+
# op: create
160+
# content: |
161+
# [debug]
162+
# level = "info"
163+
extraMachineFiles: []

‎charts/generic/templates/_helpers.tpl‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,27 @@ machine:
3434
- {{ . }}
3535
{{- end }}
3636
{{- end }}
37+
{{- with .Values.extraKubeletExtraArgs }}
38+
extraConfig:
39+
{{- toYaml . | nindent 6 }}
40+
{{- end }}
3741
{{- with .Values.certSANs }}
3842
certSANs:
3943
{{- toYaml . | nindent 2 }}
4044
{{- end }}
45+
{{- with .Values.extraSysctls }}
46+
sysctls:
47+
{{- toYaml . | nindent 4 }}
48+
{{- end }}
49+
{{- with .Values.extraKernelModules }}
50+
kernel:
51+
modules:
52+
{{- toYaml . | nindent 6 }}
53+
{{- end }}
54+
{{- with .Values.extraMachineFiles }}
55+
files:
56+
{{- toYaml . | nindent 2 }}
57+
{{- end }}
4158
install:
4259
{{- (include "talm.discovered.disks_info" .) | nindent 4 }}
4360
disk: {{ include "talm.discovered.system_disk_name" . | quote }}

‎charts/generic/values.yaml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,3 +87,40 @@ serviceSubnets:
8787
advertisedSubnets: []
8888

8989
certSANs: []
90+
91+
# Operator-supplied extension points. The generic preset ships no
92+
# defaults for any of the keys below, so each rendered block appears
93+
# only when the corresponding value is non-empty. Keep parity with the
94+
# cozystack preset's `extra*` keys so a values file is portable
95+
# between presets.
96+
97+
# Kernel modules to load on every node. Each entry is a Talos kernel
98+
# module spec — see Talos docs for the full shape. Example:
99+
# extraKernelModules:
100+
# - name: nf_conntrack
101+
# - name: foo
102+
# parameters:
103+
# - bar=baz
104+
extraKernelModules: []
105+
106+
# Kubelet command-line args, rendered as machine.kubelet.extraConfig.
107+
# Talos expects string values. Example:
108+
# extraKubeletExtraArgs:
109+
# maxPods: "256"
110+
# feature-gates: "NodeSwap=true"
111+
extraKubeletExtraArgs: {}
112+
113+
# Kernel sysctls, rendered as machine.sysctls. Values must be YAML
114+
# strings (Talos expects strings even for numeric sysctls). Example:
115+
# extraSysctls:
116+
# net.core.somaxconn: "65535"
117+
extraSysctls: {}
118+
119+
# Machine files to drop on every node. Each entry is a Talos
120+
# machine.files spec (content / path / permissions / op). Example:
121+
# extraMachineFiles:
122+
# - path: /etc/example.conf
123+
# op: create
124+
# content: |
125+
# hello = "world"
126+
extraMachineFiles: []

‎docs/manual-test-plan.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,76 @@ Expected: both renders print "OK" — no `address: 169.254.…` lines anywhere i
246246

247247
Regression anchor: a regression that re-introduces the v1.11-only `host`-scope filter would let link-local addresses leak into the legacy `machine.network.interfaces[].addresses` block on a future COSI schema bump that emitted them on the default-gateway-bearing link. Pinned by `TestContract_NetworkLegacy_DefaultAddressesFilterLinkScope` with a `linkScopedAddressOnDefaultGatewayLookup` fixture, but the live render is still useful as a sanity check against real cluster discovery output.
248248

249+
### B7. `extra*` operator extension points (cozystack: append to defaults)
250+
251+
The cozystack preset ships hardcoded defaults for kernel modules, sysctls, kubelet extraConfig, and machine.files. The matching `extra*` values keys let an operator extend each block without forking the chart. Set every key, re-render, and inspect each affected section:
252+
253+
```yaml
254+
# values.yaml additions:
255+
extraKernelModules:
256+
- name: nf_conntrack
257+
extraKubeletExtraArgs:
258+
feature-gates: "NodeSwap=true"
259+
extraSysctls:
260+
net.core.somaxconn: "65535"
261+
extraMachineFiles:
262+
- path: /etc/example/operator.conf
263+
op: create
264+
content: |
265+
hello = "world"
266+
```
267+
268+
```bash
269+
talm template -f nodes/node0.yaml > /tmp/render.yaml
270+
yq '.machine.kernel.modules' /tmp/render.yaml
271+
yq '.machine.kubelet.extraConfig' /tmp/render.yaml
272+
yq '.machine.sysctls' /tmp/render.yaml
273+
yq '.machine.files[].path' /tmp/render.yaml
274+
```
275+
276+
Expected:
277+
278+
- `.machine.kernel.modules` lists the built-in six (`openvswitch`, `drbd` with `usermode_helper=disabled`, `zfs`, `spl`, `vfio_pci`, `vfio_iommu_type1`) AND `nf_conntrack` — append, never override.
279+
- `.machine.kubelet.extraConfig` carries the built-ins (`cpuManagerPolicy: static`, `maxPods: 512`) AND `feature-gates: NodeSwap=true`. Operator keys MUST NOT collide with built-ins; a collision fails the render at template time.
280+
- `.machine.sysctls` carries the built-in `gc_thresh1/2/3` and `vm.nr_hugepages` (when set) AND `net.core.somaxconn`. Same rejection-on-collision rule.
281+
- `.machine.files[].path` lists `/etc/cri/conf.d/20-customization.part`, `/etc/lvm/lvm.conf`, AND `/etc/example/operator.conf`.
282+
283+
Verify the rejection path explicitly. Set an operator key that collides with a built-in (e.g. `extraSysctls: { "net.ipv4.neigh.default.gc_thresh1": "9000" }`) and re-render:
284+
285+
```bash
286+
talm template -f nodes/node0.yaml 2>&1 | grep -E 'extraSysctls\.net\.ipv4\.neigh\.default\.gc_thresh1 collides'
287+
```
288+
289+
Expected: render fails with `Error: values.yaml: extraSysctls.net.ipv4.neigh.default.gc_thresh1 collides with the cozystack preset's built-in machine.sysctls — keys never override (yaml.v3 rejects duplicate map keys on decode). Remove the entry from extraSysctls, or fork the chart preset if you need a different default.` Same shape for `extraKubeletExtraArgs.maxPods` etc. The "fork the preset" escape hatch is the only supported path to change a built-in default.
290+
291+
Regression anchor: contract tests `TestContract_Machine_Extra{KernelModules,KubeletExtraArgs,Sysctls,MachineFiles}_Cozystack_*` pin append / merge semantics for each section. The `_RejectsCollisionWithBuiltin` tests pin the failure mode for every built-in key. A regression that swaps the chart-level `fail` for a silent `toYaml | nindent` would render duplicate map keys; the matching `_MergeInto*` test round-trip-decodes through yaml.v3 (which refuses duplicates), and the `_RejectsCollisionWithBuiltin` cases would also fail.
292+
293+
### B8. `extra*` operator extension points (generic: emit-only-when-set)
294+
295+
The generic preset ships no defaults for the same four blocks. Each `machine.*` block appears in the render ONLY when its `extra*` values key is non-empty.
296+
297+
With all `extra*` keys at their empty defaults:
298+
299+
```bash
300+
talm template -f nodes/node0.yaml > /tmp/render.yaml
301+
for k in kernel sysctls files; do
302+
if [ "$(yq ".machine.$k" /tmp/render.yaml)" = "null" ]; then
303+
echo "OK: no $k block"
304+
else
305+
echo "FAIL: $k block leaked into default generic render"
306+
fi
307+
done
308+
if [ "$(yq '.machine.kubelet.extraConfig' /tmp/render.yaml)" = "null" ]; then
309+
echo "OK: no extraConfig"
310+
else
311+
echo "FAIL: extraConfig leaked"
312+
fi
313+
```
314+
315+
Expected: all four prints "OK". Set any single `extra*` key non-empty, re-render — only the matching block appears; the other three stay absent. The `yq ... = "null"` form is exact-match: `yq` emits the literal string `null` on a single line for an absent path, so a leaking empty block (`modules: []`, `extraConfig: {}`) shows up as a non-`null` value and trips the FAIL branch.
316+
317+
Regression anchor: contract tests `TestContract_Machine_Extra*_Generic_NonEmptyEmitsBlock` pin the on-state for each block; `TestContract_Machine_NoCozystackOpinionsOnGeneric` pins the off-state at default values. A regression that emits an empty `modules: []` / `sysctls: {}` / `files: []` block in the default render would fail the latter.
318+
249319
## C. Apply (auth path)
250320

251321
This section is the smoke-test for the apply pipe itself; the per-gate matrix lives in **Section C-safety** below.

0 commit comments

Comments
 (0)