Repository navigation
Commit b507cb5
committed
feat(kubernetes): allow a per-node-group Talos schematic
A Talos schematic is a fixed set of system extensions baked into one
image, and Talos refuses to finish booting when an extension service in
it cannot start. ext-nvidia-persistenced and ext-nvidia-cdi-gen require
an NVIDIA card, so a node with no GPU that boots an NVIDIA schematic
fails startAllServices and reboots, indefinitely.
talos.schematicID was cluster-wide, so a cluster mixing GPU and non-GPU
node groups had no correct value: the GPU group needs the extensions and
every other group is broken by them. Setting the NVIDIA schematic for the
sake of the GPU group put every other group into a reboot cycle of about
70 minutes per node. Nothing reports it — kubelet starts before the
failing phase, so the node holds Ready and only the pod restart counters,
identical across a node, betray it. Seen in production on 1.6.0.
Add an optional per-node-group schematicID falling back to
talos.schematicID, so the NVIDIA schematic can be scoped to the group
that has the cards. Both consumers resolve it through one helper: the
boot disk image the DataVolume pulls, and the installer image in the
TalosConfigTemplate. They have to agree, or an in-place Talos upgrade
swaps a node's extension set out from under it.
Deliberately not derived from gpus, unlike kernelModules in the previous
commit: a schematic ID is an opaque image-factory digest, so the chart
cannot know which one carries the NVIDIA extensions or synthesise one.
Unset renders byte-identically to before, so the content-hash-named
KubevirtMachineTemplate keeps its name and no worker is rolled. Setting
it does roll that group's workers, which is inherent — changing a node's
boot image means replacing the node.
Refs: cozystack#3563
Assisted-By: Claude <[email protected]>
Signed-off-by: Mattia Eleuteri <[email protected]>1 parent 41f4cdc commit b507cb5
19 files changed
Lines changed: 508 additions & 60 deletions
File tree
- api/apps/v1alpha1
- kubernetesnodes
- kubernetes
- packages
- apps
- kubernetes-nodes
- templates
- tests
- kubernetes
- templates
- talos
- tests
- system
- kubernetes-nodes-rd/cozyrds
- kubernetes-rd/cozyrds
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
110 | 154 | | |
111 | 155 | | |
112 | 156 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| 44 | + | |
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
| |||
82 | 83 | | |
83 | 84 | | |
84 | 85 | | |
85 | | - | |
| 86 | + | |
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
332 | 332 | | |
333 | 333 | | |
334 | 334 | | |
335 | | - | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
336 | 340 | | |
337 | 341 | | |
338 | 342 | | |
| |||
400 | 404 | | |
401 | 405 | | |
402 | 406 | | |
| 407 | + | |
403 | 408 | | |
404 | 409 | | |
405 | 410 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
229 | 229 | | |
230 | 230 | | |
231 | 231 | | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
232 | 252 | | |
233 | | - | |
| 253 | + | |
234 | 254 | | |
235 | 255 | | |
236 | 256 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
111 | 116 | | |
112 | 117 | | |
113 | 118 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
62 | 65 | | |
63 | 66 | | |
64 | 67 | | |
| |||
0 commit comments