You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 125948a
Browse filesBrowse the repository at this point in the historyBrowse files
feat(apply): first -f file anchors project root; later files are patches
ValidateAndDetectRootsForFiles previously iterated EVERY -f file
and rejected the chain if any file lacked a Chart.yaml + secrets
ancestor OR resolved to a divergent project root. That gate
blocked the natural side-patch UX: 'talm apply -f nodes/n.yaml
-f /tmp/extra.yaml' should layer /tmp/extra.yaml on top of the
project anchored by nodes/n.yaml, but the gate refused the chain
because /tmp/extra.yaml has no project root.
Switch to first-file-anchors semantics: only filePaths[0] is
required to resolve a project root, and that root is returned
as the anchor. Subsequent files skip detection entirely and are
loaded as patches downstream. Reversing the chain (orphan first,
rooted second) is rejected with a hint pointing at the FIRST file
and instructing the operator to reorder, not to move the file.
Single-file orphans continue to error out identically to pre-
change behaviour.
Existing tests that pinned the strict-consistency rule
(_DifferentRootsError, _OrphanFileError, _FilesInDifferentRootsError)
are rewritten as positive contracts under the new semantics; the
new tests cover the side-patch acceptance, the reorder-hint
rejection, and the no-regression single-orphan case.
Assisted-By: Claude <[email protected]>
Signed-off-by: Aleksei Sviridkin <[email protected]>
Copy file name to clipboardExpand all lines: README.md
+26-1Lines changed: 26 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,7 +14,7 @@ While developing Talm, we aimed to achieve the following goals:
14
14
15
15
-**GitOps Friendly**: The patches generated do not contain sensitive data, allowing them to be stored in Git in an unencrypted, open format. For scenarios requiring complete configurations, the `--full` option allows the obtain a complete config that can be used for matchbox and other solutions.
16
16
17
-
-**Simplicity of Use**: You no longer need to pass connection options for each specific server; they are saved along with the templating results into a separate file. This allows you to easily apply one or multiple files in batch using a syntax similar to `kubectl apply -f node1.yaml -f node2.yaml`.
17
+
-**Simplicity of Use**: You no longer need to pass connection options for each specific server; they are saved along with the templating results into a separate file. Per-node configuration sits in a single modeline-annotated `nodes/<name>.yaml`; the first `-f` file is the anchor and any subsequent `-f` files stack onto its rendered config as side-patches (see "Apply with side-patches" below).
18
18
19
19
-**Compatibility with talosctl**: We strive to maintain compatibility with the upstream project in patches and configurations. The configurations you obtain can be used with the official tools like talosctl and Omni.
> `talm template -f node.yaml` (with or without `-I`) does **not** apply the same overlay: its output is the rendered template plus the modeline and the auto-generated warning, byte-identical to what the template alone would produce. Routing it through the patcher would drop every YAML comment (including the modeline) and re-sort keys, breaking downstream commands that read the file back. Use `apply --dry-run` if you want to preview the exact bytes that will be sent to the node.
218
218
219
+
## Apply with side-patches
220
+
221
+
`talm apply -f`accepts a chain of files. The FIRST `-f` is the **anchor** — it must carry a `# talm: nodes=[…], templates=[…]` modeline and live under a `talm init`'d project (Chart.yaml + secrets.yaml). Any subsequent `-f` files are **side-patches**: they are merged in order on top of the anchor's rendered config, and a single `ApplyConfiguration` is issued per node carrying the composed result.
222
+
223
+
```bash
224
+
# Single node file (anchor only):
225
+
talm apply -f nodes/cp01.yaml
226
+
227
+
# Side-patch stacked on top — useful for one-shot overlays
Side-patches do not need to live under the project root; the first file anchors detection. Reversing the order is an error — the first file must be the rooted anchor.
233
+
234
+
If you were relying on the previous "apply N independent node files" shape (`talm apply -f n1.yaml -f n2.yaml -f n3.yaml` triggering three separate applies), invoke `talm apply` once per node file instead:
235
+
236
+
```bash
237
+
for n in nodes/*.yaml; do talm apply -f "$n"; done
238
+
```
239
+
240
+
The current shape rejects this misuse loudly: if any of the subsequent `-f` files carries its own `# talm: …` modeline, the apply errors out before any RPC fires with a hint pointing at the shell-loop pattern above. Stripping the modeline turns a file into a legitimate side-patch.
241
+
242
+
Side-patches with a non-empty body are restricted to **single-node anchors**. The same body cannot be distinguished from per-node fields (hostname, address, VIP) versus cluster-wide knobs (NTP servers, KubeProxy mode) by static inspection, and stamping per-node fields across N machines is the original foot-gun the per-node-body guard was designed to prevent. If your anchor's `nodes=[…]` lists more than one target and your side-patch is non-empty, talm rejects the apply early with a hint pointing at the per-file shell loop. For cluster-wide overlays on multi-node anchors, fold the overlay into `values.yaml` or templates rather than passing it as a side-patch; for per-node overrides, generate per-node files via `talm template -I` and feed them into the per-file shell loop.
243
+
219
244
## Using talosctl commands
220
245
221
246
Talm offers a similar set of commands to those provided by talosctl. However, you can specify the --file option for them.
Expected: single-wrapped error with hint. Inner cause `preset not found in Chart.yaml dependencies` is surfaced verbatim with hint `add a preset chart (e.g. cozystack) to Chart.yaml dependencies or pass --preset`. No outer rewrap line like `preset is required: use --preset flag or ensure Chart.yaml has a preset dependency` — the rewrap was double-messaging the same condition before the fix.
115
+
116
+
Regression anchor: `errors.GetAllHints(err)` (or `2>&1 | grep "^hint:"`) MUST still surface the inner `add a preset chart` hint after the unwrap. A regression that drops the hint along with the rewrap is a UX regression.
117
+
101
118
**Regression anchor**: A6's error must reference the missing-key path explicitly. A bare `read key file: open ...: no such file or directory` with no follow-up hint is a UX regression — the operator should see a clear recovery path (`run \`talm init\` to generate a new key, or restore your backup`).
Expected: `Updated.` on stdout. The diff shows that the rendered body replaces the previous contents — **including any user-added comments**. This is by design (`-I` is rewrite, not merge), but operators often trip on it; note in your test report.
157
+
Expected: `Updated.` on stdout. The rendered body replaces the previous body of the file — but **operator-authored comments above the modeline are preserved verbatim** (post-#178 behavior). Comments embedded in the YAML body still get overwritten, since `-I` re-renders the body and Helm has no way to round-trip user-edits made there.
158
+
159
+
Regression anchor: write `nodes/node0.yaml` as `# Operator note A\n# Operator note B\n# talm: ...\n<body>`. After `template -I -f nodes/node0.yaml`, the first two lines (`# Operator note A`, `# Operator note B`) MUST still be there, followed by the modeline, the talm-rendered warning header, then the body. Re-run idempotent: a second `template -I` keeps the same prefix structure — leading comments don't drift, multiply, or disappear.
141
160
142
161
### B5. Render with stale chart preset
143
162
@@ -205,6 +224,77 @@ Expected: same drift preview as C5, but the secret paths render verbatim — no
205
224
206
225
Regression anchor: `--show-secrets-in-drift` is operator opt-in, never default. Verify by running `talm apply --help` and confirming the flag default is `false`.
207
226
227
+
### C6a. Apply chain — first `-f` anchors project root, later `-f` files stack as side-patches
228
+
229
+
```bash
230
+
cat > /tmp/side-ntp.yaml <<'EOF'
231
+
machine:
232
+
time:
233
+
servers:
234
+
- 2.example.test
235
+
EOF
236
+
/tmp/talm-safety apply --dry-run \
237
+
-f nodes/node0.yaml -f /tmp/side-ntp.yaml
238
+
```
239
+
240
+
Expected: progress line includes `side-patches=[/tmp/side-ntp.yaml]` alongside `nodes=[…]` and `endpoints=[…]`. Drift preview shows both the anchor's rendered config diff AND the side-patch's `machine.time.servers: added [2.example.test]` mutation. The side-patch is stacked via `engine.MergeFileAsPatch`; last writer wins on overlapping keys.
241
+
242
+
Regression anchor: `/tmp/side-ntp.yaml` lives outside the project root. The first `-f` file (`nodes/node0.yaml`) is the SOLE anchor for root detection; later files don't get root-validated. Verify by reversing order — `apply --dry-run -f /tmp/side-ntp.yaml -f nodes/node0.yaml` should fail because the first file has no project root.
243
+
244
+
### C6b. Non-modelined anchor with side-patches rejected with reorder hint
Expected: error `first -f file nodes/orphan.yaml lacks a modeline; side-patches require a modelined anchor` with hint that the first `-f` file must carry `# talm: nodes=[…], templates=[…]` so talm knows what to render before stacking. Note: the orphan file lives inside the project tree because the first `-f` file must have a Chart.yaml + secrets.yaml ancestor for root detection — see C6b-pre below for the root-detection failure mode.
254
+
255
+
### C6b-pre. Anchor outside any project root is rejected at root detection
Expected: error `failed to detect project root for first file /tmp/orphan.yaml (Chart.yaml and secrets.yaml not found)` with hint `the first -f file anchors the project root; place it inside a talm init'd project, or reorder the -f chain so a rooted file comes first`. This gate fires BEFORE the modeline-vs-orphan dispatch, so any orphan placed outside a project root can never reach the direct-patch path.
265
+
266
+
### C6c. Orphan file inside project root dispatches to direct-patch when context provides nodes
267
+
268
+
```bash
269
+
# Build a talosconfig with explicit nodes field
270
+
cp talosconfig /tmp/talosconfig-with-nodes
271
+
# (edit to add nodes: [...] under contexts.<active>.nodes — YAML or yq)
272
+
/tmp/talm-safety apply --dry-run \
273
+
--talosconfig /tmp/talosconfig-with-nodes \
274
+
-f nodes/orphan.yaml
275
+
```
276
+
277
+
Expected: progress line `- talm: file=nodes/orphan.yaml, nodes=[…], endpoints=[…]` — targets resolved from the talosconfig context, no early "no nodes available" rejection. The dispatch reaches the direct-patch path (chart-render-with-orphan-as-patch). Note: the orphan must live inside the project tree to clear root detection (see C6b-pre).
278
+
279
+
### C6d. Orphan file with no source for nodes fails late with three-way hint
280
+
281
+
```bash
282
+
# Same talosconfig but WITHOUT a nodes: field in the active context
283
+
/tmp/talm-safety apply --dry-run \
284
+
--talosconfig /tmp/talosconfig-no-nodes \
285
+
-f nodes/orphan.yaml
286
+
```
287
+
288
+
Expected: `no nodes to target for nodes/orphan.yaml` with hint `the file lacks a # talm: nodes=[…] modeline, no --nodes flag was passed, and the active talosconfig context carries no nodes; pass --nodes explicitly or supply a modelined node file`. The hint must name all three sources so the operator knows which to populate.
289
+
290
+
Regression anchor: the "no nodes" check fires AFTER the talosconfig context fallback runs, not before. A regression that re-introduces the early `len(GlobalArgs.Nodes) == 0` gate would block C6c silently.
291
+
292
+
### C6e. Progress lines use comma-joined bracketed lists
293
+
294
+
Inspect any apply progress line from C1, C3, C6a, C6c. Expect `nodes=[n1,n2,n3]` and `endpoints=[e1,e2]` — comma-joined, bracket-framed, no `[n1 n2 n3]` Go slice format artifact.
295
+
296
+
Regression anchor: the format uses `strings.Join(slice, ",")` with surrounding brackets in the format string; `%s` or `%v` directly on a `[]string` produces the space-separated `[n1 n2 n3]` artifact and is a regression. Verify with a multi-node modeline (`# talm: nodes=["1.2.3.4","5.6.7.8"], ...`) → progress line shows `nodes=[1.2.3.4,5.6.7.8]`.
297
+
208
298
### C7. Phase 1 walker rejects malformed net-addr fields before the RPC
209
299
210
300
When a rendered MachineConfig carries a malformed value in any of the new walker-covered fields, Phase 1 must block before the apply RPC fires:
@@ -505,6 +595,25 @@ Expected: refuses with `FailedPrecondition: blockdevice "<dev>" is in use by dis
Then exercise each target. Each expectation below is a forward-looking check the operator runs interactively (or via `__complete <subcommand> <flag-value-or-positional> ""` for scripted assertions).
608
+
609
+
-`talm init --preset <TAB>` → curated list including `cozystack`. Sourced from `pkg/generated/presets.go::AvailablePresets()`; no generic file completion.
-`talm apply --file <TAB>` → only `nodes/*.yaml` files that carry a valid `# talm: …` modeline. Non-modelined yaml files in the project tree do NOT surface. Same for `talm template --file <TAB>` and `talm upgrade --file <TAB>`.
-`talm --nodes <TAB>` / `talm --endpoints <TAB>` → union of nodes / endpoints declared across every context in the active talosconfig.
614
+
615
+
Regression anchor: positional completion (`talm apply <TAB>` without a flag) MUST NOT surface anything — apply / template / upgrade declare `cobra.NoArgs` so the positional slot is empty. A regression that wires `ValidArgsFunction` to a populated completer would silently re-introduce a dead-completion code path.
616
+
508
617
### I1. Generate completion for each shell
509
618
510
619
```bash
@@ -531,6 +640,20 @@ Expected: with the post-#163 chart, fails fast with `talm: floatingIP "0700" is
531
640
532
641
**Operator footgun**: `--set floatingIP=198.51.100.1`*may* be parsed as the float `198.51 × 100.1` by Helm's loose type-coercion. For IPs use `--set-string floatingIP="198.51.100.1"` or put it in `values.yaml`.
533
642
643
+
Post-#182, `--set` emits a non-fatal warning to stderr when the RHS matches an IP-shaped or semver-shaped literal:
# Expected: no warning emitted — operator opt-in to verbatim string.
650
+
```
651
+
652
+
Regression anchors:
653
+
654
+
- Warning fires for IPv4 (`10.0.0.1`, `192.0.2.1/24`) and v-prefixed semver (`v1.2.3`, `v1.2`). Bare two-component decimals (`1.5`, `2.0`) and colon-separated literals (`00:11:22:33:44:55`) do NOT trigger — those aren't strvals-coerced footguns.
655
+
- Warning is non-fatal — command still proceeds. A regression that turns it into a hard error would block legitimate `--set` flows.
Expected: `error parsing JSON array for key nodes` with a hint about the expected syntax.
916
+
Expected: `error parsing JSON array for key nodes` with a hint about the expected syntax. The error surfaces the modeline parse step — the file is NOT silently routed to direct-patch mode where the malformed modeline would be invisible.
917
+
918
+
### M1a. Orphan file (no modeline) inside project root — classified as orphan, not malformed
Expected: dispatch into direct-patch mode — progress line `- talm: file=nodes/_orphan-commented.yaml, nodes=[$NODE], endpoints=[$NODE]` followed by drift preview. NO "parsing modeline" error, NO "non-comment line found before modeline" error. The classifier (`FindAndParseModeline`) must distinguish "no `# talm:` line anywhere" (orphan, → `ErrModelineNotFound`) from "modeline present but parse failed" (malformed).
928
+
929
+
Note: the orphan must live inside the project tree (e.g. under `nodes/`) so root detection finds the project's Chart.yaml / secrets.yaml. An orphan placed at `/tmp/` is rejected at root detection before the modeline classifier runs — see C6b-pre for that path.
930
+
931
+
### M1b. Misplaced modeline (modeline below YAML) — rejected with a specific hint
Expected: error `modeline found below non-comment content: first non-comment line was "machine:"` with hint `the # talm: … modeline must precede any YAML content; only #-prefixed comments and blank lines may sit above it`. Distinct from both M1 (malformed value) and M1a (orphan).
944
+
945
+
Regression anchor 1: the classifier lookahead must scan the entire file for a `# talm:` line before deciding orphan vs misplaced. A regression that returns `ErrModelineNotFound` on first non-comment encounter (without lookahead) would silently route misplaced-modeline files into direct-patch mode, hiding the operator error.
946
+
947
+
Regression anchor 2: the lookahead is column-0-strict. An indented `# talm:` line inside a YAML body (e.g. ` # talm: see docs for nodes/templates wiring`) is operator-authored documentation, NOT a misplaced modeline; it MUST NOT trip the misplaced-modeline error. Verify by writing a node file with ` # talm: comment` indented under a YAML key and confirming the file routes to orphan / direct-patch mode rather than failing on misplaced-modeline.
0 commit comments