Skip to content

Commit c3d7ace

Browse files
authored
Merge pull request #212 from cozystack/fix/stdout-stderr-and-lookup-diagnostics
fix(cli, engine): emit progress on stderr, enrich lookup error chain, retry transient failures
2 parents bf869e4 + cc0239d commit c3d7ace

10 files changed

Lines changed: 1878 additions & 22 deletions

‎docs/manual-test-plan.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1502,6 +1502,112 @@ Expected: per-entry findings count exactly, valid forms produce zero findings, a
15021502

15031503
Regression anchor: the no-overlap unit test `TestMultidocNetAddrHandlers_NoOverlapWithRefHandlers` pins the dispatch-map disjointness contract. A future entry that lands in BOTH `multidocHandlers` and `multidocNetAddrHandlers` produces double findings — verify via the unit suite before manual smokes.
15041504

1505+
## N. Output discipline & error diagnostics
1506+
1507+
These scenarios pin two operator-facing contracts: (1) stdout stays clean enough to be piped into another file without filtering progress text out; (2) when a `lookup` against the live Talos API fails, the error chain names the dialed endpoint, the resource kind, and a class-specific remedy.
1508+
1509+
### N.1 stdout cleanliness — `talm template --file X > Y`
1510+
1511+
```bash
1512+
cd $PROJECT
1513+
talm template --offline -f nodes/node0.yaml > /tmp/rendered.yaml
1514+
head -1 /tmp/rendered.yaml
1515+
# Expected: the first line is `# talm: nodes=[...]` (rendered modeline comment), NOT `- talm: file=...`.
1516+
# A leading `- talm:` would mean the per-file progress line is leaking into stdout.
1517+
1518+
# Same redirect without --offline against a reachable node:
1519+
talm template -f nodes/node0.yaml > /tmp/rendered.yaml
1520+
head -1 /tmp/rendered.yaml
1521+
# Expected: same `# talm: nodes=[...]` first line; progress is on stderr only.
1522+
1523+
# Confirm progress lands on stderr:
1524+
talm template --offline -f nodes/node0.yaml 1>/dev/null 2>/tmp/stderr.txt
1525+
grep '^- talm: file=' /tmp/stderr.txt
1526+
# Expected: one matching line per --file argument; stderr is the only channel that carries it.
1527+
```
1528+
1529+
Run the same shape against `talm apply --dry-run -f nodes/node0.yaml > /tmp/rendered.yaml`. Note that `talm apply` writes nothing to stdout in normal mode (only `--debug` emits the recipe stream); the assertion here is **defensive** — verifying that nothing new starts leaking onto stdout in the future, not filtering an existing stream. The progress line must still go to stderr; assert stdout is empty and stderr carries `- talm: file=...`.
1530+
1531+
### N.2 Lookup error diagnostics — node unreachable
1532+
1533+
Run with `--endpoints` pointing at an address that is reachable on TCP but does not run Talos (or a local closed port), and WITHOUT `--offline`:
1534+
1535+
```bash
1536+
talm template -f nodes/node0.yaml --endpoints 192.0.2.123 || true
1537+
# Expected error chain MUST contain:
1538+
# - `looking up resource kind="disks" namespace="" id="" on endpoints=[192.0.2.123]`
1539+
# - a class hint mentioning the dialed endpoint (192.0.2.123)
1540+
# - one of the remedies depending on class:
1541+
# * TLS handshake failure → mentions "cert SANs" / "maintenance mode"
1542+
# * connection refused → mentions "firewall" / "wrong port"
1543+
# * timeout → mentions "timed out"
1544+
# * authn rejection → mentions "talosconfig"
1545+
# - a closing sentence pointing at `--offline` as the safe escape ("If live discovery is not required, pass --offline").
1546+
```
1547+
1548+
Repeat with `talm apply --dry-run` against the same unreachable endpoint:
1549+
1550+
```bash
1551+
talm apply --dry-run -f nodes/node0.yaml --endpoints 192.0.2.123 || true
1552+
# Expected: same class hint shape, but the remedy clause MUST say
1553+
# "Fix node reachability before re-running apply." and MUST NOT mention `--offline`
1554+
# (apply has no --offline flag — suggesting it would teach a non-existent workflow).
1555+
```
1556+
1557+
### N.3 Lookup retry policy — transient connectivity
1558+
1559+
Bring up a node, then drop its API briefly mid-template (e.g. `systemctl stop apid` for ~300ms on a Talos test node, or use `iptables -A INPUT -p tcp --dport 50000 -j DROP` for half a second on a non-prod target). The expectation is that `talm template -f nodes/node0.yaml` succeeds on the second or third attempt without the operator seeing an error — retry budget is 3 attempts with 200ms/400ms exponential backoff (~600ms total).
1560+
1561+
```bash
1562+
# In one shell:
1563+
talm template -f nodes/node0.yaml > /tmp/rendered.yaml
1564+
1565+
# In another (within ~300ms of the first):
1566+
# Briefly block the talos API:
1567+
ssh ${NODE} iptables -A INPUT -p tcp --dport 50000 -j DROP
1568+
sleep 0.3
1569+
ssh ${NODE} iptables -D INPUT -p tcp --dport 50000 -j DROP
1570+
```
1571+
1572+
Expected: template render succeeds after retry (1 to 2 retried attempts visible only if `--debug` is on); operator sees the rendered output, no error message.
1573+
1574+
Same shape against a node that stays down longer than the budget. Use `REJECT --reject-with tcp-reset` rather than `-j DROP` — DROP silently discards SYN packets, the talos client retries the TCP handshake until its own deadline expires, and the lookup classifies as deadline (timeout) instead of refused. REJECT sends an RST so the kernel surfaces ECONNREFUSED immediately, matching what an operator typically sees from a firewall-blocked or stopped-service node:
1575+
1576+
```bash
1577+
ssh ${NODE} iptables -A INPUT -p tcp --dport 50000 -j REJECT --reject-with tcp-reset
1578+
talm template -f nodes/node0.yaml || true
1579+
ssh ${NODE} iptables -D INPUT -p tcp --dport 50000 -j REJECT --reject-with tcp-reset
1580+
```
1581+
1582+
Expected: failure after ~600ms+ (3 attempts × backoff); operator sees the `connection refused` class hint pointing at firewall / port / `--offline`.
1583+
1584+
(If you intentionally want the deadline-class path instead, swap REJECT for `-j DROP` — the expected hint then mentions `timed out` rather than `connection refused`.)
1585+
1586+
Ctrl+C should interrupt retries well before the full retry budget:
1587+
1588+
```bash
1589+
ssh ${NODE} iptables -A INPUT -p tcp --dport 50000 -j REJECT --reject-with tcp-reset
1590+
talm template -f nodes/node0.yaml &
1591+
sleep 0.1
1592+
kill -INT $!
1593+
ssh ${NODE} iptables -D INPUT -p tcp --dport 50000 -j REJECT --reject-with tcp-reset
1594+
```
1595+
1596+
Expected: `talm` exits well under 1s (the cancellation interrupts the inter-attempt backoff sleep immediately) rather than the full ~600ms retry budget. A hard sub-100ms bound is too brittle across ssh hops and busy hosts — anything visibly faster than the full budget proves the cancellation path is wired correctly.
1597+
1598+
### N.4 Lookup error diagnostics — non-network failure
1599+
1600+
Forge a chart that calls `lookup` for an unknown resource kind (e.g. add `{{- (lookup "nonexistent_resource_kind" "" "").items }}` to a helper template), then run `talm template -f nodes/node0.yaml` against a reachable Talos node:
1601+
1602+
```bash
1603+
talm template -f nodes/node0.yaml || true
1604+
# Expected error chain MUST contain:
1605+
# - `looking up resource kind="nonexistent_resource_kind" ...`
1606+
# - hint mentioning "chart bug" / "unsupported Talos resource"
1607+
# - `talosctl get nonexistent_resource_kind` as the verification step
1608+
# - MUST NOT suggest `--offline` (would mask the chart bug by rendering against empty discovery).
1609+
```
1610+
15051611
## Sanity-check block
15061612

15071613
Run after every destructive section (E, F, H, and anything that touches `--mode=reboot` / `--mode=staged` / `apply -I`):

‎pkg/commands/apply.go‎

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,12 @@ import (
4242
const parentDir = ".."
4343

4444
// applyCommandName labels this subcommand inside engine.Options for
45-
// FailIfMultiNodes error wording. Centralised so the template
46-
// rendering options block and any test asserting against the field
47-
// share a single canonical value.
48-
const applyCommandName = "talm apply"
45+
// FailIfMultiNodes error wording and for the engine's apply-vs-template
46+
// hint branching. Aliased to engine.CommandNameApply so the engine
47+
// constant remains the single source of truth — drift between the
48+
// two would let apply error hints silently start suggesting the
49+
// non-existent --offline flag.
50+
const applyCommandName = engine.CommandNameApply
4951

5052
// roleAnchor and roleSidePatch label the role of a config file in
5153
// the apply chain for operator-facing error messages
@@ -417,12 +419,16 @@ func applyOneFileTemplateMode(configFile string, sidePatches, modelineTemplates
417419
// dominant invocation shape is `talm apply -f nodes/<name>.yaml`
418420
// without side-patches; reporting `side-patches=[]` on every line
419421
// was visible noise without operator value.
422+
// Progress line goes to stderr. Apply normally writes nothing to
423+
// stdout (only `--debug` emits the recipe stream via
424+
// engine.debugPhase); routing progress here matches the
425+
// stdout-cleanliness contract already in effect for `talm template
426+
// > file.yaml` and keeps the `--debug` recipe stream
427+
// uncontaminated.
420428
if len(sidePatches) == 0 {
421-
//nolint:forbidigo // CLI progress line surfaces the file-to-target mapping for the operator
422-
fmt.Printf("- talm: file=%s, nodes=[%s], endpoints=[%s]\n", configFile, strings.Join(nodes, ","), strings.Join(GlobalArgs.Endpoints, ","))
429+
fmt.Fprintf(os.Stderr, "- talm: file=%s, nodes=[%s], endpoints=[%s]\n", configFile, strings.Join(nodes, ","), strings.Join(GlobalArgs.Endpoints, ","))
423430
} else {
424-
//nolint:forbidigo // CLI progress line surfaces the file-to-target mapping for the operator
425-
fmt.Printf("- talm: file=%s, side-patches=[%s], nodes=[%s], endpoints=[%s]\n", configFile, strings.Join(sidePatches, ","), strings.Join(nodes, ","), strings.Join(GlobalArgs.Endpoints, ","))
431+
fmt.Fprintf(os.Stderr, "- talm: file=%s, side-patches=[%s], nodes=[%s], endpoints=[%s]\n", configFile, strings.Join(sidePatches, ","), strings.Join(nodes, ","), strings.Join(GlobalArgs.Endpoints, ","))
426432
}
427433

428434
applyClosure := buildApplyClosure()
@@ -567,8 +573,8 @@ func applyOneFileDirectPatchMode(configFile, withSecretsPath string) error {
567573
return err
568574
}
569575

570-
//nolint:forbidigo // CLI progress line surfaces the file-to-target mapping for the operator
571-
fmt.Printf("- talm: file=%s, nodes=[%s], endpoints=[%s]\n", configFile, strings.Join(targetNodes, ","), strings.Join(GlobalArgs.Endpoints, ","))
576+
// Progress line goes to stderr; stdout is reserved for rendered output.
577+
fmt.Fprintf(os.Stderr, "- talm: file=%s, nodes=[%s], endpoints=[%s]\n", configFile, strings.Join(targetNodes, ","), strings.Join(GlobalArgs.Endpoints, ","))
572578

573579
read := cosiVersionReader(c)
574580

@@ -1010,6 +1016,7 @@ func buildApplyRenderOptions(modelineTemplates []string, withSecretsPath string)
10101016
Root: Config.RootDir,
10111017
TemplateFiles: resolvedTemplates,
10121018
CommandName: applyCommandName,
1019+
TalosEndpoints: append([]string(nil), GlobalArgs.Endpoints...),
10131020
}
10141021
}
10151022

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
// Copyright Cozystack Authors
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
// Contract: applyCommandName is the exact engine.CommandNameApply
16+
// value. Drift between the two would cause apply error hints to
17+
// silently start suggesting the non-existent `--offline` flag — the
18+
// apply hint formatter in pkg/engine/lookup_classify.go branches on
19+
// equality with engine.CommandNameApply to omit the `--offline`
20+
// remedy clause.
21+
//
22+
// `applyCommandName` is declared as `= engine.CommandNameApply` in
23+
// pkg/commands/apply.go so a Go compile error catches drift at the
24+
// declaration site already. This test is the belt-and-suspenders
25+
// safety net: a future refactor that re-introduces a literal would
26+
// fail here too.
27+
28+
package commands
29+
30+
import (
31+
"testing"
32+
33+
"github.com/cozystack/talm/pkg/engine"
34+
)
35+
36+
func TestContract_ApplyCommandName_MatchesEngineConstant(t *testing.T) {
37+
if applyCommandName != engine.CommandNameApply {
38+
t.Errorf("applyCommandName=%q drifted from engine.CommandNameApply=%q — apply hints will silently suggest --offline",
39+
applyCommandName, engine.CommandNameApply)
40+
}
41+
}

0 commit comments

Comments
 (0)