Skip to content

Commit 0dd28fc

Browse files
committed
fix(apply): mask the node's side of the dry-run diff
The dry-run diff the node returns prints the config it runs next to the one being applied. talm masked it by the secret values of the rendered config only, so when the two carried different secrets, as after a CA rotation or with a regenerated secrets.yaml, the node's CA keys, tokens and encryption secrets printed on the removed side in the clear, on every contract. On a dry run, the only apply whose response carries that diff, the apply paths now read the node's config before applying, as the drift preview already does, and mask its secret values too. A node on a maintenance connection has no config to read. A read or decode failure leaves that side unmasked with a warning but never fails the apply. A test runs a real Talos config diff between two configs with different secrets through the print path, on v1.13 and v1.14, and requires every field Talos redacts on either side to be masked. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 5ec4e98 commit 0dd28fc

4 files changed

Lines changed: 231 additions & 7 deletions

File tree

‎docs/manual-test-plan.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -857,7 +857,9 @@ talm apply --dry-run --show-secrets-in-drift -f nodes/node0.yaml 2>&1 | grep -cF
857857

858858
Expected: in the `Config diff:` block, a user secret renders as `value: ***` and a bootstrap `*.key` / `token` / encryption-secret renders as `key: ***` (etc.) by default. `--show-secrets-in-drift` prints the block verbatim (the same flag that governs the structured drift preview governs this surface too). A leak here is a security-class bug: `talm` prints the server-computed diff verbatim, so an unredacted `ModeDetails` would expose CA private keys and user secrets in CI logs.
859859

860-
Regression anchor: the `Config diff:` shows secrets as context lines too (an unchanged `key:` adjacent to a change hunk), not only on `+`/`-` lines — value-based masking covers both. Public material adjacent to a secret slice (an `acceptedCAs[].crt`) is redacted alongside its key by design (the slice is masked whole); `--show-secrets-in-drift` restores it.
860+
Both sides, on a project pinned at `talosVersion: "v1.14"`: keep a copy of `secrets.yaml`, regenerate it so the rendered secrets differ from the ones the node runs, and run `talm apply --dry-run -f nodes/node0.yaml 2>&1`. Expected: none of the `-` lines (the node's current keys) and none of the `+` lines (the regenerated ones) shows key material. The CA and service-account keys are multi-line PEM there, so every line between `-----BEGIN` and `-----END` in the `issuingCA.key` and `issuer.privateKey` blocks reads `***`, while the armor lines stay. If talm cannot read the node's config (a COSI timeout, an RPC error), the dry run prints `warning: the node's side of the dry-run diff is not masked` and the apply itself is unaffected. Restore the original `secrets.yaml` before applying anything.
861+
862+
Regression anchor: the `Config diff:` shows secrets as context lines too (an unchanged `key:` adjacent to a change hunk), not only on `+`/`-` lines — value-based masking covers both. `TestModeDetails_PrintedDiffMasksSecretsOnBothSides` runs a real Talos config diff between two configs with different secrets through the print path, on v1.13 and v1.14. Public material adjacent to a secret slice (an `acceptedCAs[].crt`) is redacted alongside its key by design (the slice is masked whole); `--show-secrets-in-drift` restores it.
861863

862864
### C6. Drift preview shows secrets with explicit opt-in
863865

‎pkg/commands/apply.go‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ import (
1818
"context"
1919
"fmt"
2020
"io"
21+
"maps"
2122
"os"
2223
"path/filepath"
2324
"slices"
@@ -486,6 +487,8 @@ func buildApplyClosure() applyFunc {
486487
return err
487488
}
488489

490+
nodeValues := nodeSecretValues(cosiCtx, cosiMachineConfigReader(c, applyCmdFlags.insecure), os.Stderr)
491+
489492
resp, err := c.ApplyConfiguration(ctx, &machineapi.ApplyConfigurationRequest{
490493
Data: data,
491494
Mode: applyCmdFlags.Mode.Mode,
@@ -496,7 +499,7 @@ func buildApplyClosure() applyFunc {
496499
return errors.Wrap(annotateApplyConfigError(err), "applying new configuration")
497500
}
498501

499-
if err := emitApplyResults(resp, data, true); err != nil {
502+
if err := emitApplyResults(resp, data, nodeValues, true); err != nil {
500503
return err
501504
}
502505

@@ -587,13 +590,17 @@ func applyOneFileDirectPatchMode(configFile, withSecretsPath string) error {
587590

588591
read := cosiVersionReader(c)
589592

593+
nodeValues := map[string]struct{}{}
594+
590595
for _, node := range targetNodes {
591596
nodeCtx := client.WithNode(ctx, node)
592597
preflightCheckTalosVersion(nodeCtx, read, applyCmdFlags.talosVersion, os.Stderr)
593598

594599
if err := runPreApplyGates(nodeCtx, c, result, node, os.Stderr, false); err != nil {
595600
return err
596601
}
602+
603+
maps.Copy(nodeValues, nodeSecretValues(nodeCtx, cosiMachineConfigReader(c, applyCmdFlags.insecure), os.Stderr))
597604
}
598605

599606
resp, err := c.ApplyConfiguration(ctx, &machineapi.ApplyConfigurationRequest{
@@ -614,7 +621,7 @@ func applyOneFileDirectPatchMode(configFile, withSecretsPath string) error {
614621
return errors.Wrap(annotateApplyConfigError(err), "applying new configuration")
615622
}
616623

617-
if err := emitApplyResults(resp, result, false); err != nil {
624+
if err := emitApplyResults(resp, result, nodeValues, false); err != nil {
618625
return err
619626
}
620627

‎pkg/commands/contract_drift_typed_secret_redaction_test.go‎

Lines changed: 166 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,15 @@ package commands
22

33
import (
44
"bytes"
5+
"context"
56
"fmt"
67
"io"
78
"strings"
89
"testing"
910

1011
"github.com/cockroachdb/errors"
12+
machineapi "github.com/siderolabs/talos/pkg/machinery/api/machine"
13+
"github.com/siderolabs/talos/pkg/machinery/config/configdiff"
1114
"github.com/siderolabs/talos/pkg/machinery/config/configloader"
1215
"github.com/siderolabs/talos/pkg/machinery/config/encoder"
1316
"github.com/siderolabs/talos/pkg/machinery/config/machine"
@@ -60,7 +63,15 @@ encryption:
6063
func v114ControlPlaneWithSecrets(t *testing.T) []byte {
6164
t.Helper()
6265

63-
configBundle, err := engine.InitializeConfigBundle(engine.Options{TalosVersion: "v1.14", KubernetesVersion: "v1.34.3"})
66+
return append(controlPlaneWithSecrets(t, "v1.14"), []byte(typedSecretDocs)...)
67+
}
68+
69+
// controlPlaneWithSecrets renders a control-plane config on the given
70+
// contract with its own freshly generated secrets.
71+
func controlPlaneWithSecrets(t *testing.T, talosVersion string) []byte {
72+
t.Helper()
73+
74+
configBundle, err := engine.InitializeConfigBundle(engine.Options{TalosVersion: talosVersion, KubernetesVersion: "v1.34.3"})
6475
if err != nil {
6576
t.Fatalf("InitializeConfigBundle: %v", err)
6677
}
@@ -70,7 +81,7 @@ func v114ControlPlaneWithSecrets(t *testing.T) []byte {
7081
t.Fatalf("Serialize: %v", err)
7182
}
7283

73-
return append(out, []byte(typedSecretDocs)...)
84+
return out
7485
}
7586

7687
// talosSecretLeaves returns every string leaf Talos itself masks in cfgBytes,
@@ -250,6 +261,159 @@ func TestModeDetails_EtcdEncryptionCollectsOnlySecretLeaves(t *testing.T) {
250261
}
251262
}
252263

264+
// The dry-run diff the node returns shows both its own config and the one
265+
// being applied. Every secret Talos redacts must be masked on both sides, as
266+
// printed: from v1.14 the CA and service-account keys are multi-line PEM,
267+
// which the diff splits into one prefixed line per line of the key.
268+
func TestModeDetails_PrintedDiffMasksSecretsOnBothSides(t *testing.T) {
269+
for _, talosVersion := range []string{"v1.13", "v1.14"} {
270+
t.Run(talosVersion, func(t *testing.T) {
271+
onNode := controlPlaneWithSecrets(t, talosVersion)
272+
rendered := controlPlaneWithSecrets(t, talosVersion)
273+
274+
diff := nodeDiff(t, onNode, rendered)
275+
276+
nodeValues, err := collectConfigSecretValues(onNode)
277+
if err != nil {
278+
t.Fatalf("collectConfigSecretValues: %v", err)
279+
}
280+
281+
values, err := modeDetailsSecretValues(rendered, nodeValues)
282+
if err != nil {
283+
t.Fatalf("modeDetailsSecretValues: %v", err)
284+
}
285+
286+
buf := &bytes.Buffer{}
287+
printApplyResultsRedacted(&machineapi.ApplyConfigurationResponse{
288+
Messages: []*machineapi.ApplyConfiguration{{ModeDetails: diff}},
289+
}, values, buf)
290+
printed := buf.String()
291+
292+
if !strings.Contains(printed, "@@") {
293+
t.Fatalf("the two configs produced no diff to check:\n%s", printed)
294+
}
295+
296+
for side, cfg := range map[string][]byte{"node": onNode, "rendered": rendered} {
297+
for kind, leaves := range talosSecretLeaves(t, cfg) {
298+
for path, value := range leaves {
299+
if line, leaked := leakedSecretLine(printed, value); leaked {
300+
t.Errorf("%s side: %s %s printed in the dry-run diff (%q)", side, kind, path, line)
301+
}
302+
}
303+
}
304+
}
305+
})
306+
}
307+
}
308+
309+
// The node's secrets are collected before the apply, so a node config talm
310+
// cannot read or decode costs the masking of that side, never the apply. A
311+
// dry run says so instead of leaking silently; a maintenance connection has
312+
// no config to read and stays quiet.
313+
func TestNodeSecretValues(t *testing.T) {
314+
origShow, origDryRun := applyCmdFlags.showSecretsInDrift, applyCmdFlags.dryRun
315+
t.Cleanup(func() { applyCmdFlags.showSecretsInDrift, applyCmdFlags.dryRun = origShow, origDryRun })
316+
317+
applyCmdFlags.dryRun = true
318+
319+
onNode := controlPlaneWithSecrets(t, "v1.14")
320+
321+
reader := func(data []byte, ok bool, err error) machineConfigReader {
322+
return func(context.Context) ([]byte, bool, error) { return data, ok, err }
323+
}
324+
325+
for _, tc := range []struct {
326+
name string
327+
show bool
328+
read machineConfigReader
329+
wantValues bool
330+
wantWarning bool
331+
}{
332+
{name: "read", read: reader(onNode, true, nil), wantValues: true},
333+
{name: "read error", read: reader(nil, false, errors.New("rpc timeout")), wantWarning: true},
334+
{name: "maintenance connection", read: reader(nil, false, nil)},
335+
{name: "undecodable config", read: reader([]byte("machine: [unterminated"), true, nil), wantWarning: true},
336+
{name: "show secrets", show: true, read: func(context.Context) ([]byte, bool, error) {
337+
t.Error("--show-secrets-in-drift must not read the node's config")
338+
339+
return nil, false, nil
340+
}},
341+
} {
342+
t.Run(tc.name, func(t *testing.T) {
343+
applyCmdFlags.showSecretsInDrift = tc.show
344+
345+
buf := &bytes.Buffer{}
346+
values := nodeSecretValues(context.Background(), tc.read, buf)
347+
348+
if got := len(values) > 0; got != tc.wantValues {
349+
t.Errorf("collected secrets = %v, want %v", got, tc.wantValues)
350+
}
351+
352+
if got := strings.Contains(buf.String(), "not masked"); got != tc.wantWarning {
353+
t.Errorf("warning printed = %v, want %v: %q", got, tc.wantWarning, buf.String())
354+
}
355+
})
356+
}
357+
358+
// Talos puts the config diff into the response of a dry run only, so a
359+
// real apply has nothing of the node's to mask and skips the read.
360+
t.Run("no read outside a dry run", func(t *testing.T) {
361+
applyCmdFlags.showSecretsInDrift, applyCmdFlags.dryRun = false, false
362+
363+
buf := &bytes.Buffer{}
364+
values := nodeSecretValues(context.Background(), func(context.Context) ([]byte, bool, error) {
365+
t.Error("a real apply must not read the node's config for redaction")
366+
367+
return onNode, true, nil
368+
}, buf)
369+
370+
if values != nil || buf.Len() != 0 {
371+
t.Errorf("a real apply collected %d values and printed %q", len(values), buf.String())
372+
}
373+
})
374+
}
375+
376+
func nodeDiff(t *testing.T, oldBytes, newBytes []byte) string {
377+
t.Helper()
378+
379+
oldCfg, err := configloader.NewFromBytes(oldBytes)
380+
if err != nil {
381+
t.Fatalf("loading node config: %v", err)
382+
}
383+
384+
newCfg, err := configloader.NewFromBytes(newBytes)
385+
if err != nil {
386+
t.Fatalf("loading rendered config: %v", err)
387+
}
388+
389+
diff, err := configdiff.DiffConfigs(oldCfg, newCfg)
390+
if err != nil {
391+
t.Fatalf("DiffConfigs: %v", err)
392+
}
393+
394+
return diff
395+
}
396+
397+
// leakedSecretLine reports a line of value that shows up in printed. A
398+
// multi-line value is checked line by line, as the diff prints it; the PEM
399+
// armor lines are not secret.
400+
func leakedSecretLine(printed, value string) (string, bool) {
401+
for _, line := range strings.Split(value, "\n") {
402+
line = strings.TrimSpace(line)
403+
if line == "" || strings.HasPrefix(line, "-----") {
404+
continue
405+
}
406+
407+
for _, out := range strings.Split(printed, "\n") {
408+
if strings.TrimSpace(strings.TrimLeft(out, "+- ")) == line || (len(line) >= 16 && strings.Contains(out, line)) {
409+
return line, true
410+
}
411+
}
412+
}
413+
414+
return "", false
415+
}
416+
253417
// The same field name is secret on one kind and harmless on another:
254418
// KubeEtcdEncryptionConfig.config holds the encryption keys,
255419
// KubeletConfig.config holds kubelet settings an operator needs to see.

‎pkg/commands/preflight_apply_safety_modedetails.go‎

Lines changed: 53 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,10 @@ package commands
1616

1717
import (
1818
"bytes"
19+
"context"
1920
"fmt"
2021
"io"
22+
"maps"
2123
"os"
2224
"sort"
2325
"strings"
@@ -272,6 +274,51 @@ func printApplyResultsRedacted(resp *machineapi.ApplyConfigurationResponse, valu
272274
}
273275
}
274276

277+
// nodeSecretValues reads the config the node runs and collects its secret
278+
// values, for masking the node's side of the dry-run diff. It runs before the
279+
// apply, so a node config talm cannot read or decode costs only that masking.
280+
// A maintenance connection has no config to read and contributes nothing;
281+
// any other failure on a dry run is reported to w, so the unmasked side is
282+
// not a silent leak.
283+
func nodeSecretValues(ctx context.Context, read machineConfigReader, w io.Writer) map[string]struct{} {
284+
// Talos puts the config diff into the response of a dry run only.
285+
if applyCmdFlags.showSecretsInDrift || !applyCmdFlags.dryRun {
286+
return nil
287+
}
288+
289+
current, ok, err := read(ctx)
290+
if err == nil && !ok {
291+
return nil
292+
}
293+
294+
var values map[string]struct{}
295+
if err == nil {
296+
values, err = collectConfigSecretValues(current)
297+
}
298+
299+
if err != nil {
300+
_, _ = fmt.Fprintf(w, "warning: the node's side of the dry-run diff is not masked, its config could not be read: %v\n", err)
301+
302+
return nil
303+
}
304+
305+
return values
306+
}
307+
308+
// modeDetailsSecretValues collects the secret values the node's dry-run diff
309+
// can show: the diff prints the node's own config next to the rendered one,
310+
// so with different secrets on the two sides both sets have to be masked.
311+
func modeDetailsSecretValues(rendered []byte, nodeValues map[string]struct{}) (map[string]struct{}, error) {
312+
values, err := collectConfigSecretValues(rendered)
313+
if err != nil {
314+
return nil, err
315+
}
316+
317+
maps.Copy(values, nodeValues)
318+
319+
return values, nil
320+
}
321+
275322
// emitApplyResults is the redacting replacement for the two
276323
// helpers.PrintApplyResults call sites on the apply paths. It collects the
277324
// secret value set the dry-run diff must hide — Talos bootstrap material from
@@ -283,14 +330,18 @@ func printApplyResultsRedacted(resp *machineapi.ApplyConfigurationResponse, valu
283330
// path feeds value files into the applied config, so only it can leak a user
284331
// secret through ModeDetails; the direct-patch path renders none, so collecting
285332
// them there would be pure overhead and would wrongly require a talm.key.
286-
func emitApplyResults(resp *machineapi.ApplyConfigurationResponse, rendered []byte, rendersUserValues bool) error {
333+
//
334+
// nodeValues are the secrets of the config the node runs before the apply,
335+
// collected by nodeSecretValues: the diff prints that config next to the
336+
// rendered one, so its secrets are masked too.
337+
func emitApplyResults(resp *machineapi.ApplyConfigurationResponse, rendered []byte, nodeValues map[string]struct{}, rendersUserValues bool) error {
287338
if applyCmdFlags.showSecretsInDrift {
288339
printApplyResultsRedacted(resp, nil, os.Stderr)
289340

290341
return nil
291342
}
292343

293-
values, err := collectConfigSecretValues(rendered)
344+
values, err := modeDetailsSecretValues(rendered, nodeValues)
294345
if err != nil {
295346
return err
296347
}

0 commit comments

Comments
 (0)