Skip to content

Commit 80aa305

Browse files
committed
fix(age): make RotateKeys atomic via backup+restore (closes #159)
Address branch-review blocker: the previous fix for #151 introduced a data-loss window between os.Remove(talm.key) and the final WriteFile of the re-encrypted secrets. Any crash in that window left the operator with an encrypted secrets file paired against either a missing key (unrecoverable) or a fresh-but-empty key (which cannot decrypt the still-old ciphertext). New shape: Phase 1 — read encrypted file, decrypt with old key, all in mem. Phase 2 — generate new identity, encrypt new ciphertext, all in mem. Phase 3 — atomic-rename originals aside as *.rotation-backup. Phase 4 — write new key + encrypted file via secureperm.WriteFile (atomic temp-then-rename, mode 0o600). Any failure here triggers a restore() helper that puts the backups back and returns an error citing the failure stage. Phase 5 — remove backups on success. Refuse to start if leftover backups from a previous interrupted run are present — the operator must inspect them before another rotation runs, otherwise the recovery state would be silently overwritten. Adjacent fix in the same function: secrets.encrypted.yaml is now written via secureperm.WriteFile so it lands at mode 0o600 (was 0o644 — pre-existing world-readable, defense-in-depth). Extracted formatKeyFile() helper so RotateKeys produces the same canonical age keygen layout as GenerateKey. New tests in pkg/age/contract_test.go: - BothFilesMode0600 — both files end up 0o600 after rotation. - NoLeftoverBackups — successful rotation removes both backup files, no clutter. - RefusesOnLeftoverBackup — second rotation refuses with a precise error pointing at the leftover path; the leftover is preserved for manual inspection. Existing ReplacesKeyAndPreservesPlaintext continues to pin the core invariant (oldPub != newPub AND plaintext round-trips). Closes #159 in addition to #151. Also picks up the docstring follow-up on normalizeEndpoint — adds IPv6 examples to the function comment so the contract is visible in godoc, not just in the implementation comment. Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent c68d5c7 commit 80aa305

3 files changed

Lines changed: 202 additions & 44 deletions

File tree

‎pkg/age/age.go‎

Lines changed: 106 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -59,21 +59,27 @@ func GenerateKey(rootDir string) (*age.X25519Identity, bool, error) {
5959
return nil, false, fmt.Errorf("failed to generate age identity: %w", err)
6060
}
6161

62-
publicKey := identity.Recipient().String()
63-
64-
// Format key file in age keygen format
65-
now := time.Now()
66-
keyData := fmt.Sprintf("# created: %s\n", now.Format(time.RFC3339))
67-
keyData += fmt.Sprintf("# public key: %s\n", publicKey)
68-
keyData += identity.String() + "\n"
69-
70-
if err := secureperm.WriteFile(keyFile, []byte(keyData)); err != nil {
62+
if err := secureperm.WriteFile(keyFile, []byte(formatKeyFile(identity, time.Now()))); err != nil {
7163
return nil, false, fmt.Errorf("failed to write key file: %w", err)
7264
}
7365

7466
return identity, true, nil
7567
}
7668

69+
// formatKeyFile renders the canonical age keygen layout: a creation
70+
// timestamp comment, a public key comment, and the AGE-SECRET-KEY-1
71+
// secret line, each terminated by a newline. Extracted from
72+
// GenerateKey so RotateKeys can produce the same layout for the
73+
// new identity it generates in memory.
74+
func formatKeyFile(identity *age.X25519Identity, now time.Time) string {
75+
return fmt.Sprintf(
76+
"# created: %s\n# public key: %s\n%s\n",
77+
now.Format(time.RFC3339),
78+
identity.Recipient().String(),
79+
identity.String(),
80+
)
81+
}
82+
7783
// LoadKey loads age identity from talm.key file
7884
// Supports both age keygen format (with comments) and plain format
7985
func LoadKey(rootDir string) (*age.X25519Identity, error) {
@@ -482,64 +488,125 @@ func decryptString(encryptedBase64 string, identity *age.X25519Identity) (string
482488
}
483489

484490
// RotateKeys rotates encryption keys in secrets.encrypted.yaml
491+
// RotateKeys atomically rotates the age key encrypting
492+
// secrets.encrypted.yaml. The old key is replaced with a freshly
493+
// generated identity, and the secrets file is re-encrypted under
494+
// the new key.
495+
//
496+
// Atomicity strategy: every disk-mutating step uses os.Rename or
497+
// secureperm.WriteFile (which is itself atomic temp+rename). The
498+
// previous key+encrypted pair is renamed aside into
499+
// `*.rotation-backup` files BEFORE the new files are committed; if
500+
// any later step fails the originals are restored. The function
501+
// only returns nil after the new pair is committed AND the
502+
// backup files have been removed. Operators who find leftover
503+
// `*.rotation-backup` files after an interrupted run can safely
504+
// rename them back into place — the new files will not be in the
505+
// way (they were never committed).
506+
//
507+
// The function uses the secureperm.WriteFile helper for both the
508+
// new key file and the new encrypted secrets file so both end up
509+
// at mode 0o600 (defense-in-depth — age encryption is the security
510+
// layer, but world-readable secrets material on shared workstations
511+
// invites mistakes).
485512
func RotateKeys(rootDir string) error {
486-
// Load old key first (before retiring it). The old identity is
487-
// kept in memory only — once the talm.key file is removed below,
488-
// the key material exists only here for the duration of the
489-
// re-encrypt pass, then goes out of scope.
513+
keyFile := filepath.Join(rootDir, keyFileName)
514+
encryptedFile := filepath.Join(rootDir, encryptedSecretsFile)
515+
keyBackup := keyFile + ".rotation-backup"
516+
encryptedBackup := encryptedFile + ".rotation-backup"
517+
518+
// Refuse to start if a previous rotation was interrupted: the
519+
// operator must inspect the leftover files and decide what to
520+
// keep before another rotation runs. Otherwise this run would
521+
// silently overwrite the recovery state.
522+
for _, p := range []string{keyBackup, encryptedBackup} {
523+
if _, err := os.Stat(p); err == nil {
524+
return fmt.Errorf("found leftover rotation backup %q from a previous interrupted run; inspect and remove (or restore) before retrying", p)
525+
}
526+
}
527+
528+
// Phase 1: read and decrypt with old key, all in memory.
490529
oldIdentity, err := LoadKey(rootDir)
491530
if err != nil {
492531
return fmt.Errorf("failed to load old key: %w", err)
493532
}
494-
495-
// Decrypt with old key
496-
encryptedFile := filepath.Join(rootDir, encryptedSecretsFile)
497-
498533
encryptedData, err := os.ReadFile(encryptedFile)
499534
if err != nil {
500535
return fmt.Errorf("failed to read encrypted file: %w", err)
501536
}
502-
503537
var encryptedSecrets map[string]any
504538
if err := yaml.Unmarshal(encryptedData, &encryptedSecrets); err != nil {
505539
return fmt.Errorf("failed to parse encrypted YAML: %w", err)
506540
}
507-
508-
// Decrypt values with old key
509541
decryptedSecrets, err := decryptYAMLValues(encryptedSecrets, oldIdentity)
510542
if err != nil {
511543
return fmt.Errorf("failed to decrypt with old key: %w", err)
512544
}
513545

514-
// Retire the old key: GenerateKey is a load-or-create helper, so
515-
// without removing talm.key first it would re-load the old
516-
// identity instead of generating a fresh one — and "rotation"
517-
// would become a no-op that leaves the same key encrypting every
518-
// secret. Remove it first, then generate so a new identity is
519-
// guaranteed to land on disk.
520-
keyFile := filepath.Join(rootDir, keyFileName)
521-
if err := os.Remove(keyFile); err != nil {
522-
return fmt.Errorf("failed to retire old key: %w", err)
523-
}
524-
525-
newIdentity, _, err := GenerateKey(rootDir)
546+
// Phase 2: generate new identity and encrypt new ciphertext —
547+
// still all in memory. No disk mutation yet.
548+
newIdentity, err := age.GenerateX25519Identity()
526549
if err != nil {
527-
return fmt.Errorf("failed to generate new key: %w", err)
550+
return fmt.Errorf("failed to generate new identity: %w", err)
528551
}
529-
530-
// Encrypt with new key
531552
encryptedSecretsNew, err := encryptYAMLValues(decryptedSecrets, newIdentity.Recipient())
532553
if err != nil {
533554
return fmt.Errorf("failed to encrypt with new key: %w", err)
534555
}
535-
536556
encryptedDataNew, err := yaml.Marshal(encryptedSecretsNew)
537557
if err != nil {
538-
return fmt.Errorf("failed to marshal encrypted secrets: %w", err)
558+
return fmt.Errorf("failed to marshal new encrypted secrets: %w", err)
559+
}
560+
561+
// Phase 3: move originals aside as backups. Atomic rename, so
562+
// either both originals exist as `*.rotation-backup` after this
563+
// block or neither move took effect (for the second rename: we
564+
// undo the first if it errors).
565+
if err := os.Rename(keyFile, keyBackup); err != nil {
566+
return fmt.Errorf("failed to back up key file before rotation: %w", err)
567+
}
568+
if err := os.Rename(encryptedFile, encryptedBackup); err != nil {
569+
// Roll back the key rename so the project is untouched.
570+
_ = os.Rename(keyBackup, keyFile)
571+
return fmt.Errorf("failed to back up encrypted file before rotation: %w", err)
572+
}
573+
574+
// restore is a recovery helper used when any later step fails:
575+
// rename the backups back into place and return the original
576+
// error wrapped with a recovery note. We never silently swallow
577+
// the restore's own error — if recovery fails the operator
578+
// needs to know.
579+
restore := func(stage string, cause error) error {
580+
_ = os.Remove(keyFile) // best-effort: remove half-written new key if any
581+
_ = os.Remove(encryptedFile) // ditto for encrypted file
582+
errKey := os.Rename(keyBackup, keyFile)
583+
errEnc := os.Rename(encryptedBackup, encryptedFile)
584+
if errKey != nil || errEnc != nil {
585+
return fmt.Errorf("rotation failed at %s: %w; AND restore from backup partially failed (key: %v, encrypted: %v) — manual recovery: rename %q -> %q and %q -> %q",
586+
stage, cause, errKey, errEnc, keyBackup, keyFile, encryptedBackup, encryptedFile)
587+
}
588+
return fmt.Errorf("rotation failed at %s: %w (originals restored)", stage, cause)
539589
}
540590

541-
if err := os.WriteFile(encryptedFile, encryptedDataNew, 0o644); err != nil {
542-
return fmt.Errorf("failed to write encrypted file: %w", err)
591+
// Phase 4: write new key, then new encrypted file. Both via
592+
// secureperm.WriteFile (atomic + 0o600). On any failure the
593+
// `restore` closure puts the originals back.
594+
if err := secureperm.WriteFile(keyFile, []byte(formatKeyFile(newIdentity, time.Now()))); err != nil {
595+
return restore("write new key", err)
596+
}
597+
if err := secureperm.WriteFile(encryptedFile, encryptedDataNew); err != nil {
598+
return restore("write new encrypted file", err)
599+
}
600+
601+
// Phase 5: rotation succeeded — remove backups. If removal
602+
// fails we still return nil (the rotation itself is committed
603+
// and verifiable on disk; leftover backups are an annoyance,
604+
// not data loss). Log to stderr so the operator can clean up.
605+
if err := os.Remove(keyBackup); err != nil {
606+
fmt.Fprintf(os.Stderr, "warning: failed to remove %q after rotation: %v\n", keyBackup, err)
607+
}
608+
if err := os.Remove(encryptedBackup); err != nil {
609+
fmt.Fprintf(os.Stderr, "warning: failed to remove %q after rotation: %v\n", encryptedBackup, err)
543610
}
544611

545612
return nil

‎pkg/age/contract_test.go‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -444,6 +444,89 @@ func TestContract_Age_RotateKeys_ReplacesKeyAndPreservesPlaintext(t *testing.T)
444444
}
445445
}
446446

447+
// Contract: after a successful rotation both the new key file and
448+
// the new encrypted secrets file are mode 0o600. Defense-in-depth
449+
// — age encryption is the security layer, but world-readable
450+
// secrets material on shared workstations invites mistakes. Pin so
451+
// a regression that reverts the secureperm.WriteFile call to a raw
452+
// os.WriteFile with 0o644 surfaces here.
453+
func TestContract_Age_RotateKeys_BothFilesMode0600(t *testing.T) {
454+
dir := t.TempDir()
455+
if err := os.WriteFile(filepath.Join(dir, "secrets.yaml"), []byte("secret: x\n"), 0o600); err != nil {
456+
t.Fatal(err)
457+
}
458+
if err := age.EncryptSecretsFile(dir); err != nil {
459+
t.Fatal(err)
460+
}
461+
if err := age.RotateKeys(dir); err != nil {
462+
t.Fatal(err)
463+
}
464+
for _, name := range []string{"talm.key", "secrets.encrypted.yaml"} {
465+
info, err := os.Stat(filepath.Join(dir, name))
466+
if err != nil {
467+
t.Fatalf("stat %s: %v", name, err)
468+
}
469+
if got := info.Mode().Perm(); got != 0o600 {
470+
t.Errorf("%s mode = %o, want 0600", name, got)
471+
}
472+
}
473+
}
474+
475+
// Contract: a successful rotation leaves NO `*.rotation-backup`
476+
// files in the project root. Backups are an in-progress recovery
477+
// artefact only; their presence after the function returns nil
478+
// would clutter the project and confuse subsequent runs.
479+
func TestContract_Age_RotateKeys_NoLeftoverBackups(t *testing.T) {
480+
dir := t.TempDir()
481+
if err := os.WriteFile(filepath.Join(dir, "secrets.yaml"), []byte("secret: x\n"), 0o600); err != nil {
482+
t.Fatal(err)
483+
}
484+
if err := age.EncryptSecretsFile(dir); err != nil {
485+
t.Fatal(err)
486+
}
487+
if err := age.RotateKeys(dir); err != nil {
488+
t.Fatal(err)
489+
}
490+
for _, name := range []string{"talm.key.rotation-backup", "secrets.encrypted.yaml.rotation-backup"} {
491+
if _, err := os.Stat(filepath.Join(dir, name)); err == nil {
492+
t.Errorf("rotation left backup file %q on disk", name)
493+
}
494+
}
495+
}
496+
497+
// Contract: if a `*.rotation-backup` file is present at the start
498+
// (operator's previous rotation crashed mid-flight), the new run
499+
// refuses to start with a precise error pointing at the leftover
500+
// path. This protects the recovery state from being silently
501+
// overwritten by the second run.
502+
func TestContract_Age_RotateKeys_RefusesOnLeftoverBackup(t *testing.T) {
503+
dir := t.TempDir()
504+
if err := os.WriteFile(filepath.Join(dir, "secrets.yaml"), []byte("secret: x\n"), 0o600); err != nil {
505+
t.Fatal(err)
506+
}
507+
if err := age.EncryptSecretsFile(dir); err != nil {
508+
t.Fatal(err)
509+
}
510+
// Simulate an interrupted previous rotation by creating a
511+
// dangling `talm.key.rotation-backup`.
512+
leftover := filepath.Join(dir, "talm.key.rotation-backup")
513+
if err := os.WriteFile(leftover, []byte("orphaned"), 0o600); err != nil {
514+
t.Fatal(err)
515+
}
516+
517+
err := age.RotateKeys(dir)
518+
if err == nil {
519+
t.Fatal("expected RotateKeys to refuse with leftover backup present")
520+
}
521+
if !strings.Contains(err.Error(), "rotation-backup") {
522+
t.Errorf("error must reference the leftover backup path, got: %v", err)
523+
}
524+
// The leftover must still be on disk — refusal must not delete it.
525+
if _, statErr := os.Stat(leftover); statErr != nil {
526+
t.Errorf("leftover backup was removed despite refusal: %v", statErr)
527+
}
528+
}
529+
447530
// === EncryptYAMLFile / DecryptYAMLFile ===
448531

449532
// Contract: the generic file-pair encrypt/decrypt accepts arbitrary

‎pkg/commands/talosctl_wrapper.go‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -216,12 +216,20 @@ func init() {
216216
}
217217
}
218218

219-
// normalizeEndpoint normalizes an endpoint by removing any existing port and protocol, then adding https:// and :6443
219+
// normalizeEndpoint normalizes an endpoint by removing any existing
220+
// port and protocol, then adding https:// and :6443. IPv6 literals
221+
// are bracketed per RFC 3986 §3.2.2 (the assembly uses
222+
// net.JoinHostPort which auto-bracket-wraps any host that contains
223+
// a colon).
224+
//
220225
// Examples:
221-
// - "1.2.3.4" -> "https://1.2.3.4:6443"
222-
// - "1.2.3.4:50000" -> "https://1.2.3.4:6443"
223-
// - "https://1.2.3.4:50000" -> "https://1.2.3.4:6443"
224-
// - "http://1.2.3.4" -> "https://1.2.3.4:6443"
226+
// - "1.2.3.4" -> "https://1.2.3.4:6443"
227+
// - "1.2.3.4:50000" -> "https://1.2.3.4:6443"
228+
// - "https://1.2.3.4:50000" -> "https://1.2.3.4:6443"
229+
// - "http://1.2.3.4" -> "https://1.2.3.4:6443"
230+
// - "node.example.com" -> "https://node.example.com:6443"
231+
// - "[2001:db8::1]:6443" -> "https://[2001:db8::1]:6443"
232+
// - "[2001:db8::1]" -> "https://[2001:db8::1]:6443"
225233
func normalizeEndpoint(endpoint string) string {
226234
// Remove protocol if present
227235
endpoint = strings.TrimPrefix(endpoint, "https://")

0 commit comments

Comments
 (0)