Skip to content

Commit 77cc49a

Browse files
committed
feat(values): decrypt encrypted user value files in memory via talm.key
Operators need to inject arbitrary secret values that chart templates consume (a registry password, a KMS secret-id) while keeping them encrypted at rest in git. talm already encrypts its fixed secret set with the age key in talm.key, but had no way to do the same for user values. Teach loadValues to decrypt a value file named *.encrypted.yaml in memory: a new age.DecryptYAMLToMap reuses the existing per-leaf age envelope (ENC[AGE,data:...]) and the project's talm.key to return the plaintext map without ever writing it to disk. Detection is name-authoritative and content-validated — a *.encrypted.yaml carrying no envelope errors out (ErrNoEncryptedValues) instead of silently rendering as plaintext, and a plaintext value that merely mentions the prefix mid-string is never mistaken for ciphertext (prefix AND suffix). A missing talm.key surfaces the existing recovery hint plus the file name. Wire values-secret.yaml <-> values-secret.encrypted.yaml into init --encrypt/--decrypt and git-ignore the plaintext sibling so the operator has a turnkey way to produce the committed encrypted file and reference it from templateOptions.valueFiles. Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 82b5ef6 commit 77cc49a

8 files changed

Lines changed: 626 additions & 29 deletions

‎docs/manual-test-plan.md‎

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -79,13 +79,19 @@ Expected: one `Overwriting <path> (--force)` line per diff; no prompt; exit zero
7979

8080
```bash
8181
cd /tmp/talm-init-test
82-
talm init --decrypt
83-
test -f secrets.yaml && test -f talosconfig
82+
echo 'registryPassword: hunter2' > values-secret.yaml
8483
talm init --encrypt
85-
test -f secrets.encrypted.yaml && test -f talosconfig.encrypted
84+
test -f secrets.encrypted.yaml && test -f talosconfig.encrypted && test -f values-secret.encrypted.yaml
85+
grep -q 'ENC\[AGE,data:' values-secret.encrypted.yaml # value encrypted
86+
! grep -q hunter2 values-secret.encrypted.yaml # plaintext not present
87+
grep -q '^values-secret.yaml$' .gitignore # plaintext git-ignored
88+
rm values-secret.yaml secrets.yaml talosconfig
89+
talm init --decrypt
90+
test -f secrets.yaml && test -f talosconfig && test -f values-secret.yaml
91+
grep -q 'registryPassword: hunter2' values-secret.yaml # decrypts back
8692
```
8793

88-
Expected: per-file `Decrypting X -> Y` / `Encrypting X -> Y` lines; both round-trips succeed.
94+
Expected: per-file `Decrypting X -> Y` / `Encrypting X -> Y` lines (now including `values-secret.yaml`); both round-trips succeed.
8995

9096
### A6. Decrypt without `talm.key`
9197

@@ -201,6 +207,25 @@ talm template -f nodes/node0.yaml \
201207

202208
Expected: `dnsDomain: overridden.local` appears in output.
203209

210+
### B2a. Render with encrypted user values (in-memory decryption)
211+
212+
Author a secret value, encrypt it, reference the encrypted file, and have a template consume it (e.g. `clusterDomain: {{ .Values.clusterDomain | quote }}`):
213+
214+
```bash
215+
cd $PROJECT
216+
echo 'clusterDomain: secret.example' > values-secret.yaml
217+
talm init --encrypt # produces values-secret.encrypted.yaml
218+
# add `valueFiles: [values-secret.encrypted.yaml]` under templateOptions in Chart.yaml
219+
talm template -f nodes/node0.yaml | grep dnsDomain # expect secret.example
220+
talm apply --dry-run -f nodes/node0.yaml # same value at apply (re-render decrypts in-memory)
221+
222+
# Recovery-hint path: with the key gone, the encrypted value file must fail loudly.
223+
mv talm.key /tmp/talm.key.bak
224+
talm template -f nodes/node0.yaml ; mv /tmp/talm.key.bak talm.key
225+
```
226+
227+
Expected: the encrypted file decrypts in memory at both template and apply — no plaintext `values-secret.yaml` is needed at render time and none is written. With `talm.key` missing, the error names `values-secret.encrypted.yaml` and carries the talm.key recovery hint (not a bare "file not found"). A value file whose name ends `.encrypted.yaml` but contains no `ENC[AGE,...]` envelope must error rather than render as empty plaintext.
228+
204229
### B3. Render against missing file
205230

206231
```bash

‎pkg/age/age.go‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ import (
2121
"io"
2222
"os"
2323
"path/filepath"
24+
"slices"
2425
"strings"
2526
"time"
2627

@@ -50,8 +51,22 @@ const (
5051
plainSecretsFile = "secrets.yaml"
5152
ageEncryptionPrefix = "ENC[AGE,data:"
5253
ageEncryptionSuffix = "]"
54+
55+
// EncryptedFileSuffix is the filename convention that marks a YAML
56+
// file as age-encrypted (secrets.encrypted.yaml, talosconfig.encrypted
57+
// is the legacy non-yaml exception). Consumers that decrypt encrypted
58+
// value files in-memory key off this suffix so the marker stays a
59+
// single source of truth shared with the init encrypt/decrypt flow.
60+
EncryptedFileSuffix = ".encrypted.yaml"
5361
)
5462

63+
// ErrNoEncryptedValues is returned by DecryptYAMLToMap when a file the caller
64+
// declared encrypted (by its .encrypted.yaml name) parses cleanly but carries
65+
// no ENC[AGE,...] envelope at all. Treating it as plaintext would silently
66+
// feed an unencrypted (or corrupt) file into rendering; surfacing the mismatch
67+
// lets the operator fix the file rather than ship a hole.
68+
var ErrNoEncryptedValues = errors.New("file is named *.encrypted.yaml but contains no ENC[AGE,...] encrypted values")
69+
5570
// GenerateKey generates a new age identity and saves it to talm.key file in age keygen format.
5671
// Returns true if a new key was created (not loaded from existing file).
5772
func GenerateKey(rootDir string) (*age.X25519Identity, bool, error) {
@@ -870,6 +885,83 @@ func DecryptYAMLFile(rootDir, encryptedFile, plainFile string) error {
870885
return decryptYAMLPair(rootDir, encryptedFile, plainFile)
871886
}
872887

888+
// isAgeEnvelope reports whether s is a complete ENC[AGE,data:...] envelope.
889+
// The check is prefix AND suffix (never a substring): a plaintext value that
890+
// merely mentions the prefix mid-string (e.g. documentation about the format)
891+
// must NOT be mistaken for ciphertext. This is the same guard decryptYAMLValues
892+
// applies before attempting a decrypt.
893+
func isAgeEnvelope(s string) bool {
894+
return strings.HasPrefix(s, ageEncryptionPrefix) && strings.HasSuffix(s, ageEncryptionSuffix)
895+
}
896+
897+
// containsEncryptedValue reports whether data carries at least one age
898+
// envelope leaf anywhere in its tree. Used to validate that a file declared
899+
// encrypted by its name actually holds ciphertext before requiring talm.key.
900+
func containsEncryptedValue(data any) bool {
901+
switch typed := data.(type) {
902+
case map[string]any:
903+
for _, value := range typed {
904+
if containsEncryptedValue(value) {
905+
return true
906+
}
907+
}
908+
909+
return false
910+
case []any:
911+
return slices.ContainsFunc(typed, containsEncryptedValue)
912+
case string:
913+
return isAgeEnvelope(typed)
914+
default:
915+
return false
916+
}
917+
}
918+
919+
// DecryptYAMLToMap reads the age-encrypted YAML at filePath, decrypts its
920+
// string-leaf values in memory with the project's talm.key (located under
921+
// rootDir), and returns the plaintext map. Unlike DecryptYAMLFile it writes
922+
// nothing to disk — it feeds encrypted user value files straight into chart
923+
// rendering so the plaintext never lands in the working tree.
924+
//
925+
// filePath is used verbatim (the caller has already resolved it); rootDir
926+
// only locates talm.key. The file is validated to contain at least one
927+
// envelope (ErrNoEncryptedValues otherwise) BEFORE the key is required, so a
928+
// mis-named or corrupt file fails with a precise cause rather than a confusing
929+
// "talm.key missing". Partially-encrypted files are fine: plaintext leaves
930+
// pass through untouched.
931+
func DecryptYAMLToMap(rootDir, filePath string) (map[string]any, error) {
932+
encryptedData, err := os.ReadFile(filePath)
933+
if err != nil {
934+
return nil, errors.Wrapf(err, "read encrypted values file %q", filePath)
935+
}
936+
937+
var encryptedYAML map[string]any
938+
939+
if err := yaml.Unmarshal(encryptedData, &encryptedYAML); err != nil {
940+
return nil, errors.Wrapf(err, "parse encrypted values file %q", filePath)
941+
}
942+
943+
if !containsEncryptedValue(encryptedYAML) {
944+
return nil, errors.Wrapf(ErrNoEncryptedValues, "%q", filePath)
945+
}
946+
947+
identity, err := LoadKey(rootDir)
948+
if err != nil {
949+
return nil, errors.Wrapf(err, "loading talm.key to decrypt %q", filePath)
950+
}
951+
952+
decrypted, err := decryptYAMLValues(encryptedYAML, identity)
953+
if err != nil {
954+
return nil, errors.Wrapf(err, "decrypt values in %q", filePath)
955+
}
956+
957+
out, ok := decrypted.(map[string]any)
958+
if !ok {
959+
return nil, errors.Wrapf(errInternalInvariant, "decryptYAMLValues returned %T for %q", decrypted, filePath)
960+
}
961+
962+
return out, nil
963+
}
964+
873965
// decryptYAMLPair is the shared implementation for DecryptSecretsFile
874966
// and DecryptYAMLFile. Both flows take an encrypted-YAML path and a
875967
// plain-YAML destination under rootDir, load the project's age key,
Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
// Copyright Cozystack Authors
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package age_test
16+
17+
import (
18+
"os"
19+
"path/filepath"
20+
"strings"
21+
"testing"
22+
23+
cerrors "github.com/cockroachdb/errors"
24+
"github.com/cozystack/talm/pkg/age"
25+
"gopkg.in/yaml.v3"
26+
)
27+
28+
// writeEncryptedValuesFile writes plain as values-secret.yaml under dir,
29+
// encrypts it (generating talm.key), and returns the encrypted file path.
30+
func writeEncryptedValuesFile(t *testing.T, dir string, plain map[string]any) string {
31+
t.Helper()
32+
33+
plainBytes, err := yaml.Marshal(plain)
34+
if err != nil {
35+
t.Fatalf("marshal plain: %v", err)
36+
}
37+
38+
if err := os.WriteFile(filepath.Join(dir, "values-secret.yaml"), plainBytes, 0o600); err != nil {
39+
t.Fatalf("write plain: %v", err)
40+
}
41+
42+
if err := age.EncryptYAMLFile(dir, "values-secret.yaml", "values-secret.encrypted.yaml"); err != nil {
43+
t.Fatalf("EncryptYAMLFile: %v", err)
44+
}
45+
46+
return filepath.Join(dir, "values-secret.encrypted.yaml")
47+
}
48+
49+
// TestContract_DecryptYAMLToMap_RoundTrip pins the core in-memory decrypt:
50+
// the map returned must equal the plaintext that was encrypted, without any
51+
// file being written back to disk.
52+
func TestContract_DecryptYAMLToMap_RoundTrip(t *testing.T) {
53+
dir := t.TempDir()
54+
plain := map[string]any{"registryPassword": "s3cr3t", "nested": map[string]any{"token": "abc123"}}
55+
56+
encPath := writeEncryptedValuesFile(t, dir, plain)
57+
58+
got, err := age.DecryptYAMLToMap(dir, encPath)
59+
if err != nil {
60+
t.Fatalf("DecryptYAMLToMap: %v", err)
61+
}
62+
63+
if got["registryPassword"] != "s3cr3t" {
64+
t.Errorf("registryPassword = %v, want s3cr3t", got["registryPassword"])
65+
}
66+
67+
nested, ok := got["nested"].(map[string]any)
68+
if !ok || nested["token"] != "abc123" {
69+
t.Errorf("nested.token = %v, want abc123 (got nested=%v)", nested["token"], got["nested"])
70+
}
71+
72+
// No plaintext sibling must be written — decryption is in-memory only.
73+
if _, err := os.Stat(filepath.Join(dir, "values-secret.yaml.decrypted")); err == nil {
74+
t.Error("DecryptYAMLToMap must not write any file to disk")
75+
}
76+
}
77+
78+
// TestContract_DecryptYAMLToMap_PartialEncryption pins that a file mixing an
79+
// encrypted leaf with a plaintext leaf decrypts the envelope and passes the
80+
// plaintext through unchanged. Operators hand-editing an encrypted file rely
81+
// on this.
82+
func TestContract_DecryptYAMLToMap_PartialEncryption(t *testing.T) {
83+
dir := t.TempDir()
84+
encPath := writeEncryptedValuesFile(t, dir, map[string]any{"password": "topsecret"})
85+
86+
// Append a plaintext leaf to the encrypted file.
87+
raw, err := os.ReadFile(encPath)
88+
if err != nil {
89+
t.Fatalf("read encrypted: %v", err)
90+
}
91+
92+
var mixed map[string]any
93+
if err := yaml.Unmarshal(raw, &mixed); err != nil {
94+
t.Fatalf("unmarshal encrypted: %v", err)
95+
}
96+
97+
mixed["note"] = "a plaintext note"
98+
99+
mixedBytes, err := yaml.Marshal(mixed)
100+
if err != nil {
101+
t.Fatalf("marshal mixed: %v", err)
102+
}
103+
104+
if err := os.WriteFile(encPath, mixedBytes, 0o600); err != nil {
105+
t.Fatalf("write mixed: %v", err)
106+
}
107+
108+
got, err := age.DecryptYAMLToMap(dir, encPath)
109+
if err != nil {
110+
t.Fatalf("DecryptYAMLToMap on partial file: %v", err)
111+
}
112+
113+
if got["password"] != "topsecret" {
114+
t.Errorf("password = %v, want topsecret (envelope must decrypt)", got["password"])
115+
}
116+
if got["note"] != "a plaintext note" {
117+
t.Errorf("note = %v, want verbatim plaintext", got["note"])
118+
}
119+
}
120+
121+
// TestContract_DecryptYAMLToMap_NoEnvelopeErrors pins the content-validation
122+
// contract: a file with no full ENC[AGE,...] envelope returns
123+
// ErrNoEncryptedValues even though it parses cleanly. A plaintext value that
124+
// merely mentions the envelope prefix mid-string MUST NOT be mistaken for
125+
// ciphertext (the prefix-AND-suffix guard).
126+
func TestContract_DecryptYAMLToMap_NoEnvelopeErrors(t *testing.T) {
127+
dir := t.TempDir()
128+
if _, _, err := age.GenerateKey(dir); err != nil {
129+
t.Fatalf("GenerateKey: %v", err)
130+
}
131+
132+
plaintext := "desc: \"the format is ENC[AGE,data:...] but this is not encrypted\"\nport: \"6443\"\n"
133+
path := filepath.Join(dir, "not-really.encrypted.yaml")
134+
if err := os.WriteFile(path, []byte(plaintext), 0o600); err != nil {
135+
t.Fatalf("write plaintext: %v", err)
136+
}
137+
138+
_, err := age.DecryptYAMLToMap(dir, path)
139+
if !cerrors.Is(err, age.ErrNoEncryptedValues) {
140+
t.Errorf("expected ErrNoEncryptedValues for an envelope-less file; got %v", err)
141+
}
142+
}
143+
144+
// TestContract_DecryptYAMLToMap_MissingKeySurfacesHint pins that when the
145+
// file is genuinely encrypted but talm.key is gone, the error names the file
146+
// and carries LoadKey's recovery hint — not a bare read failure.
147+
func TestContract_DecryptYAMLToMap_MissingKeySurfacesHint(t *testing.T) {
148+
dir := t.TempDir()
149+
encPath := writeEncryptedValuesFile(t, dir, map[string]any{"password": "topsecret"})
150+
151+
if err := os.Remove(filepath.Join(dir, "talm.key")); err != nil {
152+
t.Fatalf("remove talm.key: %v", err)
153+
}
154+
155+
_, err := age.DecryptYAMLToMap(dir, encPath)
156+
if err == nil {
157+
t.Fatal("expected error when talm.key is missing")
158+
}
159+
160+
if !strings.Contains(err.Error(), "values-secret.encrypted.yaml") {
161+
t.Errorf("error must name the offending file; got %v", err)
162+
}
163+
164+
hints := strings.ToLower(strings.Join(cerrors.GetAllHints(err), "\n"))
165+
if !strings.Contains(hints, "talm.key") {
166+
t.Errorf("missing-key error must surface LoadKey's recovery hint; got hints: %s", hints)
167+
}
168+
}

‎pkg/commands/contract_chart_gitignore_test.go‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -157,10 +157,12 @@ func TestContract_ReadChartYamlPreset_MalformedYAMLError(t *testing.T) {
157157
// === writeGitignoreFile ===
158158

159159
// Contract: writeGitignoreFile creates .gitignore from scratch
160-
// containing the four secrets-bearing files talm manages:
161-
// secrets.yaml, talosconfig, talm.key, kubeconfig (default name).
160+
// containing the secrets-bearing files talm manages: secrets.yaml,
161+
// talosconfig, talm.key, values-secret.yaml, kubeconfig (default name).
162162
// Without this list a fresh `talm init` followed by `git init &&
163-
// git add .` would commit private cluster material.
163+
// git add .` would commit private cluster material — including the
164+
// plaintext values-secret.yaml whose only committed form must be its
165+
// encrypted sibling.
164166
func TestContract_WriteGitignoreFile_CreatesWithRequiredEntries(t *testing.T) {
165167
dir := t.TempDir()
166168
setRoot(t, dir)
@@ -177,7 +179,7 @@ func TestContract_WriteGitignoreFile_CreatesWithRequiredEntries(t *testing.T) {
177179
t.Fatalf("read .gitignore: %v", err)
178180
}
179181
content := string(data)
180-
for _, want := range []string{"secrets.yaml", "talosconfig", "talm.key", "kubeconfig"} {
182+
for _, want := range []string{"secrets.yaml", "talosconfig", "talm.key", "values-secret.yaml", "kubeconfig"} {
181183
if !strings.Contains(content, want) {
182184
t.Errorf(".gitignore missing %q in:\n%s", want, content)
183185
}
@@ -253,6 +255,7 @@ func TestContract_WriteGitignoreFile_IdempotentOnFullList(t *testing.T) {
253255
secrets.yaml
254256
talosconfig
255257
talm.key
258+
values-secret.yaml
256259
kubeconfig
257260
`
258261
gitignore := filepath.Join(dir, ".gitignore")

0 commit comments

Comments
 (0)