Skip to content

Commit 0b3f5dd

Browse files
authored
fix(keycloak)!: make the KMS proxy keep its keys across restarts (#4818)
## What this PR does Two fixes for the optional KMS proxy of the platform Keycloak (`encryption.*`). The default static backend never started. The chart stored 32 alphanumeric characters as the KEK, but the proxy wants base64 of 32 bytes and exits with `config: KKP_KEK must decode to 32 bytes (got 24)`. The KEK is now generated with `randBytes`, and a stored key of the old form is replaced. The proxy never ran with such a key, so nothing was encrypted under it. Any other stored key is kept if the proxy accepts it and fails the render otherwise, so the chart never overwrites key material. The KEK Secret now carries `helm.sh/resource-policy: keep`: the DEK set is wrapped with it, so it has to survive turning encryption off or reinstalling the package. Without `deksetSecretName` the proxy mints a new DEK set on every start (`buildCipher` in `cmd/proxy/main.go` of keycloak-kms-proxy) and keeps it only in memory. After the first restart of the proxy pod it cannot decrypt anything it wrote before. The values comment said this is safe on a fresh DB, it is not. The chart now refuses to render the proxy without a DEK set unless `fields: disabled`. Passthrough still works without one, so the static backend can get its KEK Secret created before the DEK set is minted with it. Breaking: a release with encryption on and no `deksetSecretName` fails to render on upgrade until a DEK set is configured. Such a setup loses its data on the next proxy restart anyway, so failing loudly is the point. A stored static KEK the proxy cannot decode fails the render too, instead of being replaced. Checked with `helm unittest` (100 tests, the new ones fail on the old template) and by starting keycloak-kms-proxy v0.2.3 with the KEK the chart renders now. With the KEK from the old template it fails as quoted above. ### Screenshots No UI changes. ### Downstream repositories - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: cozystack/website#742 documents the setup with a DEK set - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The other repositories do not touch the keycloak chart values. ### Release note ```release-note fix(keycloak)!: The KMS proxy of the platform Keycloak now requires `encryption.deksetSecretName` unless `encryption.fields` is `disabled`. Without it the proxy generated a new data key on every start and could not read data encrypted before a restart. The static backend now generates a KEK the proxy accepts; before, the proxy could not start with it. The KEK Secret is no longer deleted when encryption is turned off or the package is removed. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Configuration** * Enabling field encryption requires a persistent DEK set. Chart rendering reports an error if it is missing; setting fields to `disabled` does not require one. * For the static backend, a valid stored KEK is reused, and a missing or legacy-format KEK is replaced. An unusable stored KEK causes rendering to fail. * The KEK Secret is retained when encryption is disabled or the package is removed. * **Documentation** * Clarified DEK set requirements and sharing across starts and replicas, and updated guidance on configuring multiple proxy replicas. * Warns against deleting the retained KEK while encrypted values remain. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents b366286 + b62f1ca commit 0b3f5dd

4 files changed

Lines changed: 283 additions & 13 deletions

File tree

‎packages/system/keycloak/templates/proxy.yaml‎

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,23 +55,57 @@
5555
{{- if and (gt $replicas 1) (not .Values.encryption.deksetSecretName) }}
5656
{{- fail "encryption.replicas > 1 requires encryption.deksetSecretName (a shared wrapped DEK set); otherwise replicas would encrypt deterministic columns under different keys" }}
5757
{{- end }}
58+
{{- /*
59+
Without KKP_DEKSET_FILE the proxy mints a fresh DEK set on every start and
60+
keeps it only in memory, so after any restart (rollout, drain, OOM) it can
61+
no longer decrypt what it wrote before. Refuse to encrypt without a
62+
persistent DEK set. fields=disabled is pure passthrough and needs no key: it
63+
is how the static backend gets its KEK Secret created before the operator
64+
mints a DEK set with it.
65+
*/}}
66+
{{- if and (ne (.Values.encryption.fields | default "") "disabled") (not .Values.encryption.deksetSecretName) }}
67+
{{- fail "encryption.deksetSecretName is required unless encryption.fields is \"disabled\": without a persistent DEK set the proxy mints a new key on every start and cannot decrypt data written before a restart. Mint one with the proxy's `backfill generate-dekset`, store it in a Secret and set encryption.deksetSecretName" }}
68+
{{- end }}
5869
{{- if eq $backend "static" }}
5970
{{- /*
6071
Static KMS: a self-generated 32-byte KEK kept in a Secret. Generated once
6172
and preserved across upgrades via lookup (same pattern as the admin
6273
credentials in sts.yaml). For production prefer kms.backend=vault-transit.
6374
*/}}
75+
{{- /*
76+
The proxy reads KKP_KEK as base64 of 32 random bytes. Earlier chart
77+
versions stored 32 alphanumeric characters instead, which decode to 24
78+
bytes: the proxy refused to start with them, so nothing was ever encrypted
79+
under such a KEK and it is safe to replace it. Any other stored value may be
80+
a key the operator restored by hand, so it is kept when the proxy accepts
81+
it and refused otherwise, never overwritten. b64dec ignores CR and LF like
82+
the proxy does, so a restored key with a trailing newline still counts.
83+
*/}}
6484
{{- $existing := lookup "v1" "Secret" .Release.Namespace "keycloak-kms-proxy-kek" }}
65-
{{- $kek := randAlphaNum 32 | b64enc }}
85+
{{- $kek := randBytes 32 | b64enc }}
86+
{{- $current := "" }}
6687
{{- if and $existing $existing.data }}
67-
{{- $kek = index $existing.data "kek" | default $kek }}
88+
{{- $current = index $existing.data "kek" | default "" }}
89+
{{- end }}
90+
{{- if $current }}
91+
{{- $stored := b64dec $current }}
92+
{{- if eq (len (b64dec $stored)) 32 }}
93+
{{- $kek = $current }}
94+
{{- else if not (regexMatch "^[A-Za-z0-9]{32}$" $stored) }}
95+
{{- fail "the kek key of Secret keycloak-kms-proxy-kek is not base64 of 32 bytes, so the KMS proxy cannot use it. The chart does not overwrite key material: restore the KEK the data was encrypted with, or delete the Secret if nothing was encrypted under it yet" }}
96+
{{- end }}
6897
{{- end }}
6998
apiVersion: v1
7099
kind: Secret
71100
metadata:
72101
name: keycloak-kms-proxy-kek
73102
labels:
74103
app: keycloak-kms-proxy
104+
annotations:
105+
# The DEK set is wrapped with this KEK, so losing it loses the data.
106+
# Keep it when encryption is disabled or the release is reinstalled;
107+
# the lookup above picks it up again on the next install.
108+
helm.sh/resource-policy: keep
75109
data:
76110
kek: {{ $kek | quote }}
77111
---

0 commit comments

Comments
 (0)