Skip to content

Commit 212d0e1

Browse files
authored
feat(keycloak): let the KMS proxy trust a private Vault CA (#3874)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The KMS-encrypting DB proxy could only reach a Vault whose certificate chains to a publicly trusted root. A Vault fronted by an internal load balancer with a self-signed certificate was therefore unreachable over HTTPS, leaving plain HTTP as the only option for the DEK wrap/unwrap traffic. `encryption.kms.vault` now accepts the CA either inline as `caBundle` (the chart renders the Secret) or as an already-seeded `caSecretName`, with `caSecretKey` naming the key to project. The CA reaches the proxy as `SSL_CERT_FILE` **and** `SSL_CERT_DIR`. `SSL_CERT_FILE` on its own would only *add* the CA to the roots the image ships: `crypto/x509` replaces its file list but then scans `SSL_CERT_DIR` unconditionally, and the pinned proxy image carries `/etc/ssl/certs/ca-certificates.crt`. Setting both collapses the pool to the mount, so the connection that guards the KEK trusts the configured CA and nothing else. The database leg is unaffected either way, because it never consults the system trust store: it is plaintext today, and will verify against an explicit `RootCAs` pool once `KKP_BACKEND_CA_FILE` is wired. Supporting guarantees: a non-PEM `caBundle` fails the render instead of being dropped silently at runtime; rotating an inline `caBundle` rolls the pods through a `checksum/vault-ca` annotation, since Go memoizes its root pool per process; both CA keys are inert unless `kms.backend=vault-transit`, matching every other `vault.*` key; and setting `caBundle` together with `caSecretName` fails the render instead of silently picking one. Existing installations are unaffected: with neither key set the proxy keeps using the image's system trust store, and no volume or env var is added. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(keycloak): the KMS-encrypting DB proxy can now trust a private CA for Vault, supplied inline as encryption.kms.vault.caBundle or as a pre-seeded Secret via encryption.kms.vault.caSecretName (encryption.kms.vault.caSecretKey names the key). The Vault connection is pinned to that CA alone, which makes a Vault behind an internal load balancer with a self-signed certificate reachable over HTTPS instead of plain HTTP. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Vault TLS CA configuration for encrypted deployments. * Supports inline certificate bundles or existing Secrets with a configurable key. * Configured secure Vault connections to use the selected CA certificate. * **Bug Fixes** * Rejects invalid certificate bundles and prevents conflicting CA sources. * Restricts custom CA settings to Vault Transit connections. * Preserves the system trust store when no custom CA is configured. * Triggers pod rollouts when inline CA bundles change. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 652231e + 517e33f commit 212d0e1

3 files changed

Lines changed: 489 additions & 9 deletions

File tree

‎packages/system/keycloak/templates/proxy.yaml‎

Lines changed: 98 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,46 @@
1111
{{- if and (ne $backend "static") (ne $backend "vault-transit") }}
1212
{{- fail (printf "encryption.kms.backend must be 'static' or 'vault-transit', got %q" $backend) }}
1313
{{- end }}
14+
{{- $vault := ($kms.vault | default dict) }}
1415
{{- /* The proxy needs its ServiceAccount token only for Vault Kubernetes auth. */}}
15-
{{- $vaultAuth := (($kms.vault | default dict).auth | default "token") }}
16+
{{- $vaultAuth := ($vault.auth | default "token") }}
1617
{{- $k8sAuth := and (eq $backend "vault-transit") (eq $vaultAuth "kubernetes") }}
18+
{{- /*
19+
CA the proxy trusts for the Vault connection: either seeded by the operator
20+
(caSecretName) or rendered here from an inline PEM (caBundle). It reaches the
21+
proxy as SSL_CERT_FILE plus SSL_CERT_DIR — see the env block for why both are
22+
needed. Neither affects the database leg, which never consults the system
23+
trust store: that connection is plaintext today, and once KKP_BACKEND_CA_FILE
24+
is wired it will verify against an explicit RootCAs pool.
25+
26+
Like every other vault.* key, both are inert unless the backend is
27+
vault-transit: the volume and mount below sit at pod level, so keying them off
28+
an ungated caSecretName would make a static-backend Deployment mount a Secret
29+
the chart never creates and wedge the proxy in ContainerCreating.
30+
*/}}
31+
{{- $vaultCASecret := "" }}
32+
{{- $vaultCASecretKey := "" }}
33+
{{- if eq $backend "vault-transit" }}
34+
{{- if and $vault.caBundle $vault.caSecretName }}
35+
{{- fail "encryption.kms.vault.caBundle and caSecretName are mutually exclusive — set one" }}
36+
{{- end }}
37+
{{- if $vault.caBundle }}
38+
{{- /*
39+
A bundle that is not PEM would be written to the Secret, mounted, and then
40+
silently dropped by AppendCertsFromPEM, leaving the proxy on whatever roots
41+
the image ships — a failure that only shows up as an unrelated TLS error, or
42+
not at all against a publicly trusted Vault. Reject it at render time.
43+
*/}}
44+
{{- if not (regexMatch "-----BEGIN CERTIFICATE-----" $vault.caBundle) }}
45+
{{- fail "encryption.kms.vault.caBundle must be a PEM bundle containing at least one -----BEGIN CERTIFICATE----- block" }}
46+
{{- end }}
47+
{{- $vaultCASecret = "keycloak-kms-proxy-vault-ca" }}
48+
{{- $vaultCASecretKey = "ca.crt" }}
49+
{{- else if $vault.caSecretName }}
50+
{{- $vaultCASecret = $vault.caSecretName }}
51+
{{- $vaultCASecretKey = ($vault.caSecretKey | default "ca.crt") }}
52+
{{- end }}
53+
{{- end }}
1754
{{- $replicas := .Values.encryption.replicas | default 1 | int }}
1855
{{- if and (gt $replicas 1) (not .Values.encryption.deksetSecretName) }}
1956
{{- fail "encryption.replicas > 1 requires encryption.deksetSecretName (a shared wrapped DEK set); otherwise replicas would encrypt deterministic columns under different keys" }}
@@ -39,6 +76,17 @@ data:
3976
kek: {{ $kek | quote }}
4077
---
4178
{{- end }}
79+
{{- if and (eq $backend "vault-transit") $vault.caBundle }}
80+
apiVersion: v1
81+
kind: Secret
82+
metadata:
83+
name: keycloak-kms-proxy-vault-ca
84+
labels:
85+
app: keycloak-kms-proxy
86+
stringData:
87+
ca.crt: {{ $vault.caBundle | quote }}
88+
---
89+
{{- end }}
4290
apiVersion: v1
4391
kind: ServiceAccount
4492
metadata:
@@ -66,6 +114,14 @@ spec:
66114
app: keycloak-kms-proxy
67115
template:
68116
metadata:
117+
{{- if and (eq $backend "vault-transit") $vault.caBundle }}
118+
annotations:
119+
# The chart owns this Secret, so it owns the rollout. crypto/x509
120+
# memoizes the system pool per process (sync.Once), so a rotated bundle
121+
# reaches the proxy only in a fresh pod — without this the upgrade
122+
# succeeds and the old CA stays in use until some unrelated restart.
123+
checksum/vault-ca: {{ $vault.caBundle | sha256sum }}
124+
{{- end }}
69125
labels:
70126
app: keycloak-kms-proxy
71127
spec:
@@ -132,18 +188,28 @@ spec:
132188
- name: KKP_LENIENT
133189
value: {{ .Values.encryption.lenient | quote }}
134190
{{- if eq $backend "vault-transit" }}
135-
{{- $vault := $kms.vault | default dict }}
136-
{{- $auth := $vault.auth | default "token" }}
137-
{{- if and (ne $auth "token") (ne $auth "approle") (ne $auth "kubernetes") }}
138-
{{- fail (printf "encryption.kms.vault.auth must be 'token', 'approle', or 'kubernetes', got %q" $auth) }}
191+
{{- if and (ne $vaultAuth "token") (ne $vaultAuth "approle") (ne $vaultAuth "kubernetes") }}
192+
{{- fail (printf "encryption.kms.vault.auth must be 'token', 'approle', or 'kubernetes', got %q" $vaultAuth) }}
139193
{{- end }}
140194
- name: KKP_VAULT_ADDR
141195
value: {{ required "encryption.kms.vault.address is required when kms.backend=vault-transit" $vault.address | quote }}
142196
- name: KKP_VAULT_KEY_NAME
143197
value: {{ required "encryption.kms.vault.keyName is required when kms.backend=vault-transit" $vault.keyName | quote }}
144198
- name: KKP_VAULT_MOUNT
145199
value: {{ $vault.mount | default "transit" | quote }}
146-
{{- if eq $auth "approle" }}
200+
{{- with $vaultCASecret }}
201+
# SSL_CERT_FILE alone would ADD this CA to the roots the image ships
202+
# (crypto/x509 replaces only its file list, then scans SSL_CERT_DIR
203+
# unconditionally, and this image carries /etc/ssl/certs). Setting
204+
# both collapses the pool to the mount, so the leg that guards the
205+
# KEK trusts this CA and nothing else. The database leg is untouched:
206+
# it never reads the system pool.
207+
- name: SSL_CERT_FILE
208+
value: /etc/kkp/vault-ca/ca.crt
209+
- name: SSL_CERT_DIR
210+
value: /etc/kkp/vault-ca
211+
{{- end }}
212+
{{- if eq $vaultAuth "approle" }}
147213
{{- $appRole := $vault.appRole | default dict }}
148214
{{- $secretName := required "encryption.kms.vault.appRole.secretName is required when vault.auth=approle" $appRole.secretName }}
149215
- name: KKP_VAULT_APPROLE_MOUNT
@@ -158,7 +224,7 @@ spec:
158224
secretKeyRef:
159225
name: {{ $secretName }}
160226
key: {{ $appRole.secretIdKey | default "secret-id" }}
161-
{{- else if eq $auth "kubernetes" }}
227+
{{- else if eq $vaultAuth "kubernetes" }}
162228
{{- $k8s := $vault.kubernetes | default dict }}
163229
# The proxy logs in with its own ServiceAccount token; bind the
164230
# Vault role to system:serviceaccount:<namespace>:keycloak-kms-proxy.
@@ -202,17 +268,40 @@ spec:
202268
tcpSocket:
203269
port: postgres
204270
periodSeconds: 15
205-
{{- if .Values.encryption.deksetSecretName }}
271+
{{- if or .Values.encryption.deksetSecretName $vaultCASecret }}
206272
volumeMounts:
273+
{{- if .Values.encryption.deksetSecretName }}
207274
- name: dekset
208275
mountPath: /etc/kkp/dekset
209276
readOnly: true
277+
{{- end }}
278+
{{- with $vaultCASecret }}
279+
- name: vault-ca
280+
mountPath: /etc/kkp/vault-ca
281+
readOnly: true
282+
{{- end }}
210283
{{- end }}
211-
{{- if .Values.encryption.deksetSecretName }}
284+
{{- if or .Values.encryption.deksetSecretName $vaultCASecret }}
212285
volumes:
286+
{{- if .Values.encryption.deksetSecretName }}
213287
- name: dekset
214288
secret:
215289
secretName: {{ .Values.encryption.deksetSecretName }}
290+
{{- end }}
291+
{{- with $vaultCASecret }}
292+
- name: vault-ca
293+
secret:
294+
secretName: {{ . }}
295+
# One key, projected under a fixed name so SSL_CERT_FILE is constant.
296+
# A pre-seeded Secret is often a cert-manager TLS Secret, whose
297+
# tls.key has no business inside the proxy. The trade-off is
298+
# deliberate: a key that does not exist wedges the pod at mount time
299+
# (vault.caSecretKey is there to point at the right one) rather than
300+
# letting an empty trust store surface later as an opaque TLS error.
301+
items:
302+
- key: {{ $vaultCASecretKey }}
303+
path: ca.crt
304+
{{- end }}
216305
{{- end }}
217306
---
218307
apiVersion: v1

0 commit comments

Comments
 (0)