Skip to content

Commit 0c52ded

Browse files
txmazinglexfrei
authored andcommitted
feat(openbao): optional static-key auto-unseal and apiserver egress label
The OpenBAO app has no auto-unseal (#2787). Every restart of a server, after a node drain, an image bump or an OOM kill, leaves it sealed until an operator enters the Shamir key shares into each replica. In HA mode the pods also hang at start (#2793). Cozystack's allow-to-apiserver policy only admits egress from pods labelled policy.cozystack.io/allow-to-apiserver, the app never set that label, and service_registration "kubernetes" cannot reach the API server without it. A new seal value selects shamir, which stays the default, or OpenBAO's seal "static". With static the chart renders the static seal stanza into the standalone and the Raft config and mounts the key Secret named in seal.secretName through the upstream chart's server.extraVolumes. The chart never generates, stores or rotates the key, because a missed lookup would replace it and leave the data unreadable. seal.previousSecretName and seal.previousKeyId carry the n-1 key that the static seal accepts during a rotation. The server pods always carry the apiserver egress label. Changing the seal of an initialized instance is a seal migration that OpenBAO performs under bao operator unseal -migrate. The chart reads the seal in use from the config ConfigMap the previous release rendered and refuses a different seal.type unless seal.allowMigration acknowledges the migration. Moving away from static needs the old stanza kept with disabled = "true", which the chart does not render, so that direction is refused regardless. The README documents the key Secret, the rotation with the pod replacement that the OnDelete update strategy requires, and the log line that confirms the re-wrap. Signed-off-by: txmazing <[email protected]> Assisted-by: LLM Signed-off-by: Yan Bondarenko <[email protected]>
1 parent 87a6407 commit 0c52ded

7 files changed

Lines changed: 565 additions & 2 deletions

File tree

‎api/apps/v1alpha1/openbao/types.go‎

Lines changed: 27 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/apps/openbao/README.md‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,53 @@ It provides identity-based secrets and encryption management for cloud infrastru
55

66
> `storageClass` is annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it.
77
8+
## Auto-unseal
9+
10+
By default OpenBAO uses Shamir key shares: after every pod restart an operator has to unseal each replica by hand. Setting `seal.type: static` switches the instance to OpenBAO's [`seal "static"`](https://openbao.org/docs/configuration/seal/static/), which unseals the barrier from a 32-byte key read from a Secret you manage. The chart never generates, stores or rotates the key; it only mounts the Secret you name.
11+
12+
1. Create the key Secret in the release namespace. The value is the base64 text of 32 random bytes, stored under the key `key`:
13+
14+
```bash
15+
kubectl -n tenant-<name> create secret generic openbao-unseal \
16+
--from-literal=key="$(head -c 32 /dev/urandom | base64 | tr -d '\n')"
17+
```
18+
19+
2. Point the instance at it:
20+
21+
```yaml
22+
seal:
23+
type: static
24+
secretName: openbao-unseal
25+
keyId: key-1
26+
```
27+
28+
`keyId` is a permanent identifier for that key material. OpenBAO refuses to start with a key whose identifier it has already seen with different material, so change `keyId` whenever you change the key.
29+
30+
To rotate the key:
31+
32+
1. Create a second Secret with the new key material and pick a new `keyId`.
33+
2. Move the current pair to `previousSecretName`/`previousKeyId`, put the new pair in `secretName`/`keyId` and upgrade the release. This only changes the StatefulSet template. The upstream chart uses the `OnDelete` update strategy and the server copies its HCL to `/tmp/storageconfig.hcl` at startup, so running pods keep the old key and the old config until they are replaced.
34+
3. Replace the pods. Standalone: delete the single pod. HA: delete the standby pods one at a time, waiting for each to rejoin and unseal, then delete the active pod so that leadership moves to a pod that already runs the new config. Do not delete the active pod first: once a node with the new config has taken over, a standby still on the old template cannot unseal after a restart.
35+
36+
```bash
37+
kubectl -n <namespace> delete pod <release>-2 # standby
38+
kubectl -n <namespace> delete pod <release>-1 # standby
39+
kubectl -n <namespace> delete pod <release>-0 # active, last
40+
```
41+
42+
4. Confirm the rotation before touching the old key. The node that becomes active with the new config re-wraps the stored barrier and recovery keys with `current_key` and logs it:
43+
44+
```bash
45+
kubectl -n <namespace> logs <active pod> | grep -E 'upgrading (stored keys|recovery key)'
46+
```
47+
48+
Until those lines have appeared, storage is still wrapped with the previous key. The static seal decrypts by the key identifier stored with the data and refuses an identifier it does not know (`unknown key id for data`), which is why both keys have to stay configured until the re-wrap is done.
49+
5. Drop the `previous*` values, upgrade, and replace the pods once more in the same order. A pod that unseals with the new key alone proves the re-wrap. Delete the old Secret last.
50+
51+
> Switching an existing Shamir-sealed instance to `static` is a seal migration and needs `bao operator unseal -migrate`; see the OpenBAO [seal migration](https://openbao.org/docs/concepts/seal/#seal-migration) documentation. The chart does not run the migration for you. It does refuse to change `seal.type` on an instance that already runs with another seal, because a plain config change would leave a barrier nobody can decrypt. Set `seal.allowMigration: true` only for the upgrade in which you run the migration, then remove it. Migrating away from `static` needs the old stanza kept with `disabled = "true"` and its key still mounted, which this chart does not render; that direction is refused regardless of `allowMigration`.
52+
53+
Keep a copy of the key outside the cluster: a Raft snapshot or a data PVC is unreadable without it.
54+
855
## Parameters
956

1057
### Common parameters
@@ -27,3 +74,16 @@ It provides identity-based secrets and encryption management for cloud infrastru
2774
| ---- | -------------------------- | ------ | ------ |
2875
| `ui` | Enable the OpenBAO web UI. | `bool` | `true` |
2976

77+
78+
### Seal
79+
80+
| Name | Description | Type | Value |
81+
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------- |
82+
| `seal` | Seal configuration. Defaults to Shamir key shares; set `type: static` for auto-unseal. | `object` | `{}` |
83+
| `seal.type` | Seal type. `shamir` keeps the current behaviour; `static` enables auto-unseal from `secretName`. | `string` | `shamir` |
84+
| `seal.secretName` | Existing Secret in the release namespace whose key `key` holds the base64 text of 32 random bytes (for example `openssl rand -base64 32`). Required when `type` is `static`. | `string` | `""` |
85+
| `seal.keyId` | Permanent identifier of the current key. Change it whenever the key material changes; OpenBAO refuses a key whose identifier it has already seen with different material. Required when `type` is `static`. | `string` | `""` |
86+
| `seal.previousSecretName` | Secret holding the previous key during an n-1 key rotation. Keep it until the pods have been replaced and the active node has logged `upgrading stored keys`; see the README. | `string` | `""` |
87+
| `seal.previousKeyId` | Identifier of the previous key. Required when `previousSecretName` is set. | `string` | `""` |
88+
| `seal.allowMigration` | Acknowledge a seal migration. The chart refuses to change `type` on an existing instance unless this is `true`; set it only while running `bao operator unseal -migrate`, then remove it. | `bool` | `false` |
89+

‎packages/apps/openbao/templates/openbao.yaml‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,44 @@
1+
{{- $seal := .Values.seal }}
2+
{{- $static := eq $seal.type "static" }}
3+
{{- if and $static (not $seal.secretName) }}
4+
{{- fail "seal.secretName is required when seal.type is \"static\"" }}
5+
{{- end }}
6+
{{- if and $static (not $seal.keyId) }}
7+
{{- fail "seal.keyId is required when seal.type is \"static\"" }}
8+
{{- end }}
9+
{{- if and $seal.previousSecretName (not $seal.previousKeyId) }}
10+
{{- fail "seal.previousKeyId is required when seal.previousSecretName is set" }}
11+
{{- end }}
12+
{{- if and $seal.previousKeyId (not $seal.previousSecretName) }}
13+
{{- fail "seal.previousSecretName is required when seal.previousKeyId is set" }}
14+
{{- end }}
15+
{{- if and $seal.previousSecretName (not $static) }}
16+
{{- fail "seal.previousSecretName only applies to seal.type \"static\"" }}
17+
{{- end }}
18+
{{- /*
19+
Refuse a silent seal change on an existing instance. Changing the seal of an
20+
initialized OpenBAO is a seal migration (`bao operator unseal -migrate`); a
21+
plain config change instead leaves a barrier nobody can decrypt. The seal in
22+
use is visible in the config ConfigMap the previous release rendered. `lookup`
23+
returns nothing on first install and on dry-run, so the check is a no-op there.
24+
*/}}
25+
{{- $cm := lookup "v1" "ConfigMap" .Release.Namespace (printf "%s-config" .Release.Name) }}
26+
{{- if $cm }}
27+
{{- $hcl := index ($cm.data | default dict) "extraconfig-from-values.hcl" | default "" }}
28+
{{- $stanza := regexFind `seal "[a-z]+"` $hcl }}
29+
{{- $liveSeal := ternary (trimAll "\"" (trimPrefix "seal " $stanza)) "shamir" (ne $stanza "") }}
30+
{{- /*
31+
Migrating away from static needs the old stanza kept with disabled = "true"
32+
and its key still mounted, which this chart does not render. Refuse the
33+
direction rather than emit a config the migration cannot run against.
34+
*/}}
35+
{{- if and (eq $liveSeal "static") (eq $seal.type "shamir") }}
36+
{{- fail "migration from seal.type \"static\" to \"shamir\" is not supported by this chart" }}
37+
{{- end }}
38+
{{- if and (ne $liveSeal $seal.type) (not $seal.allowMigration) }}
39+
{{- fail (printf "seal.type %q differs from the seal in use (%q). Changing the seal of an existing instance is a seal migration (bao operator unseal -migrate); set seal.allowMigration: true only while performing it." $seal.type $liveSeal) }}
40+
{{- end }}
41+
{{- end }}
142
apiVersion: helm.toolkit.fluxcd.io/v2
243
kind: HelmRelease
344
metadata:
@@ -27,6 +68,12 @@ spec:
2768
tlsDisable: true
2869
server:
2970
podManagementPolicy: Parallel
71+
# Cozystack's CiliumNetworkPolicy allow-to-apiserver only admits egress
72+
# to kube-apiserver for pods carrying this label. Without it,
73+
# service_registration "kubernetes" cannot reach the API server and
74+
# the HA pods hang on startup. See issue #2793.
75+
extraLabels:
76+
policy.cozystack.io/allow-to-apiserver: "true"
3077
resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 10 }}
3178
# Ensure the StatefulSet's data PVC is removed when the release is
3279
# deleted. Without this, the upstream chart leaves the PVC orphaned
@@ -42,6 +89,16 @@ spec:
4289
{{- with .Values.storageClass }}
4390
storageClass: {{ . }}
4491
{{- end }}
92+
{{- if $static }}
93+
# The upstream chart mounts each Secret at /openbao/userconfig/<name>/.
94+
extraVolumes:
95+
- type: secret
96+
name: {{ $seal.secretName }}
97+
{{- with $seal.previousSecretName }}
98+
- type: secret
99+
name: {{ . }}
100+
{{- end }}
101+
{{- end }}
45102
{{- if gt (int .Values.replicas) 1 }}
46103
standalone:
47104
enabled: false
@@ -68,6 +125,17 @@ spec:
68125
}
69126
{{- end }}
70127
}
128+
{{- if $static }}
129+
130+
seal "static" {
131+
current_key_id = {{ $seal.keyId | quote }}
132+
current_key = "file:///openbao/userconfig/{{ $seal.secretName }}/key"
133+
{{- if $seal.previousSecretName }}
134+
previous_key_id = {{ $seal.previousKeyId | quote }}
135+
previous_key = "file:///openbao/userconfig/{{ $seal.previousSecretName }}/key"
136+
{{- end }}
137+
}
138+
{{- end }}
71139
72140
service_registration "kubernetes" {}
73141
{{- else }}
@@ -85,6 +153,17 @@ spec:
85153
storage "file" {
86154
path = "/openbao/data"
87155
}
156+
{{- if $static }}
157+
158+
seal "static" {
159+
current_key_id = {{ $seal.keyId | quote }}
160+
current_key = "file:///openbao/userconfig/{{ $seal.secretName }}/key"
161+
{{- if $seal.previousSecretName }}
162+
previous_key_id = {{ $seal.previousKeyId | quote }}
163+
previous_key = "file:///openbao/userconfig/{{ $seal.previousSecretName }}/key"
164+
{{- end }}
165+
}
166+
{{- end }}
88167
# Note: service_registration "kubernetes" {} is intentionally omitted
89168
# in standalone mode — it requires an HA-capable storage backend and
90169
# causes a fatal error with storage "file".

0 commit comments

Comments
 (0)