Skip to content

Commit 407ab9c

Browse files
author
txmazing
committed
feat(openbao): optional static-key auto-unseal and apiserver egress label
Two problems with the built-in OpenBAO app: - #2787: the app has no auto-unseal. Every pod restart (node drain, image bump, OOM) leaves the instance sealed until an operator types the Shamir key shares into each replica. - #2793: in HA mode the pods hang on startup because service_registration "kubernetes" cannot reach kube-apiserver. Cozystack's CiliumNetworkPolicy allow-to-apiserver only admits egress from pods labelled policy.cozystack.io/allow-to-apiserver=true, and the wrapper never set that label, so every tenant had to hand-craft a network policy. Related: #2059, #2483. Design: - A new `seal` value with `type: shamir` (default) or `type: static`. With `static` the chart appends OpenBAO's `seal "static"` stanza to the server config and mounts the operator-owned Secret named in `seal.secretName` through the upstream chart's `server.extraVolumes` (mounted at /openbao/userconfig/<secret>/). The chart never generates, stores or rotates the key; the operator creates the Secret (32 random bytes, base64 text under key `key`) and names it. `previousSecretName` / `previousKeyId` allow the n-1 rotation the static seal supports. The stanza is rendered in both the Raft (HA) and the standalone configuration, since the static seal works with either storage. - The template fails fast when `static` is chosen without `secretName` or `keyId`, or when `previousSecretName` is set without `previousKeyId`, so a misconfiguration surfaces at reconcile time rather than as a sealed pod. - `server.extraLabels` now always carries policy.cozystack.io/allow-to-apiserver: "true". The upstream chart renders it onto the StatefulSet pods, which is exactly what the platform policy keys on. The default stays `shamir`, so existing instances render the same HCL as before apart from the new pod label. README, values.schema.json, api/apps/v1alpha1/openbao/types.go and the openbao-rd ApplicationDefinition are regenerated with `make generate`; a helm-unittest suite covers the label, both seal renderings, rotation and the three failure cases. Upstream docs: https://cozystack.io/docs/v1.6/applications/openbao/ (parameter reference only; says nothing about unsealing or the apiserver egress label) Upstream docs: https://openbao.org/docs/configuration/seal/static/ (current_key_id/current_key/previous_key_id/previous_key, 32-byte key, file:// prefix, n-1 rotation) Upstream docs: https://openbao.org/docs/concepts/seal/#seal-migration (switching an existing instance between Shamir and static needs `unseal -migrate` and downtime; the chart does not do this) Upstream docs: https://github.com/openbao/openbao-helm (charts/openbao values server.extraVolumes and server.extraLabels; mount path /openbao/userconfig/<name> in templates/_helpers.tpl) Signed-off-by: txmazing <[email protected]> Assisted-by: LLM
1 parent 9e577a5 commit 407ab9c

7 files changed

Lines changed: 310 additions & 2 deletions

File tree

‎api/apps/v1alpha1/openbao/types.go‎

Lines changed: 24 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/apps/openbao/README.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,34 @@ It provides identity-based secrets and encryption management for cloud infrastru
55

66
> `storageClass` is annotated as immutable in the chart schema — see [`docs/storage-immutability.md`](../../../docs/storage-immutability.md) for the contract and which consumers enforce it.
77
8+
## Auto-unseal
9+
10+
By default OpenBAO uses Shamir key shares: after every pod restart an operator has to unseal each replica by hand. Setting `seal.type: static` switches the instance to OpenBAO's [`seal "static"`](https://openbao.org/docs/configuration/seal/static/), which unseals the barrier from a 32-byte key read from a Secret you manage. The chart never generates, stores or rotates the key; it only mounts the Secret you name.
11+
12+
1. Create the key Secret in the release namespace. The value is the base64 text of 32 random bytes, stored under the key `key`:
13+
14+
```bash
15+
kubectl -n tenant-<name> create secret generic openbao-unseal \
16+
--from-literal=key="$(head -c 32 /dev/urandom | base64 | tr -d '\n')"
17+
```
18+
19+
2. Point the instance at it:
20+
21+
```yaml
22+
seal:
23+
type: static
24+
secretName: openbao-unseal
25+
keyId: key-1
26+
```
27+
28+
`keyId` is a permanent identifier for that key material. OpenBAO refuses to start with a key whose identifier it has already seen with different material, so change `keyId` whenever you change the key.
29+
30+
To rotate the key, create a second Secret, move the current pair to `previousSecretName`/`previousKeyId` and put the new pair in `secretName`/`keyId`. This is the n-1 rotation the static seal documents: `current_key` is used for new seal operations while `previous_key` still decrypts what was written before. Keep both Secrets in place until OpenBAO has re-wrapped storage with the new key, then drop the `previous*` values and delete the old Secret.
31+
32+
> Switching an existing Shamir-sealed instance to `static` (or back) is a seal migration and needs `bao operator unseal -migrate`; see the OpenBAO [seal migration](https://openbao.org/docs/concepts/seal/#seal-migration) documentation. The chart does not run the migration for you.
33+
34+
Keep a copy of the key outside the cluster: a Raft snapshot or a data PVC is unreadable without it.
35+
836
## Parameters
937

1038
### Common parameters
@@ -27,3 +55,15 @@ It provides identity-based secrets and encryption management for cloud infrastru
2755
| ---- | -------------------------- | ------ | ------ |
2856
| `ui` | Enable the OpenBAO web UI. | `bool` | `true` |
2957

58+
59+
### Seal
60+
61+
| Name | Description | Type | Value |
62+
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------- |
63+
| `seal` | Seal configuration. Defaults to Shamir key shares; set `type: static` for auto-unseal. | `object` | `{}` |
64+
| `seal.type` | Seal type. `shamir` keeps the current behaviour; `static` enables auto-unseal from `secretName`. | `string` | `shamir` |
65+
| `seal.secretName` | Existing Secret in the release namespace whose key `key` holds the base64 text of 32 random bytes (`head -c 32 /dev/urandom | base64 | tr -d '\n'`). Required when `type` is `static`. | `string` | `""` |
66+
| `seal.keyId` | Permanent identifier of the current key. Change it whenever the key material changes; OpenBAO refuses a key whose identifier it has already seen with different material. Required when `type` is `static`. | `string` | `""` |
67+
| `seal.previousSecretName` | Secret holding the previous key during an n-1 key rotation. | `string` | `""` |
68+
| `seal.previousKeyId` | Identifier of the previous key. Required when `previousSecretName` is set. | `string` | `""` |
69+

‎packages/apps/openbao/templates/openbao.yaml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
{{- $seal := .Values.seal }}
2+
{{- $static := eq $seal.type "static" }}
3+
{{- if and $static (not $seal.secretName) }}
4+
{{- fail "seal.secretName is required when seal.type is \"static\"" }}
5+
{{- end }}
6+
{{- if and $static (not $seal.keyId) }}
7+
{{- fail "seal.keyId is required when seal.type is \"static\"" }}
8+
{{- end }}
9+
{{- if and $seal.previousSecretName (not $seal.previousKeyId) }}
10+
{{- fail "seal.previousKeyId is required when seal.previousSecretName is set" }}
11+
{{- end }}
112
apiVersion: helm.toolkit.fluxcd.io/v2
213
kind: HelmRelease
314
metadata:
@@ -27,6 +38,12 @@ spec:
2738
tlsDisable: true
2839
server:
2940
podManagementPolicy: Parallel
41+
# Cozystack's CiliumNetworkPolicy allow-to-apiserver only admits egress
42+
# to kube-apiserver for pods carrying this label. Without it,
43+
# service_registration "kubernetes" cannot reach the API server and
44+
# the HA pods hang on startup. See issue #2793.
45+
extraLabels:
46+
policy.cozystack.io/allow-to-apiserver: "true"
3047
resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 10 }}
3148
# Ensure the StatefulSet's data PVC is removed when the release is
3249
# deleted. Without this, the upstream chart leaves the PVC orphaned
@@ -42,6 +59,16 @@ spec:
4259
{{- with .Values.storageClass }}
4360
storageClass: {{ . }}
4461
{{- end }}
62+
{{- if $static }}
63+
# The upstream chart mounts each Secret at /openbao/userconfig/<name>/.
64+
extraVolumes:
65+
- type: secret
66+
name: {{ $seal.secretName }}
67+
{{- with $seal.previousSecretName }}
68+
- type: secret
69+
name: {{ . }}
70+
{{- end }}
71+
{{- end }}
4572
{{- if gt (int .Values.replicas) 1 }}
4673
standalone:
4774
enabled: false
@@ -68,6 +95,17 @@ spec:
6895
}
6996
{{- end }}
7097
}
98+
{{- if $static }}
99+
100+
seal "static" {
101+
current_key_id = {{ $seal.keyId | quote }}
102+
current_key = "file:///openbao/userconfig/{{ $seal.secretName }}/key"
103+
{{- if $seal.previousSecretName }}
104+
previous_key_id = {{ $seal.previousKeyId | quote }}
105+
previous_key = "file:///openbao/userconfig/{{ $seal.previousSecretName }}/key"
106+
{{- end }}
107+
}
108+
{{- end }}
71109
72110
service_registration "kubernetes" {}
73111
{{- else }}
@@ -85,6 +123,17 @@ spec:
85123
storage "file" {
86124
path = "/openbao/data"
87125
}
126+
{{- if $static }}
127+
128+
seal "static" {
129+
current_key_id = {{ $seal.keyId | quote }}
130+
current_key = "file:///openbao/userconfig/{{ $seal.secretName }}/key"
131+
{{- if $seal.previousSecretName }}
132+
previous_key_id = {{ $seal.previousKeyId | quote }}
133+
previous_key = "file:///openbao/userconfig/{{ $seal.previousSecretName }}/key"
134+
{{- end }}
135+
}
136+
{{- end }}
88137
# Note: service_registration "kubernetes" {} is intentionally omitted
89138
# in standalone mode — it requires an HA-capable storage backend and
90139
# causes a fatal error with storage "file".
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
suite: seal and apiserver egress
2+
3+
# Guards two fixes:
4+
#
5+
# - issue #2793: Cozystack's CiliumNetworkPolicy allow-to-apiserver only admits
6+
# egress to kube-apiserver for pods labelled
7+
# policy.cozystack.io/allow-to-apiserver=true. The upstream chart renders
8+
# server.extraLabels onto the StatefulSet pods, so the wrapper must always
9+
# pass the label through or service_registration "kubernetes" hangs.
10+
#
11+
# - issue #2787: optional auto-unseal with OpenBAO's `seal "static"`. The key
12+
# lives in an operator-owned Secret mounted via server.extraVolumes (the
13+
# upstream chart mounts each entry at /openbao/userconfig/<name>/).
14+
15+
templates:
16+
- templates/openbao.yaml
17+
18+
release:
19+
name: openbao-test
20+
namespace: tenant-test
21+
22+
tests:
23+
- it: "always labels the server pods for apiserver egress"
24+
documentSelector:
25+
path: kind
26+
value: HelmRelease
27+
asserts:
28+
- equal:
29+
path: spec.values.openbao.server.extraLabels["policy.cozystack.io/allow-to-apiserver"]
30+
value: "true"
31+
32+
- it: "keeps shamir by default: no seal stanza and no extra volumes"
33+
documentSelector:
34+
path: kind
35+
value: HelmRelease
36+
asserts:
37+
- notExists:
38+
path: spec.values.openbao.server.extraVolumes
39+
- notMatchRegex:
40+
path: spec.values.openbao.server.standalone.config
41+
pattern: 'seal "static"'
42+
43+
- it: "renders the static seal stanza and the key volume in HA mode"
44+
set:
45+
replicas: 3
46+
seal:
47+
type: static
48+
secretName: openbao-unseal
49+
keyId: key-1
50+
documentSelector:
51+
path: kind
52+
value: HelmRelease
53+
asserts:
54+
- equal:
55+
path: spec.values.openbao.server.extraVolumes
56+
value:
57+
- type: secret
58+
name: openbao-unseal
59+
- matchRegex:
60+
path: spec.values.openbao.server.ha.raft.config
61+
pattern: 'seal "static" \{\n\s+current_key_id = "key-1"\n\s+current_key = "file:///openbao/userconfig/openbao-unseal/key"\n\s*\}'
62+
- notMatchRegex:
63+
path: spec.values.openbao.server.ha.raft.config
64+
pattern: previous_key
65+
66+
- it: "renders the static seal stanza in standalone mode too"
67+
set:
68+
seal:
69+
type: static
70+
secretName: openbao-unseal
71+
keyId: key-1
72+
documentSelector:
73+
path: kind
74+
value: HelmRelease
75+
asserts:
76+
- matchRegex:
77+
path: spec.values.openbao.server.standalone.config
78+
pattern: 'seal "static" \{\n\s+current_key_id = "key-1"\n\s+current_key = "file:///openbao/userconfig/openbao-unseal/key"\n\s*\}'
79+
80+
- it: "mounts the previous key and renders previous_key_* during rotation"
81+
set:
82+
replicas: 3
83+
seal:
84+
type: static
85+
secretName: openbao-unseal
86+
keyId: key-2
87+
previousSecretName: openbao-unseal-old
88+
previousKeyId: key-1
89+
documentSelector:
90+
path: kind
91+
value: HelmRelease
92+
asserts:
93+
- equal:
94+
path: spec.values.openbao.server.extraVolumes
95+
value:
96+
- type: secret
97+
name: openbao-unseal
98+
- type: secret
99+
name: openbao-unseal-old
100+
- matchRegex:
101+
path: spec.values.openbao.server.ha.raft.config
102+
pattern: 'previous_key_id = "key-1"\n\s+previous_key = "file:///openbao/userconfig/openbao-unseal-old/key"'
103+
104+
- it: "rejects static without secretName"
105+
set:
106+
seal:
107+
type: static
108+
keyId: key-1
109+
asserts:
110+
- failedTemplate:
111+
errorMessage: 'seal.secretName is required when seal.type is "static"'
112+
113+
- it: "rejects static without keyId"
114+
set:
115+
seal:
116+
type: static
117+
secretName: openbao-unseal
118+
asserts:
119+
- failedTemplate:
120+
errorMessage: 'seal.keyId is required when seal.type is "static"'
121+
122+
- it: "rejects previousSecretName without previousKeyId"
123+
set:
124+
seal:
125+
type: static
126+
secretName: openbao-unseal
127+
keyId: key-2
128+
previousSecretName: openbao-unseal-old
129+
asserts:
130+
- failedTemplate:
131+
errorMessage: "seal.previousKeyId is required when seal.previousSecretName is set"

‎packages/apps/openbao/values.schema.json‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,47 @@
131131
"description": "Enable the OpenBAO web UI.",
132132
"type": "boolean",
133133
"default": true
134+
},
135+
"seal": {
136+
"description": "Seal configuration. Defaults to Shamir key shares; set `type: static` for auto-unseal.",
137+
"type": "object",
138+
"default": {},
139+
"required": [
140+
"keyId",
141+
"secretName",
142+
"type"
143+
],
144+
"properties": {
145+
"keyId": {
146+
"description": "Permanent identifier of the current key. Change it whenever the key material changes; OpenBAO refuses a key whose identifier it has already seen with different material. Required when `type` is `static`.",
147+
"type": "string",
148+
"default": ""
149+
},
150+
"previousKeyId": {
151+
"description": "Identifier of the previous key. Required when `previousSecretName` is set.",
152+
"type": "string",
153+
"default": ""
154+
},
155+
"previousSecretName": {
156+
"description": "Secret holding the previous key during an n-1 key rotation.",
157+
"type": "string",
158+
"default": ""
159+
},
160+
"secretName": {
161+
"description": "Existing Secret in the release namespace whose key `key` holds the base64 text of 32 random bytes (`head -c 32 /dev/urandom | base64 | tr -d '\\n'`). Required when `type` is `static`.",
162+
"type": "string",
163+
"default": ""
164+
},
165+
"type": {
166+
"description": "Seal type. `shamir` keeps the current behaviour; `static` enables auto-unseal from `secretName`.",
167+
"type": "string",
168+
"default": "shamir",
169+
"enum": [
170+
"shamir",
171+
"static"
172+
]
173+
}
174+
}
134175
}
135176
}
136177
}

‎packages/apps/openbao/values.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,3 +81,26 @@ external: false
8181

8282
## @param {bool} ui - Enable the OpenBAO web UI.
8383
ui: true
84+
85+
##
86+
## @section Seal
87+
##
88+
89+
## @enum {string} SealType - How the OpenBAO barrier is unsealed.
90+
## @value shamir - Shamir key shares. An operator has to unseal every replica after each pod restart.
91+
## @value static - OpenBAO `seal "static"`: the barrier unseals itself from a 32-byte key read from a Secret you manage.
92+
93+
## @typedef {struct} Seal - Seal configuration. With `static`, the key is read from a Secret you create in the release namespace; the chart never generates or rotates it.
94+
## @field {SealType} type - Seal type. `shamir` keeps the current behaviour; `static` enables auto-unseal from `secretName`.
95+
## @field {string} secretName - Existing Secret in the release namespace whose key `key` holds the base64 text of 32 random bytes (`head -c 32 /dev/urandom | base64 | tr -d '\n'`). Required when `type` is `static`.
96+
## @field {string} keyId - Permanent identifier of the current key. Change it whenever the key material changes; OpenBAO refuses a key whose identifier it has already seen with different material. Required when `type` is `static`.
97+
## @field {string} [previousSecretName] - Secret holding the previous key during an n-1 key rotation.
98+
## @field {string} [previousKeyId] - Identifier of the previous key. Required when `previousSecretName` is set.
99+
100+
## @param {Seal} seal - Seal configuration. Defaults to Shamir key shares; set `type: static` for auto-unseal.
101+
seal:
102+
type: shamir
103+
secretName: ""
104+
keyId: ""
105+
previousSecretName: ""
106+
previousKeyId: ""

0 commit comments

Comments
 (0)