Skip to content

Commit cc59638

Browse files
scooby87scooby87
committed
fix(postgres,mariadb): apply rotations promptly, document restore-copy creds
mariadb-operator re-applies a password from passwordSecretKeyRef only when the referenced Secret carries the k8s.mariadb.com/watch label; without it a passwordRotation bump lands on the operator's --requeue-sql tick (10h plus up to 1h of jitter) instead of promptly, which is backwards for rotating after a leak. Label the credentials Secret so the operator reacts to the change immediately. Document that a Postgres restore into a copy advertises freshly generated application-user passwords in the target credentials Secret that do not match the roles recovered from the backup until the target leaves bootstrap, so the mismatch is a known, converging state rather than a silent failure. Clarify that any change to passwordRotation, not only an increment, triggers a rotation; pin the mariadb-operator root-at-bootstrap rationale to the vendored version; and look the mariadb example password up with a bracketed jsonpath so a username containing a dot still resolves. Assisted-by: LLM Signed-off-by: Alexey Artamonov <[email protected]>
1 parent 9191068 commit cc59638

14 files changed

Lines changed: 47 additions & 28 deletions

File tree

‎api/apps/v1alpha1/mariadb/types.go‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/apps/v1alpha1/postgresql/types.go‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/operations/backup-classes.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,12 @@ The strategy CR `cozy-default-foundationdb` is shipped, but it is **not** bound
5151

5252
**Cleanup gotcha (zombie backup_agent).** Unlike CNPG/MariaDB/Altinity (one-shot operator-side Backup CRs), the FoundationDB driver creates an `apps.foundationdb.org/FoundationDBBackup` CR that drives a **long-lived** `backup_agent` Deployment streaming continuously to S3. Deleting a Cozystack `Backup` (e.g. via retention sweeping) does NOT stop that Deployment — the agent keeps writing until the next BackupJob's `stopOtherFoundationDBBackups` call swaps it out, until an admin invokes `examples/backups/foundationdb/cleanup.sh`, or until the operator-side CR is deleted by hand. If a tenant deletes their last Cozystack Backup and never submits another BackupJob, the agent pods will continue running indefinitely and accumulate S3 PUTs. This is intentional today (the driver has no RBAC verb to stop the operator-side CR on Cozystack-Backup deletion) but admins should be aware of it.
5353

54+
## Postgres: a restored copy converges its application passwords only after leaving bootstrap
55+
56+
Managed Postgres no longer accepts a plaintext `users[].password`; every application-user password is chart-generated into `<release>-credentials` and read only from there. The CNPG backup snapshot therefore carries no passwords — deliberate, since a snapshot that did was the one place a tenant password sat in cleartext.
57+
58+
The consequence for a **restore into a copy** (a differently-named target): the recovered database keeps the source roles with their source password hashes, but the target chart generates fresh random passwords into the target's `<release>-credentials`, and the role-reconciling init-job is skipped while `bootstrap.enabled` is set on the restored app. Until the target leaves bootstrap — clear `bootstrap.enabled` on the Postgres app, or run a later user-driven `helm upgrade` — the passwords advertised in `<target>-credentials` do not match the recovered roles, so a client reading that Secret cannot log in as an application user. The data is intact and reachable as the CNPG superuser; only the chart-managed application logins lag, and they converge (the init-job runs `ALTER ROLE … WITH PASSWORD` onto the target Secret's values) as soon as bootstrap is cleared. After a to-copy restore, converge deliberately rather than assuming the advertised application password works immediately; an in-place restore is unaffected because the release keeps its existing `<release>-credentials`. Automating this convergence (clearing `bootstrap.enabled` once recovery is healthy) is tracked as follow-up.
59+
5460
## ClickHouse: opt-in to the system bucket
5561

5662
The `clickhouse-backup` sidecar runs inside the ClickHouse Pod itself, so the Helm chart is what wires its S3 credentials. Existing tenants on the legacy `backup.s3*` values continue to work unchanged. To switch a release onto the platform bucket, set:

‎examples/backups/mariadb/00-helpers.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,7 @@ mysql_exec() {
157157
pod=$(mariadb_primary_pod "$cr")
158158
[[ -n "$pod" ]] || { log_error "no primary pod for MariaDB CR '$cr'"; return 1; }
159159
pass=$(kubectl -n "$NAMESPACE" get secret "${cr}-credentials" \
160-
-o "jsonpath={.data.${MARIADB_APP_USER}}" | base64 -d)
160+
-o "jsonpath={.data['${MARIADB_APP_USER}']}" | base64 -d)
161161
[[ -n "$pass" ]] || { log_error "no password for '$MARIADB_APP_USER' in ${cr}-credentials"; return 1; }
162162
kubectl -n "$NAMESPACE" exec "$pod" -c mariadb -- \
163163
mariadb -u"$MARIADB_APP_USER" -p"$pass" -h "${cr}-primary" -e "$sql"

0 commit comments

Comments
 (0)