You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit fd17745
Browse filesBrowse the repository at this point in the historyBrowse files
docs(operations): correct the cozy-default override path and address review findings (next + v1.5)
The admin-override example documented a path the cozystack operator
silently ignores (cozystack/cozystack#3245): that PackageSource exposes
only the platform component. cozystack/cozystack#3333 wires the working
path — spec.components.platform.values.backupStorage — which the
platform chart forwards into the backupstrategy-controller component.
Update the example in both next/ and v1.5/, explain the forwarding,
call out the two plausible-looking paths that do not work, use the same
platform Package path in the external-S3 section, and add the new
backupStorage key to the platform-package values reference tables.
Review fixes from PR #622 (IvanHunters, gemini, coderabbit):
- add aliases frontmatter on backup-classes.md for the two deleted
pages, so published permalinks (release announcements link them as
absolute URLs) keep resolving
- replace the malformed BSL readiness check (jsonpath '=' 'Available'
passed as extra kubectl args) with kubectl wait
--for=jsonpath='{.status.phase}'=Available, and rephrase the
tenant-facing variant to 'returns Available'
- namespace the flux reconcile example (-n cozy-backup-controller, the
namespace the backupstrategy-controller HelmRelease actually lives
in per packages/core/platform/sources/backupstrategy-controller.yaml)
- link the FoundationDB caveat section instead of quoting its title
- 'After v1.4' -> 'Starting with v1.4' in the upgrade note
- mention Etcd in the tenant guide intro/description and add
etcd.aenix.io/EtcdBackup to the escalation diagnostics list
- merge the duplicated Backup Classes bullet in See also
Assisted-By: Claude <[email protected]>
Co-Authored-By: Claude Fable 5 <[email protected]>
Signed-off-by: Andrey Kolkov <[email protected]>
Copy file name to clipboardExpand all lines: content/en/docs/next/applications/backup-and-recovery.md
+4-5Lines changed: 4 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
---
2
2
title: "Application Backup and Recovery"
3
3
linkTitle: "Backup and Recovery"
4
-
description: "Back up and restore managed databases (Postgres, MariaDB, ClickHouse) with BackupJob, Plan, and RestoreJob."
4
+
description: "Back up and restore managed databases (Postgres, MariaDB, ClickHouse, Etcd) with BackupJob, Plan, and RestoreJob."
5
5
weight: 4
6
6
---
7
7
8
-
This guide covers backing up and restoring **Cozystack-managed databases** — Postgres, MariaDB, and ClickHouse — as a tenant user: running one-off and scheduled backups, checking status, and restoring from a backup either in place or into a separate target instance.
8
+
This guide covers backing up and restoring **Cozystack-managed databases** — Postgres, MariaDB, ClickHouse, and Etcd — as a tenant user: running one-off and scheduled backups, checking status, and restoring from a backup either in place or into a separate target instance.
9
9
10
10
{{% alert color="info" %}}
11
11
**Storage and the `BackupClass` are platform-provisioned.** Cozystack ships a single cluster-scoped `BackupClass` named `cozy-default` that covers Postgres, MariaDB, ClickHouse, Etcd, VMInstance, and VMDisk via a per-Kind `strategies[]` array. You reference it by name from `BackupJob` / `Plan` / `RestoreJob` — there is no per-application `BackupClass`, and you do **not** create or supply S3 credentials, endpoints, or paths. The platform projects a credentials Secret (`cozy-backups-creds`) into your tenant namespace automatically right before each BackupJob runs.
kubectl -n tenant-user get events --field-selector involvedObject.name=my-postgres-adhoc
201
201
```
202
202
203
-
If those do not explain the failure, the next layer of diagnostics lives on the operator-native CR the driver created (`cnpg.io/Backup`, `k8s.mariadb.com/Backup`, or the ClickHouse strategy `Pod` logs). These resources are not reachable under the tenant kubeconfig — hand the `BackupJob` name to your administrator and they will follow [Backup Classes]({{% ref "/docs/next/operations/services/backup-classes" %}}).
203
+
If those do not explain the failure, the next layer of diagnostics lives on the operator-native CR the driver created (`cnpg.io/Backup`, `k8s.mariadb.com/Backup`, `etcd.aenix.io/EtcdBackup`, or the ClickHouse strategy `Pod` logs). These resources are not reachable under the tenant kubeconfig — hand the `BackupJob` name to your administrator and they will follow [Backup Classes]({{% ref "/docs/next/operations/services/backup-classes" %}}).
204
204
205
205
## See also
206
206
207
-
- [Backup Classes]({{% ref "/docs/next/operations/services/backup-classes" %}}) — how administrators define strategies and `BackupClass` resources.
207
+
- [Backup Classes]({{% ref "/docs/next/operations/services/backup-classes" %}}) — how administrators define strategies and `BackupClass` resources for databases and VMs.
208
208
- [Backup and Recovery (VMs)]({{% ref "/docs/next/virtualization/backup-and-recovery" %}}) — the parallel guide for VMInstance / VMDisk backups (HelmRelease + CRs + PVC snapshots).
209
-
- [Backup Classes]({{% ref "/docs/next/operations/services/backup-classes" %}}) — administrator setup for the Velero-driven VM backups.
Copy file name to clipboardExpand all lines: content/en/docs/next/operations/configuration/platform-package.md
+6Lines changed: 6 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -157,6 +157,12 @@ gateway:
157
157
| --- | --- | --- |
158
158
| `scheduling.globalAppTopologySpreadConstraints` | `""` | Global pod topology spread constraints applied to all managed applications. |
159
159
160
+
#### Backup storage
161
+
162
+
| Value | Default | Description |
163
+
| --- | --- | --- |
164
+
| `backupStorage` | `{}` | S3 coordinates for the platform-managed `cozy-default` BackupClass. The whole block is forwarded into the `backupstrategy-controller` component and deep-merged over its chart defaults; keys include `provisionBucket`, `bucketName`, `endpoint`, `region`, `forcePathStyle`, `systemSecretName`, and `systemNamespaces`. See [Backup Classes]({{% ref "/docs/next/operations/services/backup-classes" %}}) for the knob-by-knob reference. |
The FoundationDB strategy CR is rendered by the chart so admins can reference it from a custom BackupClass once the operator-side plumbing (mounting `cozy-backups-creds` into the `cozy-foundationdb-operator` Deployment) is wired manually. See "FoundationDB caveat" below.
35
+
The FoundationDB strategy CR is rendered by the chart so admins can reference it from a custom BackupClass once the operator-side plumbing (mounting `cozy-backups-creds` into the `cozy-foundationdb-operator` Deployment) is wired manually. See the [FoundationDB caveat](#foundationdb-caveat) below.
33
36
34
37
### Endpoint format per driver
35
38
@@ -93,15 +96,15 @@ The bucket lives in `tenant-root` and is provisioned through the `apps.cozystack
93
96
94
97
### Bootstrap window
95
98
96
-
On a fresh-cluster install, the Velero `BackupStorageLocation` `cozy-default` is rendered before the credentials projector has had a chance to copy `cozy-backups-creds` into `cozy-velero`. The BSL reports `Unavailable` until the projector's first synchronous round completes (which happens immediately when the `backupstrategy-controller` Pod becomes Ready — typically tens of seconds after `helm install` returns, not minutes). Velero rejects new `Backup` AND `Restore` requests against `storageLocation: cozy-default` during that window. Plan VM backup automation accordingly, or wait for `kubectl -n cozy-velero get bsl cozy-default -o jsonpath='{.status.phase}' = Available` before submitting backups.
99
+
On a fresh-cluster install, the Velero `BackupStorageLocation` `cozy-default` is rendered before the credentials projector has had a chance to copy `cozy-backups-creds` into `cozy-velero`. The BSL reports `Unavailable` until the projector's first synchronous round completes (which happens immediately when the `backupstrategy-controller` Pod becomes Ready — typically tens of seconds after `helm install` returns, not minutes). Velero rejects new `Backup` AND `Restore` requests against `storageLocation: cozy-default` during that window. Plan VM backup automation accordingly, or wait for the BSL to become ready before submitting backups: `kubectl -n cozy-velero wait backupstoragelocation cozy-default --for=jsonpath='{.status.phase}'=Available --timeout=5m`.
97
100
98
101
**Note on controller restarts.** The BSL flickers `Unavailable` on every `backupstrategy-controller` pod restart while the projector replays its first synchronous round. The window is short (single-digit seconds) but operators who alert on BSL availability should suppress alerts during the controller's `kube_pod_container_status_restarts_total{container=backupstrategy-controller}` events or use a longer evaluation window than the projector tick (60s).
99
102
100
103
### Cozy-default Bucket bootstrap
101
104
102
105
`cozy-default`ships an `apps.cozystack.io/Bucket cozy-backups` CR in `tenant-root`, which the bucket-application chart turns into a `BucketClaim`; the COSI driver then assigns the real S3 bucket name and writes it to the BucketClaim's `.status.bucketName`. The strategy templates and the Velero BSL all read that real bucket name (Helm `lookup` against the BucketClaim). On a fresh install the BucketClaim takes a short reconcile cycle to populate its status — until it does, the strategy templates render empty and only the `Bucket` CR + `BackupClass` are present in the cluster. Flux re-renders the HelmRelease on its standard interval (default 10 minutes), at which point the populated BucketClaim status causes the missing strategy templates to materialise.
103
106
104
-
If you need the BackupClass functional immediately (e.g. an e2e), trigger a Flux reconcile (`flux reconcile helmrelease backupstrategy-controller`) once you see `kubectl get bucketclaim -n tenant-root bucket-cozy-backups -o jsonpath='{.status.bucketName}'` non-empty.
107
+
If you need the BackupClass functional immediately (e.g. an e2e), trigger a Flux reconcile (`flux reconcile helmrelease backupstrategy-controller -n cozy-backup-controller`) once you see `kubectl get bucketclaim -n tenant-root bucket-cozy-backups -o jsonpath='{.status.bucketName}'` non-empty.
105
108
106
109
### Observability
107
110
@@ -114,7 +117,7 @@ Alert on `rate(failures_total) > 0` or `absent_over_time(successes_total[10m])`
114
117
115
118
## Admin overrides for `cozy-default`
116
119
117
-
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-foundationdb`, the two `cozy-default-velero-*`). The supported override path is the cozystack `Package` CR, which lets admins inject Helm values into platform components:
120
+
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-foundationdb`, the two `cozy-default-velero-*`). The supported override path is the `backupStorage` block on the **`platform` component** of the `cozystack.cozystack-platform` Package CR:
118
121
119
122
```yaml
120
123
apiVersion: cozystack.io/v1alpha1
@@ -123,7 +126,7 @@ metadata:
123
126
name: cozystack.cozystack-platform
124
127
spec:
125
128
components:
126
-
backupstrategy-controller:
129
+
platform:
127
130
values:
128
131
backupStorage:
129
132
provisionBucket: true # default; set false for external S3
@@ -136,6 +139,8 @@ spec:
136
139
- cozy-velero
137
140
```
138
141
142
+
The platform chart forwards this block into the child `Package cozystack.backupstrategy-controller` as component values, from where the cozystack operator merges it into the `backupstrategy-controller` HelmRelease over the chart defaults. Two paths that look plausible do **not** work: `spec.components.backupstrategy-controller`on the `cozystack.cozystack-platform` Package is silently ignored (the only component under that PackageSource is `platform`), and patching the child `Package cozystack.backupstrategy-controller` directly is reverted whenever the platform helm-reconcile re-renders it.
143
+
139
144
| Knob | Effect |
140
145
|---|---|
141
146
| `provisionBucket` | Toggle creation of the in-cluster `apps.cozystack.io/Bucket` CR. Set `false` for external S3 (see [Disabling the platform-managed bucket](#disabling-the-platform-managed-bucket)). |
@@ -164,13 +169,13 @@ The system-managed credentials Secret is the **only** way for in-cluster strateg
164
169
165
170
## Disabling the platform-managed bucket
166
171
167
-
If a deployment runs against an external S3 (no SeaweedFS), set `backupStorage.provisionBucket: false` in the `backupstrategy-controller` values and create the source credentials Secret in `tenant-root` manually (flat-key format: `accessKey` / `secretKey` / `endpoint` / `bucketName`; or the raw COSI `BucketInfo` JSON). Update `backupStorage.endpoint`, `backupStorage.region`, and (for VM backups) the chart's Velero BSL settings to point at the external S3.
172
+
If a deployment runs against an external S3 (no SeaweedFS), set `backupStorage.provisionBucket: false` through the same platform Package path as above (`spec.components.platform.values.backupStorage`) and create the source credentials Secret in `tenant-root` manually (flat-key format: `accessKey` / `secretKey` / `endpoint` / `bucketName`; or the raw COSI `BucketInfo` JSON). In the same `backupStorage` block, update `endpoint`, `region`, and (for VM backups) the chart's Velero BSL settings to point at the external S3.
168
173
169
174
## Upgrade notes from chart-managed backups
170
175
171
176
> **Postgres `backup.enabled: true` with placeholder credentials no longer renders `barmanObjectStore` on upgrade.**
172
177
>
173
-
> The pre-v1.4 defaults for `backup.s3AccessKey` / `backup.s3SecretKey` in `packages/apps/postgres/values.yaml` were the literal `"<your-access-key>"` / `"<your-secret-key>"` placeholders, so the Postgres chart still rendered `spec.backup.barmanObjectStore` on the `cnpg.io/Cluster` (with junk credentials, `archive_command` failing at runtime). After v1.4 those defaults are empty strings and the chart NO LONGER renders the backup block at all when the placeholders are unmodified. Tenants on the legacy chart-managed flow who relied on those placeholders see their `barmanObjectStore` disappear from the live `Cluster` on `helm upgrade`. Action — pick one:
178
+
> The pre-v1.4 defaults for `backup.s3AccessKey` / `backup.s3SecretKey` in `packages/apps/postgres/values.yaml` were the literal `"<your-access-key>"` / `"<your-secret-key>"` placeholders, so the Postgres chart still rendered `spec.backup.barmanObjectStore` on the `cnpg.io/Cluster` (with junk credentials, `archive_command` failing at runtime). Starting with v1.4 those defaults are empty strings and the chart NO LONGER renders the backup block at all when the placeholders are unmodified. Tenants on the legacy chart-managed flow who relied on those placeholders see their `barmanObjectStore` disappear from the live `Cluster` on `helm upgrade`. Action — pick one:
174
179
>
175
180
> - **Move to the platform flow (recommended).** Set `backup.useSystemBucket: true`; the chart leaves `barmanObjectStore` unset and the CNPG backup driver SSA-patches it onto the live `Cluster` at first BackupJob time. No tenant-side keys required.
176
181
> - **Stay on the legacy chart-managed flow.** Supply real `backup.s3AccessKey` / `backup.s3SecretKey` (or a pre-existing `backup.s3CredentialsSecret.name`); the chart renders `barmanObjectStore` exactly as before.
Copy file name to clipboardExpand all lines: content/en/docs/next/virtualization/backup-and-recovery.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -42,7 +42,7 @@ cozy-default 14m
42
42
`cozy-default` is the platform-shipped BackupClass; its `strategies[]` array binds the Velero driver for both `VMInstance` and `VMDisk`. Use this name when creating a BackupJob or Plan, or substitute a sibling class name if your administrator has created one.
43
43
44
44
{{% alert color="info" %}}
45
-
**Fresh-cluster bootstrap window.** On a fresh-cluster install, the Velero `BackupStorageLocation``cozy-default` reports `Unavailable` for tens of seconds after `helm install` returns, until the platform's credentials projector lands `cozy-backups-creds` into `cozy-velero`. Velero rejects new `Backup` and `Restore` requests against `storageLocation: cozy-default` during that window. If a BackupJob you submit fails immediately with a Velero error referencing storage, wait and retry, or ask your administrator to check `kubectl -n cozy-velero get bsl cozy-default -o jsonpath='{.status.phase}' = Available`. See the [Backup Classes admin guide]({{% ref "/docs/next/operations/services/backup-classes" %}}) for details.
45
+
**Fresh-cluster bootstrap window.** On a fresh-cluster install, the Velero `BackupStorageLocation``cozy-default` reports `Unavailable` for tens of seconds after `helm install` returns, until the platform's credentials projector lands `cozy-backups-creds` into `cozy-velero`. Velero rejects new `Backup` and `Restore` requests against `storageLocation: cozy-default` during that window. If a BackupJob you submit fails immediately with a Velero error referencing storage, wait and retry, or ask your administrator to check that `kubectl -n cozy-velero get bsl cozy-default -o jsonpath='{.status.phase}'` returns `Available`. See the [Backup Classes admin guide]({{% ref "/docs/next/operations/services/backup-classes" %}}) for details.
Copy file name to clipboardExpand all lines: content/en/docs/v1.5/applications/backup-and-recovery.md
+4-5Lines changed: 4 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
---
2
2
title: "Application Backup and Recovery"
3
3
linkTitle: "Backup and Recovery"
4
-
description: "Back up and restore managed databases (Postgres, MariaDB, ClickHouse) with BackupJob, Plan, and RestoreJob."
4
+
description: "Back up and restore managed databases (Postgres, MariaDB, ClickHouse, Etcd) with BackupJob, Plan, and RestoreJob."
5
5
weight: 4
6
6
---
7
7
8
-
This guide covers backing up and restoring **Cozystack-managed databases** — Postgres, MariaDB, and ClickHouse — as a tenant user: running one-off and scheduled backups, checking status, and restoring from a backup either in place or into a separate target instance.
8
+
This guide covers backing up and restoring **Cozystack-managed databases** — Postgres, MariaDB, ClickHouse, and Etcd — as a tenant user: running one-off and scheduled backups, checking status, and restoring from a backup either in place or into a separate target instance.
9
9
10
10
{{% alert color="info" %}}
11
11
**Storage and the `BackupClass` are platform-provisioned.** Cozystack ships a single cluster-scoped `BackupClass` named `cozy-default` that covers Postgres, MariaDB, ClickHouse, Etcd, VMInstance, and VMDisk via a per-Kind `strategies[]` array. You reference it by name from `BackupJob` / `Plan` / `RestoreJob` — there is no per-application `BackupClass`, and you do **not** create or supply S3 credentials, endpoints, or paths. The platform projects a credentials Secret (`cozy-backups-creds`) into your tenant namespace automatically right before each BackupJob runs.
kubectl -n tenant-user get events --field-selector involvedObject.name=my-postgres-adhoc
201
201
```
202
202
203
-
If those do not explain the failure, the next layer of diagnostics lives on the operator-native CR the driver created (`cnpg.io/Backup`, `k8s.mariadb.com/Backup`, or the ClickHouse strategy `Pod` logs). These resources are not reachable under the tenant kubeconfig — hand the `BackupJob` name to your administrator and they will follow [Backup Classes]({{% ref "/docs/v1.5/operations/services/backup-classes" %}}).
203
+
If those do not explain the failure, the next layer of diagnostics lives on the operator-native CR the driver created (`cnpg.io/Backup`, `k8s.mariadb.com/Backup`, `etcd.aenix.io/EtcdBackup`, or the ClickHouse strategy `Pod` logs). These resources are not reachable under the tenant kubeconfig — hand the `BackupJob` name to your administrator and they will follow [Backup Classes]({{% ref "/docs/v1.5/operations/services/backup-classes" %}}).
204
204
205
205
## See also
206
206
207
-
- [Backup Classes]({{% ref "/docs/v1.5/operations/services/backup-classes" %}}) — how administrators define strategies and `BackupClass` resources.
207
+
- [Backup Classes]({{% ref "/docs/v1.5/operations/services/backup-classes" %}}) — how administrators define strategies and `BackupClass` resources for databases and VMs.
208
208
- [Backup and Recovery (VMs)]({{% ref "/docs/v1.5/virtualization/backup-and-recovery" %}}) — the parallel guide for VMInstance / VMDisk backups (HelmRelease + CRs + PVC snapshots).
209
-
- [Backup Classes]({{% ref "/docs/v1.5/operations/services/backup-classes" %}}) — administrator setup for the Velero-driven VM backups.
Copy file name to clipboardExpand all lines: content/en/docs/v1.5/operations/configuration/platform-package.md
+6Lines changed: 6 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -151,6 +151,12 @@ gateway:
151
151
| --- | --- | --- |
152
152
| `scheduling.globalAppTopologySpreadConstraints` | `""` | Global pod topology spread constraints applied to all managed applications. |
153
153
154
+
#### Backup storage
155
+
156
+
| Value | Default | Description |
157
+
| --- | --- | --- |
158
+
| `backupStorage` | `{}` | S3 coordinates for the platform-managed `cozy-default` BackupClass. The whole block is forwarded into the `backupstrategy-controller` component and deep-merged over its chart defaults; keys include `provisionBucket`, `bucketName`, `endpoint`, `region`, `forcePathStyle`, `systemSecretName`, and `systemNamespaces`. See [Backup Classes]({{% ref "/docs/v1.5/operations/services/backup-classes" %}}) for the knob-by-knob reference. |
0 commit comments