|
| 1 | +--- |
| 2 | +title: "Application Backup and Recovery" |
| 3 | +linkTitle: "Backup and Recovery" |
| 4 | +description: "Back up and restore managed databases (Postgres, MariaDB, ClickHouse) with BackupJob, Plan, and RestoreJob." |
| 5 | +weight: 4 |
| 6 | +--- |
| 7 | + |
| 8 | +This guide covers backing up and restoring **Cozystack-managed databases** — Postgres, MariaDB, and ClickHouse — as a tenant user: running one-off and scheduled backups, checking status, and restoring from a backup either in place or into a separate target instance. |
| 9 | + |
| 10 | +{{% alert color="info" %}} |
| 11 | +**Storage, credentials, and the `BackupClass` are admin-provisioned.** Before you can run a `BackupJob`, an administrator provisions the S3 storage and the per-application credential Secrets your driver expects, and creates the cluster-scoped `BackupClass` you reference. Ask your administrator for: |
| 12 | + |
| 13 | +- the `BackupClass` name to use for your application Kind (you cannot list `BackupClass` resources under your tenant kubeconfig — they are cluster-scoped); |
| 14 | +- confirmation that the per-application credential Secrets exist in your namespace for every managed-DB application you want to back up. |
| 15 | + |
| 16 | +Admins follow the [Managed Application Backup Configuration]({{% ref "/docs/next/operations/services/managed-app-backup-configuration" %}}) guide. |
| 17 | +{{% /alert %}} |
| 18 | + |
| 19 | +{{% alert color="warning" %}} |
| 20 | +**These backups are data-only.** Each strategy snapshots the database contents through the operator's native mechanism (CloudNativePG barman, mariadb-operator dumps, Altinity `clickhouse-backup`). They do **not** capture the `apps.cozystack.io/*` CR, its `HelmRelease`, chart values, or operator-managed Secrets. |
| 21 | + |
| 22 | +To restore you must either: |
| 23 | +- keep the source application alive and restore in place (each driver re-bootstraps data into the existing operator-managed cluster), **or** |
| 24 | +- pre-provision an empty target application of the same Kind, then restore into it. |
| 25 | + |
| 26 | +For backups that include the application's Helm release, CRs, and PVC snapshots (used for VMInstance / VMDisk), see [Backup and Recovery (VMs)]({{% ref "/docs/next/virtualization/backup-and-recovery" %}}). |
| 27 | +{{% /alert %}} |
| 28 | + |
| 29 | +## Prerequisites |
| 30 | + |
| 31 | +- A `BackupClass` name handed to you by your administrator (for example, `postgres-data-backup` for a `Postgres` application). |
| 32 | +- An existing managed-DB application (`Postgres`, `MariaDB`, or `ClickHouse`) in your tenant namespace. |
| 33 | +- `kubectl` and a tenant kubeconfig with the `tenant-<ns>-admin` role. |
| 34 | + |
| 35 | +The examples below assume `tenant-user` for the tenant namespace; substitute your own. |
| 36 | + |
| 37 | +## Run a backup |
| 38 | + |
| 39 | +### One-off backup |
| 40 | + |
| 41 | +Use a `BackupJob` for an ad-hoc backup (for example, before a risky change): |
| 42 | + |
| 43 | +```yaml |
| 44 | +apiVersion: backups.cozystack.io/v1alpha1 |
| 45 | +kind: BackupJob |
| 46 | +metadata: |
| 47 | + name: my-postgres-adhoc |
| 48 | + namespace: tenant-user |
| 49 | +spec: |
| 50 | + applicationRef: |
| 51 | + apiGroup: apps.cozystack.io |
| 52 | + kind: Postgres |
| 53 | + name: my-postgres |
| 54 | + backupClassName: postgres-data-backup |
| 55 | +``` |
| 56 | +
|
| 57 | +```bash |
| 58 | +kubectl apply -f backupjob.yaml |
| 59 | +kubectl -n tenant-user get backupjobs |
| 60 | +kubectl -n tenant-user describe backupjob my-postgres-adhoc |
| 61 | +``` |
| 62 | + |
| 63 | +When the `BackupJob` reaches `phase: Succeeded`, the driver creates a `Backup` object with the same name. That name is what you reference when restoring. |
| 64 | + |
| 65 | +Replace `Postgres` / `postgres-data-backup` with `MariaDB` / `mariadb-data-backup` or `ClickHouse` / `clickhouse-data-backup` for the other drivers. |
| 66 | + |
| 67 | +### Scheduled backup |
| 68 | + |
| 69 | +Use a `Plan` for cron-driven recurring backups: |
| 70 | + |
| 71 | +```yaml |
| 72 | +apiVersion: backups.cozystack.io/v1alpha1 |
| 73 | +kind: Plan |
| 74 | +metadata: |
| 75 | + name: my-postgres-daily |
| 76 | + namespace: tenant-user |
| 77 | +spec: |
| 78 | + applicationRef: |
| 79 | + apiGroup: apps.cozystack.io |
| 80 | + kind: Postgres |
| 81 | + name: my-postgres |
| 82 | + backupClassName: postgres-data-backup |
| 83 | + schedule: |
| 84 | + type: cron |
| 85 | + cron: "0 */6 * * *" # every 6 hours |
| 86 | +``` |
| 87 | +
|
| 88 | +Each scheduled run creates a `BackupJob` (and, on success, a `Backup`) named after the `Plan` with a timestamp suffix. |
| 89 | + |
| 90 | +```bash |
| 91 | +kubectl apply -f plan.yaml |
| 92 | +kubectl -n tenant-user get plans |
| 93 | +kubectl -n tenant-user get backupjobs -l backups.cozystack.io/plan=my-postgres-daily |
| 94 | +``` |
| 95 | + |
| 96 | +## Check backup status |
| 97 | + |
| 98 | +List `BackupJob` and `Backup` resources in the namespace: |
| 99 | + |
| 100 | +```bash |
| 101 | +kubectl -n tenant-user get backupjobs |
| 102 | +kubectl -n tenant-user get backups |
| 103 | +``` |
| 104 | + |
| 105 | +Inspect a failed run: |
| 106 | + |
| 107 | +```bash |
| 108 | +kubectl -n tenant-user get backupjob my-postgres-adhoc -o jsonpath='{.status.message}' |
| 109 | +kubectl -n tenant-user describe backupjob my-postgres-adhoc |
| 110 | +kubectl -n tenant-user get events --field-selector involvedObject.name=my-postgres-adhoc |
| 111 | +``` |
| 112 | + |
| 113 | +If `status.message` does not pinpoint the failure, hand the `BackupJob` name to your administrator and they will inspect the operator-native CR the driver created (see [Tenant escalation: driver-side diagnostics]({{% ref "/docs/next/operations/services/managed-app-backup-configuration#tenant-escalation-driver-side-diagnostics" %}}) in the admin guide). |
| 114 | + |
| 115 | +## Restore in place |
| 116 | + |
| 117 | +An **in-place restore** replays the backup into the **same** application. Use this to roll back accidental deletion or corruption on a live database you intend to keep using under the same name. |
| 118 | + |
| 119 | +{{% alert color="warning" %}} |
| 120 | +In-place restore is **destructive**. Each driver wipes or replaces existing data on the source application; any writes since the backup point are lost. If you cannot afford to lose recent writes, use [Restore to a copy](#restore-to-a-copy) instead. |
| 121 | +{{% /alert %}} |
| 122 | + |
| 123 | +```yaml |
| 124 | +apiVersion: backups.cozystack.io/v1alpha1 |
| 125 | +kind: RestoreJob |
| 126 | +metadata: |
| 127 | + name: my-postgres-restore-inplace |
| 128 | + namespace: tenant-user |
| 129 | +spec: |
| 130 | + backupRef: |
| 131 | + name: my-postgres-adhoc |
| 132 | + # targetApplicationRef omitted: driver restores into Backup.spec.applicationRef. |
| 133 | + # options: |
| 134 | + # recoveryTime: "2026-05-01T12:00:00Z" # Postgres only; RFC3339 PITR |
| 135 | +``` |
| 136 | + |
| 137 | +```bash |
| 138 | +kubectl apply -f restorejob.yaml |
| 139 | +kubectl -n tenant-user get restorejobs |
| 140 | +kubectl -n tenant-user describe restorejob my-postgres-restore-inplace |
| 141 | +``` |
| 142 | + |
| 143 | +### Per-driver caveats |
| 144 | + |
| 145 | +- **Postgres (CNPG)** — the driver deletes the live `cnpg.io/Cluster` and its PVCs, then re-bootstraps from the Barman archive. Connections drop for the duration. `spec.options.recoveryTime` (RFC3339) is supported for point-in-time recovery; omit it to restore to the latest WAL. |
| 146 | +- **MariaDB** — the operator replays the logical dump into the live `MariaDB` via `mariadb-import`. Pre-existing tables will collide; pre-truncate the relevant schemas if your dump does not include `DROP TABLE`. |
| 147 | +- **ClickHouse** — the Altinity strategy does **not** pass `clickhouse-backup --rm`. You are responsible for dropping conflicting tables on the source before submitting the `RestoreJob`; otherwise the operation fails with a duplicate-table error. |
| 148 | + |
| 149 | +## Restore to a copy |
| 150 | + |
| 151 | +A **to-copy restore** replays the backup into a **different**, freshly-provisioned application of the same Kind. Use this for disaster-recovery drills, side-by-side validation, branch databases, or migrating to a new version of the upstream operator. |
| 152 | + |
| 153 | +First, provision an empty target application with the same Kind. For example, an empty `Postgres`: |
| 154 | + |
| 155 | +```yaml |
| 156 | +apiVersion: apps.cozystack.io/v1alpha1 |
| 157 | +kind: Postgres |
| 158 | +metadata: |
| 159 | + name: my-postgres-restored |
| 160 | + namespace: tenant-user |
| 161 | +spec: |
| 162 | + # ...same shape as the source, no bootstrap data required... |
| 163 | +``` |
| 164 | + |
| 165 | +Wait for the target to become Ready, then submit a `RestoreJob` that points at it: |
| 166 | + |
| 167 | +```yaml |
| 168 | +apiVersion: backups.cozystack.io/v1alpha1 |
| 169 | +kind: RestoreJob |
| 170 | +metadata: |
| 171 | + name: my-postgres-restore-to-copy |
| 172 | + namespace: tenant-user |
| 173 | +spec: |
| 174 | + backupRef: |
| 175 | + name: my-postgres-adhoc |
| 176 | + targetApplicationRef: |
| 177 | + apiGroup: apps.cozystack.io |
| 178 | + kind: Postgres |
| 179 | + name: my-postgres-restored |
| 180 | +``` |
| 181 | + |
| 182 | +The source application stays untouched. Cross-namespace restores are **not** supported — `targetApplicationRef` is a local reference; the target must live in the same namespace as the `RestoreJob`. |
| 183 | + |
| 184 | +## Limitations and lifecycle |
| 185 | + |
| 186 | +- **Data-only scope.** Application CRs, HelmReleases, chart values, and operator-managed Secrets (e.g. `cnpg.io` superuser secret, `clickhouse-installation` users) are not captured. Pre-provision the target application before a to-copy restore. |
| 187 | +- **Archive retention is driver-owned.** Deleting a Cozystack `Backup` CR removes the artefact reference but leaves the actual S3 object intact. Each driver enforces its own retention: |
| 188 | + - CNPG: `retentionPolicy` on the strategy (admin-owned; default `30d` in the admin example). |
| 189 | + - MariaDB: `cleanupStrategy` on the operator-side `Backup` CR or rotation at the bucket level (admin-owned). |
| 190 | + - ClickHouse: governed by the in-pod sidecar's retention configuration. To purge an archive ahead of schedule, ask your administrator — the call goes against the `clickhouse-backup` HTTP API on the sidecar. |
| 191 | +- **ClickHouse depends on the in-chart sidecar.** The Altinity strategy is a thin HTTP client; the backup itself runs inside each `chi-*` Pod via `clickhouse-backup`. Disabling `backup.enabled` on the application also disables the BackupClass flow. |
| 192 | + |
| 193 | +## Troubleshooting |
| 194 | + |
| 195 | +If a `BackupJob` or `RestoreJob` ends in `phase: Failed`, start with what you can see in your namespace: |
| 196 | + |
| 197 | +```bash |
| 198 | +kubectl -n tenant-user get backupjob my-postgres-adhoc -o jsonpath='{.status.message}' |
| 199 | +kubectl -n tenant-user get restorejob my-postgres-restore-inplace -o jsonpath='{.status.message}' |
| 200 | +kubectl -n tenant-user describe backupjob my-postgres-adhoc |
| 201 | +kubectl -n tenant-user get events --field-selector involvedObject.name=my-postgres-adhoc |
| 202 | +``` |
| 203 | + |
| 204 | +If those do not explain the failure, the next layer of diagnostics lives on the operator-native CR the driver created (`cnpg.io/Backup`, `k8s.mariadb.com/Backup`, or the ClickHouse strategy `Pod` logs). These resources are not reachable under the tenant kubeconfig — hand the `BackupJob` name to your administrator and they will follow [Tenant escalation: driver-side diagnostics]({{% ref "/docs/next/operations/services/managed-app-backup-configuration#tenant-escalation-driver-side-diagnostics" %}}). |
| 205 | + |
| 206 | +## See also |
| 207 | + |
| 208 | +- [Managed Application Backup Configuration]({{% ref "/docs/next/operations/services/managed-app-backup-configuration" %}}) — how administrators define strategies and `BackupClass` resources. |
| 209 | +- [Backup and Recovery (VMs)]({{% ref "/docs/next/virtualization/backup-and-recovery" %}}) — the parallel guide for VMInstance / VMDisk backups (HelmRelease + CRs + PVC snapshots). |
| 210 | +- [Velero Backup Configuration]({{% ref "/docs/next/operations/services/velero-backup-configuration" %}}) — administrator setup for the Velero-driven VM backups. |
0 commit comments