You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit a0b9e5e
Browse filesBrowse the repository at this point in the historyBrowse files
docs(backups): describe BackupRetentionPolicy in backups core API design
Fold the retention design into the backups core API DESIGN.md as §4.6 instead
of a standalone proposal file, so it lives beside the resources it extends and
realises the "higher-level retention policies" §4.4 earmarks.
BackupRetentionPolicy is a namespaced CRD whose spec is a flat minCount/
maxCount/maxAge; a Backup joins one through a single retention-policy label
stamped from a retentionPolicyName field common to Plan, BackupJob and
BackupClass, the last acting as the namespace default. Matching is
namespace-local so identically named policies never collide, and artifacts move
between policies by relabel. §4.6 also spells out when each knob applies (floor
vs ceiling vs TTL, and the recovery-window caveat when a count ceiling meets a
frequent schedule). maxAge is a string with an h/d/w suffix so a window reads as
"90d" rather than a metav1.Duration capped at hours.
The Backup section declares that the platform owns backup cleanup end to end:
it never delegates expiry to an application operator's TTL or retention window,
drives deletion through the Backup object and its finalizer, and requires each
driver to reclaim physically under a single-authority, idempotent, serialized
contract.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <[email protected]>
Copy file name to clipboardExpand all lines: api/backups/v1alpha1/DESIGN.md
+170-1Lines changed: 170 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -374,10 +374,22 @@ type BackupStatus struct {
374
374
375
375
* List backups for a given application/plan.
376
376
* Anchor `RestoreJob` operations.
377
-
* Implement higher-level policies (retention) if needed.
377
+
* Implement higher-level retention policies — see §4.6.
378
378
379
379
**Note:** Parameters are resolved from `BackupClass` when the `BackupJob` is created. The driver uses these parameters to determine where to store backups. The storage location itself is managed by the driver (e.g., Velero's `BackupStorageLocation` CRD) and is not directly referenced in the `Backup` resource. When restoring, the driver resolves the storage location from the original `BackupClass` parameters or from the driver's own metadata.
380
380
381
+
**Backup cleanup**
382
+
383
+
Cozystack owns the lifecycle of a `Backup` artifact end to end. The platform is the single authority that decides when a backup is removed, and it MUST NOT delegate that decision to any mechanism controlled by the application's own operator — not a TTL, not a retention window, not an object-store lifecycle rule (CNPG `Cluster.spec.backup.retentionPolicy`, MariaDB `maxRetention`, and the like are left unset). Retention is expressed only through `BackupRetentionPolicy` (§4.6) and enforced only by the core sweep, so an operator upgrade or a changed operator default can never silently expire, keep, or resurrect a backup the platform still accounts for.
384
+
385
+
Deletion is driven through the `Backup` object: removing it fires the `backups.cozystack.io/cleanup` finalizer, and the strategy driver reclaims both the record and the physical archive on the platform's behalf.
386
+
387
+
Every strategy driver MUST honor this contract:
388
+
389
+
1.**Platform-owned, single authority.** The application operator's native retention MUST be unset; the core retention sweep is the only mechanism that deletes backup artifacts, so the operator and the platform never race over the same archive.
390
+
2.**Idempotency and finalizer-hold.** The `backups.cozystack.io/cleanup` finalizer holds the `Backup` until the driver confirms the archive is gone; a retried or duplicated deletion of the same backup is a no-op.
391
+
3.**Serialization.** Physical deletion for a server MUST NOT run concurrently with an in-flight backup or WAL archiving for that same server.
392
+
381
393
---
382
394
383
395
### 4.5 RestoreJob
@@ -447,6 +459,162 @@ Drivers must not modify `RestoreJob.spec` or delete `RestoreJob`.
Cap how many `Backup` artifacts survive for an application and for how long — the higher-level retention earmarked in §4.4. A policy is a named set of rules; membership is decided by a label on `Backup`, not by a selector.
469
+
470
+
**Key fields (spec)**
471
+
472
+
```go
473
+
typeBackupRetentionPolicySpecstruct {
474
+
// Never delete below this many newest Ready backups; overrides MaxAge. Defaults to 1.
475
+
MinCountint32`json:"minCount"`
476
+
// Keep at most this many newest Ready backups. Unset means unbounded.
477
+
MaxCount *int32`json:"maxCount,omitempty"`
478
+
// Delete backups older than this, by spec.takenAt, bounded by MinCount.
479
+
// Suffix h/d/w (e.g. "7d", "90d"); metav1.Duration is not used because it stops at hours.
480
+
MaxAgestring`json:"maxAge,omitempty"`
481
+
}
482
+
```
483
+
484
+
There is intentionally no selector: a `Backup` carries exactly one policy name in a label — a scalar — so it is never claimed by two policies at once.
485
+
486
+
**Use cases**
487
+
488
+
The three knobs are orthogonal, and picking the right one starts from the intent:
489
+
490
+
*`minCount` is a floor — "keep at least N newest, whatever their age". It guarantees a minimum but sets no ceiling: with many young backups it keeps them all. Example: a fresh backup plus a six-month-old manual one under `minCount: 2, maxAge: 30d` — the old one survives because it is among the two newest, even though it is past `maxAge`.
491
+
*`maxAge` is a TTL — the natural knob for regular scheduled backups, expressing a recovery window in time.
492
+
*`maxCount` is a ceiling — "keep at most N". It is the only knob that bounds the object count (and therefore storage cost) independently of schedule frequency, which matters in three situations `minCount`/`maxAge` cannot cover: an unbounded S3 footprint under a frequent cron, a burst of ad-hoc backups before a migration, and a misconfigured "every minute" schedule. In all three the backups are still young, so `maxAge` does not fire and only a count ceiling trims them.
493
+
494
+
`minCount` cannot substitute for `maxCount`: a floor never trims a large set of young backups.
495
+
496
+
| State |`minCount=2, maxAge=30d`|`maxCount=5`|
497
+
|---|---|---|
498
+
| 2 backups (fresh + 180d) | both kept | both kept |
499
+
| 20 young backups (<30d, burst or frequent cron) | all 20 kept | 5 kept |
500
+
501
+
**Recovery-window caveat.** On a continuous-archiving driver `maxCount` indirectly bounds the recovery window: under an hourly `Plan`, `maxCount: 3` collapses PITR to roughly three hours, because dropping the oldest base backups drops the WALs they anchored. This is intuitive for `maxAge` but surprising for `maxCount` at high frequency — surface the effective window rather than only the count.
502
+
503
+
**Binding fields**
504
+
505
+
A `Backup` joins a policy through the `backups.cozystack.io/retention-policy` label, stamped by the core controller when the `Backup` is created. One new optional field, `retentionPolicyName`, feeds it on `Plan`, `BackupJob`, and `BackupClass`, capped at 63 characters because the value lands in a DNS-1123 label:
506
+
507
+
```go
508
+
// Plan.spec, BackupJob.spec, and BackupClass.spec (BackupClass as the namespace default).
Resolution when a `Backup` is created, highest precedence first: `BackupJob.spec.retentionPolicyName` (or `Plan.spec.retentionPolicyName`, inherited onto the BackupJob), then `BackupClass.spec.retentionPolicyName`, then none — a `Backup` with no label is never swept.
513
+
514
+
**Retention semantics**
515
+
516
+
For a policy `P` in namespace `N`:
517
+
518
+
1. List `Backup` in `N` labeled `retention-policy=P.name`; keep only `status.phase == Ready`. `Failed`/`Pending` backups never consume the floor and are pruned only by `MaxAge`.
519
+
2. Sort by `spec.takenAt` descending.
520
+
3. Protect the newest `MinCount` unconditionally.
521
+
4. From the rest, delete a backup when its index `>= MaxCount`**or**`now - takenAt > MaxAge`. `MinCount` always wins over `MaxAge`.
522
+
5. Delete via the `Backup` object; the `backups.cozystack.io/cleanup` finalizer runs the driver's cleanup path.
523
+
524
+
The sweep is an idempotent set difference — safe to run on every reconcile and on `Backup` create/delete/label events.
525
+
526
+
**Namespace-local matching**
527
+
528
+
Both `Backup` and `BackupRetentionPolicy` are namespaced, and matching is scoped to the policy's own namespace. A policy named `default` in one namespace and one so named in another are distinct objects governing disjoint backup sets, so identical names never collide. A name identifies a policy only within a namespace, which suffices because a `Backup` and its policy always share one.
529
+
530
+
**Migration between policies**
531
+
532
+
Moving artifacts to another policy is a label change, with no `Plan` edit:
Changing `retentionPolicyName` on a `Plan` moves only future backups; existing ones stay until relabeled. Relabeling onto a stricter policy may delete artifacts on the next sweep; onto a laxer one, it spares them.
541
+
542
+
**Examples**
543
+
544
+
Platform default, shipped per tenant namespace and wired as the BackupClass default:
0 commit comments