Skip to content

Commit a0b9e5e

Browse files
committed
docs(backups): describe BackupRetentionPolicy in backups core API design
Fold the retention design into the backups core API DESIGN.md as §4.6 instead of a standalone proposal file, so it lives beside the resources it extends and realises the "higher-level retention policies" §4.4 earmarks. BackupRetentionPolicy is a namespaced CRD whose spec is a flat minCount/ maxCount/maxAge; a Backup joins one through a single retention-policy label stamped from a retentionPolicyName field common to Plan, BackupJob and BackupClass, the last acting as the namespace default. Matching is namespace-local so identically named policies never collide, and artifacts move between policies by relabel. §4.6 also spells out when each knob applies (floor vs ceiling vs TTL, and the recovery-window caveat when a count ceiling meets a frequent schedule). maxAge is a string with an h/d/w suffix so a window reads as "90d" rather than a metav1.Duration capped at hours. The Backup section declares that the platform owns backup cleanup end to end: it never delegates expiry to an application operator's TTL or retention window, drives deletion through the Backup object and its finalizer, and requires each driver to reclaim physically under a single-authority, idempotent, serialized contract. Assisted-by: LLM Signed-off-by: Andrey Kolkov <[email protected]>
1 parent 56e10b1 commit a0b9e5e

1 file changed

Lines changed: 170 additions & 1 deletion

File tree

‎api/backups/v1alpha1/DESIGN.md‎

Lines changed: 170 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -374,10 +374,22 @@ type BackupStatus struct {
374374

375375
* List backups for a given application/plan.
376376
* Anchor `RestoreJob` operations.
377-
* Implement higher-level policies (retention) if needed.
377+
* Implement higher-level retention policies — see §4.6.
378378

379379
**Note:** Parameters are resolved from `BackupClass` when the `BackupJob` is created. The driver uses these parameters to determine where to store backups. The storage location itself is managed by the driver (e.g., Velero's `BackupStorageLocation` CRD) and is not directly referenced in the `Backup` resource. When restoring, the driver resolves the storage location from the original `BackupClass` parameters or from the driver's own metadata.
380380

381+
**Backup cleanup**
382+
383+
Cozystack owns the lifecycle of a `Backup` artifact end to end. The platform is the single authority that decides when a backup is removed, and it MUST NOT delegate that decision to any mechanism controlled by the application's own operator — not a TTL, not a retention window, not an object-store lifecycle rule (CNPG `Cluster.spec.backup.retentionPolicy`, MariaDB `maxRetention`, and the like are left unset). Retention is expressed only through `BackupRetentionPolicy` (§4.6) and enforced only by the core sweep, so an operator upgrade or a changed operator default can never silently expire, keep, or resurrect a backup the platform still accounts for.
384+
385+
Deletion is driven through the `Backup` object: removing it fires the `backups.cozystack.io/cleanup` finalizer, and the strategy driver reclaims both the record and the physical archive on the platform's behalf.
386+
387+
Every strategy driver MUST honor this contract:
388+
389+
1. **Platform-owned, single authority.** The application operator's native retention MUST be unset; the core retention sweep is the only mechanism that deletes backup artifacts, so the operator and the platform never race over the same archive.
390+
2. **Idempotency and finalizer-hold.** The `backups.cozystack.io/cleanup` finalizer holds the `Backup` until the driver confirms the archive is gone; a retried or duplicated deletion of the same backup is a no-op.
391+
3. **Serialization.** Physical deletion for a server MUST NOT run concurrently with an in-flight backup or WAL archiving for that same server.
392+
381393
---
382394

383395
### 4.5 RestoreJob
@@ -447,6 +459,162 @@ Drivers must not modify `RestoreJob.spec` or delete `RestoreJob`.
447459

448460
---
449461

462+
### 4.6 BackupRetentionPolicy
463+
464+
**Group/Kind**
465+
`backups.cozystack.io/v1alpha1, Kind=BackupRetentionPolicy` (namespaced)
466+
467+
**Purpose**
468+
Cap how many `Backup` artifacts survive for an application and for how long — the higher-level retention earmarked in §4.4. A policy is a named set of rules; membership is decided by a label on `Backup`, not by a selector.
469+
470+
**Key fields (spec)**
471+
472+
```go
473+
type BackupRetentionPolicySpec struct {
474+
// Never delete below this many newest Ready backups; overrides MaxAge. Defaults to 1.
475+
MinCount int32 `json:"minCount"`
476+
// Keep at most this many newest Ready backups. Unset means unbounded.
477+
MaxCount *int32 `json:"maxCount,omitempty"`
478+
// Delete backups older than this, by spec.takenAt, bounded by MinCount.
479+
// Suffix h/d/w (e.g. "7d", "90d"); metav1.Duration is not used because it stops at hours.
480+
MaxAge string `json:"maxAge,omitempty"`
481+
}
482+
```
483+
484+
There is intentionally no selector: a `Backup` carries exactly one policy name in a label — a scalar — so it is never claimed by two policies at once.
485+
486+
**Use cases**
487+
488+
The three knobs are orthogonal, and picking the right one starts from the intent:
489+
490+
* `minCount` is a floor — "keep at least N newest, whatever their age". It guarantees a minimum but sets no ceiling: with many young backups it keeps them all. Example: a fresh backup plus a six-month-old manual one under `minCount: 2, maxAge: 30d` — the old one survives because it is among the two newest, even though it is past `maxAge`.
491+
* `maxAge` is a TTL — the natural knob for regular scheduled backups, expressing a recovery window in time.
492+
* `maxCount` is a ceiling — "keep at most N". It is the only knob that bounds the object count (and therefore storage cost) independently of schedule frequency, which matters in three situations `minCount`/`maxAge` cannot cover: an unbounded S3 footprint under a frequent cron, a burst of ad-hoc backups before a migration, and a misconfigured "every minute" schedule. In all three the backups are still young, so `maxAge` does not fire and only a count ceiling trims them.
493+
494+
`minCount` cannot substitute for `maxCount`: a floor never trims a large set of young backups.
495+
496+
| State | `minCount=2, maxAge=30d` | `maxCount=5` |
497+
|---|---|---|
498+
| 2 backups (fresh + 180d) | both kept | both kept |
499+
| 20 young backups (<30d, burst or frequent cron) | all 20 kept | 5 kept |
500+
501+
**Recovery-window caveat.** On a continuous-archiving driver `maxCount` indirectly bounds the recovery window: under an hourly `Plan`, `maxCount: 3` collapses PITR to roughly three hours, because dropping the oldest base backups drops the WALs they anchored. This is intuitive for `maxAge` but surprising for `maxCount` at high frequency — surface the effective window rather than only the count.
502+
503+
**Binding fields**
504+
505+
A `Backup` joins a policy through the `backups.cozystack.io/retention-policy` label, stamped by the core controller when the `Backup` is created. One new optional field, `retentionPolicyName`, feeds it on `Plan`, `BackupJob`, and `BackupClass`, capped at 63 characters because the value lands in a DNS-1123 label:
506+
507+
```go
508+
// Plan.spec, BackupJob.spec, and BackupClass.spec (BackupClass as the namespace default).
509+
RetentionPolicyName string `json:"retentionPolicyName,omitempty"`
510+
```
511+
512+
Resolution when a `Backup` is created, highest precedence first: `BackupJob.spec.retentionPolicyName` (or `Plan.spec.retentionPolicyName`, inherited onto the BackupJob), then `BackupClass.spec.retentionPolicyName`, then none — a `Backup` with no label is never swept.
513+
514+
**Retention semantics**
515+
516+
For a policy `P` in namespace `N`:
517+
518+
1. List `Backup` in `N` labeled `retention-policy=P.name`; keep only `status.phase == Ready`. `Failed`/`Pending` backups never consume the floor and are pruned only by `MaxAge`.
519+
2. Sort by `spec.takenAt` descending.
520+
3. Protect the newest `MinCount` unconditionally.
521+
4. From the rest, delete a backup when its index `>= MaxCount` **or** `now - takenAt > MaxAge`. `MinCount` always wins over `MaxAge`.
522+
5. Delete via the `Backup` object; the `backups.cozystack.io/cleanup` finalizer runs the driver's cleanup path.
523+
524+
The sweep is an idempotent set difference — safe to run on every reconcile and on `Backup` create/delete/label events.
525+
526+
**Namespace-local matching**
527+
528+
Both `Backup` and `BackupRetentionPolicy` are namespaced, and matching is scoped to the policy's own namespace. A policy named `default` in one namespace and one so named in another are distinct objects governing disjoint backup sets, so identical names never collide. A name identifies a policy only within a namespace, which suffices because a `Backup` and its policy always share one.
529+
530+
**Migration between policies**
531+
532+
Moving artifacts to another policy is a label change, with no `Plan` edit:
533+
534+
```bash
535+
kubectl label backup -n tenant-acme \
536+
-l backups.cozystack.io/retention-policy=default \
537+
backups.cozystack.io/retention-policy=keep-90d --overwrite
538+
```
539+
540+
Changing `retentionPolicyName` on a `Plan` moves only future backups; existing ones stay until relabeled. Relabeling onto a stricter policy may delete artifacts on the next sweep; onto a laxer one, it spares them.
541+
542+
**Examples**
543+
544+
Platform default, shipped per tenant namespace and wired as the BackupClass default:
545+
546+
```yaml
547+
apiVersion: backups.cozystack.io/v1alpha1
548+
kind: BackupRetentionPolicy
549+
metadata:
550+
name: default
551+
namespace: tenant-acme
552+
spec:
553+
minCount: 3
554+
maxAge: 7d
555+
---
556+
apiVersion: backups.cozystack.io/v1alpha1
557+
kind: BackupClass
558+
metadata:
559+
name: cozy-default # cluster-scoped platform BackupClass
560+
spec:
561+
retentionPolicyName: default
562+
strategies:
563+
- application: { apiGroup: apps.cozystack.io, kind: Postgres }
564+
strategyRef:
565+
apiGroup: strategy.backups.cozystack.io
566+
kind: CNPG
567+
name: cozy-default-cnpg
568+
```
569+
570+
A tenant creating one policy and using it from both a scheduled `Plan` and an ad-hoc `BackupJob`:
571+
572+
```yaml
573+
apiVersion: backups.cozystack.io/v1alpha1
574+
kind: BackupRetentionPolicy
575+
metadata: { name: keep-90d, namespace: tenant-acme }
576+
spec:
577+
minCount: 5
578+
maxCount: 60
579+
maxAge: 90d
580+
---
581+
apiVersion: backups.cozystack.io/v1alpha1
582+
kind: Plan
583+
metadata: { name: pg-src-daily, namespace: tenant-acme }
584+
spec:
585+
applicationRef: { apiGroup: apps.cozystack.io, kind: Postgres, name: pg-src }
586+
backupClassName: cozy-default
587+
schedule: { type: cron, cron: "0 */6 * * *" }
588+
retentionPolicyName: keep-90d # overrides BackupClass default
589+
---
590+
apiVersion: backups.cozystack.io/v1alpha1
591+
kind: BackupJob
592+
metadata: { name: pg-src-adhoc, namespace: tenant-acme }
593+
spec:
594+
applicationRef: { apiGroup: apps.cozystack.io, kind: Postgres, name: pg-src }
595+
backupClassName: cozy-default
596+
retentionPolicyName: keep-90d # same policy for an ad-hoc run
597+
```
598+
599+
The resulting `Backup`, with the label stamped by the controller:
600+
601+
```yaml
602+
apiVersion: backups.cozystack.io/v1alpha1
603+
kind: Backup
604+
metadata:
605+
name: pg-src-20260910-1040
606+
namespace: tenant-acme
607+
labels:
608+
backups.cozystack.io/retention-policy: keep-90d
609+
spec:
610+
applicationRef: { apiGroup: apps.cozystack.io, kind: Postgres, name: pg-src }
611+
planRef: { name: pg-src-daily }
612+
strategyRef: { apiGroup: strategy.backups.cozystack.io, kind: CNPG, name: cozy-default-cnpg }
613+
takenAt: "2026-09-10T10:40:00Z"
614+
```
615+
616+
---
617+
450618
## 5. Strategy drivers (high-level)
451619

452620
Strategy drivers are separate controllers that:
@@ -491,5 +659,6 @@ The Cozystack backups core API:
491659
* **Execution** (BackupJob) – created by Plan when schedule fires, resolves BackupClass to get strategy and parameters, then delegates to driver.
492660
* **What backup artifacts exist** (Backup) – driver-created but cluster-visible.
493661
* **Restore lifecycle** (RestoreJob) – shared contract boundary.
662+
* **Retention** (BackupRetentionPolicy) – label-scoped, namespaced pruning of Backup artifacts by min/max count and age.
494663
* Allows multiple strategy drivers to implement backup/restore logic without entangling their implementation with the core API.
495664

0 commit comments

Comments
 (0)