Skip to content

Commit 534e20b

Browse files
androndoclaude
andcommitted
feat(backups): add RabbitMQ definitions backup via a dedicated strategy driver
RabbitMQ's cluster operator ships no backup CRD and RabbitMQ has no consistent online export of message payloads, so back up broker DEFINITIONS (vhosts, users, permissions, queues, exchanges, bindings, policies, parameters) over the management HTTP API and store them in the system bucket. Message payloads and PITR are out of scope - a full-data backup is a Velero volume snapshot, a separate strategy. Add a first-class strategy.backups.cozystack.io/Rabbitmq driver, following the Altinity pattern (a dedicated driver that renders a one-shot Job for an HTTP-API engine): - reconcileRabbitmq / reconcileRabbitmqRestore render the strategy's PodTemplateSpec (curl GET/POST /api/definitions, push/pull to S3 with --aws-sigv4) as a batch/v1.Job, for both in-place and to-copy restore. - A Go-read precondition gates each run on the RabbitmqCluster reporting AllReplicasReady (via the rabbitmqtypes partial type), surfacing a precise Ready=False wait instead of a Job that starts and fails. - The produced Backup records the object location on status.artifact.uri (rendered from spec.artifactURITemplate; the Backup CRD has no status subresource, so it persists on Create, as in the Etcd driver). Wire dispatch in both controllers + supportedBackupStrategyKinds, register rabbitmqtypes in the scheme, and grant read on rabbitmqclusters. Ship cozy-default-rabbitmq (a $bucketName-gated Rabbitmq strategy) and route apps.cozystack.io/RabbitMQ to it in the cozy-default BackupClass. The application chart is untouched. Add unit tests for the driver's pure helpers and the precondition gate, a runnable example, and an honest-roundtrip Chainsaw e2e that seeds a sentinel vhost, backs it up, drops it, and asserts it round-trips through object storage on both restore flows. The Job pod reuses the platform-migrations image (curl + ca-certificates) under a new rabbitmqBackupClientImage key, stamped in lockstep with chBackupClientImage so the repository stays pinned at one digest; a dedicated slim image is a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Andrey Kolkov <[email protected]>
1 parent eef95f4 commit 534e20b

30 files changed

Lines changed: 2558 additions & 4 deletions
Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
// SPDX-License-Identifier: Apache-2.0
2+
// Package v1alpha1 defines strategy.backups.cozystack.io API types.
3+
//
4+
// Group: strategy.backups.cozystack.io
5+
// Version: v1alpha1
6+
package v1alpha1
7+
8+
import (
9+
corev1 "k8s.io/api/core/v1"
10+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
11+
"k8s.io/apimachinery/pkg/runtime"
12+
)
13+
14+
func init() {
15+
SchemeBuilder.Register(func(s *runtime.Scheme) error {
16+
s.AddKnownTypes(GroupVersion,
17+
&Rabbitmq{},
18+
&RabbitmqList{},
19+
)
20+
return nil
21+
})
22+
}
23+
24+
const (
25+
RabbitmqStrategyKind = "Rabbitmq"
26+
)
27+
28+
// +kubebuilder:object:root=true
29+
// +kubebuilder:subresource:status
30+
// +kubebuilder:resource:scope=Cluster
31+
32+
// Rabbitmq defines a backup strategy that exports and imports RabbitMQ broker
33+
// DEFINITIONS (vhosts, users, permissions, queues, exchanges, bindings,
34+
// policies, parameters) via the management HTTP API, run as a one-shot
35+
// batch/v1.Job per BackupJob / RestoreJob. Message payloads are out of scope -
36+
// RabbitMQ has no consistent online export of them, so a full-data backup is a
37+
// Velero volume snapshot, not this strategy. The driver gates on the target
38+
// RabbitmqCluster being ready, wraps the templated PodTemplateSpec in a Job,
39+
// and records the stored object on the Cozystack Backup artifact.
40+
//
41+
// Restore is a MERGE, not a reset: it POSTs the definitions to /api/definitions,
42+
// which creates or updates what is in the export and never deletes anything, so
43+
// objects created after the backup survive the restore and the broker is not
44+
// returned to its exact backup-time state. A to-copy restore also imports the
45+
// source's users (with password hashes) into the target.
46+
type Rabbitmq struct {
47+
metav1.TypeMeta `json:",inline"`
48+
metav1.ObjectMeta `json:"metadata,omitempty"`
49+
50+
Spec RabbitmqSpec `json:"spec,omitempty"`
51+
Status RabbitmqStatus `json:"status,omitempty"`
52+
}
53+
54+
// +kubebuilder:object:root=true
55+
56+
// RabbitmqList contains a list of Rabbitmq backup strategies.
57+
type RabbitmqList struct {
58+
metav1.TypeMeta `json:",inline"`
59+
metav1.ListMeta `json:"metadata,omitempty"`
60+
Items []Rabbitmq `json:"items"`
61+
}
62+
63+
// RabbitmqSpec specifies the desired Rabbitmq-driven backup strategy.
64+
type RabbitmqSpec struct {
65+
// Template is the PodTemplateSpec the driver wraps in a batch/v1.Job for
66+
// both backup and restore runs. Helm-style Go templates are supported in
67+
// every string field. The available context is:
68+
// .Application - the application object (apps.cozystack.io/RabbitMQ)
69+
// .Release.Name, .Release.Namespace - shorthand for the application
70+
// name/namespace
71+
// .Mode - "backup" or "restore"
72+
// .BackupName - the per-run identity used to scope the object key so
73+
// distinct backups of one application do not overwrite
74+
// each other: the BackupJob name on backup, the source
75+
// Backup name on restore.
76+
// .Parameters - map[string]string from the matched BackupClassStrategy
77+
// .Backup - the source Backup metadata (only set on restore);
78+
// exposes .Name, .Namespace, and
79+
// .ApplicationRef.{APIGroup,Kind,Name} so a to-copy
80+
// restore can address the source release's object key.
81+
//
82+
// The template is stored schemaless (its content is preserved but not
83+
// validated by the apiserver): the driver renders and validates the pod
84+
// spec, and generating the full PodTemplateSpec OpenAPI schema here would
85+
// add ~590 KB to this CRD. The backupstrategy-controller chart inlines
86+
// every definitions/*.yaml into one Helm release, whose state Secret is
87+
// capped at 1 MiB - a fourth full PodTemplateSpec schema (after Altinity,
88+
// Job and FoundationDB) pushes that Secret over the cap and fails install.
89+
//
90+
// +kubebuilder:validation:Schemaless
91+
// +kubebuilder:pruning:PreserveUnknownFields
92+
Template corev1.PodTemplateSpec `json:"template"`
93+
94+
// ArtifactURITemplate, when set, is rendered against the same context as
95+
// Template on a successful backup and recorded on the produced Backup's
96+
// status.artifact.uri, so the stored object's location is introspectable
97+
// from the Backup rather than living only inside the pod script. Ignored on
98+
// restore; omitted leaves status.artifact unset.
99+
// +optional
100+
ArtifactURITemplate string `json:"artifactURITemplate,omitempty"`
101+
}
102+
103+
// RabbitmqStatus reports observed state for the strategy CR.
104+
type RabbitmqStatus struct {
105+
// Conditions holds the latest available observations.
106+
// +optional
107+
// +listType=map
108+
// +listMapKey=type
109+
Conditions []metav1.Condition `json:"conditions,omitempty"`
110+
}

‎api/backups/strategy/v1alpha1/zz_generated.deepcopy.go‎

Lines changed: 97 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎cmd/backupstrategy-controller/main.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ import (
5252
"github.com/cozystack/cozystack/internal/backupcontroller/mongodbapp"
5353
"github.com/cozystack/cozystack/internal/backupcontroller/postgresapp"
5454
"github.com/cozystack/cozystack/internal/backupcontroller/psmdbtypes"
55+
"github.com/cozystack/cozystack/internal/backupcontroller/rabbitmqtypes"
5556
velerov1 "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
5657
// +kubebuilder:scaffold:imports
5758
)
@@ -77,6 +78,7 @@ func init() {
7778
utilruntime.Must(foundationdbapp.AddToScheme(scheme))
7879
utilruntime.Must(etcdtypes.AddToScheme(scheme))
7980
utilruntime.Must(etcdapp.AddToScheme(scheme))
81+
utilruntime.Must(rabbitmqtypes.AddToScheme(scheme))
8082
// +kubebuilder:scaffold:scheme
8183
}
8284

‎docs/operations/backup-classes.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Tenants reference `cozy-default` from `BackupJob`, `Plan`, and `RestoreJob` reso
1515
| `apps.cozystack.io/ClickHouse` | Altinity `clickhouse-backup` sidecar | `strategy.backups.cozystack.io/Altinity` `cozy-default-altinity` |
1616
| `apps.cozystack.io/MongoDB` | Percona psmdb operator (pbm) dump | `strategy.backups.cozystack.io/MongoDB` `cozy-default-mongodb` |
1717
| `apps.cozystack.io/Etcd` | etcd-operator snapshot | `strategy.backups.cozystack.io/Etcd` `cozy-default-etcd` |
18+
| `apps.cozystack.io/RabbitMQ` | RabbitMQ definitions (management API) | `strategy.backups.cozystack.io/Rabbitmq` `cozy-default-rabbitmq` |
1819
| `apps.cozystack.io/VMInstance` | Velero + kubevirt-velero-plugin | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vminstance` |
1920
| `apps.cozystack.io/VMDisk` | Velero | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vmdisk` |
2021

@@ -119,7 +120,7 @@ The bucket lives in `tenant-root` and is provisioned through the `apps.cozystack
119120

120121
| Key | Consumer |
121122
|-----------------------------------------------|-------------------------------------------|
122-
| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | CNPG, MariaDB, Etcd |
123+
| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | CNPG, MariaDB, Etcd, RabbitMQ |
123124
| `accessKey` / `secretKey` (plus `bucketName`, `endpoint`, `region`) | ClickHouse sidecar |
124125
| `cloud` | Velero (AWS credentials file format) |
125126
| `blob_credentials.json` | FoundationDB backup_agent |
@@ -196,7 +197,7 @@ The default-objects gate emits three more:
196197

197198
## Admin overrides for `cozy-default`
198199

199-
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-mongodb`, `cozy-default-foundationdb`, the two `cozy-default-velero-*`). The supported override path is the `backupStorage` block on the **`platform` component** of the `cozystack.cozystack-platform` Package CR:
200+
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-mongodb`, `cozy-default-foundationdb`, `cozy-default-rabbitmq`, the two `cozy-default-velero-*`). The supported override path is the `backupStorage` block on the **`platform` component** of the `cozystack.cozystack-platform` Package CR:
200201

201202
```yaml
202203
apiVersion: cozystack.io/v1alpha1
@@ -243,6 +244,7 @@ The defaults aim at a reasonable middle (30-day retention, gzip compression wher
243244
- **MongoDB strategy**: `storageName` (which `spec.backup.storages` entry on the psmdb cluster to use), `type` (`logical`), `compressionType` / `compressionLevel`. The S3 target itself is tuned via `backup.*` values on the MongoDB release (see [MongoDB: the application owns the backup storage](#mongodb-the-application-owns-the-backup-storage)).
244245
- **Altinity strategy**: tune the `clickhouse-backup` sidecar via `backup.*` values on the ClickHouse release; the strategy Pod is a thin HTTP client. When the S3 endpoint's certificate is signed by a private CA rather than a publicly-trusted one — SeaweedFS's in-cluster `:8333` being the case in point — point `backup.endpointCA` at a Secret holding that CA bundle; the chart mounts it into the sidecar and adds it to the trust store via `SSL_CERT_DIR`, which supplements the system CA set rather than replacing it.
245246
- **FoundationDB strategy**: `snapshotPeriodSeconds`, `agentCount`, `urlParameters[]`.
247+
- **Rabbitmq strategy**: `artifactURITemplate` (the `<namespace>/<application>/<backup-name>/definitions.json` object-key layout) and the pod `template` (image, resources). The backup is a definitions export over the management API — vhosts, users, permissions, queues, exchanges, bindings, policies, parameters — so message payloads are out of scope and there is no point-in-time recovery; a full message-data backup is a Velero volume snapshot instead. **Restore is a merge, not a reset**: importing definitions (`POST /api/definitions`) creates or updates what the export contains and never deletes, so anything created after the backup survives the restore and the broker is not returned to its exact backup-time state. A to-copy restore imports the source's users (with password hashes) into the target, so those source credentials become valid logins there.
246248
- **Velero strategy (VMInstance / VMDisk)**: `ttl`, `includedResources[]`, `excludedResources[]`.
247249
- **Etcd strategy**: today the strategy is path-only; combine with `Plan.spec.retentionPolicy` for trim cadence.
248250

0 commit comments

Comments
 (0)