Skip to content

Commit f196787

Browse files
androndoclaude
androndo
and
claude
committed
feat(backups): add RabbitMQ definitions backup via a dedicated strategy driver
RabbitMQ's cluster operator ships no backup CRD and RabbitMQ has no consistent online export of message payloads, so back up broker DEFINITIONS (vhosts, users, permissions, queues, exchanges, bindings, policies, parameters) over the management HTTP API and store them in the system bucket. Message payloads and PITR are out of scope - a full-data backup is a Velero volume snapshot, a separate strategy. Add a first-class strategy.backups.cozystack.io/Rabbitmq driver, following the Altinity pattern (a dedicated driver that renders a one-shot Job for an HTTP-API engine): - reconcileRabbitmq / reconcileRabbitmqRestore render the strategy's PodTemplateSpec (curl GET/POST /api/definitions, push/pull to S3 with --aws-sigv4) as a batch/v1.Job, for both in-place and to-copy restore. - A Go-read precondition gates each run on the RabbitmqCluster reporting AllReplicasReady (via the rabbitmqtypes partial type), surfacing a precise Ready=False wait instead of a Job that starts and fails. - The produced Backup records the object location on status.artifact.uri (rendered from spec.artifactURITemplate; the Backup CRD has no status subresource, so it persists on Create, as in the Etcd driver). Wire dispatch in both controllers + supportedBackupStrategyKinds, register rabbitmqtypes in the scheme, and grant read on rabbitmqclusters. Ship cozy-default-rabbitmq (a $bucketName-gated Rabbitmq strategy) and route apps.cozystack.io/RabbitMQ to it in the cozy-default BackupClass. The application chart is untouched. Add unit tests for the driver's pure helpers and the precondition gate, a runnable example, and an honest-roundtrip Chainsaw e2e that seeds a sentinel vhost, backs it up, drops it, and asserts it round-trips through object storage on both restore flows. The Job pod reuses the platform-migrations image (curl + ca-certificates) under a new rabbitmqBackupClientImage key, stamped in lockstep with chBackupClientImage so the repository stays pinned at one digest; a dedicated slim image is a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Andrey Kolkov <[email protected]>
1 parent eef95f4 commit f196787

30 files changed

Lines changed: 2433 additions & 4 deletions

‎api/backups/strategy/v1alpha1/rabbitmq_types.go‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
// SPDX-License-Identifier: Apache-2.0
2+
// Package v1alpha1 defines strategy.backups.cozystack.io API types.
3+
//
4+
// Group: strategy.backups.cozystack.io
5+
// Version: v1alpha1
6+
package v1alpha1
7+
8+
import (
9+
corev1 "k8s.io/api/core/v1"
10+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
11+
"k8s.io/apimachinery/pkg/runtime"
12+
)
13+
14+
func init() {
15+
SchemeBuilder.Register(func(s *runtime.Scheme) error {
16+
s.AddKnownTypes(GroupVersion,
17+
&Rabbitmq{},
18+
&RabbitmqList{},
19+
)
20+
return nil
21+
})
22+
}
23+
24+
const (
25+
RabbitmqStrategyKind = "Rabbitmq"
26+
)
27+
28+
// +kubebuilder:object:root=true
29+
// +kubebuilder:subresource:status
30+
// +kubebuilder:resource:scope=Cluster
31+
32+
// Rabbitmq defines a backup strategy that exports and imports RabbitMQ broker
33+
// DEFINITIONS (vhosts, users, permissions, queues, exchanges, bindings,
34+
// policies, parameters) via the management HTTP API, run as a one-shot
35+
// batch/v1.Job per BackupJob / RestoreJob. Message payloads are out of scope -
36+
// RabbitMQ has no consistent online export of them, so a full-data backup is a
37+
// Velero volume snapshot, not this strategy. The driver gates on the target
38+
// RabbitmqCluster being ready, wraps the templated PodTemplateSpec in a Job,
39+
// and records the stored object on the Cozystack Backup artifact.
40+
type Rabbitmq struct {
41+
metav1.TypeMeta `json:",inline"`
42+
metav1.ObjectMeta `json:"metadata,omitempty"`
43+
44+
Spec RabbitmqSpec `json:"spec,omitempty"`
45+
Status RabbitmqStatus `json:"status,omitempty"`
46+
}
47+
48+
// +kubebuilder:object:root=true
49+
50+
// RabbitmqList contains a list of Rabbitmq backup strategies.
51+
type RabbitmqList struct {
52+
metav1.TypeMeta `json:",inline"`
53+
metav1.ListMeta `json:"metadata,omitempty"`
54+
Items []Rabbitmq `json:"items"`
55+
}
56+
57+
// RabbitmqSpec specifies the desired Rabbitmq-driven backup strategy.
58+
type RabbitmqSpec struct {
59+
// Template is the PodTemplateSpec the driver wraps in a batch/v1.Job for
60+
// both backup and restore runs. Helm-style Go templates are supported in
61+
// every string field. The available context is:
62+
// .Application - the application object (apps.cozystack.io/RabbitMQ)
63+
// .Release.Name, .Release.Namespace - shorthand for the application
64+
// name/namespace
65+
// .Mode - "backup" or "restore"
66+
// .BackupName - the per-run identity used to scope the object key so
67+
// distinct backups of one application do not overwrite
68+
// each other: the BackupJob name on backup, the source
69+
// Backup name on restore.
70+
// .Parameters - map[string]string from the matched BackupClassStrategy
71+
// .Backup - the source Backup metadata (only set on restore);
72+
// exposes .Name, .Namespace, and
73+
// .ApplicationRef.{APIGroup,Kind,Name} so a to-copy
74+
// restore can address the source release's object key.
75+
//
76+
// The template is stored schemaless (its content is preserved but not
77+
// validated by the apiserver): the driver renders and validates the pod
78+
// spec, and generating the full PodTemplateSpec OpenAPI schema here would
79+
// add ~590 KB to this CRD. The backupstrategy-controller chart inlines
80+
// every definitions/*.yaml into one Helm release, whose state Secret is
81+
// capped at 1 MiB - a fourth full PodTemplateSpec schema (after Altinity,
82+
// Job and FoundationDB) pushes that Secret over the cap and fails install.
83+
//
84+
// +kubebuilder:validation:Schemaless
85+
// +kubebuilder:pruning:PreserveUnknownFields
86+
Template corev1.PodTemplateSpec `json:"template"`
87+
88+
// ArtifactURITemplate, when set, is rendered against the same context as
89+
// Template on a successful backup and recorded on the produced Backup's
90+
// status.artifact.uri, so the stored object's location is introspectable
91+
// from the Backup rather than living only inside the pod script. Ignored on
92+
// restore; omitted leaves status.artifact unset.
93+
// +optional
94+
ArtifactURITemplate string `json:"artifactURITemplate,omitempty"`
95+
}
96+
97+
// RabbitmqStatus reports observed state for the strategy CR.
98+
type RabbitmqStatus struct {
99+
// Conditions holds the latest available observations.
100+
// +optional
101+
// +listType=map
102+
// +listMapKey=type
103+
Conditions []metav1.Condition `json:"conditions,omitempty"`
104+
}

‎api/backups/strategy/v1alpha1/zz_generated.deepcopy.go‎

Lines changed: 97 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎cmd/backupstrategy-controller/main.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ import (
5252
"github.com/cozystack/cozystack/internal/backupcontroller/mongodbapp"
5353
"github.com/cozystack/cozystack/internal/backupcontroller/postgresapp"
5454
"github.com/cozystack/cozystack/internal/backupcontroller/psmdbtypes"
55+
"github.com/cozystack/cozystack/internal/backupcontroller/rabbitmqtypes"
5556
velerov1 "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
5657
// +kubebuilder:scaffold:imports
5758
)
@@ -77,6 +78,7 @@ func init() {
7778
utilruntime.Must(foundationdbapp.AddToScheme(scheme))
7879
utilruntime.Must(etcdtypes.AddToScheme(scheme))
7980
utilruntime.Must(etcdapp.AddToScheme(scheme))
81+
utilruntime.Must(rabbitmqtypes.AddToScheme(scheme))
8082
// +kubebuilder:scaffold:scheme
8183
}
8284

‎docs/operations/backup-classes.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Tenants reference `cozy-default` from `BackupJob`, `Plan`, and `RestoreJob` reso
1515
| `apps.cozystack.io/ClickHouse` | Altinity `clickhouse-backup` sidecar | `strategy.backups.cozystack.io/Altinity` `cozy-default-altinity` |
1616
| `apps.cozystack.io/MongoDB` | Percona psmdb operator (pbm) dump | `strategy.backups.cozystack.io/MongoDB` `cozy-default-mongodb` |
1717
| `apps.cozystack.io/Etcd` | etcd-operator snapshot | `strategy.backups.cozystack.io/Etcd` `cozy-default-etcd` |
18+
| `apps.cozystack.io/RabbitMQ` | RabbitMQ definitions (management API) | `strategy.backups.cozystack.io/Rabbitmq` `cozy-default-rabbitmq` |
1819
| `apps.cozystack.io/VMInstance` | Velero + kubevirt-velero-plugin | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vminstance` |
1920
| `apps.cozystack.io/VMDisk` | Velero | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vmdisk` |
2021

@@ -119,7 +120,7 @@ The bucket lives in `tenant-root` and is provisioned through the `apps.cozystack
119120

120121
| Key | Consumer |
121122
|-----------------------------------------------|-------------------------------------------|
122-
| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | CNPG, MariaDB, Etcd |
123+
| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | CNPG, MariaDB, Etcd, RabbitMQ |
123124
| `accessKey` / `secretKey` (plus `bucketName`, `endpoint`, `region`) | ClickHouse sidecar |
124125
| `cloud` | Velero (AWS credentials file format) |
125126
| `blob_credentials.json` | FoundationDB backup_agent |
@@ -196,7 +197,7 @@ The default-objects gate emits three more:
196197

197198
## Admin overrides for `cozy-default`
198199

199-
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-mongodb`, `cozy-default-foundationdb`, the two `cozy-default-velero-*`). The supported override path is the `backupStorage` block on the **`platform` component** of the `cozystack.cozystack-platform` Package CR:
200+
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-mongodb`, `cozy-default-foundationdb`, `cozy-default-rabbitmq`, the two `cozy-default-velero-*`). The supported override path is the `backupStorage` block on the **`platform` component** of the `cozystack.cozystack-platform` Package CR:
200201

201202
```yaml
202203
apiVersion: cozystack.io/v1alpha1
@@ -243,6 +244,7 @@ The defaults aim at a reasonable middle (30-day retention, gzip compression wher
243244
- **MongoDB strategy**: `storageName` (which `spec.backup.storages` entry on the psmdb cluster to use), `type` (`logical`), `compressionType` / `compressionLevel`. The S3 target itself is tuned via `backup.*` values on the MongoDB release (see [MongoDB: the application owns the backup storage](#mongodb-the-application-owns-the-backup-storage)).
244245
- **Altinity strategy**: tune the `clickhouse-backup` sidecar via `backup.*` values on the ClickHouse release; the strategy Pod is a thin HTTP client. When the S3 endpoint's certificate is signed by a private CA rather than a publicly-trusted one — SeaweedFS's in-cluster `:8333` being the case in point — point `backup.endpointCA` at a Secret holding that CA bundle; the chart mounts it into the sidecar and adds it to the trust store via `SSL_CERT_DIR`, which supplements the system CA set rather than replacing it.
245246
- **FoundationDB strategy**: `snapshotPeriodSeconds`, `agentCount`, `urlParameters[]`.
247+
- **Rabbitmq strategy**: `artifactURITemplate` (the `<namespace>/<application>/<backup-name>/definitions.json` object-key layout) and the pod `template` (image, resources). The backup is a definitions export over the management API — vhosts, users, permissions, queues, exchanges, bindings, policies, parameters — so message payloads are out of scope and there is no point-in-time recovery; a full message-data backup is a Velero volume snapshot instead.
246248
- **Velero strategy (VMInstance / VMDisk)**: `ttl`, `includedResources[]`, `excludedResources[]`.
247249
- **Etcd strategy**: today the strategy is path-only; combine with `Plan.spec.retentionPolicy` for trim cadence.
248250

0 commit comments

Comments
 (0)