Skip to content

Commit dff56aa

Browse files
androndoclaude
androndo
and
claude
committed
feat(backups): add a native MongoDB (psmdb) backup-strategy driver
Adds a MongoDB (Percona Server for MongoDB / psmdb) backup-strategy driver so `apps.cozystack.io/MongoDB` reaches feature parity with the CNPG, MariaDB and ClickHouse native drivers — logical backups, restore-to-differently-named instances, and point-in-time recovery through the unified BackupJob/RestoreJob/BackupClass/Plan interface (closes #3545). The driver orchestrates the operator's PerconaServerMongoDBBackup / PerconaServerMongoDBRestore CRDs: - BackupJob: creates a one-shot PerconaServerMongoDBBackup, polls status.state, and persists the S3 destination + storage descriptor (by-reference, no credentials) onto the Cozystack Backup artifact. - RestoreJob: drives PerconaServerMongoDBRestore via backupSource (read back from the source backup status, snapshot fallback) so the same artifact restores in-place or into a differently-named instance; PITR via spec.options.pitr (date/latest). Because psmdb resolves the backup storage by name against the cluster's spec.backup.storages, the driver names the chart's storage (s3-storage) via the cozy-default-mongodb Strategy rather than injecting it, and surfaces a clear Ready=False precondition when a target has backups disabled. Wiring: - api/backups/strategy/v1alpha1: MongoDB strategy type + generated CRD/deepcopy. - internal/backupcontroller: psmdbstrategy_controller + psmdbtypes/mongodbapp partial-schema mirrors; dispatch, supported-kinds and scheme registration. - backupstrategy-controller: cozy-default MongoDB route, cozy-default-mongodb Strategy CR, and psmdb.percona.com RBAC. - apps/mongodb: backup.insecureSkipTLSVerify knob (psmdb s3 storage has no CA-bundle field) so backups can target Cozystack's self-signed seaweedfs. Tests + e2e: - unit tests for the driver (validation, ensure-idempotency, precondition, backupSource resolution, snapshot round-trip, PITR option parsing). - examples/backups/mongodb + a mongodb-2-backup-roundtrip Chainsaw suite that drives run-all.sh end-to-end (sentinel document round-trips through S3 into a restored copy while the source stays untouched), per docs/agents/e2e-testing.md. - docs/operations/backup-classes.md updated. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Andrey Kolkov <[email protected]>
1 parent a242f34 commit dff56aa

37 files changed

Lines changed: 3319 additions & 18 deletions

‎api/apps/v1alpha1/mongodb/types.go‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/backups/strategy/v1alpha1/mongodb_types.go‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
// SPDX-License-Identifier: Apache-2.0
2+
// Package v1alpha1 defines strategy.backups.cozystack.io API types.
3+
//
4+
// Group: strategy.backups.cozystack.io
5+
// Version: v1alpha1
6+
package v1alpha1
7+
8+
import (
9+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
10+
"k8s.io/apimachinery/pkg/runtime"
11+
)
12+
13+
func init() {
14+
SchemeBuilder.Register(func(s *runtime.Scheme) error {
15+
s.AddKnownTypes(GroupVersion,
16+
&MongoDB{},
17+
&MongoDBList{},
18+
)
19+
return nil
20+
})
21+
}
22+
23+
const (
24+
MongoDBStrategyKind = "MongoDB"
25+
)
26+
27+
// +kubebuilder:object:root=true
28+
// +kubebuilder:subresource:status
29+
// +kubebuilder:resource:scope=Cluster
30+
31+
// MongoDB defines a backup strategy that delegates execution to the Percona
32+
// Server for MongoDB operator (psmdb.percona.com). The strategy carries a
33+
// templated logical-backup configuration; the driver materialises
34+
// PerconaServerMongoDBBackup objects per BackupJob and surfaces them as
35+
// Cozystack Backup artifacts. Restores create a PerconaServerMongoDBRestore
36+
// CR with clusterName pointing at the target cluster and a backupSource
37+
// carrying the S3 destination read back from the source backup, so the same
38+
// artifact restores in-place or into a differently-named instance.
39+
//
40+
// Unlike the MariaDB strategy, the storage config is NOT inlined here: psmdb
41+
// resolves spec.storageName against the source cluster's spec.backup.storages,
42+
// so the S3/credentials live on the PerconaServerMongoDB CR (the mongodb chart
43+
// declares them when backup.enabled=true) and the strategy only names the
44+
// storage to use.
45+
type MongoDB struct {
46+
metav1.TypeMeta `json:",inline"`
47+
metav1.ObjectMeta `json:"metadata,omitempty"`
48+
49+
Spec MongoDBSpec `json:"spec,omitempty"`
50+
Status MongoDBStatus `json:"status,omitempty"`
51+
}
52+
53+
// +kubebuilder:object:root=true
54+
55+
// MongoDBList contains a list of MongoDB backup strategies.
56+
type MongoDBList struct {
57+
metav1.TypeMeta `json:",inline"`
58+
metav1.ListMeta `json:"metadata,omitempty"`
59+
Items []MongoDB `json:"items"`
60+
}
61+
62+
// MongoDBSpec specifies the desired psmdb-operator-driven backup strategy.
63+
type MongoDBSpec struct {
64+
// Template carries the templated PerconaServerMongoDBBackup configuration.
65+
// String fields support Helm-style Go templating with two top-level
66+
// values:
67+
// .Application - the application object (apps.cozystack.io/MongoDB)
68+
// .Parameters - the parameters from the matched BackupClassStrategy.
69+
// These values MUST NOT carry credentials — the psmdb
70+
// storage (bucket/endpoint/credentialsSecret) lives on the
71+
// PerconaServerMongoDB CR, referenced here only by name.
72+
Template MongoDBTemplate `json:"template"`
73+
}
74+
75+
// MongoDBTemplate describes the templated PerconaServerMongoDBBackup shape the
76+
// driver renders per BackupJob.
77+
type MongoDBTemplate struct {
78+
// StorageName names a storage entry declared in the source cluster's
79+
// spec.backup.storages. The mongodb chart declares "s3-storage" when
80+
// backup.enabled=true; leave empty to use that default. Templating is
81+
// supported.
82+
// +optional
83+
StorageName string `json:"storageName,omitempty"`
84+
85+
// Type selects the backup type. Only "logical" is supported (logical
86+
// dumps are portable across clusters, which is what makes
87+
// restore-to-differently-named instances work). Defaults to "logical"
88+
// when empty.
89+
// +kubebuilder:validation:Enum=logical
90+
// +optional
91+
Type string `json:"type,omitempty"`
92+
93+
// CompressionType is the pbm compression algorithm (e.g. "gzip", "zstd",
94+
// "s2", "snappy", "lz4", "pgzip", "none"). Defaults to the operator-side
95+
// default when empty. Templating is supported.
96+
// +optional
97+
CompressionType string `json:"compressionType,omitempty"`
98+
99+
// CompressionLevel tunes the compression algorithm (algorithm-specific).
100+
// Left unset the operator picks its default.
101+
// +optional
102+
CompressionLevel *int `json:"compressionLevel,omitempty"`
103+
}
104+
105+
// MongoDBStatus reports observed state for the strategy CR. Driver controllers
106+
// surface diagnostic conditions here (e.g. validation issues).
107+
type MongoDBStatus struct {
108+
// Conditions holds the latest available observations.
109+
// +optional
110+
// +listType=map
111+
// +listMapKey=type
112+
Conditions []metav1.Condition `json:"conditions,omitempty"`
113+
}

‎api/backups/strategy/v1alpha1/zz_generated.deepcopy.go‎

Lines changed: 117 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎cmd/backupstrategy-controller/main.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,9 @@ import (
4949
"github.com/cozystack/cozystack/internal/backupcontroller/foundationdbtypes"
5050
"github.com/cozystack/cozystack/internal/backupcontroller/mariadbapp"
5151
"github.com/cozystack/cozystack/internal/backupcontroller/mariadbtypes"
52+
"github.com/cozystack/cozystack/internal/backupcontroller/mongodbapp"
5253
"github.com/cozystack/cozystack/internal/backupcontroller/postgresapp"
54+
"github.com/cozystack/cozystack/internal/backupcontroller/psmdbtypes"
5355
velerov1 "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
5456
// +kubebuilder:scaffold:imports
5557
)
@@ -69,6 +71,8 @@ func init() {
6971
utilruntime.Must(postgresapp.AddToScheme(scheme))
7072
utilruntime.Must(mariadbtypes.AddToScheme(scheme))
7173
utilruntime.Must(mariadbapp.AddToScheme(scheme))
74+
utilruntime.Must(psmdbtypes.AddToScheme(scheme))
75+
utilruntime.Must(mongodbapp.AddToScheme(scheme))
7276
utilruntime.Must(foundationdbtypes.AddToScheme(scheme))
7377
utilruntime.Must(foundationdbapp.AddToScheme(scheme))
7478
utilruntime.Must(etcdtypes.AddToScheme(scheme))

‎docs/operations/backup-classes.md‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ Tenants reference `cozy-default` from `BackupJob`, `Plan`, and `RestoreJob` reso
1313
| `apps.cozystack.io/Postgres` | CloudNativePG (barman) | `strategy.backups.cozystack.io/CNPG` `cozy-default-cnpg` |
1414
| `apps.cozystack.io/MariaDB` | mariadb-operator dump | `strategy.backups.cozystack.io/MariaDB` `cozy-default-mariadb` |
1515
| `apps.cozystack.io/ClickHouse` | Altinity `clickhouse-backup` sidecar | `strategy.backups.cozystack.io/Altinity` `cozy-default-altinity` |
16+
| `apps.cozystack.io/MongoDB` | Percona psmdb operator (pbm) dump | `strategy.backups.cozystack.io/MongoDB` `cozy-default-mongodb` |
1617
| `apps.cozystack.io/Etcd` | etcd-operator snapshot | `strategy.backups.cozystack.io/Etcd` `cozy-default-etcd` |
1718
| `apps.cozystack.io/VMInstance` | Velero + kubevirt-velero-plugin | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vminstance` |
1819
| `apps.cozystack.io/VMDisk` | Velero | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vmdisk` |
@@ -34,6 +35,7 @@ Different operators expect different endpoint shapes; the strategy templates ren
3435
| CNPG (Postgres) | `barmanObjectStore.endpointURL` | full URL (scheme preserved) |
3536
| Etcd | `destination.s3.endpoint` | full URL (scheme preserved) |
3637
| MariaDB | `storage.s3.endpoint` | bare host:port (scheme stripped); `tls.enabled` derived from the scheme |
38+
| MongoDB | n/a — storage lives on the app (`backup.endpointURL`) | full URL (scheme preserved), configured on the MongoDB application, not the strategy |
3739
| FoundationDB | `blobStoreConfiguration.accountName` + `urlParameters.secure_connection` | bare host:port + derived secure flag |
3840
| Velero | `BackupStorageLocation.spec.config.s3Url` | full URL (scheme preserved) |
3941
| ClickHouse sidecar | `S3_ENDPOINT` env | bare host:port (from projected Secret) |
@@ -66,6 +68,43 @@ When `useSystemBucket: true`:
6668

6769
`s3Region`, `s3Bucket`, `endpoint`, `s3AccessKey`, `s3SecretKey`, and `s3CredentialsSecret` are ignored in this mode.
6870

71+
## MongoDB: the application owns the backup storage
72+
73+
Unlike CNPG / MariaDB / Altinity, the MongoDB driver cannot inject its S3 target per-backup: the Percona psmdb operator only runs the percona-backup-mongodb (pbm) agents and services `PerconaServerMongoDBBackup` CRs when the `PerconaServerMongoDB` cluster has `spec.backup.enabled: true` and a storage declared, and a `PerconaServerMongoDBBackup` references that storage only by name. So the MongoDB application must **opt into backups** and point at the bucket in its own chart values:
74+
75+
```yaml
76+
apiVersion: apps.cozystack.io/v1alpha1
77+
kind: MongoDB
78+
metadata:
79+
name: orders-db
80+
spec:
81+
backup:
82+
enabled: true
83+
destinationPath: "s3://<bucket>/orders-db/"
84+
endpointURL: "https://seaweedfs-s3.tenant-root:8333"
85+
insecureSkipTLSVerify: true # self-signed in-cluster seaweedfs; psmdb s3 storage has no CA-bundle field
86+
s3AccessKey: "<key>"
87+
s3SecretKey: "<secret>"
88+
```
89+
90+
The chart declares that storage as `s3-storage`, which the `cozy-default-mongodb` strategy names (`storageName: s3-storage`, `type: logical`). The driver then drives on-demand backups and restores through the unified `BackupJob` / `RestoreJob` / `Plan` interface on top of the operator's storage config — adding restore-to-differently-named instances (via `PerconaServerMongoDBRestore.spec.backupSource`) and point-in-time recovery that the raw scheduled-task / `mongodump` paths do not offer. When a target cluster has backups disabled, the driver surfaces a clear `Ready=False` precondition on the BackupJob/RestoreJob rather than hanging until the deadline.
91+
92+
A full, scripted example (write a marker document, back up, restore to a copy, assert the round-trip while the source stays untouched) is in [`examples/backups/mongodb/`](../../examples/backups/mongodb/) — driven by `run-all.sh`.
93+
94+
### Point-in-time recovery (MongoDB)
95+
96+
psmdb records an oplog stream between logical backups. A MongoDB `RestoreJob` recovers to a timestamp via `spec.options.pitr`:
97+
98+
```yaml
99+
spec:
100+
options:
101+
pitr:
102+
type: date
103+
date: "2026-08-05 12:34:56" # UTC, YYYY-MM-DD HH:MM:SS
104+
```
105+
106+
Use `type: latest` (no `date`) to replay to the newest restorable point. `spec.options.restoreTimeoutSeconds` caps how long the driver waits for the operator restore before failing (default 30m).
107+
69108
## Inspecting the defaults
70109

71110
```bash
@@ -157,7 +196,7 @@ The default-objects gate emits three more:
157196

158197
## Admin overrides for `cozy-default`
159198

160-
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-foundationdb`, the two `cozy-default-velero-*`). The supported override path is the `backupStorage` block on the **`platform` component** of the `cozystack.cozystack-platform` Package CR:
199+
`cozy-default` is rendered by the `backupstrategy-controller` chart and owned by Flux's helm-controller. **Direct `kubectl edit backupclass cozy-default` is overwritten on the next helm reconcile** — the same applies to its companion `strategy.backups.cozystack.io/*` CRs (`cozy-default-cnpg`, `cozy-default-etcd`, `cozy-default-mariadb`, `cozy-default-altinity`, `cozy-default-mongodb`, `cozy-default-foundationdb`, the two `cozy-default-velero-*`). The supported override path is the `backupStorage` block on the **`platform` component** of the `cozystack.cozystack-platform` Package CR:
161200

162201
```yaml
163202
apiVersion: cozystack.io/v1alpha1
@@ -201,6 +240,7 @@ The defaults aim at a reasonable middle (30-day retention, gzip compression wher
201240

202241
- **CNPG strategy**: `barmanObjectStore.retentionPolicy`, `data.compression`, `wal.compression`.
203242
- **MariaDB strategy**: `compression`, `maxRetention`, `databases[]`.
243+
- **MongoDB strategy**: `storageName` (which `spec.backup.storages` entry on the psmdb cluster to use), `type` (`logical`), `compressionType` / `compressionLevel`. The S3 target itself is tuned via `backup.*` values on the MongoDB release (see [MongoDB: the application owns the backup storage](#mongodb-the-application-owns-the-backup-storage)).
204244
- **Altinity strategy**: tune the `clickhouse-backup` sidecar via `backup.*` values on the ClickHouse release; the strategy Pod is a thin HTTP client. When the S3 endpoint's certificate is signed by a private CA rather than a publicly-trusted one — SeaweedFS's in-cluster `:8333` being the case in point — point `backup.endpointCA` at a Secret holding that CA bundle; the chart mounts it into the sidecar and adds it to the trust store via `SSL_CERT_DIR`, which supplements the system CA set rather than replacing it.
205245
- **FoundationDB strategy**: `snapshotPeriodSeconds`, `agentCount`, `urlParameters[]`.
206246
- **Velero strategy (VMInstance / VMDisk)**: `ttl`, `includedResources[]`, `excludedResources[]`.

‎examples/backups/mongodb/00-bucket.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Cozystack-managed S3 bucket for storing MongoDB backups. The Bucket
2+
# controller provisions a Secret named "bucket-<bucket>-<username>" per user
3+
# (the bucket app carries the "bucket-" release prefix); run-all.sh reads the
4+
# "backup" user's BucketInfo and projects the access keys into the MongoDB
5+
# application's backup config.
6+
---
7+
apiVersion: apps.cozystack.io/v1alpha1
8+
kind: Bucket
9+
metadata:
10+
name: mongodb-backups
11+
spec:
12+
users:
13+
backup:
14+
readonly: false

0 commit comments

Comments
 (0)