Skip to content

Commit be7523a

Browse files
committed
test(mongodb): pin the restore credential swap and document the opt-out
The restore-time decision to re-point a backup's credentialsSecret at the target cluster lived inline in reconcileMongoDBRestore with no test, so the same-bucket guard that keeps a cross-flow restore from authenticating against the wrong bucket could be dropped unnoticed. Extract it into mongodbRestoreCredentialsSecret and cover all four {legacy,system}x{legacy,system} combinations; neutralising the bucket match now reddens the suite. The projected-secret skip is kept as well: a target storage sharing the platform bucket but naming a different secret is better restored with the platform-projected cozy-backups-creds than with that secret. Document the reverse toggle: flipping useSystemBucket true->false leaves the driver-injected s3-storage on the live PerconaServerMongoDB under the driver's field manager, which the chart does not own and never prunes. It is inert once backups are disabled; the operator must delete it by hand to return the release to the legacy flow. Signed-off-by: Andrey Kolkov <[email protected]> Assisted-By: LLM
1 parent 21f49ad commit be7523a

3 files changed

Lines changed: 112 additions & 21 deletions

File tree

‎docs/operations/backup-classes.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,8 @@ When `useSystemBucket: true`:
119119

120120
Because the chart drops `spec.backup.tasks` and `spec.backup.pitr`, the psmdb operator's own scheduled backups and oplog PITR do **not** run on this flow — `backup.schedule` and `backup.retentionPolicy` stay in the schema for the default flow but have no effect here, and the `recoveryTime` PITR restores described below are not available. Migrate scheduled backups to a `backups.cozystack.io/Plan` against `cozy-default` instead. **Flipping this flag on a running app** removes the storage, tasks and pitr the app had, so a release that was taking scheduled backups + PITR before the flip stops until a `Plan` takes over.
121121

122+
**Opting back out** (`useSystemBucket` true→false) is not automatic in reverse: the driver stops injecting, but the `s3-storage` entry it already applied stays on the live `PerconaServerMongoDB` under the driver's own field manager (`cozystack-psmdb-backup-driver`) — the chart does not own it and so never prunes it. It is inert once `backup.enabled=false`; to fully return a release to the legacy flow, delete that storage entry from the `PerconaServerMongoDB` by hand before re-declaring the tenant's own `backup.*` S3 values.
123+
122124
### Point-in-time recovery (MongoDB)
123125

124126
psmdb records an oplog stream between logical backups. A MongoDB `RestoreJob` recovers to a timestamp via `spec.options.recoveryTime` — the same option name and RFC3339 format the Postgres/CNPG driver uses, so the two are uniform:

‎internal/backupcontroller/psmdbstrategy_controller.go‎

Lines changed: 44 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -401,6 +401,39 @@ func psmdbTargetCredentialsSecret(cluster *psmdbtypes.PerconaServerMongoDB, pref
401401
return ""
402402
}
403403

404+
// mongodbRestoreCredentialsSecret picks the s3 credentialsSecret a restore into
405+
// targetCluster should authenticate with. It returns the source's own reference
406+
// unless a safe swap applies, so the caller can compare and re-point only when
407+
// the value actually changes. Two guards make the swap correct across the legacy
408+
// and useSystemBucket flows:
409+
// - When the source already names the platform-projected cozy-backups-creds
410+
// (useSystemBucket), keep it: that Secret is projected into the restore
411+
// namespace and outlives the source app, so there is nothing to repair, and
412+
// repointing it at a legacy target's own creds would send a platform-bucket
413+
// restore at the wrong credentials.
414+
// - Otherwise adopt a target credential only when its storage is on the SAME
415+
// bucket as the archive (source.S3.Bucket): the backupSource's bucket stays
416+
// the source's, so a credential for a different bucket (a system-bucket
417+
// target's cozy-backups-creds against a legacy tenant bucket) would fail the
418+
// restore with AccessDenied. When no same-bucket target credential is
419+
// discoverable the source reference is kept (in-place restore into the
420+
// still-present source resolves to the same Secret, so this is a no-op).
421+
//
422+
// Returns "" only when the source carries no s3 reference at all.
423+
func mongodbRestoreCredentialsSecret(source *psmdbtypes.BackupSource, targetCluster *psmdbtypes.PerconaServerMongoDB) string {
424+
if source == nil || source.S3 == nil {
425+
return ""
426+
}
427+
current := source.S3.CredentialsSecret
428+
if current == psmdbDefaultCredentialsSecret {
429+
return current
430+
}
431+
if cred := psmdbTargetCredentialsSecret(targetCluster, source.StorageName, source.S3.Bucket); cred != "" {
432+
return cred
433+
}
434+
return current
435+
}
436+
404437
// psmdbStorageS3 extracts .s3.bucket and .s3.credentialsSecret from one psmdb
405438
// spec.backup.storages entry. Returns ("","") for an empty/non-object entry or
406439
// a non-s3 storage. The bucket lets the restore path refuse to hand a restore a
@@ -772,27 +805,17 @@ func (r *RestoreJobReconciler) reconcileMongoDBRestore(ctx context.Context, rest
772805
fmt.Sprintf("waiting for target psmdb.percona.com/PerconaServerMongoDB %s/%s to enable backups", target.Namespace, targetPSMDBName))
773806
}
774807

775-
// Re-point the restore at the TARGET cluster's own S3 credentials. On the
776-
// legacy flow the backupSource carries the SOURCE release's -s3-creds Secret,
777-
// which is deleted together with the source app — exactly the disaster-
778-
// recovery case backups exist for — so a restore into a fresh, differently-
779-
// named target must not depend on it. Two guards keep the swap correct across
780-
// flows:
781-
// - Skip it when the source already names the platform-projected
782-
// cozy-backups-creds (the useSystemBucket flow): that Secret is projected
783-
// into the restore namespace and outlives the source app, so there is
784-
// nothing to repair, and repointing it at a legacy target's own creds
785-
// would send a platform-bucket restore at the wrong credentials.
786-
// - Only adopt a target credential whose storage is on the SAME bucket as
787-
// the archive (source.S3.Bucket). The backupSource's bucket/prefix/
788-
// endpoint always stay the source's (that is where the archive lives);
789-
// swapping in a credential for a different bucket (a system-bucket
790-
// target's cozy-backups-creds against a legacy tenant bucket) would fail
791-
// the restore with AccessDenied. When no same-bucket target credential is
792-
// discoverable the source reference is kept (in-place restore into the
793-
// still-present source resolves to the same Secret, so this is a no-op).
794-
if source.S3 != nil && source.S3.CredentialsSecret != psmdbDefaultCredentialsSecret {
795-
if cred := psmdbTargetCredentialsSecret(targetCluster, source.StorageName, source.S3.Bucket); cred != "" && cred != source.S3.CredentialsSecret {
808+
// Re-point the restore at the TARGET cluster's own S3 credentials when a safe
809+
// swap applies. On the legacy flow the backupSource carries the SOURCE
810+
// release's -s3-creds Secret, which is deleted together with the source app —
811+
// exactly the disaster-recovery case backups exist for — so a restore into a
812+
// fresh, differently-named target must not depend on it. mongodbRestoreCreden-
813+
// tialsSecret encodes the cross-flow guards (skip when the source already
814+
// names the projected cozy-backups-creds; adopt a target credential only on
815+
// the same bucket as the archive); it is unit-tested across the four flow
816+
// combinations.
817+
if source.S3 != nil {
818+
if cred := mongodbRestoreCredentialsSecret(source, targetCluster); cred != "" && cred != source.S3.CredentialsSecret {
796819
logger.Debug("re-pointing MongoDB restore credentials at target cluster storage",
797820
"restorejob", restoreJob.Name, "from", source.S3.CredentialsSecret, "to", cred)
798821
source.S3.CredentialsSecret = cred

‎internal/backupcontroller/psmdbstrategy_controller_test.go‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,72 @@ func TestPsmdbTargetCredentialsSecret(t *testing.T) {
204204
}
205205
}
206206

207+
// TestMongoDBRestoreCredentialsSecret pins the restore credential-swap decision
208+
// across the four {legacy,system}×{legacy,system} flow combinations, so neither
209+
// cross-flow guard can be dropped unnoticed: the projected-secret skip (a system
210+
// source keeps cozy-backups-creds rather than adopting a legacy target's own
211+
// Secret) and the same-bucket match (a legacy source is not handed a credential
212+
// for a bucket its archive does not live in).
213+
func TestMongoDBRestoreCredentialsSecret(t *testing.T) {
214+
s3Storage := func(bucket, secret string) runtime.RawExtension {
215+
return runtime.RawExtension{Raw: []byte(fmt.Sprintf(`{"type":"s3","s3":{"bucket":%q,"credentialsSecret":%q}}`, bucket, secret))}
216+
}
217+
target := func(storages map[string]runtime.RawExtension) *psmdbtypes.PerconaServerMongoDB {
218+
return &psmdbtypes.PerconaServerMongoDB{
219+
Spec: psmdbtypes.PerconaServerMongoDBSpec{
220+
Backup: psmdbtypes.PerconaServerMongoDBBackupConfig{Storages: storages},
221+
},
222+
}
223+
}
224+
src := func(bucket, cred string) *psmdbtypes.BackupSource {
225+
return &psmdbtypes.BackupSource{StorageName: "s3-storage", S3: &psmdbtypes.BackupStorageS3{Bucket: bucket, CredentialsSecret: cred}}
226+
}
227+
cases := []struct {
228+
name string
229+
source *psmdbtypes.BackupSource
230+
target *psmdbtypes.PerconaServerMongoDB
231+
want string
232+
}{
233+
{
234+
name: "legacy source -> legacy target on the same bucket: swap to target creds (DR after source delete)",
235+
source: src("tenant-bucket", "src-s3-creds"),
236+
target: target(map[string]runtime.RawExtension{"s3-storage": s3Storage("tenant-bucket", "target-s3-creds")}),
237+
want: "target-s3-creds",
238+
},
239+
{
240+
name: "legacy source -> legacy target on a different bucket: keep source creds (no cross-bucket swap)",
241+
source: src("tenant-bucket", "src-s3-creds"),
242+
target: target(map[string]runtime.RawExtension{"s3-storage": s3Storage("other-bucket", "target-s3-creds")}),
243+
want: "src-s3-creds",
244+
},
245+
{
246+
name: "system source -> legacy target: keep projected cozy-backups-creds (outlives source, reads the platform bucket)",
247+
source: src("cozy-backups-PLATFORM", "cozy-backups-creds"),
248+
target: target(map[string]runtime.RawExtension{"s3-storage": s3Storage("tenant-bucket", "target-s3-creds")}),
249+
want: "cozy-backups-creds",
250+
},
251+
{
252+
name: "legacy source -> system-bucket target: keep source creds (target storage is on the platform bucket, not the archive's)",
253+
source: src("tenant-bucket", "src-s3-creds"),
254+
target: target(map[string]runtime.RawExtension{"s3-storage": s3Storage("cozy-backups-PLATFORM", "cozy-backups-creds")}),
255+
want: "src-s3-creds",
256+
},
257+
{
258+
name: "no s3 source: empty (caller keeps the source reference)",
259+
source: &psmdbtypes.BackupSource{StorageName: "s3-storage"},
260+
target: target(map[string]runtime.RawExtension{"s3-storage": s3Storage("tenant-bucket", "target-s3-creds")}),
261+
want: "",
262+
},
263+
}
264+
for _, tc := range cases {
265+
t.Run(tc.name, func(t *testing.T) {
266+
if got := mongodbRestoreCredentialsSecret(tc.source, tc.target); got != tc.want {
267+
t.Errorf("mongodbRestoreCredentialsSecret: got %q want %q", got, tc.want)
268+
}
269+
})
270+
}
271+
}
272+
207273
// ---------------------------------------------------------------------------
208274
// Backup-side ensure idempotency + CR shape
209275
// ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)