Skip to content

Commit 61a7a25

Browse files
authored
fix(backups): fail Velero RestoreJob rename for non-VMInstance kinds (cozystack#4426)
## What this PR does Fixes cozystack#3537. A `RestoreJob` with `spec.targetApplicationRef.name` set to a new name and `options.targetNamespace` set to another namespace reported `phase: Succeeded` for every Velero-backed application, but only a `VMInstance` was actually renamed. `postRestoreRename` looks up the restored HelmRelease under the `vm-instance-` prefix, gets `NotFound` for any other kind and returns `nil`, so a Postgres, MariaDB, ClickHouse, MongoDB or standalone VMDisk copy came up under the **source** name and nothing was surfaced. Renaming those kinds through the Velero strategy cannot work by construction: Velero DataUpload restores volumes under their original names, and only a `VMInstance` keeps using them after its HelmRelease is renamed (its disks keep their names and CDI adopts the restored PVCs). A renamed database `Cluster` or `VMDisk` renders fresh volumes for the new name and comes up empty next to orphaned source volumes. This PR takes the short-term, correctness half of the issue: the rename is rejected up front for every kind other than `VMInstance`, before a new RestoreJob enters `Running` or a Velero `Restore` is created, with a `Ready=False` message that names the kind and points at omitting `targetApplicationRef.name` or, for kinds that have one, using the application's engine-native strategy (for Postgres that is the CNPG strategy, which supports restore-to-copy). The existing same-namespace guard keeps precedence, so its message is unchanged. A rename RestoreJob that is already `Running` when a controller with this check starts is failed on its next reconcile, after its Velero `Restore` may have completed. Such a job says in its message that the application may already sit in the target namespace under its source name (a retry without the rename would otherwise trip `failIfTargetExists` on that copy), and its resource-modifier ConfigMap is released immediately rather than at the job's deletion. Jobs already `Succeeded` or `Failed` are never revisited. Rename-on-restore for non-VM applications through engine-native recovery (the longer-term feature in the issue) is out of scope here. A standalone `VMDisk` is rejected as well: its rename needs a post-restore PVC rebind and DataVolume replacement that do not exist yet, tracked in cozystack#4428. Tests: `TestVeleroRenameSupported` and `TestReconcileVeleroRestore_RenameGuard`. Rejected inputs: Postgres and VMDisk cross-namespace rename, and a VMDisk backup restored with a `VMInstance` target kind (the guard keys on the backup's kind, since that is what `postRestoreRename` acts on) → `Failed` with the new message; Postgres same-namespace rename → the pre-existing DataUpload message. Accepted inputs: VMInstance cross-namespace rename, a VMDisk copy that keeps its name, and a VMDisk in-place restore without `targetApplicationRef` → `Running`. The rejected cases fail on `main` (phase `Running` instead of `Failed`); the accepted cases fail if the `IsRenamed` conjunct is dropped. `TestReconcileVeleroRestore_RenameGuard_InFlight` covers the already-running case for VMDisk (`Failed`, message names the source-name copy, ConfigMap gone) and VMInstance (`Succeeded`). ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff: two Go files under `internal/backupcontroller/`, no API, CRD, chart, values, schema or docs change. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(backups): a Velero-strategy RestoreJob that requests a different targetApplicationRef.name for an application kind other than VMInstance (Postgres, MariaDB, ClickHouse, MongoDB, VMDisk) now fails with an explicit message instead of reporting Succeeded with the application restored under its original name. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Velero restores now fail with a clear message when a requested application rename isn’t supported for that application type. * If an unsupported rename is detected after a restore has started, the restore is stopped and its temporary configuration is cleaned up. The message notes that resources may already have been restored under the original name. * Renames for VM instances and restores that keep the original application name continue to work. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents e9868b4 + d50385f commit 61a7a25

2 files changed

Lines changed: 327 additions & 0 deletions

File tree

‎internal/backupcontroller/velerostrategy_controller.go‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -606,6 +606,27 @@ func (r *RestoreJobReconciler) reconcileVeleroRestore(ctx context.Context, resto
606606
"Use options.targetNamespace to restore into a different namespace")
607607
}
608608

609+
// Validate: rename is only implemented for VMInstance. Failing here beats a
610+
// Succeeded RestoreJob whose application silently keeps the source name.
611+
if target.IsRenamed && !veleroRenameSupported(backup.Spec.ApplicationRef.Kind) {
612+
msg := fmt.Sprintf(
613+
"restoring %s %q under a different name (%q) is not supported by the Velero strategy: "+
614+
"Velero DataUpload always restores volumes under their original names and only a VMInstance "+
615+
"can adopt them. Omit targetApplicationRef.name to keep the source name",
616+
backup.Spec.ApplicationRef.Kind, backup.Spec.ApplicationRef.Name, target.AppName)
617+
if backup.Spec.ApplicationRef.Kind != vmDiskAppKind {
618+
msg += ", or use the application's engine-native strategy to restore into a new application"
619+
}
620+
if restoreJob.Status.StartedAt != nil {
621+
// The job predates this check: its Velero Restore may already have
622+
// restored the application under the source name.
623+
r.cleanupResourceModifierConfigMaps(ctx, restoreJob)
624+
msg += fmt.Sprintf(". This RestoreJob was already running, so %q may have been restored into namespace %q under its source name",
625+
backup.Spec.ApplicationRef.Name, target.Namespace)
626+
}
627+
return r.markRestoreJobFailed(ctx, restoreJob, msg)
628+
}
629+
609630
// Step 1: On first reconcile, set startedAt and phase = Running
610631
if restoreJob.Status.StartedAt == nil {
611632
logger.Debug("setting RestoreJob StartedAt and phase to Running")
@@ -958,6 +979,16 @@ var (
958979
dataVolumeGVR = schema.GroupVersionResource{Group: "cdi.kubevirt.io", Version: "v1beta1", Resource: "datavolumes"}
959980
)
960981

982+
// veleroRenameSupported reports whether postRestoreRename can restore the
983+
// given application kind under a new name. Velero DataUpload writes volumes
984+
// under their original names, and only a VMInstance re-attaches them after its
985+
// HelmRelease is renamed: its disks keep their names and CDI adopts the restored
986+
// PVCs. Every other kind (a standalone VMDisk, a database operator's Cluster)
987+
// renders fresh volumes for the new name and comes up empty.
988+
func veleroRenameSupported(appKind string) bool {
989+
return appKind == vmInstanceKind
990+
}
991+
961992
// helmReleaseNameForApp returns the HelmRelease name for the given application
962993
// kind and name. Returns empty string for unsupported kinds.
963994
func helmReleaseNameForApp(appKind, appName string) string {

‎internal/backupcontroller/velerostrategy_controller_test.go‎

Lines changed: 296 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,11 @@ package backupcontroller
33
import (
44
"context"
55
"encoding/json"
6+
"strings"
67
"testing"
78

89
corev1 "k8s.io/api/core/v1"
10+
"k8s.io/apimachinery/pkg/api/errors"
911
"k8s.io/apimachinery/pkg/api/meta"
1012
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
1113
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@@ -1122,6 +1124,300 @@ func TestHelmReleaseNameForApp(t *testing.T) {
11221124
}
11231125
}
11241126

1127+
func TestVeleroRenameSupported(t *testing.T) {
1128+
tests := []struct {
1129+
kind string
1130+
want bool
1131+
}{
1132+
{vmInstanceKind, true},
1133+
{vmDiskAppKind, false},
1134+
{"Postgres", false},
1135+
{"MariaDB", false},
1136+
{"ClickHouse", false},
1137+
{"MongoDB", false},
1138+
{"", false},
1139+
}
1140+
for _, tt := range tests {
1141+
t.Run(tt.kind, func(t *testing.T) {
1142+
if got := veleroRenameSupported(tt.kind); got != tt.want {
1143+
t.Errorf("veleroRenameSupported(%q) = %v, want %v", tt.kind, got, tt.want)
1144+
}
1145+
})
1146+
}
1147+
}
1148+
1149+
// TestReconcileVeleroRestore_RenameGuard covers the fix for a RestoreJob that
1150+
// asked for a new application name via targetApplicationRef.name but whose
1151+
// kind cannot be renamed by postRestoreRename: it used to report Succeeded
1152+
// with the application restored under the source name.
1153+
func TestReconcileVeleroRestore_RenameGuard(t *testing.T) {
1154+
const (
1155+
sourceNS = "tenant-src"
1156+
targetNS = "tenant-dst"
1157+
)
1158+
1159+
tests := []struct {
1160+
name string
1161+
appKind string
1162+
targetKind string // targetApplicationRef.kind; defaults to appKind
1163+
targetName string // targetApplicationRef.name; "" omits targetApplicationRef
1164+
targetNS string
1165+
wantPhase backupsv1alpha1.RestoreJobPhase
1166+
wantMessage string
1167+
wantMissing string // a remedy the message must not offer for this kind
1168+
}{
1169+
{
1170+
name: "postgres cross-namespace rename is rejected",
1171+
appKind: "Postgres",
1172+
targetName: "dst",
1173+
targetNS: targetNS,
1174+
wantPhase: backupsv1alpha1.RestoreJobPhaseFailed,
1175+
wantMessage: `restoring Postgres "src" under a different name ("dst") is not supported by the Velero strategy`,
1176+
},
1177+
{
1178+
name: "postgres rejection points at the engine-native strategy",
1179+
appKind: "Postgres",
1180+
targetName: "dst",
1181+
targetNS: targetNS,
1182+
wantPhase: backupsv1alpha1.RestoreJobPhaseFailed,
1183+
wantMessage: "engine-native strategy",
1184+
},
1185+
{
1186+
name: "vmdisk cross-namespace rename is rejected",
1187+
appKind: vmDiskAppKind,
1188+
targetName: "dst",
1189+
targetNS: targetNS,
1190+
wantPhase: backupsv1alpha1.RestoreJobPhaseFailed,
1191+
wantMessage: `restoring VMDisk "src" under a different name ("dst") is not supported by the Velero strategy`,
1192+
wantMissing: "engine-native strategy",
1193+
},
1194+
{
1195+
name: "vmdisk rename is rejected by the backup kind, not the target kind",
1196+
appKind: vmDiskAppKind,
1197+
targetKind: vmInstanceKind,
1198+
targetName: "dst",
1199+
targetNS: targetNS,
1200+
wantPhase: backupsv1alpha1.RestoreJobPhaseFailed,
1201+
wantMessage: `restoring VMDisk "src" under a different name ("dst") is not supported by the Velero strategy`,
1202+
},
1203+
{
1204+
name: "postgres same-namespace rename keeps the DataUpload message",
1205+
appKind: "Postgres",
1206+
targetName: "dst",
1207+
targetNS: "",
1208+
wantPhase: backupsv1alpha1.RestoreJobPhaseFailed,
1209+
wantMessage: "restoring to the same namespace with a different application name is not supported",
1210+
},
1211+
{
1212+
name: "vminstance cross-namespace rename passes the guard",
1213+
appKind: vmInstanceKind,
1214+
targetName: "dst",
1215+
targetNS: targetNS,
1216+
wantPhase: backupsv1alpha1.RestoreJobPhaseRunning,
1217+
},
1218+
{
1219+
name: "vmdisk cross-namespace copy keeping the source name passes the guard",
1220+
appKind: vmDiskAppKind,
1221+
targetName: "src",
1222+
targetNS: targetNS,
1223+
wantPhase: backupsv1alpha1.RestoreJobPhaseRunning,
1224+
},
1225+
{
1226+
name: "vmdisk in-place restore without targetApplicationRef passes the guard",
1227+
appKind: vmDiskAppKind,
1228+
targetNS: "",
1229+
wantPhase: backupsv1alpha1.RestoreJobPhaseRunning,
1230+
},
1231+
}
1232+
for _, tt := range tests {
1233+
t.Run(tt.name, func(t *testing.T) {
1234+
backup := &backupsv1alpha1.Backup{
1235+
ObjectMeta: metav1.ObjectMeta{Name: "src-sb", Namespace: sourceNS},
1236+
Spec: backupsv1alpha1.BackupSpec{
1237+
ApplicationRef: corev1.TypedLocalObjectReference{
1238+
APIGroup: stringPtr("apps.cozystack.io"),
1239+
Kind: tt.appKind,
1240+
Name: "src",
1241+
},
1242+
StrategyRef: corev1.TypedLocalObjectReference{Kind: "Velero", Name: "cozy-default-velero"},
1243+
},
1244+
}
1245+
restoreJob := &backupsv1alpha1.RestoreJob{
1246+
ObjectMeta: metav1.ObjectMeta{Name: "src-restore", Namespace: sourceNS},
1247+
Spec: backupsv1alpha1.RestoreJobSpec{
1248+
BackupRef: corev1.LocalObjectReference{Name: "src-sb"},
1249+
},
1250+
}
1251+
if tt.targetName != "" {
1252+
targetKind := tt.targetKind
1253+
if targetKind == "" {
1254+
targetKind = tt.appKind
1255+
}
1256+
restoreJob.Spec.TargetApplicationRef = &corev1.TypedLocalObjectReference{
1257+
APIGroup: stringPtr("apps.cozystack.io"),
1258+
Kind: targetKind,
1259+
Name: tt.targetName,
1260+
}
1261+
}
1262+
if tt.targetNS != "" {
1263+
raw, err := json.Marshal(RestoreOptions{CommonRestoreOptions: CommonRestoreOptions{TargetNamespace: tt.targetNS}})
1264+
if err != nil {
1265+
t.Fatalf("marshal options: %v", err)
1266+
}
1267+
restoreJob.Spec.Options = &runtime.RawExtension{Raw: raw}
1268+
}
1269+
targetNamespace := &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: targetNS}}
1270+
1271+
reconciler := newTestRestoreJobReconcilerWithDynamic(t, nil, restoreJob, backup, targetNamespace)
1272+
reconciler.Client = newRestoreJobTestClient(t, restoreJob, backup, targetNamespace)
1273+
ctx := context.Background()
1274+
1275+
if _, err := reconciler.reconcileVeleroRestore(ctx, restoreJob, backup); err != nil {
1276+
t.Fatalf("reconcileVeleroRestore() error = %v", err)
1277+
}
1278+
1279+
got := &backupsv1alpha1.RestoreJob{}
1280+
if err := reconciler.Get(ctx, client.ObjectKeyFromObject(restoreJob), got); err != nil {
1281+
t.Fatalf("get RestoreJob: %v", err)
1282+
}
1283+
if got.Status.Phase != tt.wantPhase {
1284+
t.Fatalf("phase = %q, want %q", got.Status.Phase, tt.wantPhase)
1285+
}
1286+
if tt.wantMessage == "" {
1287+
return
1288+
}
1289+
ready := meta.FindStatusCondition(got.Status.Conditions, "Ready")
1290+
if ready == nil {
1291+
t.Fatalf("Ready condition missing, conditions: %+v", got.Status.Conditions)
1292+
}
1293+
if !strings.Contains(ready.Message, tt.wantMessage) {
1294+
t.Errorf("Ready message = %q, want it to contain %q", ready.Message, tt.wantMessage)
1295+
}
1296+
if tt.wantMissing != "" && strings.Contains(ready.Message, tt.wantMissing) {
1297+
t.Errorf("Ready message = %q, must not offer %q", ready.Message, tt.wantMissing)
1298+
}
1299+
})
1300+
}
1301+
}
1302+
1303+
// TestReconcileVeleroRestore_RenameGuard_InFlight covers a rename RestoreJob
1304+
// that was already Running when the guard was introduced: its Velero Restore
1305+
// has completed under the source name, so the job must fail with a message
1306+
// saying so and release its resource-modifier ConfigMap instead of waiting
1307+
// for the job's deletion. A VMInstance in the same state still succeeds.
1308+
func TestReconcileVeleroRestore_RenameGuard_InFlight(t *testing.T) {
1309+
const (
1310+
sourceNS = "tenant-src"
1311+
targetNS = "tenant-dst"
1312+
)
1313+
tests := []struct {
1314+
name string
1315+
appKind string
1316+
wantPhase backupsv1alpha1.RestoreJobPhase
1317+
wantMessage string
1318+
}{
1319+
{
1320+
name: "vmdisk in-flight rename fails and names the source-name copy",
1321+
appKind: vmDiskAppKind,
1322+
wantPhase: backupsv1alpha1.RestoreJobPhaseFailed,
1323+
wantMessage: `This RestoreJob was already running, so "src" may have been restored into namespace "tenant-dst" under its source name`,
1324+
},
1325+
{
1326+
name: "vminstance in-flight rename still succeeds",
1327+
appKind: vmInstanceKind,
1328+
wantPhase: backupsv1alpha1.RestoreJobPhaseSucceeded,
1329+
},
1330+
}
1331+
for _, tt := range tests {
1332+
t.Run(tt.name, func(t *testing.T) {
1333+
started := metav1.Now()
1334+
backup := &backupsv1alpha1.Backup{
1335+
ObjectMeta: metav1.ObjectMeta{Name: "src-sb", Namespace: sourceNS},
1336+
Spec: backupsv1alpha1.BackupSpec{
1337+
ApplicationRef: corev1.TypedLocalObjectReference{
1338+
APIGroup: stringPtr("apps.cozystack.io"),
1339+
Kind: tt.appKind,
1340+
Name: "src",
1341+
},
1342+
StrategyRef: corev1.TypedLocalObjectReference{Kind: "Velero", Name: "cozy-default-velero"},
1343+
DriverMetadata: map[string]string{veleroBackupNameMetadataKey: "vb"},
1344+
},
1345+
}
1346+
raw, err := json.Marshal(RestoreOptions{CommonRestoreOptions: CommonRestoreOptions{TargetNamespace: targetNS}})
1347+
if err != nil {
1348+
t.Fatalf("marshal options: %v", err)
1349+
}
1350+
restoreJob := &backupsv1alpha1.RestoreJob{
1351+
ObjectMeta: metav1.ObjectMeta{Name: "src-restore", Namespace: sourceNS},
1352+
Spec: backupsv1alpha1.RestoreJobSpec{
1353+
BackupRef: corev1.LocalObjectReference{Name: "src-sb"},
1354+
TargetApplicationRef: &corev1.TypedLocalObjectReference{
1355+
APIGroup: stringPtr("apps.cozystack.io"),
1356+
Kind: tt.appKind,
1357+
Name: "dst",
1358+
},
1359+
Options: &runtime.RawExtension{Raw: raw},
1360+
},
1361+
Status: backupsv1alpha1.RestoreJobStatus{
1362+
Phase: backupsv1alpha1.RestoreJobPhaseRunning,
1363+
StartedAt: &started,
1364+
},
1365+
}
1366+
owningLabels := map[string]string{
1367+
backupsv1alpha1.OwningJobNameLabel: restoreJob.Name,
1368+
backupsv1alpha1.OwningJobNamespaceLabel: restoreJob.Namespace,
1369+
}
1370+
veleroRestore := &velerov1.Restore{
1371+
ObjectMeta: metav1.ObjectMeta{Name: "vr", Namespace: veleroNamespace, Labels: owningLabels},
1372+
Status: velerov1.RestoreStatus{Phase: velerov1.RestorePhaseCompleted},
1373+
}
1374+
modifiers := &corev1.ConfigMap{
1375+
ObjectMeta: metav1.ObjectMeta{Name: "restore-modifiers-tenant-src-src-restore", Namespace: veleroNamespace, Labels: owningLabels},
1376+
}
1377+
strategy := &strategyv1alpha1.Velero{ObjectMeta: metav1.ObjectMeta{Name: "cozy-default-velero"}}
1378+
targetNamespace := &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: targetNS}}
1379+
1380+
reconciler := newTestRestoreJobReconcilerWithDynamic(t, nil)
1381+
testScheme := runtime.NewScheme()
1382+
_ = scheme.AddToScheme(testScheme)
1383+
_ = backupsv1alpha1.AddToScheme(testScheme)
1384+
_ = velerov1.AddToScheme(testScheme)
1385+
_ = strategyv1alpha1.AddToScheme(testScheme)
1386+
reconciler.Client = clientfake.NewClientBuilder().
1387+
WithScheme(testScheme).
1388+
WithObjects(restoreJob, backup, veleroRestore, modifiers, strategy, targetNamespace).
1389+
WithStatusSubresource(&backupsv1alpha1.RestoreJob{}).
1390+
Build()
1391+
ctx := context.Background()
1392+
1393+
if _, err := reconciler.reconcileVeleroRestore(ctx, restoreJob, backup); err != nil {
1394+
t.Fatalf("reconcileVeleroRestore() error = %v", err)
1395+
}
1396+
1397+
got := &backupsv1alpha1.RestoreJob{}
1398+
if err := reconciler.Get(ctx, client.ObjectKeyFromObject(restoreJob), got); err != nil {
1399+
t.Fatalf("get RestoreJob: %v", err)
1400+
}
1401+
if got.Status.Phase != tt.wantPhase {
1402+
t.Fatalf("phase = %q, want %q", got.Status.Phase, tt.wantPhase)
1403+
}
1404+
if err := reconciler.Get(ctx, client.ObjectKeyFromObject(modifiers), &corev1.ConfigMap{}); !errors.IsNotFound(err) {
1405+
t.Errorf("resource-modifier ConfigMap should be deleted, get err = %v", err)
1406+
}
1407+
if tt.wantMessage == "" {
1408+
return
1409+
}
1410+
ready := meta.FindStatusCondition(got.Status.Conditions, "Ready")
1411+
if ready == nil {
1412+
t.Fatalf("Ready condition missing, conditions: %+v", got.Status.Conditions)
1413+
}
1414+
if !strings.Contains(ready.Message, tt.wantMessage) {
1415+
t.Errorf("Ready message = %q, want it to contain %q", ready.Message, tt.wantMessage)
1416+
}
1417+
})
1418+
}
1419+
}
1420+
11251421
func TestTargetHelmReleaseExists_VMDisk(t *testing.T) {
11261422
ns := "tenant-copy"
11271423
hr := makeUnstructuredHelmRelease("vm-disk-ubuntu-source", ns, nil)

0 commit comments

Comments
 (0)