Skip to content

Commit 0d04bcc

Browse files
androndoclaude
andcommitted
test(vmdisk): run the VMDisk backup/restore round-trip un-gated in CI
Make vmdisk-2-backup-roundtrip actually exercise a Velero backup/restore in CI instead of exiting on the gate. It is the in-CI proof that the cozy-default-velero-vmdisk path works end to end: a CSI VolumeSnapshot of the LINSTOR `replicated` block disk, moved to S3 by Velero's data mover (kopia), then restored back to a Bound, Ready disk. The platform ships a default VolumeSnapshotClass for linstor.csi.linbit.com and runs Velero with features: EnableCSI + deployNodeAgent, so the machinery is present in the sandbox. VMDisk (not VMInstance) is the un-gated CI case on purpose: it drives the same Velero + CSI-data-mover + S3 path but captures only HelmRelease + PVC (+ configmaps) — no nested VM boot, no guest-agent freeze, no 20Gi image — so it is fast and reliable enough for kind. The heavier VMInstance round-trip stays gated behind VMI_E2E_S3_ROUNDTRIP=1. The platform cozy-default BSL derives the EXTERNAL S3 ingress endpoint, which is unroutable inside kind (the same reason the postgres round-trip targets the in-cluster endpoint). So the harness stands up a throwaway BSL cozy-e2e-vmdisk pointing at the in-cluster seaweedfs endpoint with insecureSkipTLSVerify (kind cannot trust the self-signed SeaweedFS CA) and checksumAlgorithm="" (the velero-plugin-for-aws v1.14 / AWS-SDK-v2 vs SeaweedFS incompatibility) — the #3236 fixes applied at the BSL — reusing the platform bucket + cozy-backups-creds under a distinct prefix. The example scripts stay generic and just consume BACKUP_STORAGE_LOCATION. Runs in tenant-root so the data-mover DataUpload Pods reach seaweedfs over same-tenant egress; the finally tears down the throwaway BSL alongside the example cleanup. Assisted-By: Claude <[email protected]> Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Andrey Kolkov <[email protected]>
1 parent d3359d5 commit 0d04bcc

1 file changed

Lines changed: 82 additions & 37 deletions

File tree

‎hack/e2e-chainsaw/vminstance/chainsaw-test.yaml‎

Lines changed: 82 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -238,28 +238,34 @@ spec:
238238
cd ../../..
239239
NAMESPACE="$NAMESPACE" examples/backups/vmi/cleanup.sh
240240
---
241-
# vmdisk-2-backup-roundtrip — full Velero backup/restore round-trip for a
242-
# STANDALONE VMDisk, gated out of CI (opt in with VMI_E2E_S3_ROUNDTRIP=1, the
243-
# same flag as the VMInstance round-trip above).
241+
# vmdisk-2-backup-roundtrip — REAL Velero backup/restore round-trip for a
242+
# standalone VMDisk, run un-gated in CI. This is the in-CI proof that the
243+
# cozy-default-velero-vmdisk path works end to end: a CSI VolumeSnapshot of the
244+
# LINSTOR `replicated` block disk, moved to S3 by Velero's data mover (kopia),
245+
# then restored back to a Bound, Ready disk.
244246
#
245-
# The other cozy-default Velero path: cozy-default binds VMDisk to a Velero
246-
# strategy (cozy-default-velero-vmdisk) just as it does VMInstance, but a
247-
# standalone disk exercises a distinct path — the strategy captures only the
248-
# HelmRelease + PVC (+ configmaps), with no VirtualMachine / VMI / launcher pod.
249-
# It delegates the flow to the validated example scripts
250-
# (examples/backups/vmdisk/run-all.sh): Velero strategy -> BackupClass -> an
251-
# imported VMDisk -> ad-hoc BackupJob (waits Succeeded) -> in-place RestoreJob
252-
# (waits Succeeded, then waits for the restored disk to be Bound + Ready). A
253-
# standalone disk is not mounted anywhere, so the round-trip proof is that the
254-
# restored VMDisk reconciles back to a Ready HelmRelease + Bound PVC; in-guest
255-
# data is exercised by vminstance-2-backup-roundtrip, which boots the VM.
247+
# VMDisk (not VMInstance) is the un-gated CI case on purpose: it exercises the
248+
# same Velero + CSI-data-mover + S3 machinery but captures only HelmRelease +
249+
# PVC (+ configmaps) — no nested VM boot, no guest-agent freeze, no 20Gi image —
250+
# so it is fast and reliable enough for the kind sandbox. The heavier VMInstance
251+
# round-trip above stays gated behind VMI_E2E_S3_ROUNDTRIP=1.
256252
#
257-
# Runs in tenant-root and is gated for the same reasons as the VMInstance
258-
# round-trip (Cilium egress to tenant-root's seaweedfs; the sandbox S3 endpoint
259-
# serves a self-signed cert and the platform BSL defaults do not complete the
260-
# flow out of the box — cozystack/cozystack#3236). Teardown is explicit
261-
# (skipDelete: true): the BackupClass and Velero strategy are cluster-scoped and
262-
# the Velero Backup/Restore live in cozy-velero, so cleanup.sh prunes them.
253+
# S3 target: the platform cozy-default BSL derives the EXTERNAL S3 ingress
254+
# endpoint, which is an unroutable placeholder inside kind (the same reason the
255+
# postgres round-trip targets the in-cluster endpoint). So the harness stands up
256+
# a throwaway BSL `cozy-e2e-vmdisk` pointing at the IN-CLUSTER seaweedfs endpoint
257+
# with insecureSkipTLSVerify (kind cannot trust the self-signed SeaweedFS CA) and
258+
# checksumAlgorithm="" (the velero-plugin-for-aws v1.14/AWS-SDK-v2 vs SeaweedFS
259+
# incompatibility) — i.e. the cozystack/cozystack#3236 fixes applied at the BSL.
260+
# It reuses the platform bucket + cozy-backups-creds (a distinct prefix isolates
261+
# the objects). The example scripts stay generic — they just consume
262+
# BACKUP_STORAGE_LOCATION.
263+
#
264+
# Runs in tenant-root: Velero's data-mover DataUpload Pods run in the source
265+
# namespace and reach tenant-root's seaweedfs over same-tenant egress. Teardown
266+
# is explicit (skipDelete: true) — cleanup.sh prunes the cluster-scoped
267+
# BackupClass/strategy and the cozy-velero Velero Backup/Restore, and the harness
268+
# deletes the throwaway BSL.
263269
apiVersion: chainsaw.kyverno.io/v1alpha1
264270
kind: Test
265271
metadata:
@@ -282,43 +288,82 @@ spec:
282288
- describe:
283289
apiVersion: backups.cozystack.io/v1alpha1
284290
kind: Backup
291+
# The CSI snapshot + data-mover work lands in these Velero CRs and the
292+
# node-agent/velero Pod logs; dump them on failure.
285293
- podLogs:
286294
namespace: cozy-velero
287295
selector: app.kubernetes.io/name=velero
288296
tail: 200
289297
steps:
290298
- name: run
291299
try:
292-
# 40m budget: Ubuntu image import into a 5Gi disk, a full-disk Velero backup
293-
# to S3, and an in-place restore. run-all.sh is `set -e` and fails fast on a
294-
# terminal BackupJob/RestoreJob phase rather than polling out the budget.
300+
# 35m budget: build the in-cluster BSL, import the Ubuntu image into a 5Gi
301+
# disk (~150s proven by the vmdisk test above), a CSI-snapshot data-mover
302+
# backup to S3, and an in-place restore. run-all.sh is `set -e` and fails
303+
# fast on a terminal BackupJob/RestoreJob phase rather than polling the
304+
# budget out. BSL setup is in this step (not a separate one) so the finally
305+
# below always tears the BSL down, even if setup fails.
295306
- script:
296-
timeout: 40m
307+
timeout: 35m
297308
content: |
298309
set -eu
299-
if [ "${VMI_E2E_S3_ROUNDTRIP:-0}" != "1" ]; then
300-
echo "# VMDisk S3 backup/restore round-trip gated out of CI: Velero"
301-
echo "# moves the disk to the SeaweedFS BackupStorageLocation, whose"
302-
echo "# in-cluster endpoint serves a self-signed cert kind cannot trust"
303-
echo "# (cozystack/cozystack#3236). Set VMI_E2E_S3_ROUNDTRIP=1 on a real"
304-
echo "# backups-enabled cluster to run it."
305-
exit 0
306-
fi
310+
# Build the in-cluster BSL the example will target. Reuse the platform
311+
# bucket (read its COSI-assigned name off cozy-default, created once
312+
# the platform BucketClaim reconciles) + cozy-backups-creds.
313+
echo "Waiting for the platform cozy-default BSL to expose its bucket name..."
314+
timeout 180 sh -ec 'until kubectl -n cozy-velero get bsl cozy-default -o jsonpath="{.spec.objectStorage.bucket}" 2>/dev/null | grep -q .; do sleep 5; done'
315+
BUCKET=$(kubectl -n cozy-velero get bsl cozy-default -o jsonpath='{.spec.objectStorage.bucket}')
316+
REGION=$(kubectl -n cozy-velero get bsl cozy-default -o jsonpath='{.spec.config.region}')
317+
[ -n "$REGION" ] || REGION=us-east-1
318+
echo "Platform bucket: $BUCKET (region $REGION)"
319+
kubectl apply -f - <<YAML
320+
apiVersion: velero.io/v1
321+
kind: BackupStorageLocation
322+
metadata:
323+
name: cozy-e2e-vmdisk
324+
namespace: cozy-velero
325+
labels:
326+
cozystack.io/e2e: vmdisk-backup
327+
spec:
328+
provider: aws
329+
default: false
330+
objectStorage:
331+
bucket: "$BUCKET"
332+
prefix: velero-e2e-vmdisk
333+
credential:
334+
name: cozy-backups-creds
335+
key: cloud
336+
config:
337+
region: "$REGION"
338+
s3ForcePathStyle: "true"
339+
s3Url: https://seaweedfs-s3.tenant-root.svc.cozy.local:8333
340+
insecureSkipTLSVerify: "true"
341+
checksumAlgorithm: ""
342+
YAML
343+
echo "Waiting for BSL cozy-e2e-vmdisk to validate Available..."
344+
timeout 180 sh -ec '
345+
until [ "$(kubectl -n cozy-velero get bsl cozy-e2e-vmdisk -o jsonpath="{.status.phase}" 2>/dev/null)" = "Available" ]; do
346+
sleep 5
347+
done'
348+
kubectl -n cozy-velero get bsl cozy-e2e-vmdisk
349+
307350
cd ../../..
308351
[ -x examples/backups/vmdisk/run-all.sh ] || { echo "vmdisk backup example scripts not found at examples/backups/vmdisk" >&2; exit 1; }
309-
NAMESPACE="$NAMESPACE" BACKUP_STORAGE_LOCATION=cozy-default \
352+
NAMESPACE="$NAMESPACE" BACKUP_STORAGE_LOCATION=cozy-e2e-vmdisk \
310353
examples/backups/vmdisk/run-all.sh
311354
check:
312355
($error == null): true
313356
finally:
314-
# Prune the harness-created resources (cluster-scoped BackupClass / Velero
315-
# strategy, namespaced BackupJob/RestoreJob/Backup which cascade to the
316-
# Velero Backup/Restore in cozy-velero, the VMDisk and any renamed original
317-
# PVC). Idempotent: a no-op when the round-trip was gated out.
357+
# Prune the harness-created resources: the example cleanup (cluster-scoped
358+
# BackupClass/strategy, namespaced BackupJob/RestoreJob/Backup which cascade
359+
# to the Velero Backup/Restore in cozy-velero, the VMDisk and any renamed
360+
# original PVC), then the throwaway BSL. Has teeth — a teardown that does not
361+
# settle fails the step.
318362
- description: vmdisk backup-flow cleanup
319363
script:
320364
timeout: 15m
321365
content: |
322366
set -eu
323367
cd ../../..
324368
NAMESPACE="$NAMESPACE" examples/backups/vmdisk/cleanup.sh
369+
kubectl -n cozy-velero delete bsl cozy-e2e-vmdisk --ignore-not-found

0 commit comments

Comments
 (0)