Skip to content

Commit 0b60f36

Browse files
committed
ci(hack): route examples/backups/{vmi,vmdisk} edits to the vminstance suite
The two backup round-trips run the examples/backups/{vmi,vmdisk} scripts as their harness, but select-e2e.sh mapped an examples/backups/<dir> edit to a suite only when <dir> was itself a suite name. The dirs are vmi/vmdisk while the suite is vminstance, so a harness-only edit selected no suite and the round-trip that runs it never ran. Translate both dirs to vminstance, matching the vm-disk-application entry in src_to_suites, and pin it with a bats case. The suite runs the vmi demo with SKIP_RESTORE_TO_COPY=1, so 07-restore-to-copy.sh stays docs-only: selecting the suite for it would pass without executing it. Both round-trips write to the in-cluster seaweedfs-s3 endpoint, so a seaweedfs change selects the vminstance suite as well. Also correct the controller- owned-teardown note in docs/agents/e2e-testing.md (only Backup/RestoreJob finalizers reach Velero; a BackupJob has none). Assisted-by: LLM Signed-off-by: Andrey Kolkov <[email protected]>
1 parent 5ecbe22 commit 0b60f36

3 files changed

Lines changed: 33 additions & 2 deletions

File tree

‎docs/agents/e2e-testing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Trap-based cleanup was the single biggest source of false failures in the BATS s
7070

7171
### 4. Do not mask cleanup or teardown failures
7272

73-
- Resources a test creates that Chainsaw cannot reclaim — because a controller, not the test, owns them (e.g. the Velero `Backup`, `BackupStorageLocation`, and credentials secret in `cozy-velero` in the disabled `hack/e2e-chainsaw/backup/` suite) — must be pruned explicitly. Do not `|| true` over a stuck delete and leave stale state for the next run.
73+
- Resources a test creates that Chainsaw cannot reclaim — because a controller, not the test, owns them (e.g. the Velero `Backup`/`Restore` in `cozy-velero`, which the `vminstance` backup round-trips prune by deleting the cozystack `Backup` and `RestoreJob` CRs, whose finalizers then clean up the Velero objects asynchronously — the `BackupJob` has no finalizer and its `Backup` carries no owner reference, so deleting a `BackupJob` reclaims nothing — plus an explicit delete of the throwaway `BackupStorageLocation`, sequenced after the Velero `Backup` is gone so it does not race the finalizer) — must be pruned explicitly. Do not `|| true` over a stuck delete and leave stale state for the next run.
7474
- For nested tenants, tear down **child → parent with a hard wait for deletion between** each. Deleting the parent while a child is still uninstalling wedges the parent's cleanup Job on the child namespace, and both stuck uninstalls occupy helm-controller workers past the end of the test — starving whichever suite runs next. The kubernetes suites encode this ordering in `hack/e2e-chainsaw/_lib/run-kubernetes.sh`.
7575
- A test that creates a tenant `Kubernetes` cluster waits for that release to become **Ready** before the assertions it ends on — not before every read of the cluster, which is a different rule and a false one. The chart's post-install hook (`packages/apps/kubernetes/templates/oidc-rbac-job.yaml`) polls for the Kamaji-issued admin kubeconfig in the modes that have one, and that Secret is what the test needs to reach the tenant API, so bringing up a worker or waiting on a child release happens while the parent is still installing. Reading mid-install is fine. What the rule is about is the COMPLETION path: a run that asserts, passes and ends while the parent is still installing turns a green result into a red one in teardown. A failure path that diagnoses and exits mid-install does the same to the install, but that run is already red, so the second red costs noise rather than a wrong verdict — and the script has a number of such exits between the CR and the wait, which the guard does not and should not treat as violations.
7676
- A Helm release cannot be uninstalled while its install action is running, and for this kind the action can outlive the fast assertions — by how much depends on the mode, per the bullet below: `release.cozystack.io/helm-install-disable-wait` on the `ApplicationDefinition` drops the wait on the main resources, but the chart's blocking post-install hook still holds the install. A test that asserts and ends therefore hands the remainder of the install to teardown and is reported red on a step that tests nothing.

‎hack/select-e2e.sh‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,7 @@ src_to_suites() {
265265
# harbor stores its registry in a BucketClaim the seaweedfs COSI driver
266266
# serves. etcd's round-trip is left out because CI gates it off. A test
267267
# derives this set from the suites.
268-
seaweedfs-application) echo "bucket clickhouse harbor kafka kafka-metadata mariadb mongodb postgres rabbitmq redis seaweedfs" ;;
268+
seaweedfs-application) echo "bucket clickhouse harbor kafka kafka-metadata mariadb mongodb postgres rabbitmq redis seaweedfs vminstance" ;;
269269
# Both suites create a KeycloakClient and wait for the operator's finalizer
270270
# to remove it. kubernetes-application has no dependsOn on keycloak-operator
271271
# (#4680), so the graph reaches only monitoring and would drop the tenant
@@ -451,6 +451,17 @@ while IFS= read -r file || [ -n "$file" ]; do
451451
# selected_apps would empty the final intersection and trip the
452452
# full-suite safety net).
453453
app=$(echo "$file" | sed -nE 's,^examples/backups/([^/]+)/.*,\1,p')
454+
# The vmi and vmdisk backup demos are both harnessed by the vminstance
455+
# suite, which owns VMDisk coverage too (see vm-disk-application in
456+
# src_to_suites), so their dirs need the same source-style translation.
457+
# The suite runs the vmi demo with SKIP_RESTORE_TO_COPY=1, so the to-copy
458+
# step is docs-only: selecting the suite would pass without executing it.
459+
case "$file" in
460+
examples/backups/vmi/07-restore-to-copy.sh) continue ;;
461+
esac
462+
case "$app" in
463+
vmi|vmdisk) app=vminstance ;;
464+
esac
454465
if echo "$all_apps" | grep -Fxq "$app"; then
455466
selected_apps="$selected_apps $app"
456467
trigger_any=1

‎hack/select-e2e_test.bats‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,26 @@ assert_full_suite() {
280280
rm -rf "$tmp"
281281
}
282282

283+
@test "vmi/vmdisk backup harness edit selects the vminstance suite" {
284+
tmp=$(mktemp -d)
285+
cp -r packages/core/platform/sources "$tmp/sources"
286+
for f in examples/backups/vmi/run-all.sh examples/backups/vmdisk/run-all.sh; do
287+
echo "$f" > "$tmp/diff"
288+
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
289+
[ "$output" = "vminstance" ]
290+
done
291+
rm -rf "$tmp"
292+
}
293+
294+
@test "vmi restore-to-copy step, which the suite skips, selects nothing" {
295+
tmp=$(mktemp -d)
296+
cp -r packages/core/platform/sources "$tmp/sources"
297+
echo "examples/backups/vmi/07-restore-to-copy.sh" > "$tmp/diff"
298+
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources") || true
299+
rm -rf "$tmp"
300+
[ -z "$output" ]
301+
}
302+
283303
@test "backup example without a matching suite selects nothing" {
284304
tmp=$(mktemp -d)
285305
cp -r packages/core/platform/sources "$tmp/sources"

0 commit comments

Comments
 (0)