Skip to content

Commit 4ac0cd5

Browse files
androndomyasnikovdaniil
authored andcommitted
ci(hack): route examples/backups/{vmi,vmdisk} edits to the vminstance suite
The two backup round-trips run the examples/backups/{vmi,vmdisk} scripts as their harness, but select-e2e.sh mapped an examples/backups/<dir> edit to a suite only when <dir> was itself a suite name. The dirs are vmi/vmdisk while the suite is vminstance, so a harness-only edit selected no suite and the round-trip that runs it never ran. Translate both dirs to vminstance, matching the vm-disk-application entry in src_to_suites, and pin it with a bats case. The suite runs the vmi demo with SKIP_RESTORE_TO_COPY=1, so 07-restore-to-copy.sh stays docs-only: selecting the suite for it would pass without executing it. Both round-trips write to the in-cluster seaweedfs-s3 endpoint, so a seaweedfs change selects the vminstance suite as well. Also correct the controller- owned-teardown note in docs/agents/e2e-testing.md (only Backup/RestoreJob finalizers reach Velero; a BackupJob has none). The two round-trips are also the only Tests that run a Velero backup and restore, so the velero PackageSource maps to the vminstance suite; it otherwise maps to a velero suite that does not exist, and a Velero change ran neither round-trip. Assisted-by: LLM Signed-off-by: Andrey Kolkov <[email protected]> (cherry picked from commit 58b4343) Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 564ca28 commit 4ac0cd5

3 files changed

Lines changed: 46 additions & 4 deletions

File tree

‎docs/agents/e2e-testing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Trap-based cleanup was the single biggest source of false failures in the BATS s
7070

7171
### 4. Do not mask cleanup or teardown failures
7272

73-
- Resources a test creates that Chainsaw cannot reclaim — because a controller, not the test, owns them (e.g. the Velero `Backup`, `BackupStorageLocation`, and credentials secret in `cozy-velero` in the disabled `hack/e2e-chainsaw/backup/` suite) — must be pruned explicitly. Do not `|| true` over a stuck delete and leave stale state for the next run.
73+
- Resources a test creates that Chainsaw cannot reclaim — because a controller, not the test, owns them (e.g. the Velero `Backup`/`Restore` in `cozy-velero`, which the `vminstance` backup round-trips prune by deleting the cozystack `Backup` and `RestoreJob` CRs, whose finalizers then clean up the Velero objects asynchronously — the `BackupJob` has no finalizer and its `Backup` carries no owner reference, so deleting a `BackupJob` reclaims nothing — plus an explicit delete of the throwaway `BackupStorageLocation`, sequenced after the Velero `Backup` is gone so it does not race the finalizer) — must be pruned explicitly. Do not `|| true` over a stuck delete and leave stale state for the next run.
7474
- For nested tenants, tear down **child → parent with a hard wait for deletion between** each. Deleting the parent while a child is still uninstalling wedges the parent's cleanup Job on the child namespace, and both stuck uninstalls occupy helm-controller workers past the end of the test — starving whichever suite runs next. The kubernetes suites encode this ordering in `hack/e2e-chainsaw/_lib/run-kubernetes.sh`.
7575
- A test that creates a tenant `Kubernetes` cluster waits for that release to become **Ready** before the assertions it ends on — not before every read of the cluster, which is a different rule and a false one. The chart's post-install hook (`packages/apps/kubernetes/templates/oidc-rbac-job.yaml`) polls for the Kamaji-issued admin kubeconfig in the modes that have one, and that Secret is what the test needs to reach the tenant API, so bringing up a worker or waiting on a child release happens while the parent is still installing. Reading mid-install is fine. What the rule is about is the COMPLETION path: a run that asserts, passes and ends while the parent is still installing turns a green result into a red one in teardown. A failure path that diagnoses and exits mid-install does the same to the install, but that run is already red, so the second red costs noise rather than a wrong verdict — and the script has a number of such exits between the CR and the wait, which the guard does not and should not treat as violations.
7676
- A Helm release cannot be uninstalled while its install action is running, and for this kind the action can outlive the fast assertions — by how much depends on the mode, per the bullet below: `release.cozystack.io/helm-install-disable-wait` on the `ApplicationDefinition` drops the wait on the main resources, but the chart's blocking post-install hook still holds the install. A test that asserts and ends therefore hands the remainder of the install to teardown and is reported red on a step that tests nothing.

‎hack/select-e2e.sh‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,10 @@ src_to_suites() {
251251
# not exist, is dropped by intersect_suites(), and a CDI change reaches
252252
# nothing runnable through it.
253253
vm-disk-application) echo vminstance ;;
254+
# The VMInstance and VMDisk backup round-trips are the only Tests that run a
255+
# Velero backup and restore; without this the source maps to a `velero`
256+
# suite that does not exist and a Velero change runs neither.
257+
velero) echo vminstance ;;
254258
# Both sources own system/ingress-nginx, and the gateway suite carries the
255259
# legacy Ingress admission regression coverage. Without these explicit
256260
# mappings their derived names point at suites that do not exist, so an
@@ -274,7 +278,7 @@ src_to_suites() {
274278
# harbor stores its registry in a BucketClaim the seaweedfs COSI driver
275279
# serves. etcd's round-trip is left out because CI gates it off. A test
276280
# derives this set from the suites.
277-
seaweedfs-application) echo "bucket clickhouse harbor kafka kafka-metadata mariadb mongodb postgres rabbitmq redis seaweedfs valkey" ;;
281+
seaweedfs-application) echo "bucket clickhouse harbor kafka kafka-metadata mariadb mongodb postgres rabbitmq redis seaweedfs valkey vminstance" ;;
278282
# Both suites create a KeycloakClient and wait for the operator's finalizer
279283
# to remove it. kubernetes-application has no dependsOn on keycloak-operator
280284
# (#4680), so the graph reaches only monitoring and would drop the tenant
@@ -462,8 +466,17 @@ while IFS= read -r file || [ -n "$file" ]; do
462466
# across suites: the valkey round-trip runs it with APP_KIND=Valkey, so an
463467
# edit there selects both.
464468
app=$(echo "$file" | sed -nE 's,^examples/backups/([^/]+)/.*,\1,p')
469+
# The vmi and vmdisk backup demos are both harnessed by the vminstance
470+
# suite, which owns VMDisk coverage too (see vm-disk-application in
471+
# src_to_suites), so their dirs need the same source-style translation.
472+
# The suite runs the vmi demo with SKIP_RESTORE_TO_COPY=1, so the to-copy
473+
# step is docs-only: selecting the suite would pass without executing it.
474+
case "$file" in
475+
examples/backups/vmi/07-restore-to-copy.sh) continue ;;
476+
esac
465477
case "$app" in
466478
redis) users="redis valkey" ;;
479+
vmi|vmdisk) users=vminstance ;;
467480
*) users=$app ;;
468481
esac
469482
for user in $users; do

‎hack/select-e2e_test.bats‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,35 @@ assert_full_suite() {
296296
rm -rf "$tmp"
297297
}
298298

299+
@test "a velero change selects the vminstance suite" {
300+
tmp=$(mktemp -d)
301+
cp -r packages/core/platform/sources "$tmp/sources"
302+
echo "packages/system/velero/values.yaml" > "$tmp/diff"
303+
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
304+
rm -rf "$tmp"
305+
case " $output " in *" vminstance "*) ;; *) echo "a velero change selected '$output'" >&2; exit 1 ;; esac
306+
}
307+
308+
@test "vmi/vmdisk backup harness edit selects the vminstance suite" {
309+
tmp=$(mktemp -d)
310+
cp -r packages/core/platform/sources "$tmp/sources"
311+
for f in examples/backups/vmi/run-all.sh examples/backups/vmdisk/run-all.sh; do
312+
echo "$f" > "$tmp/diff"
313+
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
314+
[ "$output" = "vminstance" ]
315+
done
316+
rm -rf "$tmp"
317+
}
318+
319+
@test "vmi restore-to-copy step, which the suite skips, selects nothing" {
320+
tmp=$(mktemp -d)
321+
cp -r packages/core/platform/sources "$tmp/sources"
322+
echo "examples/backups/vmi/07-restore-to-copy.sh" > "$tmp/diff"
323+
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources") || true
324+
rm -rf "$tmp"
325+
[ -z "$output" ]
326+
}
327+
299328
@test "backup example without a matching suite selects nothing" {
300329
tmp=$(mktemp -d)
301330
cp -r packages/core/platform/sources "$tmp/sources"
@@ -495,7 +524,7 @@ assert_full_suite() {
495524
# escalated to the full run.
496525
tmp=$(mktemp -d)
497526
cp -r packages/core/platform/sources "$tmp/sources"
498-
echo "hack/e2e-chainsaw/backup/chainsaw-test.yaml.disabled" > "$tmp/diff"
527+
echo "hack/e2e-chainsaw/kubernetes-proxmox/chainsaw-test.yaml.disabled" > "$tmp/diff"
499528
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
500529
[ -z "$output" ]
501530
rm -rf "$tmp"
@@ -507,7 +536,7 @@ assert_full_suite() {
507536
# that would silently unselect real work committed in the same change.
508537
tmp=$(mktemp -d)
509538
cp -r packages/core/platform/sources "$tmp/sources"
510-
printf '%s\n' hack/e2e-chainsaw/backup/chainsaw-test.yaml.disabled \
539+
printf '%s\n' hack/e2e-chainsaw/kubernetes-proxmox/chainsaw-test.yaml.disabled \
511540
packages/apps/postgres/values.yaml > "$tmp/diff"
512541
output=$(hack/select-e2e.sh "$tmp/diff" "$tmp/sources")
513542
[ "$output" = "postgres" ]

0 commit comments

Comments
 (0)