Skip to content

Commit 1f6e59d

Browse files
test(e2e): add chainsaw-native release upgrade testing lane
Reimplements the intent of #2401 (previous-minor-stable -> current upgrade test) over the post-Chainsaw-migration codebase. The old #2401 was a single 895-line BATS file, now obsolete; this builds a two-phase Chainsaw lane bracketing an external helm upgrade, reusing existing files throughout. Flow (make upgrade-cozystack, in the e2e sandbox): install-previous (bats, prev OCI chart) -> seed (chainsaw, skipDelete) -> helm upgrade to in-tree chart (bats) -> verify (chainsaw, assert-only). DRY over existing files: - 4 composite actions (.github/actions/e2e-{download-assets,prepare,collect, teardown}) shared by both the e2e and new upgrade-e2e jobs; the e2e job is refactored onto them (behavior-preserving). - hack/e2e-wait-hr-ready.sh: the all-HR-Ready gate extracted from e2e-install-cozystack.bats so install and upgrade share one legible gate. - hack/upgrade-prev-version.sh (+ unit test) resolves the baseline version. - seed suites apply the existing app manifests by relative path (no new CRs). Trigger: an upgrade-e2e job in pull-requests.yaml, parallel to e2e, gated on the release label or a maintainer upgrade-e2e label. Advisory / non-blocking. Coverage: postgres + mariadb canary rows, redis/VM volume survival, migration stamp advance, all-HRs-Ready + no-new-unbound-PV. The tenant Kubernetes survival suites (#2931 path) are env-gated OFF (UPGRADE_E2E_TENANT_K8S) as the flakiest surface, pending dev-stand validation. Co-Authored-By: Claude Opus 4.8 <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 4689b2b commit 1f6e59d

27 files changed

Lines changed: 1672 additions & 60 deletions
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: "E2E sandbox collect"
2+
description: >-
3+
Collect the cozyreport bundle (operator/Flux/LINSTOR/Talos logs + per-failed-
4+
test crust-gather snapshots) and the image list from the sandbox and upload
5+
them as artifacts. Every step is `if: always()` so diagnostics are captured
6+
even after a failed run. Shared by the e2e and upgrade-e2e jobs; runs BEFORE
7+
teardown so the sandbox is still alive. The chainsaw JUnit report is uploaded
8+
by each job separately (its path differs per lane).
9+
10+
inputs:
11+
sandbox-name:
12+
description: "Sandbox container name (from the e2e-prepare action output)."
13+
required: true
14+
artifact-prefix:
15+
description: >-
16+
Prefix for uploaded artifact names so parallel jobs don't collide
17+
(e.g. "" for e2e, "upgrade-" for upgrade-e2e).
18+
required: false
19+
default: ""
20+
21+
runs:
22+
using: composite
23+
steps:
24+
- name: Collect report
25+
if: always()
26+
shell: bash
27+
run: |
28+
cd "/tmp/${{ inputs.sandbox-name }}"
29+
make -C packages/core/testing SANDBOX_NAME=${{ inputs.sandbox-name }} collect-report || true
30+
31+
- name: Upload cozyreport.tgz
32+
if: always()
33+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
34+
with:
35+
name: ${{ inputs.artifact-prefix }}cozyreport
36+
path: /tmp/${{ inputs.sandbox-name }}/_out/cozyreport.tgz
37+
38+
- name: Collect images list
39+
if: always()
40+
shell: bash
41+
run: |
42+
cd "/tmp/${{ inputs.sandbox-name }}"
43+
make -C packages/core/testing SANDBOX_NAME=${{ inputs.sandbox-name }} collect-images || true
44+
45+
- name: Upload image list
46+
if: always()
47+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
48+
with:
49+
name: ${{ inputs.artifact-prefix }}image-list
50+
path: /tmp/${{ inputs.sandbox-name }}/_out/images.txt
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
name: "E2E asset download"
2+
description: >-
3+
Fetch the Talos disk (+ pr.patch) this e2e run needs. Regular PR: pull the
4+
talos-image and pr-patch artifacts built earlier in the run and apply the
5+
patch. Release PR: download the nocloud disk from the draft release. Shared by
6+
the e2e and upgrade-e2e jobs so the asset paths live in one place.
7+
8+
inputs:
9+
app-token:
10+
description: "GitHub App token, used only on the release-PR path to fetch the draft-release asset."
11+
required: false
12+
default: ""
13+
disk-id:
14+
description: "Release asset id of nocloud-amd64.raw.xz (release PR path; from the resolve_assets job)."
15+
required: false
16+
default: ""
17+
18+
runs:
19+
using: composite
20+
steps:
21+
# ▸ Regular PR path — artifacts from build-talos + finalize in this run.
22+
- name: Download Talos image (regular PR)
23+
if: "!contains(github.event.pull_request.labels.*.name, 'release')"
24+
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
25+
with:
26+
name: talos-image
27+
path: _out/assets
28+
29+
- name: Download PR patch (regular PR)
30+
if: "!contains(github.event.pull_request.labels.*.name, 'release')"
31+
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
32+
with:
33+
name: pr-patch
34+
path: _out/assets
35+
36+
- name: Apply patch (regular PR)
37+
if: "!contains(github.event.pull_request.labels.*.name, 'release')"
38+
shell: bash
39+
run: git apply _out/assets/pr.patch
40+
41+
# ▸ Release PR path — the draft-release disk for the promoted version.
42+
- name: Download assets from draft release (release PR)
43+
if: contains(github.event.pull_request.labels.*.name, 'release')
44+
shell: bash
45+
env:
46+
APP_TOKEN: ${{ inputs.app-token }}
47+
DISK_ID: ${{ inputs.disk-id }}
48+
run: |
49+
mkdir -p _out/assets
50+
curl -sSL -H "Authorization: token ${APP_TOKEN}" -H "Accept: application/octet-stream" \
51+
-o _out/assets/nocloud-amd64.raw.xz \
52+
"https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/assets/${DISK_ID}"
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: "E2E sandbox prepare"
2+
description: >-
3+
Compute the sandbox id, copy the workspace into /tmp, and provision the Talos
4+
e2e sandbox (make prepare-env with the sanctioned 3x infra retry — the one
5+
retry justified by docs/agents/e2e-testing.md #1). Shared by the e2e and
6+
upgrade-e2e jobs so the sandbox bring-up lives in exactly one place.
7+
8+
inputs:
9+
sandbox-suffix:
10+
description: >-
11+
Appended to the sandbox id so jobs that run in parallel on the same
12+
workflow/ref (e2e vs upgrade-e2e) get distinct sandbox names. Empty
13+
reproduces the original single-job id byte-for-byte.
14+
required: false
15+
default: ""
16+
17+
outputs:
18+
sandbox-name:
19+
description: "The computed sandbox container name (also exported as SANDBOX_NAME)."
20+
value: ${{ steps.set.outputs.sandbox-name }}
21+
22+
runs:
23+
using: composite
24+
steps:
25+
- name: Set sandbox ID
26+
id: set
27+
shell: bash
28+
run: |
29+
key="${GITHUB_REPOSITORY}:${GITHUB_WORKFLOW}:${GITHUB_REF}"
30+
# Only extend the hash key when a suffix is given, so the default
31+
# (e2e) sandbox name is identical to the pre-composite behaviour.
32+
[ -n "${{ inputs.sandbox-suffix }}" ] && key="${key}:${{ inputs.sandbox-suffix }}"
33+
name="cozy-e2e-sandbox-$(echo "$key" | sha256sum | cut -c1-10)"
34+
echo "SANDBOX_NAME=$name" >> "$GITHUB_ENV"
35+
echo "sandbox-name=$name" >> "$GITHUB_OUTPUT"
36+
37+
- name: Prepare workspace
38+
shell: bash
39+
run: |
40+
rm -rf "/tmp/$SANDBOX_NAME"
41+
cp -r "${{ github.workspace }}" "/tmp/$SANDBOX_NAME"
42+
43+
- name: Prepare environment
44+
shell: bash
45+
run: |
46+
cd "/tmp/$SANDBOX_NAME"
47+
attempt=0
48+
until make SANDBOX_NAME=$SANDBOX_NAME prepare-env; do
49+
attempt=$((attempt + 1))
50+
if [ $attempt -ge 3 ]; then
51+
echo "Attempt $attempt failed, exiting..."
52+
exit 1
53+
fi
54+
echo "Attempt $attempt failed, retrying..."
55+
done
56+
echo "Prepare environment completed after $((attempt + 1)) attempts"
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: "E2E sandbox teardown"
2+
description: >-
3+
Tear down the sandbox container and remove its workspace copy. Both steps are
4+
`if: always()`. Kept separate from e2e-collect so a job can slot other
5+
always-run steps (e.g. the SSH breakpoint) BETWEEN collecting diagnostics and
6+
destroying the sandbox. Shared by the e2e and upgrade-e2e jobs.
7+
8+
inputs:
9+
sandbox-name:
10+
description: "Sandbox container name (from the e2e-prepare action output)."
11+
required: true
12+
13+
runs:
14+
using: composite
15+
steps:
16+
- name: Tear down sandbox
17+
if: always()
18+
shell: bash
19+
run: make -C packages/core/testing SANDBOX_NAME=${{ inputs.sandbox-name }} delete || true
20+
21+
- name: Remove workspace
22+
if: always()
23+
shell: bash
24+
run: rm -rf "/tmp/${{ inputs.sandbox-name }}"

‎.github/labels.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -302,6 +302,10 @@
302302
color: '704479'
303303
description: Debugging in progress
304304

305+
- name: upgrade-e2e
306+
color: '1d76db'
307+
description: Run the release upgrade E2E test (previous minor stable -> this build) on this PR
308+
305309
- name: sponsored
306310
color: '00ff00'
307311
description: Sponsored work

0 commit comments

Comments
 (0)