Skip to content

[Backport release-1.6] ci(e2e): run nightly and release-tag e2e on the container lane - #4913

Merged
myasnikovdaniil merged 2 commits into
release-1.6from
backport-4828-to-release-1.6
Oct 14, 2026
Merged

myasnikovdaniil merged 2 commits into
release-1.6from
backport-4828-to-release-1.6

Conversation

@myasnikovdaniil

Copy link
Copy Markdown
Contributor

What this PR does

Backport of #4828 to release-1.6.

PR lane on this line runs on Talos containers since the backport of #4020, but e2e-tag.yaml (the rc gate that tags.yaml calls for every rc from release-1.6) and nightly.yaml still boot QEMU guests. Every rc tag run here since v1.6.2-rc.1 is red. v1.6.4-rc.1 failed with kubernetes-latest and kubernetes-previous spending about 32 minutes each at node-join deadline (https://github.com/cozystack/cozystack/actions/runs/36124680874), while the same 41 suites passed on container lane in the v1.6.4 promotion PR, in a 93 minute job (https://github.com/cozystack/cozystack/actions/runs/36383277279).

e2e plumbing on release-1.6 diverged from main after #4020, so the cherry-pick of the first commit has conflicts resolved against this branch, and there is one prerequisite commit:

  • e2e-tag.yaml and nightly.yaml get the same host module step, prepare-env-container, endpoint and storage class env and two-part teardown as pull-requests.yaml here. e2e-tag.yaml still requires the release to carry its nocloud disk but doesn't download it anymore, and a dispatch against a tag cut before container lane (v1.6.3 and older) fails early and says to dispatch with --ref <tag>.
  • nightly moves to 32-vCPU runner, because hack/e2e-container-up.sh refuses a 24-vCPU host (main did the same earlier in db2ddfa).

Tests: platform-packages_test.bats checks container lane env on all three lanes, promote-gate-contract.bats covers the new guard (on main it lives in collect-images_test.bats, there is no such file here).

Differences from main: every lane keeps timeout-minutes: 180, the follow-up comment fix about 215-minute budget is not included (this branch has no such comment and no such value), fork lane wording is dropped because there is no fork split here, docs edits cover only paragraphs that exist on this branch.

Coverage lost on this line: DRBD and the replicated StorageClass, Cozystack Talos node image with its extensions, CDI stock 600M worker memory limit. Docs and comments that pointed at the rc run as that coverage now say so. nightly.yaml runs only from main, so the change there is for parity.

tags.yaml takes e2e-tag.yaml from the commit of the rc tag, so this has to merge before v1.6.5-rc.1 is cut. If rc is cut first, dispatching e2e-tag.yaml from release-1.6 with that tag gives the same evidence.

make bats-unit-tests passes locally (487 ok, base has 486), but the workflows themselves were not run yet. This stays a draft until e2e-tag.yaml dispatched from this branch against v1.6.4-rc.1 is green, I will put the run link here.

Downstream repositories

Release note

ci(e2e): on release-1.6, the release-tag e2e gate and nightly run on Talos containers instead of QEMU guests. DRBD, the replicated StorageClass and the Cozystack Talos node image are no longer covered by any e2e lane on this line.

myasnikovdaniil and others added 2 commits October 13, 2026 20:01
The nightly e2e job runs on a 24-vCPU runner while its three
management nodes take 8 vCPU each. On QEMU that is a 1:1 commit with
nothing left for the host. On the container lane the job would not
start at all: hack/e2e-container-up.sh refuses a host that does not
have strictly more CPUs than the nodes take, and nightly checks out
main's tree, which carries the same check.

The next commit moves nightly onto the container lane, so move the job
first to the 32-vCPU class the pull request and tag lanes on this
branch already use. main made the same move in db2ddfa, together
with a QEMU-specific headroom test that is not carried here.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <[email protected]>
Nightly and the rc gate still booted the management nodes as QEMU
guests, the substrate the merge-gating lanes left behind. On it the
tenant Kubernetes suites have missed their node-join deadline on most
nights since 2026-10-02, each failure spends the whole deadline, and
the job reaches its timeout before the alphabetically later suites
run. Over the same days those suites passed on every PR run that
selected them, where the nodes are Talos containers and a tenant
worker sits one virtualization level shallower.

Both lanes now use the PR lanes' container substrate. Nightly's e2e no
longer waits for the nocloud disk, which it does not boot; the disk is
still built and published. A manual e2e-tag dispatch against a tag
that predates the container lane fails early and names the ref to
dispatch from instead.

DRBD, the replicated StorageClass and the Cozystack Talos node image
lose their last e2e coverage with this. The docs and comments that
pointed at nightly and the rc gate as that coverage now say so.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>
(cherry picked from commit da28ea7)
Signed-off-by: Myasnikov Daniil <[email protected]>
@github-actions github-actions Bot added area/release Issues or PRs related to release tooling (changelog, backport, release pipeline) area/testing Issues or PRs related to testing (e2e, bats, unit tests) labels Oct 14, 2026
@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: cozystack/cozystack/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 68ab5853-91b3-4f01-8efe-c9cb7d7cde79

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the size/L This PR changes 100-499 lines, ignoring generated files label Oct 14, 2026
@myasnikovdaniil
myasnikovdaniil marked this pull request as ready for review October 14, 2026 08:19
@myasnikovdaniil
myasnikovdaniil merged commit 68d5e5c into release-1.6 Oct 14, 2026
44 checks passed
@myasnikovdaniil
myasnikovdaniil deleted the backport-4828-to-release-1.6 branch October 14, 2026 09:13
@myasnikovdaniil

Copy link
Copy Markdown
Contributor Author

backport-audit: complete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/release Issues or PRs related to release tooling (changelog, backport, release pipeline) area/testing Issues or PRs related to testing (e2e, bats, unit tests) size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants