Skip to content

release-1.7: unit tests red since release prep rewrote two image refs the tests read #5178

Open

Description

Unit & controller tests fails on release-1.7 itself, so every backport PR to that branch shows red for reasons outside its diff. Two tests read image references that the release flow rewrites on the release branch, and neither expects the rewritten form.

Both failures reproduce on a clean checkout of release-1.7 at a32334a (chore(release): promote v1.7.0-rc.3 -> v1.7.0), and #5147 was green on the branch before the release commits. Example run on a backport: 41162009892.

1. hack/cni-plugins-staging-contract.bats, test 567 ("the kube-ovn release the CNI plugin pin was checked against is the one vendored"). The test compares its marker (kube-ovn v1.15.26) with the tag in packages/system/kubeovn/values.yaml. On main that tag is v1.15.26@sha256:…. On release-1.7 the release flow re-tagged the first-party kube-ovn image to v1.7.0@sha256:…, so the test reads the Cozystack release version as if it were the upstream kube-ovn version and fails:

the multus CNI plugin pin was checked against kube-ovn v1.15.26,
but packages/system/kubeovn/values.yaml now vendors v1.7.0.

2. packages/system/monitoring/tests/vtraces_collector_test.yaml ("pins the collector image through the cozy-lib.image helper"). The assertion expects opentelemetry-collector-contrib:.*@sha256:, while on release-1.7 the collector renders as ghcr.io/cozystack/cozystack/otelcol-tracing:v0.136.0-cozystack.0@sha256:….

Following the message of test 1 and moving its marker to v1.7.0 would be wrong: v1.7.0 is a release tag, not a kube-ovn version. I think both tests need to read something the release flow does not rewrite, or the release flow needs to update them in the same commit. That choice belongs with whoever owns release prep. A test that asserts behaviour instead of the image string would avoid the second class of failure altogether.

Activity

  1. dosubot commented on Nov 1, 2026

    @dosubot

    There are already two useful precedents in the tree:

    • For kube-ovn, packages/system/kubeovn/charts/kube-ovn/Chart.yaml has appVersion: "1.15.26". The package’s update target uses that field to stamp the Dockerfile’s upstream VERSION and TAG, and hack/check-kubeovn-image-copies.bats already checks its upstream fixture against it. Comparing the CNI marker with v + that unquoted appVersion would follow the existing upstream-version contract rather than the release image tag. [1] [2] [3]
    • MetalLB’s Helm tests explicitly account for release-prep rewriting image refs: they assert the repository and a full SHA-256 digest while leaving the tag free. That is an existing pattern for a separate collector pinning assertion; the current collector regex alone does not demonstrate helper behavior. [4] [5]

    To reply, just mention @dosu.


    Share context across your team and agents. Try Dosu.

    Leave Feedback Learn about cozystack with Dosu Add Dosu to your team

  2. themoriarti commented on Nov 1, 2026

    @themoriarti
    Collaborator

    Fix is in #5179 (opened against release-1.7 a few minutes after this issue, so it did not link itself):

    • cni-plugins-staging-contract.bats reads the vendored kube-ovn release from appVersion in packages/system/kubeovn/charts/kube-ovn/Chart.yaml, the field packages/system/kubeovn/Makefile builds the image from. The release flow leaves it alone, so the marker stays a kube-ovn version.
    • vtraces_collector_test.yaml now asserts behaviour, as suggested here: it sets _cluster.images-registry and checks that the registry lands in front of the rendered image and that the image stays digest-pinned. The image name is not in the test any more. Dropping the cozy-lib.image call from the template fails the case; the old regex did not notice that.

    Measured on a clean release-1.7 (a32334a): bats 9/10, helm unittest FAIL; with #5179: 10/10 and 28/28. I also grepped every image/tag literal the Prepare release v1.7.0-rc.3 commit rewrote (41 of them) across *_test.yaml, *.bats and helm snapshots: the only other hit is a fixture inside hack/promote-rewrite-tags_test.bats, which builds its own input. So these two were the whole class on this branch.

    Why it stayed hidden: the promote PR (#5161) carries the release label, and pull-requests.yaml skips the Unit & controller tests job on that label. The stamped tree therefore never met the unit suite until the first backport PR. The e2e skip on that label is justified (the rc was already verified); the unit suite has no cluster and takes ~16 min. Dropping release from that job's if: would have shown this on #5161 before merge. That is the release-prep owners' call, so I am only proposing it here; happy to open the PR for it if wanted. The same two-line test fix applies to main and I will open that separately, otherwise release-1.8 repeats this.

  3. added 2 commits that reference this issue on Nov 9, 2026
    ec30722
    da7f2c2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

triage/needs-triageIndicates an issue needs triage by a maintainer

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions