Skip to content

Commit 6748014

Browse files
authored
fix(ci): overlay the right line's refs for a PR on a feature branch (#4460)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does A PR stacked on a feature branch now overlays the image refs of the line it grew from, so its E2E no longer falls back to the refs committed in the tree. The Finalize job pulls `cozystack-packages:<base branch>` and overlays its image refs onto the packages the PR did not rebuild. For a stacked PR the base is something like `fix/foo`. The slash is not valid in an OCI tag, and no workflow publishes an artifact for that branch anyway. So the pull failed, the step only printed a warning, and every unbuilt package ran its committed release ref. That is how stacked PRs got the Grafana plugin 404 described in #4391. Now `main` and `release-X.Y` bases read their own artifact. Any other base reads the artifact of whichever of main and the release lines it has the fewest commits on top of, and a notice says which one. A stack on a release line keeps that line's images, for example #3895, which sits on `release-1.5`. On a tie main wins, so a stack on a line cut from main reads main's artifact until the stack contains a commit of the line's own. Always taking main there would put main's images under the line's charts, and that is the install failure from #3437. The new bats case builds a small git fixture and checks the tag pulled for six bases, one of them a deleted branch. The other half of #4391, the stale `grafana-dashboards.tag`, is a release-managed digest pin and is not touched here. It gets re-stamped at release prep. Part of #4391 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(ci): E2E for a PR stacked on a feature branch overlays the image refs of the line the stack grew from, instead of falling back to the committed release refs. ```
2 parents b981ea7 + 9c5c5f9 commit 6748014

2 files changed

Lines changed: 76 additions & 8 deletions

File tree

‎.github/workflows/pull-requests.yaml‎

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -639,24 +639,46 @@ jobs:
639639
# cozystack-packages:<line> for every maintained release-<major>.<minor>
640640
# branch, so each base branch has its own artifact to read.
641641
#
642+
# A base that is neither main nor a line (a stacked PR's feature branch) has
643+
# no artifact, and a `/` in it is not even a valid OCI tag. It reads the
644+
# artifact of whichever of main and the lines it has the fewest commits on
645+
# top of. A tie goes to main, so a stack on a line keeps that line's
646+
# generation only once the line has a commit of its own: a line fresh off
647+
# main is topologically the same as main at that commit.
648+
#
642649
# A missing artifact stays a no-op (packages keep their committed refs), which
643650
# also covers a line whose first build-release run has not landed yet.
644651
- name: Pull base-branch packages tree
645652
run: |
653+
ARTIFACT_REF="$BASE_REF"
654+
if [ "$BASE_REF" != "main" ] \
655+
&& ! printf '%s' "$BASE_REF" | grep -qE '^release-[0-9]+\.[0-9]+$'; then
656+
ARTIFACT_REF=main
657+
# A re-run can outlive its base: the payload keeps a branch deleted since.
658+
if git rev-parse --verify --quiet "origin/${BASE_REF}" >/dev/null; then
659+
best="$(git rev-list --count "origin/main..origin/${BASE_REF}")"
660+
for line in $(git for-each-ref --format='%(refname:lstrip=3)' 'refs/remotes/origin/release-*' \
661+
| grep -E '^release-[0-9]+\.[0-9]+$'); do
662+
n="$(git rev-list --count "origin/${line}..origin/${BASE_REF}")"
663+
if [ "$n" -lt "$best" ]; then best="$n"; ARTIFACT_REF="$line"; fi
664+
done
665+
fi
666+
echo "::notice::Base ${BASE_REF} publishes no packages artifact; overlaying cozystack-packages:${ARTIFACT_REF}."
667+
fi
646668
rm -rf _out/mainpkgs
647669
mkdir -p _out/mainpkgs
648-
if ! flux pull artifact "oci://${REGISTRY}/cozystack-packages:${BASE_REF}" \
670+
if ! flux pull artifact "oci://${REGISTRY}/cozystack-packages:${ARTIFACT_REF}" \
649671
--output _out/mainpkgs; then
650672
rm -rf _out/mainpkgs
651673
# Degrading to committed refs is safe, but SAY so: on a release line a
652674
# missing artifact means build-release.yaml has not published this line
653675
# yet (first build pending, or its branch filter does not match this
654676
# branch), and the PR is then testing the line's last release instead of
655677
# its tip. Silent would look identical to a working overlay.
656-
if [ "${BASE_REF}" = "main" ]; then
657-
echo "cozystack-packages:${BASE_REF} pull failed — keeping committed refs"
678+
if [ "${ARTIFACT_REF}" = "main" ]; then
679+
echo "cozystack-packages:${ARTIFACT_REF} pull failed — keeping committed refs"
658680
else
659-
echo "::warning title=No packages artifact for ${BASE_REF}::Keeping committed refs, so unbuilt packages use this line's last release rather than its tip. Check that build-release.yaml has run for ${BASE_REF}."
681+
echo "::warning title=No packages artifact for ${ARTIFACT_REF}::Keeping committed refs, so unbuilt packages use this line's last release rather than its tip. Check that build-release.yaml has run for ${ARTIFACT_REF}."
660682
fi
661683
fi
662684
env:

‎hack/overlay-main-images_test.bats‎

Lines changed: 50 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -204,10 +204,10 @@
204204
inside')"
205205
[ -n "$block" ] || { echo "the base-branch packages pull step is missing from $wf" >&2; exit 1; }
206206

207-
# The artifact tag must come from the base branch, passed via env.
208-
printf '%s\n' "$block" | grep -qF 'cozystack-packages:${BASE_REF}' || {
209-
echo "the packages artifact tag is not the PR's base branch. Reading a fixed" >&2
210-
echo "tag (e.g. :main) hands a release-line PR another generation's images." >&2
207+
# The artifact tag must be derived from the base branch, passed via env.
208+
printf '%s\n' "$block" | grep -qF 'cozystack-packages:${ARTIFACT_REF}' || {
209+
echo "the packages artifact tag is not derived from the PR's base branch. Reading" >&2
210+
echo "a fixed tag (e.g. :main) hands a release-line PR another generation's images." >&2
211211
exit 1; }
212212
printf '%s\n' "$block" | grep -qF 'BASE_REF: ${{ github.base_ref }}' || {
213213
echo "BASE_REF is not wired to github.base_ref in the pull step's env." >&2; exit 1; }
@@ -223,6 +223,52 @@
223223
return 0
224224
}
225225

226+
# A stacked PR's base is a feature branch: `fix/x` is not a valid OCI tag, and
227+
# no artifact is published for it anyway. Such a PR must read the artifact of
228+
# the line its stack grew from, not fall through to the committed release refs
229+
# (#4391), and not main's when the stack sits on a release line (#3437).
230+
@test "the pull step maps the base branch to a published artifact tag" {
231+
root=$(pwd)
232+
w="${BATS_TEST_TMPDIR:-$(mktemp -d)}"
233+
git init --quiet --initial-branch=main "$w/repo"
234+
cd "$w/repo"
235+
git config user.email [email protected]
236+
git config user.name ci
237+
git config commit.gpgsign false
238+
c() { git commit --quiet --allow-empty --message "$1"; }
239+
c m1
240+
git checkout --quiet -b release-1.5; c q1
241+
git checkout --quiet main; git checkout --quiet -b release-1.6; c r1
242+
git checkout --quiet -b stack/x; c s1
243+
git checkout --quiet release-1.6; git checkout --quiet -b release-1.6.1; c p1
244+
git checkout --quiet main; c m2; c m3
245+
# release-1.7 and fix/some-branch both grow from m3, so they tie: main must win.
246+
git checkout --quiet -b release-1.7; c x1
247+
git checkout --quiet main; git checkout --quiet -b fix/some-branch; c f1
248+
for b in main release-1.5 release-1.6 release-1.7 stack/x release-1.6.1 fix/some-branch; do
249+
git update-ref "refs/remotes/origin/$b" "$b"
250+
done
251+
252+
script="$(awk '
253+
$0 == " - name: Pull base-branch packages tree" { step = 1; next }
254+
step && /^ - name: / { exit }
255+
step && $0 == " run: |" { body = 1; next }
256+
body && /^ [^ ]/ { exit }
257+
body { sub(/^ /, ""); print }' "$root/.github/workflows/pull-requests.yaml")"
258+
[ -n "$script" ] || { echo "cannot extract the pull step's script" >&2; exit 1; }
259+
260+
mkdir -p "$w/bin"
261+
printf '#!/bin/sh\nprintf "%%s\\n" "$3" > "%s/pulled"\nexit 1\n' "$w" > "$w/bin/flux"
262+
chmod +x "$w/bin/flux"
263+
for pair in main=main release-1.6=release-1.6 fix/some-branch=main \
264+
stack/x=release-1.6 release-1.6.1=release-1.6 gone/branch=main; do
265+
rm -f "$w/pulled"
266+
BASE_REF="${pair%%=*}" REGISTRY=r PATH="$w/bin:$PATH" bash -eo pipefail -c "$script" >/dev/null
267+
[ "$(cat "$w/pulled")" = "oci://r/cozystack-packages:${pair#*=}" ] || {
268+
echo "base ${pair%%=*}: pulled '$(cat "$w/pulled")', want tag ${pair#*=}" >&2; exit 1; }
269+
done
270+
}
271+
226272
@test "every maintained release line publishes its own packages artifact" {
227273
root=$(pwd)
228274
wf="$root/.github/workflows/build-release.yaml"

0 commit comments

Comments
 (0)