Skip to content

Commit 3333b32

Browse files
committed
fix(ci): preserve a PR's own edits to non-rebuilt package image refs
The current-main overlay skipped only rebuilt units (plus talos/installer), so for a package the PR edited but did not rebuild — an upstream image bump in a non-build-unit package such as keycloak or cert-manager — the committed ref differed from the main artifact on image-reference lines only, the drift check passed, and the overlay reverted the PR's edit back to current-main. e2e and the installer artifact then ran against the stale image. Pass the PR's edited package dirs (derived from the plan job's git diff) into the overlay skip set so a PR's own image-ref edits win over the artifact. Add tests pinning the preserve behaviour and covering the --image= arg form and the file-absent-in-PR-tree branch. Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 52f7d05 commit 3333b32

3 files changed

Lines changed: 83 additions & 9 deletions

File tree

‎.github/workflows/pull-requests.yaml‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ jobs:
3131
code: ${{ steps.p.outputs.code }}
3232
matrix: ${{ steps.p.outputs.matrix }}
3333
any: ${{ steps.p.outputs.any }}
34+
touched: ${{ steps.p.outputs.touched }}
3435
steps:
3536
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
3637
with:
@@ -50,6 +51,14 @@ jobs:
5051
matrix="$(hack/build-matrix.sh /tmp/changed.txt)"
5152
echo "Build matrix: $matrix"
5253
echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
54+
# Package dirs the PR edited (packages/<area>/<name>). Their committed
55+
# refs are the PR's intent and must survive the current-main overlay —
56+
# otherwise an upstream image bump in a non-build-unit package would be
57+
# silently reverted in e2e/the installer artifact (see
58+
# hack/overlay-main-images.sh). Whitespace-separated, no trailing slash.
59+
touched="$(sed -nE 's#^(packages/[^/]+/[^/]+)/.*#\1#p' /tmp/changed.txt | sort -u | tr '\n' ' ')"
60+
echo "Touched packages: $touched"
61+
echo "touched=$touched" >> "$GITHUB_OUTPUT"
5362
if [ "$matrix" = "[]" ]; then
5463
echo "any=false" >> "$GITHUB_OUTPUT"
5564
else
@@ -361,14 +370,20 @@ jobs:
361370
# behaviour), never block the PR. overlay-main-images.sh already handles
362371
# the expected paths (missing artifact, drift, absent unit) internally.
363372
run: |
364-
hack/overlay-main-images.sh _out/mainpkgs "$BUILD_MATRIX" \
373+
hack/overlay-main-images.sh _out/mainpkgs "$BUILD_MATRIX" "$PR_TOUCHED" \
365374
|| echo "::warning::overlay-main-images failed; unbuilt packages keep committed refs"
366375
env:
367376
# The set of packages THIS PR rebuilt; their pr-<N>-<sha> refs are
368377
# authoritative, so overlay-main-images.sh skips them. Computed by the
369378
# plan job from the Makefile (a JSON array of package dirs), not from
370379
# untrusted PR input — passed via env per workflow-injection hygiene.
371380
BUILD_MATRIX: ${{ needs.plan.outputs.matrix }}
381+
# Packages THIS PR edited (changed-file dirs from the plan job's git
382+
# diff). Their committed refs are the PR's intent — an upstream image
383+
# bump in a non-build-unit package would otherwise be silently reverted
384+
# by the overlay — so overlay-main-images.sh preserves them. Also
385+
# env-passed, derived from the diff, not from untrusted PR input.
386+
PR_TOUCHED: ${{ needs.plan.outputs.touched }}
372387
# buildkitd-config routes docker.io base-image pulls through mirror.gcr.io
373388
# (hack/buildkitd.toml) so the installer build dodges Docker Hub's
374389
# anonymous-pull rate limit; see the build job for the full rationale.

‎hack/overlay-main-images.sh‎

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@
2121
# Skipped:
2222
# - units the PR itself rebuilt (BUILT_JSON, the plan job's matrix): their
2323
# pr-<N>-<sha> refs, already applied in finalize, are authoritative.
24+
# - packages the PR EDITED (TOUCHED, the plan job's changed-package dirs): the
25+
# PR's committed refs are its intent (e.g. an upstream image bump in a
26+
# non-build-unit package), so they must win over the artifact — never overlay.
2427
# - packages/core/talos and packages/core/installer: rebuilt unconditionally
2528
# by their dedicated jobs (build-talos / the finalize installer build),
2629
# which own those files — never overlay them.
@@ -39,25 +42,38 @@
3942
# out the pull_request merge commit (current main + PR), so for an unbuilt file
4043
# the config already matches current main and only ref lines differ.
4144
#
42-
# Usage: hack/overlay-main-images.sh <mainpkgs-dir> <built-matrix-json>
43-
# <mainpkgs-dir> = extracted cozystack-packages:main tree; its root holds
44-
# apps/ system/ core/ extra/ ... (the CONTENTS of
45-
# packages/).
45+
# Usage: hack/overlay-main-images.sh <mainpkgs-dir> <built-matrix-json> [touched-pkg-dirs]
46+
# <mainpkgs-dir> = extracted cozystack-packages:main tree; its root holds
47+
# apps/ system/ core/ extra/ ... (the CONTENTS of
48+
# packages/).
49+
# <built-matrix-json> = JSON array of package dirs the PR rebuilt (skipped).
50+
# [touched-pkg-dirs] = whitespace-separated package dirs the PR edited
51+
# (no trailing slash); their committed refs are kept.
4652
# (run from the repo root)
4753
set -eu
4854

49-
MAINPKGS="${1:?usage: overlay-main-images.sh <mainpkgs-dir> <built-matrix-json>}"
55+
MAINPKGS="${1:?usage: overlay-main-images.sh <mainpkgs-dir> <built-matrix-json> [touched-pkg-dirs]}"
5056
MAINPKGS="${MAINPKGS%/}"
5157
BUILT_JSON="${2:-[]}"
58+
# Package dirs the PR itself EDITED (whitespace-separated, no trailing slash).
59+
# Their committed refs ARE the PR's intent — e.g. an upstream image bump in a
60+
# package that is NOT a build unit (keycloak, cert-manager, ingress-nginx, …) —
61+
# so they must win over the artifact. Without this, such an edit differs from the
62+
# artifact only on image-reference lines, the drift check passes, and the overlay
63+
# would silently revert the PR's bump (e2e then tests the stale image). The plan
64+
# job derives this from `git diff --name-only` and passes it via env, not from
65+
# untrusted PR input.
66+
TOUCHED="${3:-}"
5267

5368
if [ ! -d "$MAINPKGS" ]; then
5469
echo "No current-main packages tree at $MAINPKGS — unbuilt packages keep committed refs"
5570
exit 0
5671
fi
5772

5873
# Dirs whose files must never be overlaid: the two units owned by dedicated
59-
# jobs, plus every unit the PR rebuilt. Space-wrapped for whole-token matching.
60-
skip=" packages/core/talos packages/core/installer $(echo "$BUILT_JSON" | tr -d '[]"' | tr ',' ' ') "
74+
# jobs, every unit the PR rebuilt, and every package the PR edited (its committed
75+
# refs are the PR's intent). Space-wrapped for whole-token matching.
76+
skip=" packages/core/talos packages/core/installer $(echo "$BUILT_JSON" | tr -d '[]"' | tr ',' ' ') $TOUCHED "
6177

6278
# A changed line is image-reference-bearing if it carries a full ref (@sha256:),
6379
# is a split ref key (image/repository/registry/tag/digest), or a `--…-image=` arg.
@@ -103,4 +119,4 @@ for new in $(find "$MAINPKGS" -type d -name charts -prune -o \
103119
fi
104120
done
105121

106-
echo "Overlay current-main images: overlaid=$overlaid same=$same skipped(rebuilt/owned)=$skipped drift=$drift failed=$failed"
122+
echo "Overlay current-main images: overlaid=$overlaid same=$same skipped(rebuilt/owned/edited)=$skipped drift=$drift failed=$failed"

‎hack/overlay-main-images_test.bats‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,3 +104,46 @@
104104
"$root/hack/overlay-main-images.sh" does-not-exist '[]'
105105
grep -q 'foo:v1.5.0@sha256:aaaa' packages/apps/foo/values.yaml
106106
}
107+
108+
@test "preserves a PR-edited non-rebuilt package ref (passed via the touched arg)" {
109+
root=$(pwd)
110+
w=$(mktemp -d); trap 'rm -rf "$w"' EXIT
111+
mkdir -p "$w/packages/system/keycloak" "$w/main/system/keycloak"
112+
# The PR bumps an upstream image in keycloak. keycloak is NOT a build unit, so
113+
# it is absent from BUILD_MATRIX; only its image line differs from the artifact.
114+
# Without the touched arg the overlay would silently revert the PR's bump.
115+
echo 'image: quay.io/keycloak/keycloak:26.7.0' > "$w/packages/system/keycloak/values.yaml"
116+
echo 'image: quay.io/keycloak/keycloak:26.6.3' > "$w/main/system/keycloak/values.yaml"
117+
cd "$w"
118+
out=$("$root/hack/overlay-main-images.sh" main '[]' 'packages/system/keycloak')
119+
# The PR's edit wins; the overlay must not touch it.
120+
grep -q 'keycloak:26.7.0' packages/system/keycloak/values.yaml
121+
echo "$out" | grep -q 'overlaid=0'
122+
}
123+
124+
@test "recognizes a --…-image= arg line (no @sha256, no image key) as an image ref" {
125+
root=$(pwd)
126+
w=$(mktemp -d); trap 'rm -rf "$w"' EXIT
127+
mkdir -p "$w/packages/system/argimg" "$w/main/system/argimg"
128+
# The only differing line is a `--…-image=` arg with neither @sha256 nor an
129+
# image/tag/repository key — it must still count as image-reference-bearing.
130+
printf 'args:\n - --provider-image=ghcr.io/cozystack/cozystack/x:v1.5.0\n' > "$w/packages/system/argimg/values.yaml"
131+
printf 'args:\n - --provider-image=iad.ocir.io/x/cozystack/x:main\n' > "$w/main/system/argimg/values.yaml"
132+
cd "$w"
133+
"$root/hack/overlay-main-images.sh" main '[]'
134+
grep -q 'x:main' packages/system/argimg/values.yaml
135+
}
136+
137+
@test "does not introduce a file present in the artifact but absent in the PR tree" {
138+
root=$(pwd)
139+
w=$(mktemp -d); trap 'rm -rf "$w"' EXIT
140+
mkdir -p "$w/packages/apps/present/images" "$w/main/apps/present/images" "$w/main/apps/ghost/images"
141+
echo 'ghcr.io/cozystack/cozystack/present:v1.5.0@sha256:aaaa' > "$w/packages/apps/present/images/present.tag"
142+
echo 'iad.ocir.io/x/cozystack/present:main@sha256:bbbb' > "$w/main/apps/present/images/present.tag"
143+
# 'ghost' exists only in the artifact tree, never in the PR's packages/.
144+
echo 'iad.ocir.io/x/cozystack/ghost:main@sha256:cccc' > "$w/main/apps/ghost/images/ghost.tag"
145+
cd "$w"
146+
"$root/hack/overlay-main-images.sh" main '[]'
147+
grep -q 'present:main@sha256:bbbb' packages/apps/present/images/present.tag
148+
[ ! -e packages/apps/ghost/images/ghost.tag ]
149+
}

0 commit comments

Comments
 (0)