Skip to content

Commit eff66a7

Browse files
myasnikovdaniilmyasnikovdaniil
committed
test(release): pin the promote-retag invocation split, and recover both halves
Two gaps in the recovery story this PR is built around. The workflow runs ./.promote-scripts/hack/promote-retag.sh with the working directory on the release tree, which works only because the script sources its helper relative to $0 while scanning packages relative to the working directory. Those two resolutions have to stay independent, and that was stated in a workflow comment and held by nothing executable: every existing case runs from the repo root, where script dir and scanned root are the same place. Add a case that copies two package values.yaml into a stand-in tree containing no hack/, changes into it, and invokes the script by absolute path. Mutation check: rewriting the source line to `. hack/lib/image-refs.sh` previously kept all eleven cases green and would have broken the next promotion after the tag and release were already irreversible; it now fails this case, and only this case. The manual fallback named only the retag. A maintainer past the 30-day re-run window would follow it and stop with the images retagged and the chart unpublished — precisely the half-finished state this job exists to make recoverable, reached by following these instructions. Name both scripts and the registry login they need, and record that both are invoked by absolute path from the release tree, as the job runs them. Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent d94cd18 commit eff66a7

2 files changed

Lines changed: 54 additions & 1 deletion

File tree

‎docs/release.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -273,7 +273,15 @@ Step 5 is the part that fails: it touches 40+ external repositories over a netwo
273273

274274
Re-running is safe at any point of failure. `promote-retag.sh` skips a tag already at the expected digest, creates a missing one, and refuses to move a stable tag that points elsewhere. `promote-publish-chart.sh` compares an already-published chart against the tree by extracted content and skips an identical one, because `helm package` is not byte-reproducible and an unconditional re-push would move the stable chart tag onto new bytes. A run in which everything already landed comes out green.
275275

276-
Re-running also picks up a *fix*, not just a retry. The job checks out the release tree at the promoted merge commit — frozen, as release content must be — but checks out the promotion scripts from `main` at run time, into `.promote-scripts/`. So the sequence for a script bug is: land the correction on `main`, press "Re-run failed jobs", and the corrected script runs against the unchanged release tree. Note GitHub allows a re-run only within 30 days of the original run; past that, or if the failure is in the workflow YAML rather than in a script (a re-run reuses the original workflow file), run the script by hand from a checkout of the release tag — `cd <tag-tree> && /path/to/main/hack/promote-retag.sh vX.Y.Z`, with `MOVE_LATEST=1` only if this is the newest stable.
276+
Re-running also picks up a *fix*, not just a retry. The job checks out the release tree at the promoted merge commit — frozen, as release content must be — but checks out the promotion scripts from `main` at run time, into `.promote-scripts/`. So the sequence for a script bug is: land the correction on `main`, press "Re-run failed jobs", and the corrected script runs against the unchanged release tree. Note GitHub allows a re-run only within 30 days of the original run; past that, or if the failure is in the workflow YAML rather than in a script (a re-run reuses the original workflow file), run the scripts by hand from a checkout of the release tag. Run **both** — retagging the images without publishing the chart leaves exactly the half-finished state this job exists to make recoverable. Log in to the registry first (`docker login` against the OCIR host, the same credentials the job uses), then, with `MOVE_LATEST=1` set only if this is the newest stable:
277+
278+
```sh
279+
cd <tag-tree>
280+
MOVE_LATEST=1 /path/to/main/hack/promote-retag.sh vX.Y.Z
281+
MOVE_LATEST=1 /path/to/main/hack/promote-publish-chart.sh vX.Y.Z
282+
```
283+
284+
Both are invoked by absolute path from a working directory that is the release tree, which is how the job runs them too: each resolves its own helpers relative to the script, and scans the package tree relative to the working directory.
277285

278286
### Phase 6 — `update-releasenotes.yaml` (sync GitHub Release body)
279287

‎hack/promote-retag_test.bats‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -362,3 +362,48 @@ EOF
362362
grep -q '429' "$tmp/err"
363363
! grep -q '^copy ' "$tmp/skopeo.log"
364364
}
365+
366+
# The workflow invokes this script as ./.promote-scripts/hack/promote-retag.sh
367+
# with the working directory pointing at the RELEASE tree — two path
368+
# resolutions that have to stay independent: the helper is sourced relative to
369+
# the script (`$(dirname "$0")/lib/image-refs.sh`) while the package scan is
370+
# relative to the working directory (`collect_image_refs packages`). Every
371+
# other case here runs from the repo root, where those two happen to be the
372+
# same directory, so none of them would notice the source line being "tidied"
373+
# to `. hack/lib/image-refs.sh` — and that tidy-up breaks the next promotion
374+
# only after the tag and the release are already irreversible.
375+
@test "runs with the script dir and the scanned tree in different places" {
376+
tmp=$(mktemp -d)
377+
trap 'rm -rf "$tmp"' EXIT
378+
root="$PWD"
379+
380+
# A stand-in release tree: only the package files the selector scans, at the
381+
# same relative paths. Deliberately no hack/ inside it, so the helper cannot
382+
# resolve from the working directory and must come from the script's own dir.
383+
for pkg in system/kamaji system/linstor; do
384+
mkdir -p "$tmp/tree/packages/$pkg"
385+
cp "packages/$pkg/values.yaml" "$tmp/tree/packages/$pkg/values.yaml"
386+
done
387+
[ ! -e "$tmp/tree/hack" ]
388+
389+
cd "$tmp/tree"
390+
rc=0
391+
env -u REGISTRY "$root/hack/promote-retag.sh" v9.9.9 --dry-run \
392+
>"$tmp/out" 2>"$tmp/err" || rc=$?
393+
cd "$root"
394+
if [ "$rc" -ne 0 ]; then
395+
echo "promote-retag.sh exited $rc invoked by absolute path from $tmp/tree" >&2
396+
echo "--- script stderr ---" >&2; cat "$tmp/err" >&2
397+
echo "--- script stdout ---" >&2; cat "$tmp/out" >&2
398+
return "$rc"
399+
fi
400+
401+
# Sourcing resolved against the script dir, and the scan against the working
402+
# directory: both packages copied into the stand-in tree are in the plan.
403+
grep -q 'docker://ghcr.io/cozystack/cozystack/kamaji@sha256:' "$tmp/out"
404+
grep -q 'docker://ghcr.io/cozystack/cozystack/linstor-csi@sha256:' "$tmp/out"
405+
406+
# And the scan really did read the stand-in tree rather than the repo it was
407+
# launched from — a package present only in the repo must be absent here.
408+
! grep -q 'docker://ghcr.io/cozystack/cozystack/redis-operator@sha256:' "$tmp/out"
409+
}

0 commit comments

Comments
 (0)