Skip to content

Commit 2b5a309

Browse files
committed
ci(retention): delete with the workflow token
The app installation token lists package versions but its DELETE of a listed version is answered with 404, while the repository's GITHUB_TOKEN with packages: write deleted the same version without any change to package settings. Deletes therefore go through GITHUB_TOKEN; listing keeps the app token. That deletion was shown on a package linked to this repository. A package linked elsewhere is not attempted: it is skipped before its versions are read and named in a warning, so it cannot turn every apply run red. A failed read of the link still fails the run. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent e01bd36 commit 2b5a309

6 files changed

Lines changed: 89 additions & 7 deletions

File tree

‎.github/workflows/retention.yaml‎

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ name: Artifact Retention
2929
# lists package versions, yet DELETE of a listed version answered "404 Package
3030
# not found" (run 37939905343). This repository's GITHUB_TOKEN, with the job's
3131
# packages: write and no change to package settings, deleted the same version
32-
# (run 37941084456) from a package linked to this repository. Applying needs
33-
# the delete call switched to that token.
32+
# (run 37941084456) from a package linked to this repository, so deletes use
33+
# it. A package linked to another repository is skipped with a warning.
3434

3535
on:
3636
schedule:
@@ -87,6 +87,8 @@ jobs:
8787
GH_TOKEN: ${{ steps.app-token.outputs.token }}
8888
# workflow_dispatch input wins; otherwise the repo variable; else dry-run.
8989
APPLY: ${{ github.event.inputs.apply || vars.RETENTION_APPLY || 'false' }}
90+
# See the admin NOTE above for why deletes do not use the app token.
91+
DELETE_TOKEN: ${{ secrets.GITHUB_TOKEN }}
9092
run: |
9193
set -euo pipefail
9294
if [ "$APPLY" = "true" ]; then echo "MODE: APPLY (deleting)"; else echo "MODE: dry-run"; fi
@@ -218,6 +220,7 @@ jobs:
218220
# swept, then fails the run at the end. The loop is fed by a
219221
# here-string rather than a pipe so the flag survives it.
220222
sweep_failed=0
223+
unlinked=""
221224
gh_err="$(mktemp)"
222225
tree="$(mktemp -d)"
223226
pkgs="$(flux pull artifact "oci://ghcr.io/${ORG}/cozystack/cozystack-packages:nightly" --output "$tree" >/dev/null \
@@ -236,6 +239,19 @@ jobs:
236239
[ -n "$pkg" ] || continue
237240
enc="$(jq -rn --arg p "$pkg" '$p|@uri')"
238241
242+
# DELETE_TOKEN was shown to delete from a package linked to this
243+
# repository; one linked elsewhere is not attempted, only named in
244+
# a warning, so it cannot fail every apply run.
245+
linked="$(gh api "/orgs/${ORG}/packages/container/${enc}" --jq '.repository.full_name // ""' 2>"$gh_err")" || {
246+
echo "::error::Could not read ${pkg}; not swept: $(cat "$gh_err")"
247+
sweep_failed=1
248+
continue
249+
}
250+
if [ "$linked" != "$GITHUB_REPOSITORY" ]; then
251+
unlinked="${unlinked:+${unlinked} }${pkg}"
252+
continue
253+
fi
254+
239255
# Version ids whose tags are non-empty AND every tag is a dated
240256
# nightly (so the floating `nightly` tag and any release tag exclude
241257
# the version). API returns versions newest-first, so the dated
@@ -261,6 +277,7 @@ jobs:
261277
262278
printf '%s\n' "$drop_ids" | xargs ./hack/retention-delete.sh "$pkg" version || sweep_failed=1
263279
done <<< "$pkgs"
280+
[ -z "$unlinked" ] || echo "::warning::not linked to ${GITHUB_REPOSITORY}, skipped: ${unlinked}"
264281
265282
# Anything unresolved above means the protected set is incomplete, so
266283
# no candidate can be called abandoned. Skip this sweep entirely and

‎docs/agents/image-refs.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ If a package stamps a reference somewhere the two globs do not cover, add that f
5959
| `hack/nightly-mirror.sh` | all three shapes | copy each digest to the public registry, then rewrite hosts in the same file set |
6060
| `hack/overlay-main-images.sh` | `values.yaml` + `*.tag` | overlay current-main references onto packages a PR did not rebuild |
6161
| `hack/stitch-multiarch.sh` | all three shapes | join each amd64 image and its `-arm64` twin into one index, repin the tree on it, then fail if any old digest survives |
62-
| `hack/retention-packages.sh` | all three shapes, in the published nightly tree | name the GHCR packages the retention workflow prunes; a ref it cannot see is a package whose nightlies are never pruned |
62+
| `hack/retention-packages.sh` | all three shapes, in the published nightly tree | name the GHCR packages the retention workflow prunes (those linked to another repository are then skipped); a ref it cannot see is a package whose nightlies are never pruned |
6363
| `hack/verify-multiarch.sh` | all three shapes, first- and third-party | fail a release unless every pinned image is an index with `linux/amd64` and `linux/arm64`, bar the repositories in `hack/multiarch-allowlist` |
6464

6565
`hack/multiarch-allowlist` is the only bypass for that gate, and it is for images that cannot exist for arm64, not for images nobody has rebuilt yet. An entry is a repository with no tag or digest, so bumping an allowed image needs no edit, and it must carry a reason after `#` on the same line: the script refuses an entry without one. Allowed refs are still printed, under their own "allowed single-arch" heading, and in tree mode an entry that matches no ref that failed the check is reported as stale without changing the exit status, so it gets removed once the image is multi-arch or gone. `--refs-file` reports no stale entries, since a list of pulled images omits everything the run did not pull.

‎docs/release.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,7 @@ A nightly is an **installable copy of `main` on GHCR — not a rebuild and not a
335335
2. **build-disk** — assemble the Talos `nocloud` disk from the upstream siderolabs imager (the profile references only `ghcr.io/siderolabs/*`, so this needs no cozystack image and no rebuild) and publish it as a GHCR OCI artifact (`cozystack-nocloud`) so a nightly is installable on real hardware, not just in e2e.
336336
3. **e2e** — stage the published GHCR closure (rewritten tree + the `cozy-installer` chart pinned to the GHCR packages artifact + the disk) and run the **full** app suite. A nightly has no diff, so Test Impact Analysis does not apply.
337337

338-
No GitHub release and no `api/apps/v1alpha1/*` Go-module tag are created for a nightly. [`retention.yaml`](../.github/workflows/retention.yaml) prunes old GHCR nightly versions (keeps the newest 14 per package the current nightly publishes, named from the published `cozystack-packages:nightly` tree by [`hack/retention-packages.sh`](../hack/retention-packages.sh); the floating `nightly`, release tags and untagged versions are never touched) and abandoned packages candidates. A candidate version is eligible only when all of these hold: it carries at least one tag and **every** tag on that manifest matches the strict `promotion-vX.Y.Z-from-vX.Y.Z-rc.N-run-<run-id>-<attempt>` form, so a manifest that has been retagged stable is excluded; it was last updated more than 30 days ago; its digest is not pinned by `packages/core/installer/values.yaml` at the head of an **open** pull request that is same-repository, authored by `cozystack-ci[bot]`, on a `release-X.Y.Z` branch and carrying the `release` label; and its digest is not pinned by that same file at the tip of `main` or of any `release-X.Y` branch — which is what keeps a candidate safe between the promote PR merging and finalize retagging it. Any such PR head whose pin is not exactly one `digest=sha256:<64 hex>` stops the candidate sweep for that run and fails the job, without affecting the nightly sweep.
338+
No GitHub release and no `api/apps/v1alpha1/*` Go-module tag are created for a nightly. [`retention.yaml`](../.github/workflows/retention.yaml) prunes old GHCR nightly versions (keeps the newest 14 per package the current nightly publishes, named from the published `cozystack-packages:nightly` tree by [`hack/retention-packages.sh`](../hack/retention-packages.sh), except packages linked to another repository, which are skipped with a warning; the floating `nightly`, release tags and untagged versions are never touched) and abandoned packages candidates. A candidate version is eligible only when all of these hold: it carries at least one tag and **every** tag on that manifest matches the strict `promotion-vX.Y.Z-from-vX.Y.Z-rc.N-run-<run-id>-<attempt>` form, so a manifest that has been retagged stable is excluded; it was last updated more than 30 days ago; its digest is not pinned by `packages/core/installer/values.yaml` at the head of an **open** pull request that is same-repository, authored by `cozystack-ci[bot]`, on a `release-X.Y.Z` branch and carrying the `release` label; and its digest is not pinned by that same file at the tip of `main` or of any `release-X.Y` branch — which is what keeps a candidate safe between the promote PR merging and finalize retagging it. Any such PR head whose pin is not exactly one `digest=sha256:<64 hex>` stops the candidate sweep for that run and fails the job, without affecting the nightly sweep.
339339

340340
Install a nightly:
341341

‎hack/promotion-retention-contract.bats‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -184,3 +184,36 @@ step_block() {
184184
false
185185
fi
186186
}
187+
188+
@test "deletes use the workflow token and skip packages linked elsewhere" {
189+
# The app token lists versions but cannot delete them; GITHUB_TOKEN deleted a
190+
# version of a package linked to this repository. A package linked elsewhere
191+
# is not attempted, so it cannot turn every apply run red: it is skipped with
192+
# a warning before its versions are read, while a failed linkage read still
193+
# fails the run.
194+
block="$(step_block 'Prune' "$RETENTION")"
195+
[ -n "$block" ]
196+
code="$(printf '%s\n' "$block" | code_lines)"
197+
198+
printf '%s\n' "$block" | grep -qF 'DELETE_TOKEN: ${{ secrets.GITHUB_TOKEN }}'
199+
200+
link_at="$(printf '%s\n' "$code" | grep -nF '/packages/container/${enc}" --jq' | awk -F: 'NR == 1 { print $1 }')"
201+
read_at="$(printf '%s\n' "$code" | grep -nF '/versions" 2>' | awk -F: 'NR == 1 { print $1 }')"
202+
[ -n "$link_at" ] && [ -n "$read_at" ]
203+
[ "$link_at" -lt "$read_at" ]
204+
window="$(printf '%s\n' "$code" | awk -v s="$link_at" -v e="$read_at" 'NR >= s && NR < e')"
205+
printf '%s\n' "$window" | grep -qF '.repository.full_name'
206+
printf '%s\n' "$window" | grep -qF 'sweep_failed=1'
207+
printf '%s\n' "$window" | grep -qF '!= "$GITHUB_REPOSITORY"'
208+
printf '%s\n' "$window" | grep -qF 'unlinked='
209+
[ "$(printf '%s\n' "$window" | grep -cF 'continue')" -ge 2 ]
210+
if printf '%s\n' "$window" | grep -qF 'retention-delete.sh'; then
211+
echo "FAIL: a delete is reachable before the linkage check"
212+
false
213+
fi
214+
215+
warn_at="$(printf '%s\n' "$code" | grep -nF 'not linked to' | awk -F: 'NR == 1 { print $1 }')"
216+
promotion_at="$(printf '%s\n' "$code" | grep -nF 'promotion_ids=' | awk -F: 'NR == 1 { print $1 }')"
217+
[ -n "$warn_at" ] && [ "$warn_at" -lt "$promotion_at" ]
218+
printf '%s\n' "$code" | sed -n "${warn_at}p" | grep -qF '::warning::'
219+
}

‎hack/retention-delete.sh‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,13 @@
11
#!/bin/sh
22
# Delete GHCR container-package versions for the retention workflow.
33
#
4-
# Usage: APPLY=true|false ORG=<org> retention-delete.sh <package> <label> <version-id>...
4+
# Usage: APPLY=true|false ORG=<org> [DELETE_TOKEN=<token>] \
5+
# retention-delete.sh <package> <label> <version-id>...
6+
#
7+
# Deletes with DELETE_TOKEN, required under APPLY=true, rather than the GH_TOKEN
8+
# that listed the versions: the app installation token can list org package
9+
# versions but has its DELETE answered with 404, while the workflow's
10+
# GITHUB_TOKEN deleted a version of a package linked to its repository.
511
#
612
# <label> names what is deleted in the log ("version", "promotion candidate
713
# version"). Without APPLY=true it only reports. A refused delete is an error,
@@ -14,6 +20,9 @@ PKG="${1:?usage: retention-delete.sh <package> <label> <version-id>...}"
1420
LABEL="${2:?usage: retention-delete.sh <package> <label> <version-id>...}"
1521
shift 2
1622
: "${ORG:?ORG is required}"
23+
if [ "${APPLY:-false}" = "true" ]; then
24+
: "${DELETE_TOKEN:?DELETE_TOKEN is required to delete}"
25+
fi
1726

1827
enc="$(printf '%s' "$PKG" | sed 's|/|%2F|g')"
1928
failed=0
@@ -23,7 +32,7 @@ for id in "$@"; do
2332
continue
2433
fi
2534
echo " ✗ deleting ${PKG} ${LABEL} ${id}"
26-
if ! err="$(gh api -X DELETE "/orgs/${ORG}/packages/container/${enc}/versions/${id}" 2>&1 >/dev/null)"; then
35+
if ! err="$(GH_TOKEN="$DELETE_TOKEN" gh api -X DELETE "/orgs/${ORG}/packages/container/${enc}/versions/${id}" 2>&1 >/dev/null)"; then
2736
echo "::error::failed to delete ${PKG} ${LABEL} ${id}: ${err}"
2837
failed=1
2938
fi

‎hack/retention-delete_test.bats‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ _stub_gh() {
1414
mkdir -p "$1"
1515
cat > "$1/gh" <<'EOF'
1616
#!/bin/sh
17-
printf '%s\n' "$*" >> "$GH_CALLS"
17+
printf '%s token=%s\n' "$*" "${GH_TOKEN:-}" >> "$GH_CALLS"
1818
case "$*" in
1919
*/versions/2*) echo 'gh: Forbidden (HTTP 403)' >&2; exit 1 ;;
2020
esac
@@ -28,6 +28,8 @@ setup() {
2828
export PATH="${BATS_TEST_TMPDIR}/bin:$PATH"
2929
export GH_CALLS="${BATS_TEST_TMPDIR}/calls"
3030
export ORG=cozystack
31+
export GH_TOKEN=app-token
32+
export DELETE_TOKEN=workflow-token
3133
: > "$GH_CALLS"
3234
}
3335

@@ -57,3 +59,24 @@ setup() {
5759
*) echo "FAIL: dry-run did not report: $output"; false ;;
5860
esac
5961
}
62+
63+
@test "apply: deletes with DELETE_TOKEN, not the token that listed the versions" {
64+
run env APPLY=true "$SCRIPT" cozystack/foo version 1
65+
[ "$status" -eq 0 ]
66+
grep -q 'versions/1 token=workflow-token$' "$GH_CALLS"
67+
}
68+
69+
@test "apply: no DELETE_TOKEN is refused before any call" {
70+
run env -u DELETE_TOKEN APPLY=true "$SCRIPT" cozystack/foo version 1
71+
[ "$status" -ne 0 ]
72+
[ ! -s "$GH_CALLS" ]
73+
case "$output" in
74+
*"DELETE_TOKEN is required"*) ;;
75+
*) echo "FAIL: refusal does not name DELETE_TOKEN: $output"; false ;;
76+
esac
77+
}
78+
79+
@test "dry-run needs no DELETE_TOKEN" {
80+
run env -u DELETE_TOKEN APPLY=false "$SCRIPT" cozystack/foo version 1
81+
[ "$status" -eq 0 ]
82+
}

0 commit comments

Comments
 (0)