Skip to content

Commit 9fa8eee

Browse files
ci(release): pin the promotion toolchain and cap the candidate gate
Three review findings on the new promotion steps. `yq` was fetched from `releases/latest/download` in all three steps this pipeline installs it in — the promote dispatch, the promote PR gate and finalize. That makes an unpinned moving dependency part of what decides whether a stable release may be created, and part of what rewrites the installer pin: two promotions of the same rc can be served different binaries. Pin it the way FLUX_VERSION already is, version-gated so a matching preinstalled yq is reused, on the version the e2e sandbox image already tracks. A version authenticates nothing about the bytes it names, though, and these are the workflows that mint write-once stable names. yq is therefore also pinned BY CONTENT: its SHA-256 is committed beside its version and verified before install. Scoped to yq deliberately. It is the one downloaded binary with write authority over what a release contains — it stamps the candidate digest into installer values at promote time and platformVersion into the stable chart at finalize. skopeo, helm and oras only move bytes that are already digest-addressed, so a substituted one is caught downstream; a substituted yq writes the wrong pin and every later check verifies that pin faithfully. The Flux installer is left as it is, on purpose. It fetches flux_${FLUX_VERSION}_checksums.txt from the pinned release and verifies the binary against it, so the binary is not the exposure — the ~6 KB bootstrap script from fluxcd.io is, and it is mutable by design and fetched identically in ten other workflows here. Pinning its bytes in these three files would break on the next regeneration of a script the project does not control while leaving the same fetch unpinned everywhere else. The comment names the residual risk instead. No precedent existed either way: nothing under .github/workflows/ or hack/ verified a downloaded tool before this, and every version-pinned download in the tree (oras 1.2.0, cozyvalues-gen 1.6.0) is unverified. This is a new convention rather than an application of one, confined to the release path and justified by write authority, and the comment says so where the next reader will be standing. The three unpinned fetches outside this pipeline (nightly.yaml, e2e-fork.yaml) are left alone. The checksum is trust-on-first-use, which is the honest description: it fixes the bytes to what was checked when the version was chosen and makes any later substitution loud. Provenance is the release's own published checksums list cross-checked against the downloaded asset; both agree on fa52a4e758c63d38299163fbdd1edfb4c4963247918bf9c1c5d31d84789eded4, and that binary self-reports v4.53.3. Verified locally against the real asset: the version gate skips a matching preinstalled yq and re-installs a mismatched one, the checksum line reports OK for the genuine binary and exits 1 for a tampered copy, which aborts the step under the runner's `bash -e`. `verify-release-candidate` had no `timeout-minutes`, so it inherited the 6-hour default while `e2e-report` waits on it through `needs` — a stalled `flux pull artifact` would hold the required `E2E Tests` status pending for the whole of it. 30 minutes, matching `checks` and `finalize`. The contract pins that all three install steps carry both versions and the checksum, run the checksum check, name no `latest/download`, and agree on one yq version and one checksum. So the unpinned one-line idiom cannot come back into a release step by copy-paste from the workflows that still use it, and bumping the version in one file without its checksum cannot pass. Both mutations are caught. Signed-off-by: Myasnikov Daniil <[email protected]> Assisted-By: Claude <[email protected]>
1 parent 2d734fb commit 9fa8eee

4 files changed

Lines changed: 118 additions & 6 deletions

File tree

‎.github/workflows/promote-rc.yaml‎

Lines changed: 37 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -393,18 +393,53 @@ jobs:
393393

394394
# Promotion publishes one non-consumer-visible packages candidate. Stable
395395
# image/chart tags are still deferred to post-merge finalize.
396+
#
397+
# Both tools are version-pinned. This step arms the only registry write
398+
# the dispatch performs, so what performs it has to be a fixed input:
399+
# `latest/download` resolves to whatever mikefarah shipped this morning,
400+
# and a yq that changed how it emits or compares a value would rewrite
401+
# installer values differently between two promotions of the same rc. The
402+
# version tracks packages/core/testing/images/e2e-sandbox/Dockerfile.
403+
#
404+
# yq is additionally pinned BY CONTENT, which nothing else in this repo
405+
# does. It earns the exception by being the one downloaded binary with
406+
# write authority over what a release contains: it is what stamps the
407+
# candidate digest into installer values. skopeo, helm and oras only move
408+
# bytes that are already digest-addressed, so a substituted one is caught
409+
# downstream; a substituted yq writes the wrong pin and everything after
410+
# it verifies that pin faithfully. A committed SHA-256 is trust-on-first-
411+
# use — it fixes the bytes to what a human checked when the version was
412+
# chosen (against this release's own published checksums list) and makes
413+
# any later substitution loud. Bump the version and this together.
414+
#
415+
# The Flux installer is deliberately NOT pinned by content. It verifies
416+
# the flux binary it fetches against flux_${FLUX_VERSION}_checksums.txt
417+
# from the pinned release, so the binary is not the exposure — the ~6 KB
418+
# bootstrap script from fluxcd.io is, and it is mutable by design and
419+
# fetched the same way in ten other workflows here. Pinning its bytes in
420+
# this file alone would break on the next regeneration of a script the
421+
# project does not control, while leaving the identical fetch unpinned
422+
# everywhere else. Named rather than half-fixed.
396423
- name: Set up toolchain (flux, yq)
397424
env:
398425
FLUX_VERSION: "2.8.6"
426+
YQ_VERSION: "4.53.3"
427+
YQ_SHA256: "fa52a4e758c63d38299163fbdd1edfb4c4963247918bf9c1c5d31d84789eded4"
399428
run: |
400429
if ! flux version --client 2>/dev/null | grep -qx "flux: v${FLUX_VERSION}"; then
401430
install_script="$(mktemp)"
402431
curl -fsSL https://fluxcd.io/install.sh -o "$install_script"
403432
sudo env "FLUX_VERSION=$FLUX_VERSION" bash "$install_script"
404433
rm -f "$install_script"
405434
fi
406-
command -v yq >/dev/null \
407-
|| { sudo curl -sSL -o /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 && sudo chmod +x /usr/local/bin/yq; }
435+
if ! yq --version 2>/dev/null | grep -q "mikefarah.* version v${YQ_VERSION}\$"; then
436+
yq_bin="$(mktemp)"
437+
curl -fsSL -o "$yq_bin" \
438+
"https://github.com/mikefarah/yq/releases/download/v${YQ_VERSION}/yq_linux_amd64"
439+
printf '%s %s\n' "$YQ_SHA256" "$yq_bin" | sha256sum -c -
440+
sudo install -m 0755 "$yq_bin" /usr/local/bin/yq
441+
rm -f "$yq_bin"
442+
fi
408443
409444
- name: Login to registry (GHCR)
410445
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3

‎.github/workflows/pull-requests-release.yaml‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,19 +108,37 @@ jobs:
108108
# the tag-rewritten tree; this guard pulls it by the committed digest and
109109
# proves it still matches the merge commit (apart from its impossible
110110
# self-reference) and still pins the rc-tested container digests.
111+
#
112+
# flux and yq are version-pinned: they are what this job verifies the
113+
# candidate with and what stamps the stable chart, so `latest/download`
114+
# would make an irreversible step depend on whatever mikefarah shipped
115+
# this morning. The version tracks
116+
# packages/core/testing/images/e2e-sandbox/Dockerfile. yq is additionally
117+
# pinned by content — it stamps the chart's platformVersion, and this step
118+
# is the last one before write-once names exist. The Flux installer is not,
119+
# because it checksums the binary it fetches and the mutable part is its
120+
# own bootstrap script. promote-rc.yaml's copy carries the full reasoning.
111121
- name: Set up promotion toolchain (flux, skopeo, yq, helm)
112122
if: ${{ !contains(steps.get_tag.outputs.tag, '-') }}
113123
env:
114124
FLUX_VERSION: "2.8.6"
125+
YQ_VERSION: "4.53.3"
126+
YQ_SHA256: "fa52a4e758c63d38299163fbdd1edfb4c4963247918bf9c1c5d31d84789eded4"
115127
run: |
116128
if ! flux version --client 2>/dev/null | grep -qx "flux: v${FLUX_VERSION}"; then
117129
install_script="$(mktemp)"
118130
curl -fsSL https://fluxcd.io/install.sh -o "$install_script"
119131
sudo env "FLUX_VERSION=$FLUX_VERSION" bash "$install_script"
120132
rm -f "$install_script"
121133
fi
122-
command -v yq >/dev/null \
123-
|| { sudo curl -sSL -o /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 && sudo chmod +x /usr/local/bin/yq; }
134+
if ! yq --version 2>/dev/null | grep -q "mikefarah.* version v${YQ_VERSION}\$"; then
135+
yq_bin="$(mktemp)"
136+
curl -fsSL -o "$yq_bin" \
137+
"https://github.com/mikefarah/yq/releases/download/v${YQ_VERSION}/yq_linux_amd64"
138+
printf '%s %s\n' "$YQ_SHA256" "$yq_bin" | sha256sum -c -
139+
sudo install -m 0755 "$yq_bin" /usr/local/bin/yq
140+
rm -f "$yq_bin"
141+
fi
124142
command -v skopeo >/dev/null \
125143
|| { sudo apt-get update -qq && sudo apt-get install -y -qq skopeo; }
126144
command -v helm >/dev/null \

‎.github/workflows/pull-requests.yaml‎

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,10 @@ jobs:
203203
verify-release-candidate:
204204
name: Verify release packages candidate
205205
runs-on: ubuntu-latest
206+
# Matching `checks` and `finalize`. Without a ceiling the job inherits the
207+
# 6-hour default, and a stalled `flux pull artifact` would hold the required
208+
# `E2E Tests` status pending for all of it, since e2e-report needs this job.
209+
timeout-minutes: 30
206210
if: |
207211
contains(github.event.pull_request.labels.*.name, 'release')
208212
&& github.event.pull_request.user.login == 'cozystack-ci[bot]'
@@ -228,18 +232,35 @@ jobs:
228232
sparse-checkout: hack
229233
persist-credentials: false
230234

235+
# Both tools are version-pinned. This job decides whether a stable release
236+
# may be created, so what it runs has to be a fixed input: `latest/download`
237+
# resolves to whatever mikefarah shipped this morning, and a yq that changed
238+
# how it emits or compares a value turns a release gate into a moving one.
239+
# The version tracks packages/core/testing/images/e2e-sandbox/Dockerfile.
240+
# yq is additionally pinned by content; the Flux installer is not, because
241+
# it checksums the binary it fetches and the mutable part is its own
242+
# bootstrap script. promote-rc.yaml's copy of this step carries the full
243+
# reasoning for both.
231244
- name: Set up promotion toolchain (flux, yq)
232245
env:
233246
FLUX_VERSION: "2.8.6"
247+
YQ_VERSION: "4.53.3"
248+
YQ_SHA256: "fa52a4e758c63d38299163fbdd1edfb4c4963247918bf9c1c5d31d84789eded4"
234249
run: |
235250
if ! flux version --client 2>/dev/null | grep -qx "flux: v${FLUX_VERSION}"; then
236251
install_script="$(mktemp)"
237252
curl -fsSL https://fluxcd.io/install.sh -o "$install_script"
238253
sudo env "FLUX_VERSION=$FLUX_VERSION" bash "$install_script"
239254
rm -f "$install_script"
240255
fi
241-
command -v yq >/dev/null \
242-
|| { sudo curl -sSL -o /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 && sudo chmod +x /usr/local/bin/yq; }
256+
if ! yq --version 2>/dev/null | grep -q "mikefarah.* version v${YQ_VERSION}\$"; then
257+
yq_bin="$(mktemp)"
258+
curl -fsSL -o "$yq_bin" \
259+
"https://github.com/mikefarah/yq/releases/download/v${YQ_VERSION}/yq_linux_amd64"
260+
printf '%s %s\n' "$YQ_SHA256" "$yq_bin" | sha256sum -c -
261+
sudo install -m 0755 "$yq_bin" /usr/local/bin/yq
262+
rm -f "$yq_bin"
263+
fi
243264
244265
- name: Login to registry (GHCR)
245266
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3

‎hack/promote-packages-contract.bats‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,44 @@ step_line() {
6969
printf '%s\n' "$prepare" | code_lines | grep -qF 'DOCKER_CONFIG: ${{ runner.temp }}/.docker'
7070
}
7171

72+
@test "every promotion toolchain install pins its tools" {
73+
# A release gate whose tooling comes from `releases/latest/download` is not a
74+
# fixed input: the bytes that decide whether vX.Y.Z may be created, and the
75+
# yq that rewrites the installer pin, change whenever upstream publishes.
76+
# Pinned by version, and pinned to STAY pinned — the unpinned one-liner is
77+
# still the idiom in the workflows outside this pipeline, so it comes back by
78+
# copy-paste rather than by anyone deciding to unpin a release step.
79+
for step_and_file in \
80+
"Set up toolchain (flux, yq)|$PROMOTE" \
81+
"Set up promotion toolchain (flux, yq)|$PULL_REQUESTS" \
82+
"Set up promotion toolchain (flux, skopeo, yq, helm)|$FINALIZE"; do
83+
block="$(step_block "${step_and_file%%|*}" "${step_and_file#*|}")"
84+
[ -n "$block" ]
85+
printf '%s\n' "$block" | code_lines | grep -qF 'FLUX_VERSION: "'
86+
printf '%s\n' "$block" | code_lines | grep -qF 'YQ_VERSION: "'
87+
count="$(printf '%s\n' "$block" | code_lines | grep -c 'releases/latest/download' || true)"
88+
[ "${count:-0}" -eq 0 ]
89+
90+
# yq is pinned by content as well as by version: it is the one downloaded
91+
# binary that WRITES what a release contains, stamping the candidate digest
92+
# into installer values and platformVersion into the chart. A version
93+
# variable authenticates nothing about the bytes it names.
94+
printf '%s\n' "$block" | code_lines | grep -qF 'YQ_SHA256: "'
95+
printf '%s\n' "$block" | code_lines | grep -qF 'sha256sum -c -'
96+
done
97+
98+
# One version AND one checksum across all three, so the yq that writes the
99+
# candidate's installer values is the yq that reads them back out of the
100+
# artifact — and so bumping the version in one file without its checksum, or
101+
# either without the other two, cannot pass.
102+
for key in YQ_VERSION YQ_SHA256; do
103+
values="$(cat "$PROMOTE" "$PULL_REQUESTS" "$FINALIZE" | code_lines \
104+
| grep -o "$key: \"[0-9a-f.]*\"" | sort -u)"
105+
[ -n "$values" ]
106+
[ "$(printf '%s\n' "$values" | wc -l | tr -d ' ')" -eq 1 ]
107+
done
108+
}
109+
72110
@test "finalize verifies the pinned candidate before the first stable tag" {
73111
verify="$(step_line 'Verify stable packages candidate' "$FINALIZE")"
74112
create="$(step_line 'Create tag on merge commit (write-once)' "$FINALIZE")"

0 commit comments

Comments
 (0)