Skip to content

Commit f49d54a

Browse files
ci(release): fix review findings; keep website docs at tag time
Dual-model review (Claude + Codex) of the previous two commits found seven substantive defects, two of them destructive. Both models independently reached three of them; the split is noted per item. Descope the website half (Claude, verified by reproduction). The docs job was moved to promote time, but the website Makefile pins the fetch ref with `override BRANCH := $(RELEASE_TAG)`, which defeats the command-line BRANCH assignment. `make -p` resolves BRANCH to v1.6.0 — a tag that cannot exist at promote time, since promote's own validate step hard-fails if it does. update_apps.sh logs the failed fetch and exits 0 AFTER truncating each destination file, so every app doc would become a stub, `changed=true` would open the PR, and the new auto-merge would land it unreviewed. Not fixable from this repo. update-website-docs returns to tags.yaml where RELEASE_TAG resolves to a real tag, and the finalize auto-merge is dropped. Opening it earlier needs a cozystack/website change first and is left as a follow-up. This also removes the unordered-sibling race Codex found, where the release PR could merge before the docs job opened its PR, leaving finalize nothing to merge and no retry. Fix the non-blocking guarantee (both models). Only the AI step carried continue-on-error; the changelog JOB did not, so a flaky `npm i -g @github/copilot` failed it and `needs` skipped open-pr entirely — staging branch pushed, release drafted, no promote PR. Gate open-pr on promote alone. Generate from a checkout the contract permits (both models). The job checked out the rc staging branch, which docs/agents/changelog.md §2 rejects — a compliant agent stops and writes nothing, so the normal path produced no changelog at all. Check out the rc tag detached and document that configuration in §2. The checkout ref now comes from the validated parse output rather than raw dispatch input. Port, don't regenerate, in the backstop (both models). For a patch release promoted onto release-X.Y the changelog never reaches main, so the backstop fired and ran a second AI pass; once its PR merged, update-releasenotes.yaml overwrote the published body with unreviewed text. Detect the file at the tag and port it verbatim; the AI runs only when it is absent from both main and the tag. Survive re-dispatch (Codex). promote force-pushes the staging branch, erasing a hand-written changelog — the documented recovery when generation fails. Preserve an existing one across the rebuild, and stop open-pr overwriting it with generated output. Validate structurally, not by size (Codex). A truncated or whitespace-only AI run passed `[ -s ]`, was committed, advertised as valid in the PR body, and published verbatim as release notes. Assert the header, the compare link and a plausible line count; finalize additionally refuses whitespace-only content rather than blanking the notes. Add hack/release-changelog-contract.bats — nine tests pinning each link in the chain, mutation-checked: removing open-pr's if:, breaking finalize's body read, and restoring paths-ignore each fail the suite. Update docs/release.md (~10 sites) and docs/agents/changelog.md §2, which described the old flow throughout. Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent ba67fea commit f49d54a

6 files changed

Lines changed: 570 additions & 194 deletions

File tree

‎.github/workflows/promote-rc.yaml‎

Lines changed: 103 additions & 136 deletions
Original file line numberDiff line numberDiff line change
@@ -48,13 +48,14 @@ jobs:
4848
packages: write
4949
pull-requests: write
5050

51-
# Consumed by the changelog / open-pr / update-website-docs jobs below, so
52-
# the rc tag is parsed and validated exactly once.
51+
# Consumed by open-pr, so the rc tag is parsed and validated exactly once.
52+
# (rc_branch is deliberately not exported: the changelog job checks out the
53+
# rc *tag*, not the staging branch, and derives it from its own validated
54+
# parse — see the checkout comment in that job.)
5355
outputs:
5456
stable_tag: ${{ steps.parse.outputs.stable_tag }}
5557
stable_version: ${{ steps.parse.outputs.stable_version }}
5658
stable_branch: ${{ steps.parse.outputs.stable_branch }}
57-
rc_branch: ${{ steps.parse.outputs.rc_branch }}
5859
line: ${{ steps.parse.outputs.line }}
5960

6061
steps:
@@ -192,8 +193,28 @@ jobs:
192193
# rc version, which was only observable by cutting a release.
193194
# The script fails if any rc reference survives it.
194195
hack/promote-rewrite-tags.sh "$RC_VERSION" "$STABLE_VERSION"
196+
# Preserve a changelog already committed on the stable staging branch
197+
# across the rebuild. -B below resets the branch to the rc tree, which
198+
# would otherwise silently discard it — and the two ways it gets there
199+
# are exactly the ones worth protecting: a maintainer hand-writing it
200+
# (the documented escape hatch when generation fails) and an earlier
201+
# dispatch's reviewed output. Losing either to a retry would replace
202+
# human-edited release notes with a fresh AI run, or with nothing.
203+
CL="docs/changelogs/v${STABLE_VERSION}.md"
204+
PRESERVED=""
205+
if git fetch -q origin "${STABLE_BRANCH}" 2>/dev/null \
206+
&& git cat-file -e "FETCH_HEAD:${CL}" 2>/dev/null; then
207+
PRESERVED="$(mktemp)"
208+
git show "FETCH_HEAD:${CL}" > "$PRESERVED"
209+
echo "::notice::Preserving existing ${CL} from ${STABLE_BRANCH} across the rebuild."
210+
fi
195211
# -B resets the local branch so a re-dispatch starts clean.
196212
git checkout -B "${STABLE_BRANCH}"
213+
if [ -n "$PRESERVED" ]; then
214+
mkdir -p docs/changelogs
215+
cp "$PRESERVED" "$CL"
216+
rm -f "$PRESERVED"
217+
fi
197218
git add -A
198219
git commit -s -m "Prepare release v${STABLE_VERSION} (promoted from ${RC_TAG})" \
199220
|| echo "No tag-string changes to commit (digests already stable)"
@@ -305,15 +326,30 @@ jobs:
305326
exit 1
306327
fi
307328
VERSION="${BASH_REMATCH[1]}.${BASH_REMATCH[2]}.${BASH_REMATCH[3]}"
329+
# Re-emit the tag from the matched groups rather than passing
330+
# inputs.rc_tag through: the checkout `ref` below then consumes a value
331+
# this step reconstructed from a strict pattern, never raw dispatch
332+
# input, so a crafted rc_tag cannot reach `actions/checkout`.
308333
{
309334
echo "stable_version=${VERSION}"
310-
echo "rc_branch=release-${VERSION}-rc.${BASH_REMATCH[4]}"
335+
echo "rc_tag=v${VERSION}-rc.${BASH_REMATCH[4]}"
311336
} >> "$GITHUB_OUTPUT"
312337
313-
- name: Checkout rc staging branch
338+
# Detached at the rc TAG, not the rc staging branch. Two reasons:
339+
#
340+
# 1. docs/agents/changelog.md §2 is a hard gate — it permits exactly two
341+
# working-tree states (on `main`, or detached at the release tag) and
342+
# instructs the agent to STOP otherwise. A staging-branch checkout is
343+
# neither, so a compliant agent would refuse to generate anything on the
344+
# normal path. §2 has been extended to name this configuration too.
345+
# 2. The staging branch carries digest-vendoring commits that are not
346+
# release content. Detaching at the tag makes "use the rc tag as the
347+
# upper bound" a property of the checkout rather than an instruction the
348+
# prompt has to talk the agent into honouring.
349+
- name: Checkout rc tag
314350
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
315351
with:
316-
ref: ${{ steps.parse.outputs.rc_branch }}
352+
ref: ${{ steps.parse.outputs.rc_tag }}
317353
fetch-depth: 0
318354
fetch-tags: true
319355
token: ${{ steps.app-token-read.outputs.token }}
@@ -360,24 +396,43 @@ jobs:
360396
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
361397
GH_TOKEN: ${{ steps.app-token-read.outputs.token }}
362398
STABLE_VERSION: ${{ steps.parse.outputs.stable_version }}
363-
RC_TAG: ${{ inputs.rc_tag }}
399+
RC_TAG: ${{ steps.parse.outputs.rc_tag }}
364400
run: |
365401
copilot \
366-
--prompt "Generate the release changelog for the upcoming stable release v${STABLE_VERSION}, which is being promoted from the release candidate ${RC_TAG}. The repository is checked out on the rc staging branch. IMPORTANT: the v${STABLE_VERSION} tag does NOT exist yet — it is created only after the promotion PR merges. Use ${RC_TAG} as the upper bound in every git command (for example 'git log v<previous>..${RC_TAG}'), never HEAD and never the staging branch, because the staging branch carries digest-vendoring commits that are not part of the release content. Write the file as docs/changelogs/v${STABLE_VERSION}.md and use v${STABLE_VERSION} in the header, in the 'Full Changelog' link, and in the trailing release-link comment. Follow the instructions in @docs/agents/changelog.md exactly, including the 'Scope and boundaries' section at the top. Your deliverable is the single file docs/changelogs/v${STABLE_VERSION}.md — write it and exit; this workflow handles committing and opening the PR." \
402+
--prompt "Generate the release changelog for the upcoming stable release v${STABLE_VERSION}, which is being promoted from the release candidate ${RC_TAG}. The repository is checked out in detached HEAD at the ${RC_TAG} tag — the 'promotion' configuration in section 2 of docs/agents/changelog.md. IMPORTANT: the v${STABLE_VERSION} tag does NOT exist yet; it is created only after the promotion PR merges. Use ${RC_TAG} as the upper bound in every git command (for example 'git log v<previous>..${RC_TAG}'), never HEAD. Write the file as docs/changelogs/v${STABLE_VERSION}.md and use v${STABLE_VERSION} — not ${RC_TAG} — in the header, in the 'Full Changelog' link, and in the trailing release-link comment. Follow the instructions in @docs/agents/changelog.md exactly, including the 'Scope and boundaries' section at the top. Your deliverable is the single file docs/changelogs/v${STABLE_VERSION}.md — write it and exit; this workflow handles committing and opening the PR." \
367403
--allow-all-tools --allow-all-paths < /dev/null
368404
405+
# Structural validation, not just "non-empty". The AI step is allowed to
406+
# fail, and a run that dies partway (timeout, quota exhaustion mid-stream)
407+
# can leave a fragment or a few bytes of whitespace behind. A bare `-s`
408+
# test would call that present, commit it, advertise it as valid in the PR
409+
# body, and finalize would publish it verbatim as the release notes. Assert
410+
# the markers a real changelog always has instead; anything failing them is
411+
# treated exactly like "not generated" and takes the degraded path.
369412
- name: Verify changelog
370413
id: changelog_status
371414
env:
372415
STABLE_VERSION: ${{ steps.parse.outputs.stable_version }}
373416
run: |
374-
if [ -s "docs/changelogs/v${STABLE_VERSION}.md" ]; then
375-
echo "present=true" >> "$GITHUB_OUTPUT"
376-
echo "✅ docs/changelogs/v${STABLE_VERSION}.md present ($(wc -l < "docs/changelogs/v${STABLE_VERSION}.md") lines)."
377-
else
417+
CL="docs/changelogs/v${STABLE_VERSION}.md"
418+
fail() {
378419
echo "present=false" >> "$GITHUB_OUTPUT"
379-
echo "::warning::Changelog generation did not produce docs/changelogs/v${STABLE_VERSION}.md. The promote PR will open WITHOUT a changelog — commit it to the PR branch before merging, or the published release will have empty notes."
380-
fi
420+
echo "::warning::Changelog not usable ($1). The promote PR will open WITHOUT a changelog — commit docs/changelogs/v${STABLE_VERSION}.md to the PR branch before merging, or the published release will have empty notes."
421+
exit 0
422+
}
423+
[ -f "$CL" ] || fail "file absent"
424+
[ -s "$CL" ] || fail "file empty"
425+
# Non-whitespace content
426+
grep -q '[^[:space:]]' "$CL" || fail "file is only whitespace"
427+
# A truncated stream usually loses the tail, so check both ends.
428+
grep -qE "^# Cozystack v${STABLE_VERSION}\b" "$CL" \
429+
|| fail "missing '# Cozystack v${STABLE_VERSION}' header"
430+
grep -qF "compare/" "$CL" \
431+
|| fail "missing the Full Changelog compare link (likely truncated)"
432+
LINES=$(wc -l < "$CL")
433+
[ "$LINES" -ge 20 ] || fail "only ${LINES} lines — implausibly short for a release changelog"
434+
echo "present=true" >> "$GITHUB_OUTPUT"
435+
echo "✅ $CL validated (${LINES} lines)."
381436
382437
# Hand the file to `open-pr`, which owns every write to the staging branch.
383438
- name: Upload changelog artifact
@@ -396,6 +451,16 @@ jobs:
396451
name: Open promote PR
397452
runs-on: ubuntu-latest
398453
needs: [promote, changelog]
454+
# `needs` alone would SKIP this job whenever `changelog` fails, which would
455+
# leave the staging branch pushed and the draft release staged but no promote
456+
# PR at all — the exact wedge the non-blocking design exists to prevent.
457+
# Only the AI *step* carries continue-on-error; everything else in that job
458+
# (token, checkout, node setup, `npm i -g @github/copilot`, artifact upload,
459+
# the job timeout) is still fatal, and an npm or Copilot outage is precisely
460+
# the scenario this must survive. Gate on `promote` alone and let the
461+
# `needs.changelog.outputs.present` guards below degrade to the documented
462+
# "PR opens with the ⚠️ body" path.
463+
if: ${{ !cancelled() && needs.promote.result == 'success' }}
399464
timeout-minutes: 15
400465
permissions:
401466
contents: write
@@ -421,15 +486,33 @@ jobs:
421486
fetch-depth: 0
422487
token: ${{ steps.app-token.outputs.token }}
423488

489+
# A changelog already on the staging branch was put there deliberately —
490+
# hand-written by a maintainer, or carried over from an earlier dispatch by
491+
# `promote`'s preserve step — and must win over a fresh AI run. Without this
492+
# check the artifact download below would silently overwrite human edits.
493+
- name: Check for an existing changelog on the branch
494+
id: existing
495+
env:
496+
STABLE_VERSION: ${{ needs.promote.outputs.stable_version }}
497+
run: |
498+
if [ -s "docs/changelogs/v${STABLE_VERSION}.md" ]; then
499+
echo "present=true" >> "$GITHUB_OUTPUT"
500+
echo "✅ Changelog already on the staging branch — keeping it, not overwriting with generated output."
501+
else
502+
echo "present=false" >> "$GITHUB_OUTPUT"
503+
fi
504+
424505
- name: Download changelog artifact
425-
if: needs.changelog.outputs.present == 'true'
506+
if: needs.changelog.outputs.present == 'true' && steps.existing.outputs.present == 'false'
426507
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
427508
with:
428509
name: changelog-v${{ needs.promote.outputs.stable_version }}
429510
path: docs/changelogs
430511

512+
# Only when the file came from the artifact. A changelog already on the
513+
# branch was committed by `promote` (preserve path) and needs no second commit.
431514
- name: Commit changelog to staging branch
432-
if: needs.changelog.outputs.present == 'true'
515+
if: needs.changelog.outputs.present == 'true' && steps.existing.outputs.present == 'false'
433516
env:
434517
APP_TOKEN: ${{ steps.app-token.outputs.token }}
435518
STABLE_VERSION: ${{ needs.promote.outputs.stable_version }}
@@ -456,7 +539,10 @@ jobs:
456539
STABLE_VERSION: ${{ needs.promote.outputs.stable_version }}
457540
RC_TAG: ${{ inputs.rc_tag }}
458541
LINE: ${{ needs.promote.outputs.line }}
459-
CHANGELOG_PRESENT: ${{ needs.changelog.outputs.present }}
542+
# True when the PR carries a changelog by EITHER route: preserved on the
543+
# branch, or generated this run. The PR body must reflect what merged,
544+
# not just what the AI produced.
545+
CHANGELOG_PRESENT: ${{ (steps.existing.outputs.present == 'true' || needs.changelog.outputs.present == 'true') && 'true' || 'false' }}
460546
run: |
461547
# Target the maintenance line if it exists, else main.
462548
BASE=main
@@ -483,122 +569,3 @@ jobs:
483569
--title "chore(release): promote ${RC_TAG} -> ${STABLE_TAG}" \
484570
--body "$BODY" --label release --label full-e2e
485571
fi
486-
487-
# Website docs, opened at promote time rather than at stable-tag push, for the
488-
# same reason as the changelog: at tag push the release is already public.
489-
# Different repository, so it cannot ride the promote PR — it can only be
490-
# opened early. pull-requests-release.yaml finalize auto-merges it once the
491-
# release is published, so the site never lags the release.
492-
#
493-
# Depends only on `promote` (it reads the release-X.Y.Z branch that step
494-
# pushed), so it runs concurrently with `changelog` and `open-pr`.
495-
update-website-docs:
496-
name: Update Website Docs
497-
runs-on: ubuntu-latest
498-
needs: [promote]
499-
timeout-minutes: 30
500-
permissions:
501-
contents: read
502-
steps:
503-
- name: Generate GitHub App token
504-
id: app-token
505-
uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1
506-
with:
507-
app-id: ${{ secrets.COZYSTACK_CI_APP_ID }}
508-
private-key: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
509-
owner: cozystack
510-
511-
- name: Checkout website repo
512-
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
513-
with:
514-
repository: cozystack/website
515-
token: ${{ steps.app-token.outputs.token }}
516-
ref: main
517-
518-
# Decide whether this release promotes the `next/` trunk to a new released
519-
# version directory. Per the website repo's contract (see website#495):
520-
# - `make release-next` runs only for new minor/major final releases
521-
# (`vX.Y/` does not yet exist on disk).
522-
# - Patch releases skip this step and let `make update-all` route on
523-
# their own.
524-
- name: Determine if this release promotes next/
525-
id: promote_next
526-
env:
527-
STABLE_TAG: ${{ needs.promote.outputs.stable_tag }}
528-
run: |
529-
if [[ ! "$STABLE_TAG" =~ ^v([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
530-
echo "promote=false" >> "$GITHUB_OUTPUT"
531-
echo "Non-stable tag '$STABLE_TAG' — skipping release-next."
532-
exit 0
533-
fi
534-
MAJOR="${BASH_REMATCH[1]}"
535-
MINOR="${BASH_REMATCH[2]}"
536-
if [[ "$MAJOR" == "0" ]]; then
537-
DOC_VERSION="v0"
538-
else
539-
DOC_VERSION="v${MAJOR}.${MINOR}"
540-
fi
541-
if [[ -d "content/en/docs/${DOC_VERSION}" ]]; then
542-
echo "promote=false" >> "$GITHUB_OUTPUT"
543-
echo "content/en/docs/${DOC_VERSION}/ already exists — patch release, skipping release-next."
544-
else
545-
echo "promote=true" >> "$GITHUB_OUTPUT"
546-
echo "New minor/major release ${DOC_VERSION} — will promote next/ → ${DOC_VERSION}/."
547-
fi
548-
549-
- name: Promote next/ to released version
550-
if: steps.promote_next.outputs.promote == 'true'
551-
env:
552-
GH_TOKEN: ${{ steps.app-token.outputs.token }}
553-
TAG: ${{ needs.promote.outputs.stable_tag }}
554-
run: make release-next RELEASE_TAG="$TAG"
555-
556-
# BRANCH is the release-X.Y.Z staging branch `promote` just pushed — the
557-
# promoted stable tree. At stable-tag push this used to read the same
558-
# branch, so the input is unchanged; only the timing moved earlier.
559-
- name: Update docs from release branch
560-
env:
561-
GH_TOKEN: ${{ steps.app-token.outputs.token }}
562-
TAG: ${{ needs.promote.outputs.stable_tag }}
563-
VERSION: ${{ needs.promote.outputs.stable_version }}
564-
run: make update-all BRANCH="release-$VERSION" RELEASE_TAG="$TAG"
565-
566-
- name: Commit and push
567-
id: commit
568-
env:
569-
VERSION: ${{ needs.promote.outputs.stable_version }}
570-
run: |
571-
git config user.name "cozystack-ci[bot]"
572-
git config user.email "274107086+cozystack-ci[bot]@users.noreply.github.com"
573-
git add content hugo.yaml data/versions
574-
if git diff --cached --quiet; then
575-
echo "No changes to commit"
576-
echo "changed=false" >> "$GITHUB_OUTPUT"
577-
exit 0
578-
fi
579-
BRANCH="update-docs-v${VERSION}"
580-
git branch -D "$BRANCH" 2>/dev/null || true
581-
git checkout -b "$BRANCH"
582-
git commit --signoff -m "[docs] Update managed apps reference for v${VERSION}"
583-
git push --force --set-upstream origin "$BRANCH"
584-
echo "changed=true" >> "$GITHUB_OUTPUT"
585-
586-
- name: Open pull request
587-
if: steps.commit.outputs.changed == 'true'
588-
env:
589-
GH_TOKEN: ${{ steps.app-token.outputs.token }}
590-
VERSION: ${{ needs.promote.outputs.stable_version }}
591-
STABLE_TAG: ${{ needs.promote.outputs.stable_tag }}
592-
run: |
593-
BRANCH="update-docs-v${VERSION}"
594-
pr_state=$(gh pr view "$BRANCH" --repo cozystack/website --json state --jq .state 2>/dev/null || echo "")
595-
if [[ "$pr_state" == "OPEN" ]]; then
596-
echo "PR already open, skipping creation."
597-
else
598-
gh pr create \
599-
--repo cozystack/website \
600-
--title "[docs] Update managed apps reference for v${VERSION}" \
601-
--body "Automated docs update for release \`${STABLE_TAG}\`. Opened at promotion time, before the release is published; pull-requests-release.yaml auto-merges this once \`${STABLE_TAG}\` goes live." \
602-
--head "$BRANCH" \
603-
--base main
604-
fi

0 commit comments

Comments
 (0)