Skip to content

Commit 2420d89

Browse files
fix(ci): generate changelog from the release tag, not from main
The generate-changelog job in .github/workflows/tags.yaml previously checked out main and ran the AI agent there. The agent followed docs/agents/changelog.md, which instructed it to compute the release range with `git log <previous_version>..HEAD`. For minor releases (vX.Y.0) cut from main this happens to give the right answer; for patch releases (vX.Y.Z, Z>0) cut from a release-X.Y branch, HEAD-on- main is a strict superset of the release tag and contains commits that were merged to main both before and after the tag. The v1.3.1 changelog generated by this workflow (#2480) demonstrated the failure mode: it included 8 PRs merged to main but never shipped in v1.3.1, 6 backport PRs that landed on release-1.3 *after* v1.3.1 was tagged, both originals and their backports as separate entries, a hallucinated 2024 PR, and the cozystack-ci bot in the contributors list. The corrected v1.3.1 changelog contains the single fix that v1.3.1 actually ships. This PR fixes the root cause and tightens the agent guardrails: * tags.yaml: check out the release tag commit (`ref: ${{ steps.tag. outputs.tag }}`) instead of `main`, so HEAD == the release commit and `git log v<prev>..HEAD` corresponds to the actual release. The Create-changelog-branch step still creates the PR branch from origin/main, so PRs continue to merge cleanly into main. * tags.yaml: update the AI prompt to state explicitly that HEAD is the release commit and that the upper bound for the release range is the new tag, not main. * docs/agents/changelog.md: replace `..HEAD` with `..v<new_version>` in every example so the instruction is unambiguous regardless of which branch is checked out. * docs/agents/changelog.md: add a hard rule that backport PRs MUST be merged into the original entry (single line, two PR numbers), never listed as a second entry. Document the edge case where the original PR is outside the range — drop the entry. * docs/agents/changelog.md: forbid using the brief description verbatim as the detailed description, and forbid inventing entries for PRs that are not in the release range. * docs/agents/changelog.md: filter bot/CI accounts (`app/*`, `*[bot]`, cozystack-ci, github-actions, dependabot, renovate) out of the human Contributors list. Bot attribution on individual entries is still valid. The corresponding correction for the v1.3.1 changelog itself is in PR #2480 on the changelog-v1.3.1 branch. Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 3977aef commit 2420d89

2 files changed

Lines changed: 31 additions & 23 deletions

File tree

‎.github/workflows/tags.yaml‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -286,10 +286,16 @@ jobs:
286286
core.setOutput('version', version);
287287
core.setOutput('tag', ref);
288288
289-
- name: Checkout main branch
289+
# Check out the tag commit, not main. Patch releases are cut from
290+
# release-X.Y branches, so HEAD-on-main contains commits that are
291+
# NOT in the release tag. Running the AI against the tag commit
292+
# makes `git log v<prev>..HEAD` correspond to the actual release
293+
# contents. The Create changelog branch step below resets the file
294+
# onto a fresh branch from origin/main for the PR.
295+
- name: Checkout release tag
290296
uses: actions/checkout@v4
291297
with:
292-
ref: main
298+
ref: ${{ steps.tag.outputs.tag }}
293299
fetch-depth: 0
294300
fetch-tags: true
295301
token: ${{ steps.app-token.outputs.token }}
@@ -325,7 +331,7 @@ jobs:
325331
VERSION: ${{ steps.tag.outputs.version }}
326332
run: |
327333
copilot \
328-
--prompt "Generate the release changelog for tag v${VERSION}. Follow the instructions in @docs/agents/changelog.md exactly, including the 'Scope and boundaries' section at the top. Your deliverable is the single file docs/changelogs/v${VERSION}.md — write it and exit; this workflow handles branching, committing, pushing, and opening the PR." \
334+
--prompt "Generate the release changelog for tag v${VERSION}. The repository is checked out at the v${VERSION} tag commit, so HEAD == the release commit. Use v${VERSION} (or HEAD) as the upper bound when comparing against the previous version — never assume HEAD is on main, because patch releases are cut from release-X.Y branches. Follow the instructions in @docs/agents/changelog.md exactly, including the 'Scope and boundaries' section at the top. Your deliverable is the single file docs/changelogs/v${VERSION}.md — write it and exit; this workflow handles branching, committing, pushing, and opening the PR." \
329335
--allow-all-tools --allow-all-paths < /dev/null
330336
331337
- name: Create changelog branch and commit

‎docs/agents/changelog.md‎

Lines changed: 22 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -103,40 +103,42 @@ git branch --show-current
103103

104104
**Important**: Determine if you're generating a changelog for a **minor release** (vX.Y.0) or a **patch release** (vX.Y.Z where Z > 0).
105105

106+
**⚠️ CRITICAL — use the tag, not HEAD:** Always compare the previous version against the **tag for the release you are generating** (e.g. `v1.3.1`), never against `HEAD`. Patch releases are cut from `release-X.Y` branches, so when CI checks out the tag and runs `git log v<prev>..HEAD` you might get the right answer — but if the tag is missing locally, or if a developer runs this on `main`, `HEAD` will contain commits merged to main that are NOT in the release. Using the tag explicitly removes this footgun.
107+
106108
**For patch releases (vX.Y.Z where Z > 0):**
107-
Get the list of commits starting from the previous patch version to HEAD:
109+
Get the list of commits between the previous patch tag and the new tag:
108110

109111
**⚠️ CRITICAL: Do NOT use --first-parent flag! It will skip merge commits including backports!**
110112

111113
```bash
112-
# Get all commits including merge commits (backports)
113-
git log <previous_version>..HEAD --pretty=format:"%h - %s (%an, %ar)"
114+
# Get all commits including merge commits (backports), bounded by the new tag
115+
git log <previous_version>..v<new_version> --pretty=format:"%h - %s (%an, %ar)"
114116
```
115117

116118
For example, if generating changelog for `v0.37.2`:
117119
```bash
118-
git log v0.37.1..HEAD --pretty=format:"%h - %s (%an, %ar)"
120+
git log v0.37.1..v0.37.2 --pretty=format:"%h - %s (%an, %ar)"
119121
```
120122

121123
**⚠️ IMPORTANT: Check for backports:**
122124
- Look for commits with "[Backport release-X.Y]" in the commit message
123125
- For backport PRs, find the original PR number mentioned in the backport commit message or PR description
124126
- Use the original PR author (not the backport PR author) when creating changelog entries
125-
- Include both the original PR number and backport PR number in the changelog entry (e.g., `#1606, #1609`)
127+
- **MUST** combine the original and backport into a **single entry** with both PR numbers (e.g. `#1606, backport #1609`). Do NOT list the original and the backport as two separate entries — that produces duplicates in the changelog.
126128

127129
**For minor releases (vX.Y.0):**
128-
Minor releases must include **all changes** from patch releases of the previous minor version. Get commits from the previous minor release:
130+
Minor releases must include **all changes** from patch releases of the previous minor version. Get commits from the previous minor release tag up to the new tag:
129131

130132
**⚠️ CRITICAL: Do NOT use --first-parent flag! It will skip merge commits including backports!**
131133

132134
```bash
133135
# For v0.38.0, get all commits since v0.37.0 (including all patch releases v0.37.1, v0.37.2, etc.)
134-
git log v<previous_minor_version>..HEAD --pretty=format:"%h - %s (%an, %ar)"
136+
git log v<previous_minor_version>..v<new_version> --pretty=format:"%h - %s (%an, %ar)"
135137
```
136138

137139
For example, if generating changelog for `v0.38.0`:
138140
```bash
139-
git log v0.37.0..HEAD --pretty=format:"%h - %s (%an, %ar)"
141+
git log v0.37.0..v0.38.0 --pretty=format:"%h - %s (%an, %ar)"
140142
```
141143

142144
This will include all commits from v0.37.1, v0.37.2, v0.37.3, etc., up to v0.38.0.
@@ -279,8 +281,9 @@ git log <previous_version>..<new_version> --format="%s%n%b" | grep -oE '#[0-9]+'
279281
- Backport PRs have format: `[Backport release-X.Y] <original title> (#BACKPORT_PR_NUMBER)`
280282
- The backport commit message or PR description usually mentions the original PR number
281283
- For backport entries in changelog, use the original PR author (not the backport PR author)
282-
- Include both original and backport PR numbers in the changelog entry (e.g., `#1606, #1609`)
284+
- **MUST** combine the original and backport PR into a **single changelog entry** with both PR numbers (e.g. `#1606, backport #1609`). NEVER list the original and the backport as two separate entries — that produces visible duplicates and was the primary defect that motivated this rule.
283285
- To find original PR from backport: Check the backport PR description or commit message for "Backport of #ORIGINAL_PR"
286+
- If a release range contains a backport but its original PR is **not** in the same range (because the original was merged earlier and already shipped in a previous release), drop the entry entirely — it has already been changelog'd.
284287

285288
**For each PR number, get the author:**
286289

@@ -437,6 +440,8 @@ Create a new changelog file in the format matching previous versions:
437440

438441
3. **Entry format:**
439442
- Use the format: `* **Brief description**: detailed description ([**@username**](https://github.com/username) in #PR_NUMBER)`
443+
- **⚠️ The brief description and the detailed description MUST NOT be the same string.** The brief description is typically the conventional-commit subject (e.g. `fix(api): drop legacy field`); the detailed description is a one-or-two-sentence explanation of what the change means for users — what is fixed, what is now possible, what behavior changed. If you find yourself writing `* **fix(foo): X**: fix(foo): X (...)` you have not done the second half of the work — go read the PR body or commit diff and write the user-facing description.
444+
- **⚠️ Do NOT invent entries.** Every entry must correspond to a commit in the `git log v<previous>..v<new>` range. If you can't find a commit for an entry you wrote, drop the entry. PR numbers from outside the release range (e.g. PRs from years ago, or PRs that were merged after the tag commit) MUST NOT appear in the changelog.
440445
- **CRITICAL - Get authorship correctly**:
441446
- **ALWAYS use PR author, not commit author**: Extract PR number from commit message, then use `gh pr view` to get the PR author. The commit author (especially for squash/merge commits) is usually the person who merged the PR (or GitHub bot), NOT the person who wrote the code.
442447
```bash
@@ -494,20 +499,17 @@ Create a new changelog file in the format matching previous versions:
494499
Since you've already generated the changelog with all PR authors correctly identified, simply extract GitHub usernames from the changelog entries:
495500
496501
```bash
497-
# Extract all GitHub usernames from the current release changelog
498-
# This method is simpler and more reliable than extracting from git history
499-
500-
# For patch releases: extract from the current changelog file
501-
grep -oE '\[@[a-zA-Z0-9_-]+\]' docs/changelogs/v<version>.md | \
502-
sed 's/\[@/@/' | sed 's/\]//' | \
503-
sort -u
504-
505-
# For minor releases: extract from the current changelog file
506-
grep -oE '\[@[a-zA-Z0-9_-]+\]' docs/changelogs/v<version>.md | \
507-
sed 's/\[@/@/' | sed 's/\]//' | \
502+
# Extract all GitHub usernames from the current release changelog,
503+
# filtering out CI/bot accounts that should not appear as human contributors.
504+
# This method is simpler and more reliable than extracting from git history.
505+
grep -oE '\[\*\*@[a-zA-Z0-9_/-]+\*\*\]' docs/changelogs/v<version>.md | \
506+
sed -E 's|\[\*\*@||; s|\*\*\]||' | \
507+
grep -viE '^(app/|.*\[bot\]$|cozystack-ci$|github-actions$|dependabot$|renovate$)' | \
508508
sort -u
509509
```
510510
511+
**⚠️ MANDATORY: Exclude bot/CI accounts from the human Contributors list.** Accounts whose login starts with `app/`, ends with `[bot]`, or matches `cozystack-ci`, `github-actions`, `dependabot`, or `renovate` represent automation, not human contributions, and must NOT appear under "Contributors". They may legitimately appear in **entry attribution** (e.g. a Renovate PR is attributed to `@app/renovate`) — but never in the contributors list.
512+
511513
**Get all previous contributors (to identify new ones):**
512514
```bash
513515
# Extract GitHub usernames from all previous changelogs

0 commit comments

Comments
 (0)