Skip to content

Commit 45523d7

Browse files
fix(release): follow the kind/ backport label rename in backport-audit
The labels this tool queries were namespaced under kind/ in 7b71053, which renamed them in place through label-sync aliases. Every historical PR now reads back as kind/backport or kind/backport-previous, so `gh pr list --label backport` returns 7 of the 290 merged PRs that carry a backport request. That is the worst shape this failure could take. Against release-1.6 and release-1.5 the audit found 2 candidates, judged both already backported, and exited 0 -- a clean bill of health for two branches that between them have 5 MISSING and 7 pending backports. A gate that reports success because it stopped seeing the work is worse than no gate at all. Query both spellings per request and fold them onto the kind/ name, which is also the name the audit reports. The bare names stay because the org-level dosubot still applies them to new PRs; backport.yaml carries the same transitional pair for the same reason, and both retire together. Deliberately not changed: the rule that resolves a label to a release line. backport.yaml dropped getLatestRelease in 6ff5b98 and now takes the two newest existing release-X.Y branches, which opens a line at its first rc rather than at its first stable. The two rules agree outside a freeze window and no window has opened since that commit, so every PR audited so far merged under the rule reproduced here. The divergence is recorded in the README and on lineOpenDates; the next rc cut is what makes it matter. After the fix: 290 candidates scanned, release-1.6 3 MISSING / 4 pending, release-1.5 2 MISSING / 3 pending / 5 dropped, exit 1. Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 52e8070 commit 45523d7

4 files changed

Lines changed: 89 additions & 50 deletions

File tree

‎cmd/backport-audit/README.md‎

Lines changed: 21 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,17 @@
22

33
Answers one question, before a release or an rc is cut: did everything labelled for backport actually land on the release branch? It prints the URLs of the ones that did not, and exits non-zero, so it can gate the cut.
44

5-
The manual alternative is `gh pr list --search "is:merged label:backport"`, which lists every PR ever labelled and says nothing about whether the change reached the branch. Answering that by hand means reading the branch history per PR, which is where backports quietly go missing.
5+
The manual alternative is `gh pr list --search "is:merged label:kind/backport"`, which lists every PR ever labelled and says nothing about whether the change reached the branch. Answering that by hand means reading the branch history per PR, which is where backports quietly go missing.
66

77
```console
88
$ go run ./cmd/backport-audit release-1.5
99

10-
=== release-1.5 === 33 candidate PRs: MISSING=4 pending=9 dropped=3 backported=17
10+
=== release-1.5 === 43 candidate PRs: MISSING=2 pending=3 dropped=5 backported=33
1111

12-
MISSING -- labelled for this line, no trace of it here (4):
13-
https://github.com/cozystack/cozystack/pull/2936
14-
#2936 fix(kubernetes): make the default md0 node group removable
15-
label=backport author=myasnikovdaniil merged=2026-06-30 -- no backport PR, nothing on branch
12+
MISSING -- labelled for this line, no trace of it here (2):
13+
https://github.com/cozystack/cozystack/pull/3449
14+
#3449 ci(release): move release validation to rc time — e2e lane, changelog button, promote gate, docs at promote
15+
label=kind/backport-previous author=myasnikovdaniil merged=2026-07-29 -- no backport PR, nothing on branch
1616
```
1717

1818
The URL is always the **original PR on main** — the thing you decide about. The evidence line names the backport PR when one exists. Anything already on the branch is counted in the header and never listed, so silence means done.
@@ -61,9 +61,13 @@ go run ./cmd/backport-audit release-1.5 || echo "do not cut yet"
6161

6262
## How a PR is matched to a release line
6363

64-
A `backport` label does not name a branch. [`backport.yaml`](../../.github/workflows/backport.yaml) resolves the target from `getLatestRelease` **at merge time**, so the same label means `release-1.5` on a PR merged in June and `release-1.6` on one merged in August.
64+
A `kind/backport` label does not name a branch. [`backport.yaml`](../../.github/workflows/backport.yaml) resolves the target from `getLatestRelease` **at merge time**, so the same label means `release-1.5` on a PR merged in June and `release-1.6` on one merged in August.
6565

66-
The audit reproduces that rule rather than trusting the label text: a PR is a candidate for `release-X.Y` when it merged carrying `backport` while `X.Y` was the current line, or `backport-previous` while `X.Y` was one minor behind. A line is treated as current from the publication of its first non-prerelease release, which is what `getLatestRelease` returns. Without this, auditing an older line reports every newer-era `backport` PR as missing.
66+
The audit reproduces that rule rather than trusting the label text: a PR is a candidate for `release-X.Y` when it merged carrying `kind/backport` while `X.Y` was the current line, or `kind/backport-previous` while `X.Y` was one minor behind. A line is treated as current from the publication of its first non-prerelease release, which is what `getLatestRelease` returns. Without this, auditing an older line reports every newer-era `kind/backport` PR as missing.
67+
68+
The labels were namespaced under `kind/` in 7b71053a0, which renamed them in place via label-sync aliases, so every historical PR reads back under the new name. The bare `backport` and `backport-previous` are still queried alongside them, because the org-level dosubot goes on applying the old spelling to new PRs — the same transitional allowance [`backport.yaml`](../../.github/workflows/backport.yaml) makes, and it retires at the same time. A PR found under either spelling is reported under the `kind/` one.
69+
70+
One divergence is open and deliberate. `backport.yaml` stopped calling `getLatestRelease` in 6ff5b98d5 and now takes the two newest existing `release-X.Y` branches, which opens a line at its first rc rather than at its first stable. The two rules agree outside a freeze window, and no window has opened since that commit, so every PR audited so far merged under the rule reproduced here — the next rc cut is what makes them disagree.
6771

6872
## How landing is established
6973

@@ -92,18 +96,18 @@ Output is an object keyed by branch, each holding one record per candidate:
9296

9397
```json
9498
{
95-
"number": 3372,
96-
"title": "fix(keycloak-configure): patch HelmRelease in release namespace on teardown",
97-
"url": "https://github.com/cozystack/cozystack/pull/3372",
98-
"author": "lexfrei",
99-
"mergedAt": "2026-07-28T15:45:47Z",
100-
"label": "backport",
101-
"backport_prs": [{"number": 3478, "state": "OPEN", "url": "..."}],
99+
"number": 3963,
100+
"title": "fix(clickhouse): scheme the backup S3_ENDPOINT on the system-bucket flow",
101+
"url": "https://github.com/cozystack/cozystack/pull/3963",
102+
"author": "androndo",
103+
"mergedAt": "2026-08-28T11:05:15Z",
104+
"label": "kind/backport",
105+
"backport_prs": [{"number": 3987, "state": "OPEN", "url": "..."}],
102106
"status": "pending",
103-
"evidence": "backport PR #3478 open"
107+
"evidence": "backport PR #3987 open"
104108
}
105109
```
106110

107111
## Limits
108112

109-
A `MISSING` verdict is a prompt to check, not proof of absence. A hand-backport that was squash-merged, reworded, and referenced no original PR is invisible to every evidence layer and reads as `MISSING`; confirm at diff level before redoing the work. A `backport` label added long after merge, once the release line has moved on, resolves to the newer line. And the audit reports *that* something is missing, never *why* — a run of `MISSING` entries clustered in time usually means the bot itself was failing during that window, which is worth checking before cherry-picking them one by one.
113+
A `MISSING` verdict is a prompt to check, not proof of absence. A hand-backport that was squash-merged, reworded, and referenced no original PR is invisible to every evidence layer and reads as `MISSING`; confirm at diff level before redoing the work. A `kind/backport` label added long after merge, once the release line has moved on, resolves to the newer line. And the audit reports *that* something is missing, never *why* — a run of `MISSING` entries clustered in time usually means the bot itself was failing during that window, which is worth checking before cherry-picking them one by one.

‎cmd/backport-audit/main.go‎

Lines changed: 66 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,12 @@ limitations under the License.
2020
// from PATH) and writes nothing anywhere.
2121
//
2222
// A PR is a candidate for release-X.Y when it merged to main carrying the
23-
// `backport` label while X.Y was the current release line, or the
24-
// `backport-previous` label while X.Y was one minor behind it. That mirrors
25-
// .github/workflows/backport.yaml, which resolves its target branches from the
26-
// latest published release at merge time rather than from the label itself, so
27-
// the same label means a different branch depending on when the PR merged.
23+
// `kind/backport` label while X.Y was the current release line, or the
24+
// `kind/backport-previous` label while X.Y was one minor behind it. That
25+
// mirrors .github/workflows/backport.yaml, which resolves its target branches
26+
// from the latest published release at merge time rather than from the label
27+
// itself, so the same label means a different branch depending on when the PR
28+
// merged.
2829
//
2930
// Landing is established from three independent kinds of evidence: the PR's
3031
// merge commit already being reachable from the release branch (it merged
@@ -38,7 +39,7 @@ limitations under the License.
3839
//
3940
// Known limits: a hand-backport that is squash-merged, rewords its subject and
4041
// references no original PR is unprovable from either side and reads as
41-
// MISSING; and a `backport` label added after the release line moved on
42+
// MISSING; and a `kind/backport` label added after the release line moved on
4243
// resolves to the newer line.
4344
package main
4445

@@ -82,8 +83,29 @@ var (
8283
releaseTagRE = regexp.MustCompile(`^v(\d+)\.(\d+)\.(\d+)$`)
8384
)
8485

85-
// backportLabels are the two labels .github/workflows/backport.yaml acts on.
86-
var backportLabels = []string{"backport", "backport-previous"}
86+
// The two backport requests .github/workflows/backport.yaml acts on, named by
87+
// the label that carries them. These names are what the audit reports.
88+
const (
89+
labelCurrent = "kind/backport"
90+
labelPrevious = "kind/backport-previous"
91+
)
92+
93+
// backportLabels maps each request to every label spelling that expresses it.
94+
//
95+
// TRANSITIONAL, matching the same allowance in backport.yaml: the labels were
96+
// namespaced under kind/ in 7b71053a0, carrying the bare names as label-sync
97+
// aliases so the rename applied in place. Historical PRs therefore read back as
98+
// `kind/backport` today, but the org-level dosubot still applies the bare names
99+
// on new PRs, so a candidate can carry either. Both are queried and folded onto
100+
// the canonical name above. When dosubot's PR labelling is switched off, the
101+
// legacy spellings here and the ones in backport.yaml can be dropped together.
102+
var backportLabels = []struct {
103+
canonical string
104+
spellings []string
105+
}{
106+
{labelCurrent, []string{labelCurrent, "backport"}},
107+
{labelPrevious, []string{labelPrevious, "backport-previous"}},
108+
}
87109

88110
// Statuses, most to least in need of attention. outstanding is the prefix of
89111
// this order that makes the audit exit non-zero.
@@ -424,8 +446,16 @@ type lineOpen struct {
424446

425447
// lineOpenDates reports when each release line became the current one, oldest
426448
// first. A line opens when its first non-prerelease release is published,
427-
// which is when getLatestRelease in backport.yaml starts resolving `backport`
428-
// to that line. Drafts and prereleases are skipped, as that API does.
449+
// which is when getLatestRelease in backport.yaml starts resolving
450+
// `kind/backport` to that line. Drafts and prereleases are skipped, as that
451+
// API does.
452+
//
453+
// backport.yaml stopped calling getLatestRelease in 6ff5b98d5 and now takes the
454+
// two newest existing release-X.Y branches instead, which opens a line at its
455+
// first rc rather than at its first stable. The two rules agree everywhere
456+
// except inside a freeze window, and no window has opened since that commit
457+
// landed, so every PR audited so far merged under the rule reproduced here.
458+
// The next rc cut is what makes them diverge.
429459
func lineOpenDates() ([]lineOpen, error) {
430460
var releases []struct {
431461
TagName string `json:"tagName"`
@@ -464,7 +494,8 @@ func lineOpenDates() ([]lineOpen, error) {
464494
}
465495

466496
// targetsAt reports the current and previous release lines as of when, which is
467-
// what the `backport` and `backport-previous` labels meant at that moment.
497+
// what the `kind/backport` and `kind/backport-previous` labels meant at that
498+
// moment.
468499
func targetsAt(opened []lineOpen, when time.Time) (current, previous *line) {
469500
var live []line
470501
for _, o := range opened {
@@ -481,25 +512,29 @@ func targetsAt(opened []lineOpen, when time.Time) (current, previous *line) {
481512
return current, previous
482513
}
483514

484-
// candidates returns the merged-to-main PRs carrying a backport label.
515+
// candidates returns the merged-to-main PRs carrying a backport label, keyed by
516+
// PR number so a PR matched under two spellings of the same request, or under
517+
// both requests, is one entry carrying both canonical names.
485518
func candidates(limit int) (map[int]*mainPR, error) {
486519
found := map[int]*mainPR{}
487-
for _, label := range backportLabels {
488-
var prs []mainPR
489-
if err := ghJSON(&prs, "pr", "list", "--base", "main", "--state", "merged",
490-
"--label", label, "--limit", strconv.Itoa(limit),
491-
"--json", "number,title,url,mergedAt,mergeCommit,labels,author"); err != nil {
492-
return nil, err
493-
}
494-
for i := range prs {
495-
pr := prs[i]
496-
existing, ok := found[pr.Number]
497-
if !ok {
498-
pr.labels = map[string]bool{}
499-
found[pr.Number] = &pr
500-
existing = &pr
520+
for _, req := range backportLabels {
521+
for _, spelling := range req.spellings {
522+
var prs []mainPR
523+
if err := ghJSON(&prs, "pr", "list", "--base", "main", "--state", "merged",
524+
"--label", spelling, "--limit", strconv.Itoa(limit),
525+
"--json", "number,title,url,mergedAt,mergeCommit,labels,author"); err != nil {
526+
return nil, err
527+
}
528+
for i := range prs {
529+
pr := prs[i]
530+
existing, ok := found[pr.Number]
531+
if !ok {
532+
pr.labels = map[string]bool{}
533+
found[pr.Number] = &pr
534+
existing = &pr
535+
}
536+
existing.labels[req.canonical] = true
501537
}
502-
existing.labels[label] = true
503538
}
504539
}
505540
return found, nil
@@ -685,10 +720,10 @@ func (cfg *config) audit(branch string, cands map[int]*mainPR, opened []lineOpen
685720
current, previous := targetsAt(opened, pr.MergedAt)
686721
label := ""
687722
switch {
688-
case pr.labels["backport"] && current != nil && *current == want:
689-
label = "backport"
690-
case pr.labels["backport-previous"] && previous != nil && *previous == want:
691-
label = "backport-previous"
723+
case pr.labels[labelCurrent] && current != nil && *current == want:
724+
label = labelCurrent
725+
case pr.labels[labelPrevious] && previous != nil && *previous == want:
726+
label = labelPrevious
692727
default:
693728
continue
694729
}

‎docs/agents/releasing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,5 @@ The release process is promotion-based — a stable release is a renamed release
2727

2828
All detailed steps, workflows, and failure modes are documented in `docs/release.md`.
2929

30-
For a patch release, or any rc cut from a `release-X.Y` branch, verify the backport state of that branch before tagging: `go run ./cmd/backport-audit release-X.Y` exits non-zero while anything labeled `backport` / `backport-previous` for the line has not landed on it, and prints the URLs. See [`cmd/backport-audit/README.md`](../../cmd/backport-audit/README.md) and [Cherry-pick triage before a patch](../release.md#cherry-pick-triage-before-a-patch). Do not report a branch as ready to cut on the strength of `gh pr list --search "is:merged label:backport"` — that lists what was labeled, not what arrived.
30+
For a patch release, or any rc cut from a `release-X.Y` branch, verify the backport state of that branch before tagging: `go run ./cmd/backport-audit release-X.Y` exits non-zero while anything labeled `kind/backport` / `kind/backport-previous` for the line has not landed on it, and prints the URLs. See [`cmd/backport-audit/README.md`](../../cmd/backport-audit/README.md) and [Cherry-pick triage before a patch](../release.md#cherry-pick-triage-before-a-patch). Do not report a branch as ready to cut on the strength of `gh pr list --search "is:merged label:kind/backport"` — that lists what was labeled, not what arrived.
3131

‎docs/release.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -401,7 +401,7 @@ git log <base>..origin/main --grep="(#" --oneline
401401
gh pr list --state open --base main --label kind/bug
402402
```
403403

404-
Step 1 covers the labeled work: [`backport-audit`](../cmd/backport-audit/README.md) resolves each `backport` / `backport-previous` PR to the line the label meant **at merge time** (the same `getLatestRelease` rule the bot applies, not the label text), then reports each one as already on the branch, `pending` on an open backport PR, `dropped` with the reason someone recorded, or `MISSING`. Listing the labeled PRs with `gh pr list --search "is:merged label:backport"` is not a substitute — it says nothing about whether the change reached the branch, which is the only thing that matters at this point in the cut. Steps 2 and 3 remain necessary because they cover what step 1 structurally cannot see: changes that were never labeled at all.
404+
Step 1 covers the labeled work: [`backport-audit`](../cmd/backport-audit/README.md) resolves each `kind/backport` / `kind/backport-previous` PR to the line the label meant **at merge time**, not from the label text, then reports each one as already on the branch, `pending` on an open backport PR, `dropped` with the reason someone recorded, or `MISSING`. It takes that line from the newest stable published at merge time — the `getLatestRelease` rule the bot applied until the branch enumeration described in [The backport bot](#the-backport-bot) replaced it — so the two agree except inside a freeze window, where the audit is one line behind the bot. Listing the labeled PRs with `gh pr list --search "is:merged label:kind/backport"` is not a substitute — it says nothing about whether the change reached the branch, which is the only thing that matters at this point in the cut. Steps 2 and 3 remain necessary because they cover what step 1 structurally cannot see: changes that were never labeled at all.
405405

406406
**Include rule:**
407407

0 commit comments

Comments
 (0)