testing: pin the label-writing workflows with contract tests #3589

Two workflows decide which labels land on issues and PRs, and neither has a test behind it. .github/workflows/pr-labeler.yaml holds a type-to-label table and a scope-to-area table, 25 label names in all, written inline in a github-script block. .github/workflows/stale.yaml holds the policy that decides what gets auto-closed: two exempt lists, the day counts, the per-run write cap. grep for either file across hack/ returns nothing. Both are hand-edited, and both fail quietly when they are wrong. A mistyped label name is not an error, it is a rule that stops matching, and nobody notices until an issue that should have been exempt gets closed.

Two live examples of that drift, both harmless today, both the kind of thing a test catches the moment it appears. docs/agents/contributing.md:73 says pr-labeler.yaml parses the PR title on opened, edited, reopened and synchronize; the workflow subscribes to [opened, reopened, synchronize] at .github/workflows/pr-labeler.yaml:9 and deliberately leaves edited out, with the reason in the comment above it. Separately, the comment at pr-labeler.yaml:25-27 lists the accepted Conventional Commits types as feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert and cites docs/agents/contributing.md as the source, but that file lists ten types at :54 and revert is not one of them. Each side is internally reasonable and they disagree.

I did check the labels themselves and they are fine right now, which is the point rather than a reason to skip this. All 12 names in stale.yaml's exempt-issue-labels, all 4 in exempt-pr-labels, and all 25 that pr-labeler.yaml can apply resolve against a name: in .github/labels.yml. Nothing keeps that true through the next edit, and when it stops being true the result is an exemption that protects nothing.

The pattern to copy is already in the tree, in the same directory, run by the same target. hack/promote-gate-contract.bats and hack/release-freeze-contract.bats pin workflow lines for this exact reason, and both open by stating the rule that makes such a test worth having: pin executable and structural lines, never prose, so a gate demoted to a comment cannot satisfy the contract. Both ship a code_lines helper that strips YAML comments before matching, and release-freeze-contract.bats adds a script_lines helper for the // comments inside a github-script block, which is exactly the shape pr-labeler.yaml is written in. hack/downstream-trigger-map.bats covers the neighbouring case of a documentation table that has to keep matching the tree. #3574 is open and adds hack/issue-triage-contract.bats alongside a new labeling workflow; whether or not it lands first, it is the closest example of what is being asked for here.

Wiring is free. Makefile:161 collects the suite with BATS_UNIT_FILES := $(filter-out hack/e2e-%.bats,$(wildcard hack/*.bats)), so a new hack/*.bats file whose name does not start with e2e- is picked up by make unit-tests with no further change.

Acceptance criteria I would suggest, as concrete assertions rather than "add coverage":

• Every kind/* and area/* label pr-labeler.yaml can apply exists as a name: in .github/labels.yml.
• Every label in stale.yaml's exempt-issue-labels and exempt-pr-labels, and each of stale-issue-label, stale-pr-label, close-issue-label and close-pr-label, exists in .github/labels.yml.
• The type-to-label rows and the scope-to-area rows in docs/agents/contributing.md agree with typeToKind and scopeToArea in the workflow, and the type list at docs/agents/contributing.md:54 agrees with the type list in the workflow comment. Fixing the revert discrepancy is part of the change.
• The event list docs/agents/contributing.md:73 claims matches the workflow's on.pull_request_target.types. Fixing the edited discrepancy is part of the change.
• stale.yaml's days-before-stale, days-before-close and operations-per-run hold their current values, so changing the policy becomes a deliberate edit to the test.
• The area/uncategorized fallback at pr-labeler.yaml:212-216 and the additive-only behaviour at :218-219 are both still there, since a labeler that starts removing labels is a different tool.
• Every assertion is verified to actually guard: break the workflow line the test pins, watch it go red, restore it. A pin that passes against a broken input is worse than no pin.

Comment stripping matters for all of the above, and the two existing contract files show how. A mapping demoted to a comment must not satisfy an assertion, and pr-labeler.yaml mixes YAML # comments with JavaScript // comments in one file, so both filters are needed.

The work needs no cluster and no credentials. make unit-tests runs the whole suite locally and wants bash, bats and python3. I can answer questions in the thread.

What I checked:

• .github/workflows/pr-labeler.yaml: :5-9 (the edited omission and the subscribed types), :25-27 (the type list comment citing docs/agents/contributing.md), :28-35 (typeToKind), :39-150 (scopeToArea), :212-216 (the area/uncategorized fallback), :218-219 (additive only).
• .github/workflows/stale.yaml: :25-28 (the four label inputs), :34-39 (exempt-issue-labels, 12 entries), :40-42 (exempt-pr-labels, 4 entries), :43-45 (days-before-stale: 60, days-before-close: 14, operations-per-run: 100).
• docs/agents/contributing.md:54 (the ten-type list) and :73 (the claimed event list, including edited).
• .github/labels.yml, 77 label names, cross-checked against all 25 labels pr-labeler.yaml can apply and all 16 in stale.yaml's exempt lists, with zero misses today.
• grep -rln 'pr-labeler\|stale.yaml\|scopeToArea\|exempt-issue-labels' hack/ docs/ Makefile .github/ returns only the two workflows themselves plus documentation and changelog files, so no test references either.
• hack/promote-gate-contract.bats (the header comment on pinning executable lines, plus job_block, job_header, input_block, code_lines), hack/release-freeze-contract.bats (same plus script_lines), hack/downstream-trigger-map.bats.
• Makefile:161 (BATS_UNIT_FILES) and the bats-unit-tests recipe below it.
• feat(ci): auto-apply triage labels to issues #3574, open, adding .github/workflows/issue-triage.yaml and hack/issue-triage-contract.bats. That file is not on main yet.