Skip to content

[ENHANCEMENT] Add community-approved label for community code approvals #1872

Closed

Description

@edelauna

Problem (one or two sentences)

Community members can review PRs, but the review workflow gives no signal for their approvals. Maintainers cannot see which PRs in awaiting-maintainer already have a fresh community approval.

Context (who is affected and when)

Maintainers choose which PR to review next from the awaiting-maintainer queue. Community reviewers who approve a PR get no visible result. This discourages community review.

Desired behavior (conceptual, not technical)

Add one advisory label: community-approved. It means: "Fresh community code approval for the current head. Maintainer review still required."

The label gives a priority signal only. It does not grant merge authority. Required checks and maintainer review stay unchanged.

Acceptance criteria

Eligibility. A reviewer qualifies when all of these are true:

  • The reviewer is a non-null human account with a valid login. Ghost accounts do not qualify.
  • The reviewer is not the PR author.
  • The reviewer does not have admin, maintain, or write permission. A permission 404 can qualify an otherwise valid reviewer.
  • The reviewer's latest decisive review is an approval on the current head commit. Comments are neutral. Select reviews by review ID, not array order.

One eligible approval is enough. Bot-authored PRs use the same rule. The existing bot-author routing does not change. This feature adds no CodeRabbit requirement.

Label policy.

Condition Action
Stable head, maintainer queue, qualifying fresh approval Add or retain
Known non-maintainer phase or no qualifying approval Remove
Known head change or invalidated approval Remove, even if another check is uncertain
Unknown mergeability alone, unchanged head, no known invalidation Preserve an existing label, do not add
Community-only lookup uncertainty, no known invalidation Preserve an existing label, do not add
Global failure clears the maintainer queue Remove the label too
Read-only fork review event Make no label changes

Known invalidation takes priority over uncertainty. The presence rule describes the result of a successful reconciliation. It is not an always-true guarantee, because GitHub events and writes are not atomic.

Permission errors.

  • Share permission results between maintainer and community classification.
  • Treat a lookup failure as unknown permission. Never treat it as community eligibility.
  • If unknown permission blocks maintainer classification, keep the existing fail-closed behavior.
  • If only the community label update fails, do not clear the queue label or fail the existing gate.

Proposed approach (one PR)

  1. Add the label definition in .github/workflows/label-pr-review-state.yml. Update the "creates missing workflow labels" test from 4 to 5 calls. Keep the description at 100 characters or fewer.
  2. Keep the label outside stateLabels. Add and remove it in one shared helper that every branch calls.
  3. Validate reviewer accounts. Reuse the existing decisive-review reduction.
  4. Add converted_to_draft to the pull_request_target types.
  5. Compare latestPrDetail.head.sha with pr.head.sha at the existing refetch before adding the label.
  6. Update the managed guide comment with the label meaning and a "do not edit manually" instruction. The guide comment does not list reviewers.
  7. Add one sentence to CONTRIBUTING.md under Review Process, with the testing-evidence format. The PR template does not change.
  8. Add focused tests in src/services/__tests__/pr-review-state-workflow.test.ts:
    • Reviewer eligibility, ghost accounts, and permission 404 handling.
    • Per-user permission failures. This needs a harness change, because permissionErrorStatus is global today.
    • Freshness, dismissal, and replacement reviews. Test multiple reviewers.
    • Head change during reconciliation. This needs a new headShaSequence option, like mergeabilitySequence.
    • Queue transitions and repeated runs.
    • Read-only fork review events.
    • Community change requests do not move a PR to awaiting-author.
    • The advisory gate stays unchanged.

Trade-offs / risks

  • Fork review events are read-only. A label can be missing or stale until a trusted run. A CodeRabbit comment sometimes causes an earlier update. The hourly sweep is the fallback.
  • Community approval is an advisory priority signal. This release does not verify that reviewer accounts represent independent people.
  • Community approval does not mean all reviewers agree. Maintainers must inspect unresolved findings.
  • Community change requests do not create author-action blockers. They do not select PRs for the 14-day author-inactivity rule in stale.yml, which only selects awaiting-author. Community review activity can still affect inactivity timing.
  • The existing hourly sweep reconciles open PRs, so no backfill work is needed.

Out of scope

  • A trusted event relay for faster fork updates
  • Real-LLM smoke-test CI
  • Auto-merge
  • Reviewer permission changes
  • Reputation scores
  • Reviewer-count requirements
  • Commit-author exclusions
  • Reviewer lists in the guide comment
  • Changes to branch protection, CODEOWNERS, publishing checks, or stale closure policy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions