Skip to content

fix(review-pr): inline mention-reply correctness; bypass draft check for user-requested reviews#183

Merged
derekmisler merged 4 commits into
docker:mainfrom
docker-agent:fix/remove-gates-for-user-requests
May 6, 2026
Merged

fix(review-pr): inline mention-reply correctness; bypass draft check for user-requested reviews#183
derekmisler merged 4 commits into
docker:mainfrom
docker-agent:fix/remove-gates-for-user-requests

Conversation

@docker-agent

@docker-agent docker-agent commented May 6, 2026

Copy link
Copy Markdown
Contributor

Two related fixes to the @docker-agent reviewer pipeline.

1. Inline mention-reply correctness

When @docker-agent is mentioned on an inline review comment (e.g. asking about a single line of YAML), the reply now lands in the same inline thread with in_reply_to set, instead of as an unrelated PR-level Issues comment with no awareness of the file/line. Reference: docker/gordon#544.

File Change
src/mention-reply/index.ts parseEventContext captures path / line / original_line / diff_hunk / in_reply_to_id into EventContext.inline. buildContextPrompt emits an [INLINE COMMENT CONTEXT] block only when present.
review-pr/agents/pr-review-mention-reply.yaml Branched instruction: post via pulls/comments with in_reply_to when the inline block is present, else stay on the Issues API. Allow-list adds repos/*/pulls/*/comments*. Both branches use jq -n --arg.
src/post-comment/index.ts New postReviewCommentReply helper. The non-org-member rejection now also lands inline on inline triggers.

comment.line falls back to original_line for multi-line comments (matching the existing reply-to-feedback pattern).

2. Bypass draft check for user-requested reviews

/review, review_requested of docker-agent, explicit pr-number input, and trigger-pattern dispatches now run on draft PRs. Automatic pull_request opened/synchronize reviews still skip drafts. Org-membership gates and authorization checks are unchanged.

Single-file diff in .github/workflows/review-pr.yml: adds a Classify trigger type step and gates Check if PR is draft on user_requested != 'true'.

Tests

  • pnpm test → 128 passing
  • pnpm lint (biome + tsc + actionlint) → clean
  • New tests: [INLINE COMMENT CONTEXT] round-trip, rejection routing for inline triggers, postReviewCommentReply helper, null-line fallback.

Changes to .github/workflows/review-pr.yml:

review job:
- Add 'Classify trigger type' step that sets user_requested=true for
  explicit invocations (issue_comment /review, pull_request
  review_requested, explicit pr-number input, resolve-context trigger
  path) and user_requested=false for automatic pull_request events.
- 'Check if PR is draft' now skips for user-requested triggers
  (condition adds: && steps.trigger-type.outputs.user_requested != 'true').
  Draft check is only relevant for automatic reviews.
- Remove 'Check if org member' step entirely — org membership is no
  longer a gate for any review path.
- 'Setup credentials', 'Create check run', 'Checkout PR head', and
  'Run PR Review' conditions updated from:
    steps.membership.outputs.is_member == 'true' &&
    steps.command.outputs.is_review != 'false' &&
    steps.draft.outputs.skip != 'true'
  to:
    steps.command.outputs.is_review != 'false' &&
    (steps.trigger-type.outputs.user_requested == 'true' ||
     steps.draft.outputs.skip != 'true')
  This keeps the draft gate for automatic reviews while bypassing ALL
  gates for user-requested reviews.

reply-to-feedback job:
- Remove 'Check authorization' step (org membership check via
  ORG_MEMBERSHIP_TOKEN).
- Remove 'Notify unauthorized user' step (fallback response to
  non-members).
- 'Build thread context', 'Checkout PR head', and 'Run reply' conditions
  simplified from:
    steps.check.outputs.is_agent == 'true' &&
    steps.auth.outputs.authorized == 'true'
  to:
    steps.check.outputs.is_agent == 'true'
  Any user replying to an agent review comment now gets a response.
- Update skip-auth comment to reflect the authorization model change.

Note: issue_comment replies to agent PR-level comments (as opposed to
pull_request_review_comment replies) are not handled by reply-to-feedback
— this pre-existing gap is not addressed in this PR.

Assisted-By: docker-agent
@docker-agent
docker-agent requested a review from a team as a code owner May 6, 2026 19:39
@docker-agent
docker-agent requested a review from derekmisler May 6, 2026 19:39
… bypassed for user-requested

Amends the previous commit. The org-membership check was removed too
broadly. The correct behaviour is:

- Automatic triggers (pull_request, not review_requested): draft check
  gates entry; org-membership check runs as before.
- User-requested triggers (review_requested, /review command, explicit
  pr-number, resolve-context path): draft check is bypassed; org-
  membership check still runs and still gates Checkout + Run steps.

The bypass works naturally: when 'Check if PR is draft' is skipped
(user_requested == 'true'), steps.draft.outputs.skip is empty string.
The original condition 'draft.skip != 'true'' therefore evaluates true,
so Setup credentials, Check if org member, and all downstream steps
proceed with their original logic — no changes needed to their ifs.

Changes vs the previous commit on this branch:
- Restore 'Check if org member' step with its original condition and
  script (was removed in previous commit).
- Restore 'Setup credentials' original condition: is_review != 'false'
  && draft.skip != 'true' (was widened unnecessarily).
- Restore 'Create check run' original condition: source in
  [event,trigger] && membership.is_member == 'true' (was changed).
- Restore 'Checkout PR head' and 'Run PR Review' original conditions:
  membership.is_member == 'true' && is_review != 'false' &&
  draft.skip != 'true' (was stripped of membership gate).
- Update the comment in 'Classify trigger type' to accurately describe
  that it only affects the draft check.

reply-to-feedback: no change — org-membership removal there is still
the intended behaviour.

Assisted-By: docker-agent

@docker-agent docker-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🔴 CRITICAL

docker-agent[bot]

This comment was marked as off-topic.

…quested triggers

Reverts all changes beyond the single intended behaviour: user-requested
review triggers (review_requested for docker-agent, /review comment,
explicit pr-number input) skip the draft check. Everything else is
verbatim original.

Net diff from upstream/main:
1. New 'Classify trigger type' step (id: trigger-type) — classifies
   the trigger as user_requested=true/false.
2. 'Check if PR is draft' condition:
     was: github.event_name != 'issue_comment'
     now: github.event_name != 'issue_comment'
          && steps.trigger-type.outputs.user_requested != 'true'

Restores in this commit:
- 'Check authorization' step in reply-to-feedback (verbatim original)
- 'Notify unauthorized user' step in reply-to-feedback (verbatim)
- 'Build thread context', 'Checkout PR head', 'Run reply' conditions in
  reply-to-feedback restored to include steps.auth.outputs.authorized
- skip-auth comment on 'Run reply' restored to original text
- All conditions in the review job beyond Check if PR is draft are
  unchanged from upstream/main (no membership gates were modified)

Assisted-By: docker-agent
@docker docker deleted a comment from docker-agent Bot May 6, 2026
When @docker-agent is mentioned on an inline PR review comment, the reply
now lands in the same inline thread on the file/line being asked about,
instead of as an unrelated PR-level Issues comment.

Four reinforcing layers fixed:

1. src/mention-reply/index.ts — parseEventContext now captures the
   inline-only fields (path, line, original_line, diff_hunk, and the
   originating comment id as in_reply_to_id) into a new EventContext.inline
   sub-object. Was previously discarded.

2. src/mention-reply/index.ts — buildContextPrompt emits a new
   [INLINE COMMENT CONTEXT] block (FILE_PATH / LINE / IN_REPLY_TO_ID +
   data-isolated diff hunk) immediately before the mention-comment block,
   only when the trigger was pull_request_review_comment.

3. review-pr/agents/pr-review-mention-reply.yaml — agent instruction is
   now branched: when [INLINE COMMENT CONTEXT] is present, post via
   POST /repos/{o}/{r}/pulls/{n}/comments with in_reply_to=IN_REPLY_TO_ID
   so GitHub renders the reply in the originating thread; otherwise keep
   the existing Issues API path. Each branch uses jq -n --arg to avoid
   the shell-interpolation injection foot-gun. The permissions allow-list
   adds repos/*/pulls/*/comments* alongside the existing issues path.

4. src/post-comment/index.ts — adds postReviewCommentReply helper
   (Octokit pulls.createReplyForReviewComment) and uses it in the
   non-org-member rejection branch so the rejection also lands in the
   originating inline thread, not as a PR-level comment.

Tests: extended src/mention-reply/__tests__ to cover the new inline
fields end-to-end (parse, prompt, run() rejection routing, run() prompt
output) and added src/post-comment/__tests__ for postReviewCommentReply.
128 unit tests passing.

Falls back to original_line when line is null (multi-line comments)
matching the existing reply-to-feedback pattern in review-pr.yml.

Refs: PR #544 on docker/gordon, where derekmisler asked
'@docker-agent, do you think this is important?' on a single line of YAML
and got a global PR-level verdict instead of a line-anchored answer.

Assisted-By: docker-agent
@docker-agent docker-agent changed the title ci(review-pr): remove org-membership gates; add user-requested trigger bypass fix(review-pr): inline mention-reply correctness; bypass draft check for user-requested reviews May 6, 2026
@derekmisler

Copy link
Copy Markdown
Contributor

/review

@docker-agent docker-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

Reviewed the two fixes in this PR:

Fix 1 — Inline mention-reply correctness (src/mention-reply/, review-pr/agents/pr-review-mention-reply.yaml, src/post-comment/)

The refactor is well-structured. parseEventContext correctly captures path, line, original_line, diff_hunk, and in_reply_to_id (using comment.id) into the new InlineCommentContext sub-object. The ?? fallback chain (line ?? originalLine'') is handled correctly: bare integer IDs always present, LINE is contextual-only and doesn't affect API correctness. buildContextPrompt inserts the [INLINE COMMENT CONTEXT] block between the PR description and the mention comment — matching the agent YAML's documented expectation. The rejection path correctly branches on ctx.inline to route inline rejections via postReviewCommentReply and top-level rejections via postComment. The postReviewCommentReply implementation correctly maps to octokit.pulls.createReplyForReviewComment with pull_number and comment_id. Data-isolation markers around the diff hunk match the established pattern.

Fix 2 — Draft bypass for user-requested reviews (.github/workflows/review-pr.yml)

The Classify trigger type step correctly references needs.resolve-context.result — the review job already declares needs: [resolve-context]. The five trigger branches cover all intended paths (issue_comment /review, review_requested, explicit pr-number, trigger-pattern dispatch, automatic pull_request). The narrowed if: on the Check if PR is draft step (github.event_name != 'issue_comment' && steps.trigger-type.outputs.user_requested != 'true') correctly extends the existing issue_comment exclusion to also exclude user-requested triggers, without disturbing the downstream steps.draft.outputs.skip gate logic.

Tests cover the new parseEventContext inline metadata, buildContextPrompt block presence/absence, the non-member rejection routing to postReviewCommentReply, and postReviewCommentReply API correctness. The 128-test suite provides good coverage of the added code paths.

No bugs found in the changed code.

@derekmisler
derekmisler merged commit 00db5f0 into docker:main May 6, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants