Bug Description
Three agentic workflow failures stem from related root causes around GitHub's security model for secrets and the bot-skip mechanism.
Problem 1: Fork PRs fail pr-review.lock.yml
The PR Review workflow (pr-review.lock.yml) triggers on pull_request events including fork PRs (forks: ["*"] in pr-review.md). GitHub never exposes repository secrets (COPILOT_GITHUB_TOKEN) to pull_request workflows from forks. The activation job fails at the Validate COPILOT_GITHUB_TOKEN secret step with:
Error: None of the following secrets are set: COPILOT_GITHUB_TOKEN
Example: PR #1296 (fork PR) → Run #24436983830
Problem 2: Dependabot workflow-file PRs fail dependency-pr-review.lock.yml
The Dependabot PR Review workflow triggers on paths including .github/workflows/*.yml. When Dependabot bumps GitHub Actions versions, the PR modifies workflow files. GitHub treats these PRs similarly to fork PRs for security (to prevent malicious action version bumps from exfiltrating secrets), stripping all secrets from the pull_request event.
Example: PR #1364 (Dependabot action bump) → Run #24412526951
Note: Dependabot PRs that only modify package manifests (package.json, pyproject.toml, etc.) succeed because those files are not executable code and GitHub does not strip secrets for them. See Run #24400483154 for a working example.
Problem 3: PR Review workflow runs on Dependabot PRs when a maintainer triggers the event
The skip-bots mechanism in pr-review.lock.yml checks github.actor (the user who triggered the event), not github.event.pull_request.user.login (the PR author). When a maintainer merges main into a Dependabot branch or re-runs the workflow, the actor is the maintainer, and the bot-skip check passes. The full PR Review then runs against a Dependabot PR, producing irrelevant "Issue Alignment" and "PR Template Compliance" findings.
Example: PR #1364 → PR Review Run #24434520785 was triggered by @WilliamBerryiii via synchronize, not by dependabot[bot]. The check_skip_bots.cjs saw a non-bot actor and allowed the workflow through. The review at PR #1364 review then flagged missing linked issues, PR template fields, and checklist items — all inappropriate for a Dependabot PR.
This confirms the findings come from pr-review.md Review Steps 1 and 2, not from dependency-reviewer.agent.md.
Expected Behavior
- Fork PRs should either skip agentic review gracefully (neutral/skipped status) or use a trigger that supports secrets.
- Dependabot workflow-file PRs should not fail the dependency review workflow.
- Dependabot PRs should never receive "Issue Alignment" or "PR Template Compliance" findings from the PR Review workflow, regardless of who triggers the event.
Proposed Solution
For fork PRs (pr-review.md)
Add an early exit in the activation or agent job that detects fork PRs and calls noop with a clear message instead of failing:
## Activation Guard
**You MUST call `noop` and stop immediately if any of these conditions are true:**
* The PR is from a fork and secrets are unavailable: call `noop` with message "Skipping: fork PR, secrets unavailable."
Alternatively, handle this at the workflow level by adding a condition to pre_activation or activation that checks github.event.pull_request.head.repo.id != github.repository_id and skips gracefully.
For Dependabot workflow-file PRs (dependency-pr-review.md)
Remove .github/workflows/*.yml from the paths filter in dependency-pr-review.md. Action version bumps are low-risk changes that do not need AI dependency review; they go through normal CI validation and Dependabot's own security assessment:
on:
pull_request:
types: [opened, synchronize]
paths:
- 'package.json'
- 'package-lock.json'
- '**/requirements.txt'
- '**/pyproject.toml'
# Remove: '.github/workflows/*.yml'
- '.devcontainer/**'
For PR Review running on Dependabot PRs (pr-review.md)
The skip-bots mechanism needs to check the PR author (github.event.pull_request.user.login), not just the actor (github.actor). Add a condition to the Activation Guard in pr-review.md:
## Activation Guard
**You MUST call `noop` and stop immediately if any of these conditions are true:**
* The PR author is `dependabot[bot]` or `github-actions[bot]`: call `noop` with message "Skipping: PR authored by bot."
If check_skip_bots.cjs is a shared gh-aw framework script, the fix may need to be upstream in the framework. If the check is configurable, update the skip-bots behavior to also match on PR author. Otherwise, add an explicit author check in the pr-review.md Activation Guard instructions so the agent exits early.
Steps to Reproduce
- Open a fork PR against
microsoft/hve-core (triggers pr-review.lock.yml → fails at secret validation)
- Have Dependabot create a PR that bumps GitHub Actions versions in
.github/workflows/*.yml (triggers dependency-pr-review.lock.yml → fails at secret validation)
- As a maintainer, merge main into a Dependabot PR branch (triggers
pr-review.lock.yml with maintainer as actor → bot-skip bypassed → irrelevant findings)
Additional Context
- GitHub security model:
pull_request events from forks and from PRs that modify workflow files run with read-only tokens and no repository secrets
- This is a platform-level restriction, not configurable via Dependabot secrets or Actions settings
pull_request_target would provide secrets but runs workflow code from the base branch (security tradeoff)
- The
check_skip_bots.cjs script is part of the gh-aw framework (github/gh-aw-actions); the fix may need coordination with that team
Bug Description
Three agentic workflow failures stem from related root causes around GitHub's security model for secrets and the bot-skip mechanism.
Problem 1: Fork PRs fail
pr-review.lock.ymlThe PR Review workflow (
pr-review.lock.yml) triggers onpull_requestevents including fork PRs (forks: ["*"]inpr-review.md). GitHub never exposes repository secrets (COPILOT_GITHUB_TOKEN) topull_requestworkflows from forks. Theactivationjob fails at theValidate COPILOT_GITHUB_TOKEN secretstep with:Example: PR #1296 (fork PR) → Run #24436983830
Problem 2: Dependabot workflow-file PRs fail
dependency-pr-review.lock.ymlThe Dependabot PR Review workflow triggers on paths including
.github/workflows/*.yml. When Dependabot bumps GitHub Actions versions, the PR modifies workflow files. GitHub treats these PRs similarly to fork PRs for security (to prevent malicious action version bumps from exfiltrating secrets), stripping all secrets from thepull_requestevent.Example: PR #1364 (Dependabot action bump) → Run #24412526951
Note: Dependabot PRs that only modify package manifests (
package.json,pyproject.toml, etc.) succeed because those files are not executable code and GitHub does not strip secrets for them. See Run #24400483154 for a working example.Problem 3: PR Review workflow runs on Dependabot PRs when a maintainer triggers the event
The
skip-botsmechanism inpr-review.lock.ymlchecksgithub.actor(the user who triggered the event), notgithub.event.pull_request.user.login(the PR author). When a maintainer merges main into a Dependabot branch or re-runs the workflow, the actor is the maintainer, and the bot-skip check passes. The full PR Review then runs against a Dependabot PR, producing irrelevant "Issue Alignment" and "PR Template Compliance" findings.Example: PR #1364 → PR Review Run #24434520785 was triggered by
@WilliamBerryiiiviasynchronize, not bydependabot[bot]. Thecheck_skip_bots.cjssaw a non-bot actor and allowed the workflow through. The review at PR #1364 review then flagged missing linked issues, PR template fields, and checklist items — all inappropriate for a Dependabot PR.This confirms the findings come from
pr-review.mdReview Steps 1 and 2, not fromdependency-reviewer.agent.md.Expected Behavior
Proposed Solution
For fork PRs (
pr-review.md)Add an early exit in the activation or agent job that detects fork PRs and calls
noopwith a clear message instead of failing:Alternatively, handle this at the workflow level by adding a condition to
pre_activationoractivationthat checksgithub.event.pull_request.head.repo.id != github.repository_idand skips gracefully.For Dependabot workflow-file PRs (
dependency-pr-review.md)Remove
.github/workflows/*.ymlfrom thepathsfilter independency-pr-review.md. Action version bumps are low-risk changes that do not need AI dependency review; they go through normal CI validation and Dependabot's own security assessment:For PR Review running on Dependabot PRs (
pr-review.md)The
skip-botsmechanism needs to check the PR author (github.event.pull_request.user.login), not just the actor (github.actor). Add a condition to the Activation Guard inpr-review.md:If
check_skip_bots.cjsis a shared gh-aw framework script, the fix may need to be upstream in the framework. If the check is configurable, update theskip-botsbehavior to also match on PR author. Otherwise, add an explicit author check in thepr-review.mdActivation Guard instructions so the agent exits early.Steps to Reproduce
microsoft/hve-core(triggerspr-review.lock.yml→ fails at secret validation).github/workflows/*.yml(triggersdependency-pr-review.lock.yml→ fails at secret validation)pr-review.lock.ymlwith maintainer as actor → bot-skip bypassed → irrelevant findings)Additional Context
pull_requestevents from forks and from PRs that modify workflow files run with read-only tokens and no repository secretspull_request_targetwould provide secrets but runs workflow code from the base branch (security tradeoff)check_skip_bots.cjsscript is part of the gh-aw framework (github/gh-aw-actions); the fix may need coordination with that team