Skip to content

fix(workflows): Agentic workflows fail when GitHub strips secrets for fork PRs and Dependabot workflow-file PRs #1365

@bindsi

Description

@bindsi

Bug Description

Three agentic workflow failures stem from related root causes around GitHub's security model for secrets and the bot-skip mechanism.

Problem 1: Fork PRs fail pr-review.lock.yml

The PR Review workflow (pr-review.lock.yml) triggers on pull_request events including fork PRs (forks: ["*"] in pr-review.md). GitHub never exposes repository secrets (COPILOT_GITHUB_TOKEN) to pull_request workflows from forks. The activation job fails at the Validate COPILOT_GITHUB_TOKEN secret step with:

Error: None of the following secrets are set: COPILOT_GITHUB_TOKEN

Example: PR #1296 (fork PR) → Run #24436983830

Problem 2: Dependabot workflow-file PRs fail dependency-pr-review.lock.yml

The Dependabot PR Review workflow triggers on paths including .github/workflows/*.yml. When Dependabot bumps GitHub Actions versions, the PR modifies workflow files. GitHub treats these PRs similarly to fork PRs for security (to prevent malicious action version bumps from exfiltrating secrets), stripping all secrets from the pull_request event.

Example: PR #1364 (Dependabot action bump) → Run #24412526951

Note: Dependabot PRs that only modify package manifests (package.json, pyproject.toml, etc.) succeed because those files are not executable code and GitHub does not strip secrets for them. See Run #24400483154 for a working example.

Problem 3: PR Review workflow runs on Dependabot PRs when a maintainer triggers the event

The skip-bots mechanism in pr-review.lock.yml checks github.actor (the user who triggered the event), not github.event.pull_request.user.login (the PR author). When a maintainer merges main into a Dependabot branch or re-runs the workflow, the actor is the maintainer, and the bot-skip check passes. The full PR Review then runs against a Dependabot PR, producing irrelevant "Issue Alignment" and "PR Template Compliance" findings.

Example: PR #1364 → PR Review Run #24434520785 was triggered by @WilliamBerryiii via synchronize, not by dependabot[bot]. The check_skip_bots.cjs saw a non-bot actor and allowed the workflow through. The review at PR #1364 review then flagged missing linked issues, PR template fields, and checklist items — all inappropriate for a Dependabot PR.

This confirms the findings come from pr-review.md Review Steps 1 and 2, not from dependency-reviewer.agent.md.

Expected Behavior

  • Fork PRs should either skip agentic review gracefully (neutral/skipped status) or use a trigger that supports secrets.
  • Dependabot workflow-file PRs should not fail the dependency review workflow.
  • Dependabot PRs should never receive "Issue Alignment" or "PR Template Compliance" findings from the PR Review workflow, regardless of who triggers the event.

Proposed Solution

For fork PRs (pr-review.md)

Add an early exit in the activation or agent job that detects fork PRs and calls noop with a clear message instead of failing:

## Activation Guard

**You MUST call `noop` and stop immediately if any of these conditions are true:**

* The PR is from a fork and secrets are unavailable: call `noop` with message "Skipping: fork PR, secrets unavailable."

Alternatively, handle this at the workflow level by adding a condition to pre_activation or activation that checks github.event.pull_request.head.repo.id != github.repository_id and skips gracefully.

For Dependabot workflow-file PRs (dependency-pr-review.md)

Remove .github/workflows/*.yml from the paths filter in dependency-pr-review.md. Action version bumps are low-risk changes that do not need AI dependency review; they go through normal CI validation and Dependabot's own security assessment:

on:
  pull_request:
    types: [opened, synchronize]
    paths:
      - 'package.json'
      - 'package-lock.json'
      - '**/requirements.txt'
      - '**/pyproject.toml'
      # Remove: '.github/workflows/*.yml'
      - '.devcontainer/**'

For PR Review running on Dependabot PRs (pr-review.md)

The skip-bots mechanism needs to check the PR author (github.event.pull_request.user.login), not just the actor (github.actor). Add a condition to the Activation Guard in pr-review.md:

## Activation Guard

**You MUST call `noop` and stop immediately if any of these conditions are true:**

* The PR author is `dependabot[bot]` or `github-actions[bot]`: call `noop` with message "Skipping: PR authored by bot."

If check_skip_bots.cjs is a shared gh-aw framework script, the fix may need to be upstream in the framework. If the check is configurable, update the skip-bots behavior to also match on PR author. Otherwise, add an explicit author check in the pr-review.md Activation Guard instructions so the agent exits early.

Steps to Reproduce

  1. Open a fork PR against microsoft/hve-core (triggers pr-review.lock.yml → fails at secret validation)
  2. Have Dependabot create a PR that bumps GitHub Actions versions in .github/workflows/*.yml (triggers dependency-pr-review.lock.yml → fails at secret validation)
  3. As a maintainer, merge main into a Dependabot PR branch (triggers pr-review.lock.yml with maintainer as actor → bot-skip bypassed → irrelevant findings)

Additional Context

  • GitHub security model: pull_request events from forks and from PRs that modify workflow files run with read-only tokens and no repository secrets
  • This is a platform-level restriction, not configurable via Dependabot secrets or Actions settings
  • pull_request_target would provide secrets but runs workflow code from the base branch (security tradeoff)
  • The check_skip_bots.cjs script is part of the gh-aw framework (github/gh-aw-actions); the fix may need coordination with that team

Metadata

Metadata

Assignees

Labels

agentsCustom chat agents (.agent.md)bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions