Skip to content

ci: a new e2e workflow is classified inert, so the PR adding it greens the gate with no suite run #3653

Open

Description

Summary

A brand-new workflow that runs e2e suites is classified inert by the test-impact selector, so the PR adding it gets an empty selection and a green E2E Tests gate without any suite ever running against it. Mechanism: inert_config_pattern marks .github/ inert as a whole directory, and the e2e workflows are exempted from that by NAME in full_suite_pattern; a workflow not yet on that list falls into the inert directory rule. Measured with probes: .github/workflows/e2e-brand-new.yaml and .github/actions/foo/action.yaml both select empty.

This pre-exists the fail-safe work in flight (#3511) and survives it by construction: that change adds a fall-through escalation for unclassified paths, but a path inside .github/ IS classified, as inert, so the fall-through never sees it. The four current e2e workflow names are enumerated correctly today; the enumeration itself is the protection, and nothing checks it stays complete.

Why it matters

Adding an e2e workflow is exactly the kind of change that most needs a full run, and it is the one change the selector waves through silently. The failure needs no mistake beyond forgetting a manual list update, and the signal it produces is a green required check.

Fix shape

Derive the exemption list at runtime instead of hand-maintaining it, e.g. from the same rg -l test-chainsaw enumeration the repo already uses to find suite-running workflows, so a new e2e workflow escalates by virtue of what it contains rather than by being remembered. Short of that, a guard test asserting that every workflow matching the suite-runner heuristic appears in full_suite_pattern would at least turn the forgotten update into a red check.

Activity

  1. added
    kind/bugCategorizes issue or PR as related to a bug
    area/ciIssues or PRs related to CI workflows, GitHub Actions, automation
    on Aug 8, 2026
  2. added
    triage/acceptedIndicates an issue is ready to be actively worked on
    priority/backlogGeneral backlog priority. Lower than priority/important-longterm
    and removed
    triage/needs-triageIndicates an issue needs triage by a maintainer
    on Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/ciIssues or PRs related to CI workflows, GitHub Actions, automationkind/bugCategorizes issue or PR as related to a bugpriority/backlogGeneral backlog priority. Lower than priority/important-longtermtriage/acceptedIndicates an issue is ready to be actively worked on

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions