Repository navigation
Show each fork CI job's status on gated pull requests - #75616
Merged
Merged
Conversation
A gated pull request showed one aggregate fork CI status, so a maintainer or contributor had to open the fork run to see which jobs were running, which failed and which passed. Reporting every job as its own check, each linking to the job in the fork, puts that on the pull request itself, where project CI shows its jobs. Generated-by: Claude Opus 5
potiuk
requested review from
amoghrajesh,
ashb,
bugraoz93,
choo121600,
ephraimbuddy,
gopidesupavan,
jason810496,
jedcunningham,
jscheffl and
vatsrahul1001
as code owners
October 25, 2026 18:23
shahar1
approved these changes
Oct 25, 2026
Contributor
Backport failed to create: v3-4-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker 2f7a7ea v3-4-testThis should apply the commit to the v3-4-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
1 task done
vatsrahul1001
pushed a commit
that referenced
this pull request
Oct 26, 2026
…75654, #75655) (#75680) * Show each fork CI job's status on gated pull requests (#75616) A gated pull request showed one aggregate fork CI status, so a maintainer or contributor had to open the fork run to see which jobs were running, which failed and which passed. Reporting every job as its own check, each linking to the job in the fork, puts that on the pull request itself, where project CI shows its jobs. Generated-by: Claude Opus 5 (cherry picked from commit 2f7a7ea) * Cancel the workflow runs of pull requests that have already merged (#75619) Merging a pull request does not stop its pull_request runs, so they keep using the project's GitHub-hosted minutes and self-hosted runners for a result nobody reads once the change is on its target branch. Until now maintainers cancelled such runs by hand, when they noticed them. Generated-by: Claude Opus 5 (cherry picked from commit e5f2c87) * Auto-approve cheap CI runs of external contributors' PRs (#75455) * Auto-approve cheap CI runs of external contributors' PRs GitHub holds the workflow runs of fork PRs until a maintainer approves them. With self-hosted runners (AIP-118) the approval gate becomes part of the trust model, so it should cover all external contributors, but approving every push by hand does not scale and makes contributors wait. Runs whose change cannot touch the CI definition and whose CI is cheap are safe to start without a maintainer. Expensive or large changes, and anything that changes how CI or runners are chosen, still get a human look first, which also keeps the load on the shared public runners down. The CI-minutes estimate lets maintainers see what a run will cost before approving it. Generated-by: Claude Opus 5 * Add inline script metadata to the CI-minutes calibration script The script uses only the standard library, so with a PEP 723 block it runs with plain `uv run dev/calibrate_ci_minutes_estimates.py` instead of syncing the whole Breeze environment. Document that invocation in the script and in .github/AGENTS.md. * Keep one PR's API or git error from aborting the approval tick Each PR is now processed in its own try block. A requests.RequestException or subprocess.CalledProcessError for one PR no longer stops the remaining PRs or skips the job summary. The failed PR is recorded with failed=True and shown as "failed" in the summary, keeping its CI-time estimate when one was computed and giving a short, single-line form of the error as the reason. The workflow also tries to add the 'need workflow approval' label to it, so it is not lost once its runs leave the query window. Nothing is approved on failure. approve_workflow_runs exits 1 after writing the summary when any PR failed, so a persistent failure turns the scheduled run red instead of passing silently. * Approve the runs of fork CI pull requests unless they change CI A pull request whose CI runs in the contributor's fork (AIP-120) still gets pull_request runs in apache/airflow, and GitHub holds them until a maintainer approves them. Those runs only execute the fork CI gate, so judging them by the cost and size of project CI left cheap gate-only runs waiting for a maintainer for no benefit, and a gated run that never ran could not be cleaned up by the reconciler. What still needs a maintainer is a change to the files that define CI, since it could remove the gate or add a workflow that does run. Using the reconciler's own list keeps the two decisions consistent, and the author is told why their runs wait instead of only seeing a label. Generated-by: Claude Opus 5 * Keep CI-minutes estimates true to the runs they approve A run selects its tests from the labels the PR had when it started, so approving it on the current labels could start a full-tests run whose label was removed while it waited. Calibrating from self-hosted and all-versions runs, and leaving out the jobs that need no CI image, made the estimate describe runs other than those it is used to approve. Generated-by: Claude Opus 5 --------- Co-authored-by: Shahar Epstein <[email protected]> (cherry picked from commit 2a3ee7f) * Approve fork CI runs that started before "use fork ci" was added (#75654) When a committer adds "use fork ci" right after a PR opens, the PR's first runs were created without it, and the workflow-run approver left them for a maintainer. It had to: the fork CI gate fell back to the labels of the run's event when it could not read the current ones, and those labels could still send the run to full project CI on code nobody had reviewed. Anyone able to exhaust the shared token quota could force that fallback. The gate now fails when it cannot read the labels, so a run only ever acts on the PR's current labels and a re-run decides again. With that, runs started before the label changed are as safe to approve as later ones. Generated-by: Claude Opus 5 (cherry picked from commit 437daf6) * Check for workflow runs to auto-approve every 15 minutes (#75655) External contributors' runs wait for the next approver pass before they start, and GitHub regularly drops scheduled runs under load, which turns the 30-minute wait into an hour or more. A pass takes about a minute, so running it twice as often costs little and halves the wait. Generated-by: Claude Opus 5 (cherry picked from commit 54f645d) * Pin the real selective-checks auto-approve test to main The test decides approval for a PR targeting main, but built SelectiveChecks with the checkout's default branch. On a release branch that skips provider tests, so the large provider change was estimated cheap and approved. Generated-by: Claude Opus 5 --------- Co-authored-by: Shahar Epstein <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A pull request gated to fork CI (AIP-120) shows one aggregate
Fork CI / Tests (AMD)status, so to see which jobs are running, which failed and which passed, a maintainer or the contributor has to open the run in the fork. This reports every job of the fork run as its own commit status on the pull request, each linking to the job in the fork, the way project CI shows its jobs.What it does
GET /repos/<fork>/actions/runs/<id>/jobs?filter=latest) and posts one status per job asFork CI job / <job name>: yellow while queued or running, green when passed, red when failed, timed out, cancelled or unable to start. Skipped jobs and the jobs every run has (Fork CI gate,Check if the run is needed,Platform: AMD,CI passed) are left out.<run id>:<attempt>:<conclusion>) is recorded in the state comment's marker, so later monitor runs don't read its jobs again. A run in progress is read on every monitor run, and a re-run in the fork is read again because its attempt changes.successwith "Not in the latest fork run - see Fork CI / Tests (AMD)", since commit statuses can't be deleted and have no neutral state.CI passedand the state machine onFork CI / Tests (AMD); neither changes.Safety
Fork CI job /prefix. No job can post the aggregate status the reconciler reads its decisions back from, orCI passed, which the merge ruleset requires. Non-printable characters are replaced, names are shortened to 100 characters, and contexts are grouped case-insensitively as GitHub compares them, keeping the worst result.CI passedthat no longer holds is still revoked.read_head_statusnow reads every page of the combined status, because with job statuses a commit can carry more than 100 contexts.Write budget, and turning it off
GraphQL has no mutation for commit statuses (it can batch check runs, but a check run created by a workflow links to that workflow's run, not to the fork job), so each status is one REST write, counted against GitHub's limit of 500 content-creating requests an hour that every workflow of
apache/airflowshares.Measured on 2026-10-25 from
apache/airflow's ownTests (AMD)PR runs (the same workflow runs in forks):apache/airflowhad 1898Tests (AMD)PR runs from 2026-10-18 to 2026-10-24, about 11 an hour. During the contributor pilot (a few gated pushes a day) job statuses use under 5% of the content-creating limit. With the gate enabled, and assuming about half of the runs come from gated pull requests, they would need about 400 writes an hour, so each monitor run posts at most 60 (at most 240 an hour, 48% of the limit). What doesn't fit is posted by later runs; once the budget is spent, no further jobs or statuses are read in that run, and the monitor posts a Slack note (fixed wording, so once while deferrals persist). The share of gated runs and the jobs per push are assumptions; the measurements and the estimate are in the "Write budget" section ofdev/breeze/doc/ci/08_fork_ci_for_external_contributors.md.job_statuses = falsein.github/fork-ci-gate.tomlturns job statuses off; the aggregate status andCI passedare posted either way.dev/AGENTS.mdsays to do that when the budget is exceeded and lists the signs.Verified
dev/breezetest suite passes (1854 passed, 7 skipped), with new tests for the job statuses, the budget, the marker key, stale contexts, case-insensitive contexts, the switch, the Slack note and the paginated head status.mypy-dev.apache/airflowon 2026-10-25 (breeze ci fork-ci-reconcile --dry-runfrom this branch, 4 GraphQL and 23 REST calls): for the one gated pull request, [test - do not merge]Update README.md #75586, it planned 7 job statuses, allsuccess. That matches its fork run 40220648518 exactly: 11 started jobs, minus the 4 that are not reported. It also planned the usualFork CI / Tests (AMD)andCI passedsuccesses and a state-comment update, with no draft changes and nothing for Slack. Nothing was written: the head commit still had no statuses afterwards.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines
🤖 Generated with Claude Code