Skip to content

Commit 693fd9e

Browse files
authored
ci: run every job on the standard runners (cost stop until ci-spot) (#415)
* ci: keep every verification job on standard runners Apply the owner cost stop of 2026-10-18 to pushes, schedules, dispatches and every pull request until the ci-spot pool is live. Replace the paid-runner routing introduced by d093560 and carried into native-live by 065ef1d; retain the standard image matrix and all existing timeouts. No tests, builds or workflows run locally per the executor brief. * ci: enforce the standard runner contract in preflight Replace the event-based runner contract and mutation cases introduced by d093560 with ubuntu-24.04 and matrix.os. Refuse pf-linux-x64-16, every *-xlarge or *-large label and every larger_runner key, including in the release matrix, while retaining the remaining workflow checks. Add rejection cases for paid labels, obsolete keys and conditional paid routing. Preserve comment tolerance and core-count checks. Tests intentionally not run under the executor no-run rule. * docs: record the standard runner cost stop and ci-spot handoff Reword the runner-selection and timing assumptions introduced by d093560 (#290) and the documentation-only runner sentences introduced by 133f571 (#308). Keep the platform job limits recorded by 2394ba2 (#373). Record the owner decision of 2026-10-18 after larger-runner costs exceeded US$3,000: every event uses free standard runners until the ci-spot pool is live. Describe the unconditional runner contract and remove obsolete larger-runner timing estimates while retaining measured standard-runner durations. No local verification run, as required by the executor brief. * test(ci): pin the standard-runner mutations precisely Target the full runs-on line in both build-runner rejection cases. The bare matrix.os expression matched the earlier leg name, leaving runs-on unchanged and producing the wrong finding in PR #415 CI run 39119916194. Preserve the build leg name and the expected runner-finding assertion. Leave scripts/ci_preflight.py unchanged. No tests or verification run locally under the executor no-run rule.
1 parent f37c8b5 commit 693fd9e

6 files changed

Lines changed: 137 additions & 157 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 10 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,8 @@ env:
3232
# catalog from the checked-in diff), which only a clone without tags needs.
3333
PINEFORGE_REQUIRE_RELEASE_TAGS: "1"
3434

35-
# The heavy jobs run on the organization's 16-core larger runner, and the macOS
36-
# legs on GitHub's M2 runner for the macos-26 image, for a push, the schedule, a
37-
# dispatch and a pull request from a branch of this repository. A larger runner
38-
# bills the organization even for a public repository, so every other event --
39-
# a fork's pull request above all -- runs on the free standard runner.
35+
# Every job uses the free standard runners for every event (owner cost stop,
36+
# 2026-10-18). The ci-spot labels replace them when that pool is live.
4037
# Each verification sizes its parallelism to the runner that took it.
4138
# scripts/ci_preflight.py pins every runner, time limit and job count; every
4239
# job's limit is the platform's own 360-minute maximum, the only hang catcher
@@ -79,7 +76,7 @@ jobs:
7976
fi
8077
8178
preflight:
82-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'pf-linux-x64-16' || 'ubuntu-24.04' }}
79+
runs-on: ubuntu-24.04
8380
# 360 minutes is GitHub's job maximum and deliberately the only limit:
8481
# its verifier-tests stage runs test_ci_verify.py across parallel unittest
8582
# shards, and no stage of this preflight carries a timeout of its own any
@@ -153,15 +150,9 @@ jobs:
153150
matrix:
154151
os: [ubuntu-24.04, macos-26]
155152
build_type: [Release, Debug]
156-
include:
157-
- os: ubuntu-24.04
158-
larger_runner: pf-linux-x64-16
159-
- os: macos-26
160-
larger_runner: macos-26-xlarge
161153

162-
# matrix.os is the standard runner a fork's pull request gets; its
163-
# larger_runner runs the same image with more cores.
164-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && matrix.larger_runner || matrix.os }}
154+
# matrix.os selects the standard runner for every event.
155+
runs-on: ${{ matrix.os }}
165156

166157
steps:
167158
- uses: actions/checkout@v4
@@ -240,7 +231,7 @@ jobs:
240231
sanitizers:
241232
needs: changes
242233
if: ${{ !cancelled() && (needs.changes.result != 'success' || needs.changes.outputs.docs_only != 'true') }}
243-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'pf-linux-x64-16' || 'ubuntu-24.04' }}
234+
runs-on: ubuntu-24.04
244235
# 360 minutes (the platform maximum). No CTest stage bound remains: main's
245236
# full set ran out of the old 30- and 60-minute stage bounds twice, and
246237
# the owner rule of 2026-10-12 removed the bounds rather than raising
@@ -304,10 +295,10 @@ jobs:
304295
kernel-only:
305296
needs: changes
306297
if: ${{ !cancelled() && (needs.changes.result != 'success' || needs.changes.outputs.docs_only != 'true') }}
307-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'pf-linux-x64-16' || 'ubuntu-24.04' }}
308-
# 22.3 min on the standard runner at 0d76a099, where a fork's pull request
309-
# still runs the whole kernel set; 360 minutes is the platform maximum, the
310-
# only hang catcher (owner rule, 2026-10-12).
298+
runs-on: ubuntu-24.04
299+
# 22.3 min on the standard runner at 0d76a099; every event runs the whole
300+
# kernel set. 360 minutes is the platform maximum, the only hang catcher
301+
# (owner rule, 2026-10-12).
311302
timeout-minutes: 360
312303
steps:
313304
- uses: actions/checkout@v4

‎.github/workflows/corpus-parity.yml‎

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -80,20 +80,17 @@ env:
8080
# pin is almost entirely cache hits.
8181
CCACHE_MAXSIZE: 2G
8282

83-
# Both jobs run on the organization's 16-core larger runner for a push, the
84-
# schedule, a dispatch and a pull request from a branch of this repository;
85-
# every other event, a fork's pull request above all (this workflow's own or
86-
# CI's: in a workflow_call the github context is the caller's), runs on the
87-
# free standard runner. The build uses every core of the runner that took the
88-
# job. scripts/ci_preflight.py pins the runners and limits.
83+
# Both jobs use the free standard runner for every event (owner cost stop,
84+
# 2026-10-18). The ci-spot labels replace it when that pool is live. The build
85+
# uses every core of the runner. scripts/ci_preflight.py pins runners and limits.
8986
jobs:
9087
# The whole population. Nightly, on demand, and on a pull request that
9188
# touches the pin or the parity tooling. Never reached by `mode: subset`.
9289
corpus-parity:
9390
if: inputs.mode != 'subset'
94-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'pf-linux-x64-16' || 'ubuntu-24.04' }}
95-
# 16.3-26.2 min on the standard runner a fork's pull request still gets;
96-
# 360 minutes is the platform maximum, the only hang catcher (owner rule,
91+
runs-on: ubuntu-24.04
92+
# 16.3-26.2 min on the standard runner; 360 minutes is the platform maximum,
93+
# the only hang catcher (owner rule,
9794
# 2026-10-12: no bound of our own may fail a run for slowness).
9895
timeout-minutes: 360
9996
steps:
@@ -168,7 +165,7 @@ jobs:
168165
# names, judged against the same pinned sha256 rows. Reached through CI.
169166
corpus-parity-subset:
170167
if: inputs.mode == 'subset'
171-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'pf-linux-x64-16' || 'ubuntu-24.04' }}
168+
runs-on: ubuntu-24.04
172169
# 1.8-4.7 min on the standard runner; 360 minutes is the platform
173170
# maximum, the only hang catcher (owner rule, 2026-10-12).
174171
timeout-minutes: 360

‎.github/workflows/native-live.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,9 @@ env:
3838
jobs:
3939
native-live:
4040
name: native-live (${{ matrix.profile }})
41-
# Same trusted-event 16-core policy as CI; forks use the standard runner.
42-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'pf-linux-x64-16' || 'ubuntu-24.04' }}
41+
# Standard runner for every event (owner cost stop, 2026-10-18), until the
42+
# ci-spot pool is live and its labels replace it.
43+
runs-on: ubuntu-24.04
4344
timeout-minutes: 360
4445
strategy:
4546
fail-fast: false

‎docs/ci.md‎

Lines changed: 28 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -887,59 +887,40 @@ does not assume a build optimization without evidence.
887887

888888
## Runners and time limits
889889

890-
The heavy jobs -- `preflight`, the four `build` legs, `sanitizers`,
891-
`kernel-only`, `native-live` and both corpus-parity jobs -- run on larger
892-
runners for a push, a manual dispatch, the nightly schedule and a pull request
893-
from a branch of this repository: the organization's `pf-linux-x64-16` (Ubuntu
894-
24.04 x64, 16 cores, 64 GB) and GitHub's `macos-26-xlarge` (the `macos-26`
895-
arm64 image on an M2, 5 cores, 14 GB). A larger runner bills the organization
896-
even for a public repository, so a pull request from a fork runs the same jobs
897-
on the free standard runners, `ubuntu-24.04` (4 vCPU, 16 GB) and `macos-26`
898-
(M1, 3 cores, 7 GB). Every heavy job selects its runner with one test that
899-
names what it trusts: a push, the schedule, a manual dispatch, or a
900-
`pull_request` whose head is a branch of this repository. Every other event
901-
gets the standard runner, whether a fork can raise it (a fork's pull request,
902-
`pull_request_target`, a review or review comment -- anyone can post one on a
903-
public repository's pull request -- a comment, a `workflow_run`) or it is
904-
merely unlisted (`merge_group`).
890+
Every job in `ci.yml`, `native-live.yml` and `corpus-parity.yml` uses the free
891+
standard runners for every event, including pushes, schedules, manual
892+
dispatches and pull requests from this repository or a fork. Linux jobs use
893+
`ubuntu-24.04` (4 vCPU, 16 GB); the macOS `build` legs use `macos-26` (M1,
894+
3 cores, 7 GB). Larger runners bill even for a public repository, and their
895+
cost exceeded US$3,000: the owner's 2026-10-18 cost stop puts every job on
896+
standard runners. This is a stopgap until the ci-spot pool is live; its labels
897+
will replace the standard runner labels when that pool is ready.
905898

906899
```yaml
907-
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'pf-linux-x64-16' || 'ubuntu-24.04' }}
900+
runs-on: ubuntu-24.04
908901
```
909902
910-
The `build` matrix pairs each image with its `larger_runner` and names its legs
911-
explicitly, so they stay `build (<image>, <type>)`. `native-live.yml` and
912-
`corpus-parity.yml` apply the test themselves; in a `workflow_call` the
913-
`github` context is the caller's, so a fork's pull request through CI gets the
914-
standard runner there too. The advisory `build` aggregate runs a few seconds of
915-
shell and stays on the standard runner for every event. So does the `changes`
916-
job, a two-commit checkout and one `git diff`.
903+
The `build` matrix selects its standard image with `runs-on: ${{ matrix.os }}`
904+
and names its four legs explicitly, so they stay `build (<image>, <type>)`.
905+
`native-live.yml` and `corpus-parity.yml` use `ubuntu-24.04` directly, whether
906+
called through CI or started independently. The advisory `build` aggregate
907+
and the `changes` job also use `ubuntu-24.04` for every event.
917908

918909
Each `ci_verify.py` call passes `--jobs "$(getconf _NPROCESSORS_ONLN)"` and
919910
`check_corpus_parity.sh` gets the same count as `JOBS`, so build and CTest
920-
parallelism follow the runner: 16 on the larger Linux runner, 5 on the M2, 4
921-
and 3 on the standard runners. The larger Linux runner is configured for at
922-
most eight jobs at once, and one CI run starts seven there (preflight, the two
923-
Ubuntu `build` legs, sanitizers, kernel-only, native-live and the parity
924-
subset); a pull request that touches the corpus pin or the parity tooling also
925-
starts the whole sweep there, an eighth. A documentation-only change starts
926-
preflight alone there. A second run that overlaps them waits
927-
for a runner. Larger runners bill the organization's Actions budget: with no
928-
payment method or a spending limit of zero, their jobs wait for a runner that
929-
never comes, since GitHub does not fall back to a standard one. The test keeps
930-
a fork's pull request that leaves the workflows alone off the larger runners;
931-
one that edits them runs its own copy, which preflight reports but does not
932-
stop, so the repository's approval rule for workflow runs from outside
933-
contributors is what bounds that case.
911+
parallelism follow the runner: 4 on Ubuntu and 3 on macOS. A documentation-only
912+
change starts preflight alone among the verification jobs.
934913

935914
`scripts/ci_preflight.py` (`ci-workflow-contract`) pins every job's runner, time
936915
limit and parallelism in the three workflows a CI run starts, the `build`
937916
strategy block included, and keeps every other workflow -- `docs.yml`,
938-
`promote-baseline.yml` and any new one -- on standard runners. Only
939-
`release.yml` is exempt, by name and while its `on:` block names only events no
940-
fork can raise (a push, the schedule, a dispatch; a dispatch alone today). A
941-
heavy job moved back to the standard runner, a test or matrix entry that lets a
942-
fork's pull request onto a larger runner, a call to a workflow outside
917+
`promote-baseline.yml` and any new one -- on standard runners. `release.yml`
918+
keeps its existing standard-runner matrix and is exempt from the fixed runner
919+
allowlist, by name and while its `on:` block names only events no fork can
920+
raise (a push, the schedule, a dispatch; a dispatch alone today). The contract
921+
refuses `pf-linux-x64-16`, any `*-xlarge` or `*-large` label, and any
922+
`larger_runner` key in every workflow, including `release.yml`. An event-based
923+
runner selection, a changed build matrix, a call to a workflow outside
943924
`.github/workflows/`, a changed time limit, a `ci_verify.py` call without
944925
exactly one core-count `--jobs` (counted over the whole job, however its
945926
commands are chained, blocked or carried, with the flag and its value on one
@@ -989,17 +970,15 @@ that replaced its old limit, so the margin the removal buys stays visible. The
989970
numbers are measured durations, not a two-times multiplier.
990971
The standard-runner times are the eight CI runs from
991972
main's 8cf3be58 (2026-09-24) to 0d76a099 (2026-09-25), pull requests included,
992-
and the corpus-parity workflow's ten runs of 2026-09-22 to 2026-09-25. Until
993-
the larger runners' own runs accumulate, the maintainers' x86-64 verification
994-
hosts at 8 to 12 jobs stand in for the Linux one on parallel work, the
995-
standard runner's speed stands in for it on serial work, and the 3-core M1
996-
bounds the M2 from above.
973+
and the corpus-parity workflow's ten runs of 2026-09-22 to 2026-09-25. The
974+
maintainers' x86-64 verification-host timings at 8 to 12 jobs remain separate
975+
measurements; the standard-runner timings describe the current runner policy.
997976

998977
| Job | Limit (min) | Measured basis |
999978
| --- | --- | --- |
1000979
| `preflight` | 360 (was 45) | 7.3-9.8 min on the standard runner through 1a0e7ea1; cancelled at 10.3 min at 0d76a099, 593 s into `verifier-tests`, which takes about 15 min there at that tree (above), so preflight about 16 min. 4.8-8.2 min on the verification hosts. Preflight supplies no stage deadline -- the old 2400 s stage bound is gone (owner rule, 2026-10-12); the native-live aggregate job names a stop at the platform's 360-minute maximum `HANG (re-run)` in its log; the check still fails. |
1001-
| `build` | 360 (was 75) | A Release leg runs its full population on every event, a fork's pull request included: 30.3 min Ubuntu and 32.0 min macOS on the standard runners at 0d76a099, where the full Debug legs took 43.5 and 37.0 min. Pull-request Debug sets 17.2-17.7 min. On the larger Linux runner a full leg is its build plus the serial `test_ci_verify` row, 976-1288 s: about 30 min. The M1's 37.0 min bounds the M2. 11.1-18.4 min on the verification hosts. |
1002-
| `sanitizers` | 360 (was 120; the verifier's CTest stage bound was 60 min full, 30 min PR) | A fork's pull-request set took 35.2 min on the standard runner, whose compile alone took 1277-1335 s (pull-request run 36139176292, and main at 0d76a099). Full runs now land only on the larger runner; on the standard runner they took 34.0-69.4 min (at 0d76a099 30 min of build and ABI providers, then 39 min of CTest, 1768 s of it the serial `test_ci_verify` row), so about 40 min there with a faster build, and 18.2-19.0 min on the verification hosts. |
980+
| `build` | 360 (was 75) | A Release leg runs its full population on every event, a fork's pull request included: 30.3 min Ubuntu and 32.0 min macOS on the standard runners at 0d76a099, where the full Debug legs took 43.5 and 37.0 min. Pull-request Debug sets 17.2-17.7 min. 11.1-18.4 min on the verification hosts. |
981+
| `sanitizers` | 360 (was 120; the verifier's CTest stage bound was 60 min full, 30 min PR) | A fork's pull-request set took 35.2 min on the standard runner, whose compile alone took 1277-1335 s (pull-request run 36139176292, and main at 0d76a099). Full runs on the standard runner took 34.0-69.4 min (at 0d76a099 30 min of build and ABI providers, then 39 min of CTest, 1768 s of it the serial `test_ci_verify` row), and 18.2-19.0 min on the verification hosts. |
1003982
| `kernel-only` | 360 (was 60) | Every event runs the whole kernel set: 7.9-22.3 min on the standard runner, 6.5-9.8 min on the verification hosts. The hosted kernel-only run of 2026-10-12 lost its job to the old 1800 s stage bound (exit 124). |
1004983
| `native-live (native)` | 360 (was 75; verification step 65) | Main runs 37398893123, 37461653250 and 37478400505 spent 35.40–40.65 min in native, including 25.32–29.53 min in full CTest. Their matching PRs took 2.39–5.67 min in native CTest with the existing slow exclusion. The longest measured `test_ci_verify` row was 1695 s there and 2178 s on a c5.4xlarge Spot host, cut off by a trial deadline. The verifier supplies no default row or stage deadline and the workflow supplies no step deadline: its old 3600 s row bound is what collided with the 2820 s stage deadline on main's push run 38217844683. These are observed GitHub and Spot timings; final-head cold proof remains required. |
1005984
| `native-live (live-sanitizers)` | 360 (was 45; verification step 35) | The same three main runs spent 15.10–16.33 min in this profile after native. Its old per-row 900 s and stage 1200 s deadlines are gone; the verifier supplies no replacement, and the platform's job maximum catches a hang. |

0 commit comments

Comments
 (0)