Repository navigation
ci: member-only PR admission, then the PR legs run in GKE runner pods - #576
Closed
luisleo526 wants to merge 7 commits into
Closed
luisleo526 wants to merge 7 commits into
luisleo526 wants to merge 7 commits into
Conversation
Move the Linux proof jobs of ci-post.yml, corpus-parity.yml and native-live.yml
from the post-merge GCP VM fleet (and, for the two arm64 builds, the hosted
ubuntu-24.04-arm runner) to the gke-runners group, one size label per job:
gke-32cpu-64gb-amd64 build (Release) x4, sanitizers x4 8 jobs
gke-16cpu-32gb-amd64 build-debug x2, kernel-only x2, native-live x2,
corpus-parity 7 jobs
gke-8cpu-16gb-amd64 preflight, preflight-slow, native-live-shards x4,
corpus-parity-subset 7 jobs
gke-16cpu-32gb-arm64 build-arm (Release, Debug) 2 jobs
A run never asks for more than the group's 8 jobs per size label.
The runner pod is non-root without sudo, so every moved job runs in a root job
container (ubuntu:24.04 pinned by multi-arch index digest) whose first step
installs the base toolchain (git, python3, build-essential, cmake, ninja ...);
the existing dependency steps keep their package lists and only lose "sudo".
Not moved: authz, changes and the aggregate jobs (hosted), ci.yml and
corpus-parity-subset.yml (pull-request reachable), docs.yml, docs-post.yml,
release.yml, promote-baseline.yml, release-ruleset-probe.yml (they hold
credentials) and every macOS leg.
scripts/ci_preflight.py pins each moved job to its exact label, the container
block and the bootstrap step; it refuses the gke-runners group or a gke- label
in any pull-request reachable workflow, a secret, id-token or environment on a
job of the group or on a job that calls a workflow holding such jobs, sudo in a
container job, a paid larger-runner label, and more than 8 jobs on one label
(matrix, include/exclude and reusable-call expansion). actionlint knows the four
labels. docs/ci.md is edited line for line and states that the jobs start only
from the three workflows at refs/heads/main.
docs/ci.md sentences replaced by the GKE routing text, named for the
documentation-revert check by the commit that introduced each:
- "Local reusable calls still cannot mix PR and post-merge verification fleets."
(b0f4341) is replaced by "Local reusable calls cannot mix PR and GKE jobs."
- "A full post run schedules 22 post-fleet jobs; the parity calls share one
concurrency group." (8154993) is replaced by the 24-job GKE count per label.
- "The amd64 post build shards use the post-merge GCP fleet; the two arm64 post
cells run on hosted `ubuntu-24.04-arm`." (8154993) now reads GKE size labels.
- "Credentialed work and the remaining full profiles retain their VM pins."
(3ea5a0d) is replaced by the credentialed-docs pin sentence.
- "The post fleet admits only push, schedule and dispatch on engine `main` and
refuses all PR events." (3a0d150) is superseded by the GKE group-ref sentence.
- "Post-fleet shards require `main` for dispatch; hosted arm64 keeps its fixed
pin." (8154993) is replaced by "GKE jobs start only from ci-post.yml,
corpus-parity.yml and native-live.yml at refs/heads/main".
- "`HOSTED_JOB_RUNNERS` pins hosted jobs and post-fleet verification arrays;
arm64 exceptions exclude PR callers." (8154993) now describes the exact GKE
mappings.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… a job container The gke-runners pods now run the engine toolchain image (git, git-lfs, python3, build-essential, cmake, ninja, clang, ccache, strace, libeigen3-dev, libsqlite3-dev, zlib1g-dev, shellcheck, actionlint, tzdata-legacy) with the base image's passwordless sudo. The jobs therefore run exactly as they did on the VM fleet, so the previous commit's root job container and its bootstrap step are removed and every apt step is again byte-identical to main (with sudo). Routing is unchanged: one size label per job, 8/7/7 jobs on the 32/16/8 CPU amd64 labels and 2 on the 16 CPU arm64 label per ci-post run, never more than the group's 8 per label. scripts/ci_preflight.py drops the container-specific rules (pinned root container, exact bootstrap step, no sudo in container jobs) and instead refuses a `container:` block on a job of the gke-runners group, because the pods carry the toolchain and a bare image would be pulled from a public registry at run time. The label pins, the pull-request-reachability, credential (jobs and their callers), paid-label, cap and timeout rules are unchanged. docs/ci.md sentences reworded by this commit, named for the documentation-revert check by the commit that introduced each (8a3eca2): - "profiles use fixed `gke-runners` size labels in pinned root containers." - "fixed architecture/size labels and a pinned Ubuntu 24.04 root container." - "secret-free post Linux jobs use GKE root containers." - "every GKE job first installs the base toolchain before checkout, without sudo." Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…blocks on GKE jobs Follow-up to the pod-native routing commit, closing four findings of its independent read: - the apt-line pin selected lines with a pattern that ignored a single-line step or one chained with &&; it now takes every line that mentions apt-get and compares the full per-job list with the literals (equal to main's lines), with mutation controls that insert an extra install in both shapes; - the job-container mutation test ran on one job only and carried a dead branch; it now applies all six container forms to every moved job (11 jobs, three workflows); - a job-level `services:` block on a job of the gke-runners group pulls bare images at run time like a job container; the checker now refuses it with its own finding, with mutation tests (block, empty, quoted key) and a check that no real job has one; - a test compares the checker's linter-step constant with the linter step of the real preflight job so a drift of either side fails. No workflow, label or docs change. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… lane output, to be completed before any push)
…s cache (working commit, one open point: the retired corpus-parity-subset.yml caller)
…subset.yml Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Collaborator
Author
|
Superseded by 581: the same change as one commit on current main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ci: run the secret-free post-merge Linux jobs on the GKE runner group
Move the Linux proof jobs of ci-post.yml, corpus-parity.yml and native-live.yml
from the post-merge GCP VM fleet (and, for the two arm64 builds, the hosted
ubuntu-24.04-arm runner) to the gke-runners group, one size label per job:
gke-32cpu-64gb-amd64 build (Release) x4, sanitizers x4 8 jobs
gke-16cpu-32gb-amd64 build-debug x2, kernel-only x2, native-live x2,
corpus-parity 7 jobs
gke-8cpu-16gb-amd64 preflight, preflight-slow, native-live-shards x4,
corpus-parity-subset 7 jobs
gke-16cpu-32gb-arm64 build-arm (Release, Debug) 2 jobs
A run never asks for more than the group's 8 jobs per size label.
The runner pod is non-root without sudo, so every moved job runs in a root job
container (ubuntu:24.04 pinned by multi-arch index digest) whose first step
installs the base toolchain (git, python3, build-essential, cmake, ninja ...);
the existing dependency steps keep their package lists and only lose "sudo".
Not moved: authz, changes and the aggregate jobs (hosted), ci.yml and
corpus-parity-subset.yml (pull-request reachable), docs.yml, docs-post.yml,
release.yml, promote-baseline.yml, release-ruleset-probe.yml (they hold
credentials) and every macOS leg.
scripts/ci_preflight.py pins each moved job to its exact label, the container
block and the bootstrap step; it refuses the gke-runners group or a gke- label
in any pull-request reachable workflow, a secret, id-token or environment on a
job of the group or on a job that calls a workflow holding such jobs, sudo in a
container job, a paid larger-runner label, and more than 8 jobs on one label
(matrix, include/exclude and reusable-call expansion). actionlint knows the four
labels. docs/ci.md is edited line for line and states that the jobs start only
from the three workflows at refs/heads/main.
docs/ci.md sentences replaced by the GKE routing text, named for the
documentation-revert check by the commit that introduced each:
(b0f4341) is replaced by "Local reusable calls cannot mix PR and GKE jobs."
concurrency group." (8154993) is replaced by the 24-job GKE count per label.
cells run on hosted
ubuntu-24.04-arm." (8154993) now reads GKE size labels.(3ea5a0d) is replaced by the credentialed-docs pin sentence.
mainandrefuses all PR events." (3a0d150) is superseded by the GKE group-ref sentence.
mainfor dispatch; hosted arm64 keeps its fixedpin." (8154993) is replaced by "GKE jobs start only from ci-post.yml,
corpus-parity.yml and native-live.yml at refs/heads/main".
HOSTED_JOB_RUNNERSpins hosted jobs and post-fleet verification arrays;arm64 exceptions exclude PR callers." (8154993) now describes the exact GKE
mappings.
ci: run the Linux proof jobs directly in the GKE runner pods, without a job container
The gke-runners pods now run the engine toolchain image (git, git-lfs, python3,
build-essential, cmake, ninja, clang, ccache, strace, libeigen3-dev,
libsqlite3-dev, zlib1g-dev, shellcheck, actionlint, tzdata-legacy) with the base
image's passwordless sudo. The jobs therefore run exactly as they did on the VM
fleet, so the previous commit's root job container and its bootstrap step are
removed and every apt step is again byte-identical to main (with sudo).
Routing is unchanged: one size label per job, 8/7/7 jobs on the 32/16/8 CPU
amd64 labels and 2 on the 16 CPU arm64 label per ci-post run, never more than the
group's 8 per label.
scripts/ci_preflight.py drops the container-specific rules (pinned root
container, exact bootstrap step, no sudo in container jobs) and instead refuses
a
container:block on a job of the gke-runners group, because the pods carrythe toolchain and a bare image would be pulled from a public registry at run
time. The label pins, the pull-request-reachability, credential (jobs and their
callers), paid-label, cap and timeout rules are unchanged.
docs/ci.md sentences reworded by this commit, named for the documentation-revert
check by the commit that introduced each (8a3eca2):
gke-runnerssize labels in pinned root containers."tests: pin every apt line, cover every moved job and refuse services blocks on GKE jobs
Follow-up to the pod-native routing commit, closing four findings of its
independent read:
or one chained with &&; it now takes every line that mentions apt-get and
compares the full per-job list with the literals (equal to main's lines), with
mutation controls that insert an extra install in both shapes;
it now applies all six container forms to every moved job (11 jobs, three
workflows);
services:block on a job of the gke-runners group pulls bareimages at run time like a job container; the checker now refuses it with its
own finding, with mutation tests (block, empty, quoted key) and a check that no
real job has one;
real preflight job so a drift of either side fails.
No workflow, label or docs change.
ci: route same-repository PR proof through admission (working commit, lane output, to be completed before any push)
ci: finish the PR admission contracts and remove the live curl Actions cache (working commit, one open point: the retired corpus-parity-subset.yml caller)
🤖 Generated with Claude Code