Skip to content

ci: member-only PR admission, then the PR legs run in GKE runner pods - #576

Closed
luisleo526 wants to merge 7 commits into
mainfrom
ci/gke-pr-admission
Closed

luisleo526 wants to merge 7 commits into
mainfrom
ci/gke-pr-admission

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

ci: run the secret-free post-merge Linux jobs on the GKE runner group

Move the Linux proof jobs of ci-post.yml, corpus-parity.yml and native-live.yml
from the post-merge GCP VM fleet (and, for the two arm64 builds, the hosted
ubuntu-24.04-arm runner) to the gke-runners group, one size label per job:

gke-32cpu-64gb-amd64 build (Release) x4, sanitizers x4 8 jobs
gke-16cpu-32gb-amd64 build-debug x2, kernel-only x2, native-live x2,
corpus-parity 7 jobs
gke-8cpu-16gb-amd64 preflight, preflight-slow, native-live-shards x4,
corpus-parity-subset 7 jobs
gke-16cpu-32gb-arm64 build-arm (Release, Debug) 2 jobs

A run never asks for more than the group's 8 jobs per size label.

The runner pod is non-root without sudo, so every moved job runs in a root job
container (ubuntu:24.04 pinned by multi-arch index digest) whose first step
installs the base toolchain (git, python3, build-essential, cmake, ninja ...);
the existing dependency steps keep their package lists and only lose "sudo".

Not moved: authz, changes and the aggregate jobs (hosted), ci.yml and
corpus-parity-subset.yml (pull-request reachable), docs.yml, docs-post.yml,
release.yml, promote-baseline.yml, release-ruleset-probe.yml (they hold
credentials) and every macOS leg.

scripts/ci_preflight.py pins each moved job to its exact label, the container
block and the bootstrap step; it refuses the gke-runners group or a gke- label
in any pull-request reachable workflow, a secret, id-token or environment on a
job of the group or on a job that calls a workflow holding such jobs, sudo in a
container job, a paid larger-runner label, and more than 8 jobs on one label
(matrix, include/exclude and reusable-call expansion). actionlint knows the four
labels. docs/ci.md is edited line for line and states that the jobs start only
from the three workflows at refs/heads/main.

docs/ci.md sentences replaced by the GKE routing text, named for the
documentation-revert check by the commit that introduced each:

  • "Local reusable calls still cannot mix PR and post-merge verification fleets."
    (b0f4341) is replaced by "Local reusable calls cannot mix PR and GKE jobs."
  • "A full post run schedules 22 post-fleet jobs; the parity calls share one
    concurrency group." (8154993) is replaced by the 24-job GKE count per label.
  • "The amd64 post build shards use the post-merge GCP fleet; the two arm64 post
    cells run on hosted ubuntu-24.04-arm." (8154993) now reads GKE size labels.
  • "Credentialed work and the remaining full profiles retain their VM pins."
    (3ea5a0d) is replaced by the credentialed-docs pin sentence.
  • "The post fleet admits only push, schedule and dispatch on engine main and
    refuses all PR events." (3a0d150) is superseded by the GKE group-ref sentence.
  • "Post-fleet shards require main for dispatch; hosted arm64 keeps its fixed
    pin." (8154993) is replaced by "GKE jobs start only from ci-post.yml,
    corpus-parity.yml and native-live.yml at refs/heads/main".
  • "HOSTED_JOB_RUNNERS pins hosted jobs and post-fleet verification arrays;
    arm64 exceptions exclude PR callers." (8154993) now describes the exact GKE
    mappings.

ci: run the Linux proof jobs directly in the GKE runner pods, without a job container

The gke-runners pods now run the engine toolchain image (git, git-lfs, python3,
build-essential, cmake, ninja, clang, ccache, strace, libeigen3-dev,
libsqlite3-dev, zlib1g-dev, shellcheck, actionlint, tzdata-legacy) with the base
image's passwordless sudo. The jobs therefore run exactly as they did on the VM
fleet, so the previous commit's root job container and its bootstrap step are
removed and every apt step is again byte-identical to main (with sudo).

Routing is unchanged: one size label per job, 8/7/7 jobs on the 32/16/8 CPU
amd64 labels and 2 on the 16 CPU arm64 label per ci-post run, never more than the
group's 8 per label.

scripts/ci_preflight.py drops the container-specific rules (pinned root
container, exact bootstrap step, no sudo in container jobs) and instead refuses
a container: block on a job of the gke-runners group, because the pods carry
the toolchain and a bare image would be pulled from a public registry at run
time. The label pins, the pull-request-reachability, credential (jobs and their
callers), paid-label, cap and timeout rules are unchanged.

docs/ci.md sentences reworded by this commit, named for the documentation-revert
check by the commit that introduced each (8a3eca2):

  • "profiles use fixed gke-runners size labels in pinned root containers."
  • "fixed architecture/size labels and a pinned Ubuntu 24.04 root container."
  • "secret-free post Linux jobs use GKE root containers."
  • "every GKE job first installs the base toolchain before checkout, without sudo."

tests: pin every apt line, cover every moved job and refuse services blocks on GKE jobs

Follow-up to the pod-native routing commit, closing four findings of its
independent read:

  • the apt-line pin selected lines with a pattern that ignored a single-line step
    or one chained with &&; it now takes every line that mentions apt-get and
    compares the full per-job list with the literals (equal to main's lines), with
    mutation controls that insert an extra install in both shapes;
  • the job-container mutation test ran on one job only and carried a dead branch;
    it now applies all six container forms to every moved job (11 jobs, three
    workflows);
  • a job-level services: block on a job of the gke-runners group pulls bare
    images at run time like a job container; the checker now refuses it with its
    own finding, with mutation tests (block, empty, quoted key) and a check that no
    real job has one;
  • a test compares the checker's linter-step constant with the linter step of the
    real preflight job so a drift of either side fails.

No workflow, label or docs change.

ci: route same-repository PR proof through admission (working commit, lane output, to be completed before any push)

ci: finish the PR admission contracts and remove the live curl Actions cache (working commit, one open point: the retired corpus-parity-subset.yml caller)

🤖 Generated with Claude Code

ci-cx and others added 7 commits October 28, 2026 01:25
Move the Linux proof jobs of ci-post.yml, corpus-parity.yml and native-live.yml
from the post-merge GCP VM fleet (and, for the two arm64 builds, the hosted
ubuntu-24.04-arm runner) to the gke-runners group, one size label per job:

  gke-32cpu-64gb-amd64  build (Release) x4, sanitizers x4                 8 jobs
  gke-16cpu-32gb-amd64  build-debug x2, kernel-only x2, native-live x2,
                        corpus-parity                                      7 jobs
  gke-8cpu-16gb-amd64   preflight, preflight-slow, native-live-shards x4,
                        corpus-parity-subset                               7 jobs
  gke-16cpu-32gb-arm64  build-arm (Release, Debug)                         2 jobs

A run never asks for more than the group's 8 jobs per size label.

The runner pod is non-root without sudo, so every moved job runs in a root job
container (ubuntu:24.04 pinned by multi-arch index digest) whose first step
installs the base toolchain (git, python3, build-essential, cmake, ninja ...);
the existing dependency steps keep their package lists and only lose "sudo".

Not moved: authz, changes and the aggregate jobs (hosted), ci.yml and
corpus-parity-subset.yml (pull-request reachable), docs.yml, docs-post.yml,
release.yml, promote-baseline.yml, release-ruleset-probe.yml (they hold
credentials) and every macOS leg.

scripts/ci_preflight.py pins each moved job to its exact label, the container
block and the bootstrap step; it refuses the gke-runners group or a gke- label
in any pull-request reachable workflow, a secret, id-token or environment on a
job of the group or on a job that calls a workflow holding such jobs, sudo in a
container job, a paid larger-runner label, and more than 8 jobs on one label
(matrix, include/exclude and reusable-call expansion). actionlint knows the four
labels. docs/ci.md is edited line for line and states that the jobs start only
from the three workflows at refs/heads/main.

docs/ci.md sentences replaced by the GKE routing text, named for the
documentation-revert check by the commit that introduced each:
- "Local reusable calls still cannot mix PR and post-merge verification fleets."
  (b0f4341) is replaced by "Local reusable calls cannot mix PR and GKE jobs."
- "A full post run schedules 22 post-fleet jobs; the parity calls share one
  concurrency group." (8154993) is replaced by the 24-job GKE count per label.
- "The amd64 post build shards use the post-merge GCP fleet; the two arm64 post
  cells run on hosted `ubuntu-24.04-arm`." (8154993) now reads GKE size labels.
- "Credentialed work and the remaining full profiles retain their VM pins."
  (3ea5a0d) is replaced by the credentialed-docs pin sentence.
- "The post fleet admits only push, schedule and dispatch on engine `main` and
  refuses all PR events." (3a0d150) is superseded by the GKE group-ref sentence.
- "Post-fleet shards require `main` for dispatch; hosted arm64 keeps its fixed
  pin." (8154993) is replaced by "GKE jobs start only from ci-post.yml,
  corpus-parity.yml and native-live.yml at refs/heads/main".
- "`HOSTED_JOB_RUNNERS` pins hosted jobs and post-fleet verification arrays;
  arm64 exceptions exclude PR callers." (8154993) now describes the exact GKE
  mappings.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… a job container

The gke-runners pods now run the engine toolchain image (git, git-lfs, python3,
build-essential, cmake, ninja, clang, ccache, strace, libeigen3-dev,
libsqlite3-dev, zlib1g-dev, shellcheck, actionlint, tzdata-legacy) with the base
image's passwordless sudo. The jobs therefore run exactly as they did on the VM
fleet, so the previous commit's root job container and its bootstrap step are
removed and every apt step is again byte-identical to main (with sudo).

Routing is unchanged: one size label per job, 8/7/7 jobs on the 32/16/8 CPU
amd64 labels and 2 on the 16 CPU arm64 label per ci-post run, never more than the
group's 8 per label.

scripts/ci_preflight.py drops the container-specific rules (pinned root
container, exact bootstrap step, no sudo in container jobs) and instead refuses
a `container:` block on a job of the gke-runners group, because the pods carry
the toolchain and a bare image would be pulled from a public registry at run
time. The label pins, the pull-request-reachability, credential (jobs and their
callers), paid-label, cap and timeout rules are unchanged.

docs/ci.md sentences reworded by this commit, named for the documentation-revert
check by the commit that introduced each (8a3eca2):
- "profiles use fixed `gke-runners` size labels in pinned root containers."
- "fixed architecture/size labels and a pinned Ubuntu 24.04 root container."
- "secret-free post Linux jobs use GKE root containers."
- "every GKE job first installs the base toolchain before checkout, without sudo."

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…blocks on GKE jobs

Follow-up to the pod-native routing commit, closing four findings of its
independent read:

- the apt-line pin selected lines with a pattern that ignored a single-line step
  or one chained with &&; it now takes every line that mentions apt-get and
  compares the full per-job list with the literals (equal to main's lines), with
  mutation controls that insert an extra install in both shapes;
- the job-container mutation test ran on one job only and carried a dead branch;
  it now applies all six container forms to every moved job (11 jobs, three
  workflows);
- a job-level `services:` block on a job of the gke-runners group pulls bare
  images at run time like a job container; the checker now refuses it with its
  own finding, with mutation tests (block, empty, quoted key) and a check that no
  real job has one;
- a test compares the checker's linter-step constant with the linter step of the
  real preflight job so a drift of either side fails.

No workflow, label or docs change.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… lane output, to be completed before any push)
…s cache (working commit, one open point: the retired corpus-parity-subset.yml caller)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@luisleo526

Copy link
Copy Markdown
Collaborator Author

Superseded by 581: the same change as one commit on current main.

@luisleo526 luisleo526 closed this Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant