Skip to content

Commit b09f79d

Browse files
luisleo526claude
andauthored
ci: run release and notify-hub hosted and compare glibc with the v1.10.0 archive (#768)
* ci: run release and notify-hub hosted and compare glibc with the v1.10.0 archive release and notify-hub move to ubuntu-24.04 and keep their release environment. The x86_64 prebuilt cell stays on its protected scale set and still builds inside the pinned Bookworm image; the arm64 cell keeps ubuntu-22.04-arm. The prebuilt glibc gate now also compares the highest numeric GLIBC_ tag of the staged libraries with the same triple's v1.10.0 release archive. The workflow downloads that archive, checks its sha256 and unpacks it, and check_prebuilt_glibc.py --baseline-root reads both trees with the same strict inspection: equal or lower passes, higher fails. The fixed 2.36 ceiling and the __isoc23_* rule still apply to the staged tree; the baseline is read, not judged, because the released x86_64 archive predates both. Preflight pins the new step and the hosted pins, still refuses removing the release environment from either job, and now also refuses a job container or services on the environment-gated release jobs, which the protected scale-set route refused before. The runner sentences in docs/ci.md (introduced by bd1fa2f) and in docs/reference/ci-runner-routing.md (introduced by e7fa559) follow the move. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * Preserve release runner contracts while guarding protected jobs Reject containers and services on protected release jobs and adapt the preflight tests to the hosted release workflow. Keep legacy admission coverage and repair the actionlint label mutation fixtures. --------- Co-authored-by: Claude Sonnet 5.5 <[email protected]>
1 parent be584eb commit b09f79d

9 files changed

Lines changed: 443 additions & 84 deletions

File tree

‎.github/actionlint.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
# The unchanged release workflow retains its protected runner labels.
1+
# Release keeps one protected runner label, on its x86_64 prebuilt cell.
22
self-hosted-runner:
33
labels:
4-
- gkp-od-4cpu-8gb-amd64
54
- gkp-od-16cpu-32gb-amd64-128gb

‎.github/workflows/release.yml‎

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ env:
7777

7878
jobs:
7979
release:
80-
runs-on: gkp-od-4cpu-8gb-amd64
80+
runs-on: ubuntu-24.04
8181
environment: release
8282

8383
outputs:
@@ -485,7 +485,32 @@ jobs:
485485
python3 scripts/release_version.py check-install \
486486
--prefix "${STAGE_DIR}" --expect "${NEXT}"
487487
488-
# Both dry-run and publication inspect every installed .a/.so/.so.*.
488+
# The glibc comparison baseline is this triple's archive from the v1.10.0
489+
# release: the real released bytes, pinned by sha256 and unpacked here.
490+
# The checker below only reads the unpacked libraries.
491+
- name: Fetch the v1.10.0 baseline archive for the glibc comparison
492+
env:
493+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
494+
TRIPLE: ${{ matrix.triple }}
495+
run: |
496+
set -euo pipefail
497+
case "${TRIPLE}" in
498+
linux-x86_64) digest=cb412fbae9e8ba515de41dc31026c77d6c5e309693247bee26bb93c67d3c3543 ;;
499+
linux-aarch64) digest=982b16da2d5f5ee7c274615895484fce36bd740052a14d59b66576dca496d7bb ;;
500+
*) echo "::error::no v1.10.0 baseline archive for ${TRIPLE}" >&2; exit 1 ;;
501+
esac
502+
archive="pineforge-v1.10.0-${TRIPLE}.tar.gz"
503+
root="${RUNNER_TEMP}/glibc-baseline-${TRIPLE}"
504+
mkdir "${root}" "${root}/download" "${root}/tree"
505+
gh release download v1.10.0 --repo "${GITHUB_REPOSITORY}" \
506+
--pattern "${archive}" --dir "${root}/download"
507+
echo "${digest} ${root}/download/${archive}" | sha256sum -c -
508+
tar -xzf "${root}/download/${archive}" -C "${root}/tree" \
509+
--no-same-owner --no-same-permissions
510+
echo "BASELINE_DIR=${root}/tree" >> "${GITHUB_ENV}"
511+
512+
# Both dry-run and publication inspect every installed .a/.so/.so.* and
513+
# compare their highest glibc tag with the baseline fetched above.
489514
# Keep raw checker output and its actual exit status, including setup errors.
490515
- name: Check every staged library's glibc floor
491516
env:
@@ -494,7 +519,7 @@ jobs:
494519
set -euo pipefail
495520
log="${RUNNER_TEMP}/prebuilt-glibc-${TRIPLE}"
496521
mkdir -p "${log}"
497-
if python3 scripts/check_prebuilt_glibc.py "${STAGE_DIR}" > "${log}/stdout.jsonl" 2> "${log}/stderr.txt"; then
522+
if python3 scripts/check_prebuilt_glibc.py "${STAGE_DIR}" --baseline-root "${BASELINE_DIR}" > "${log}/stdout.jsonl" 2> "${log}/stderr.txt"; then
498523
checker_rc=0
499524
else checker_rc=$?; fi
500525
printf '%s\n' "${checker_rc}" > "${log}/exitcode.txt"
@@ -565,7 +590,7 @@ jobs:
565590
notify-hub:
566591
needs: [release, prebuilt]
567592
if: ${{ !inputs.dry_run }}
568-
runs-on: gkp-od-4cpu-8gb-amd64
593+
runs-on: ubuntu-24.04
569594
environment: release
570595
steps:
571596
- name: Mint org App token (hub dispatch)

‎docs/ci.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ Python refuse a system-wide install (PEP 668), and `.venv/` is ignored. The
113113
and fails, never skips, without it.
114114

115115
Actionlint only accepts declared custom self-hosted runner labels; preflight
116-
checks the declarations in `.github/actionlint.yaml`, which now declare only the two protected `gkp-od-*` labels of `release.yml`, including its x86_64 release matrix cell. Hosted workflows declare nothing. Before a workflow moves onto a `gke-*` label, add its ref to `.github/runner-group-allowlist.txt` (group 14, currently empty because no workflow uses one); protected `gkp-*` refs belong in `.github/runner-group-15-allowlist.txt` (group 15, currently only `release.yml`). Both mirrors are binding; a maintainer must confirm the corresponding live org workflow/ref entries before enabling the routes.
116+
checks the declarations in `.github/actionlint.yaml`, which now declare only the protected `gkp-od-*` label of `release.yml`'s x86_64 release matrix cell. Hosted workflows declare nothing. Before a workflow moves onto a `gke-*` label, add its ref to `.github/runner-group-allowlist.txt` (group 14, currently empty because no workflow uses one); protected `gkp-*` refs belong in `.github/runner-group-15-allowlist.txt` (group 15, currently only `release.yml`). Both mirrors are binding; a maintainer must confirm the corresponding live org workflow/ref entries before enabling the routes.
117117

118118
ORG-ACTIONPINS is enforced by preflight's `ci-workflow-contract`: every
119119
third-party `uses:` in `.github/workflows/*.yml` (also `.yaml`) and nested
@@ -1644,7 +1644,7 @@ The native matrix is exactly `1..4` with `--ctest-shard I/4`; live sanitizers ar
16441644
| `corpus-parity.yml` | Schedule, dispatch, post CI callers | `authz`, full parity and subset: `ubuntu-24.04` |
16451645
| `native-live.yml` | Dispatch and post CI callers on `main` | `authz`, both aggregates and the native shards: `ubuntu-24.04`; `matrix.runner`: live-sanitizers and live-tsan both `ubuntu-24.04` |
16461646
| `docs-post.yml` | Push to `main`, dispatch | `authz` and credentialed `build`: `ubuntu-24.04` |
1647-
| `release.yml` | Dispatch from `main` | `release` and `notify-hub`: `gkp-od-4cpu-8gb-amd64`; x86_64 `prebuilt`: `gkp-od-16cpu-32gb-amd64-128gb`; arm64 `prebuilt`: hosted `ubuntu-22.04-arm` |
1647+
| `release.yml` | Dispatch from `main` | `release` and `notify-hub`: hosted `ubuntu-24.04`; x86_64 `prebuilt`: `gkp-od-16cpu-32gb-amd64-128gb`; arm64 `prebuilt`: hosted `ubuntu-22.04-arm` |
16481648
| `promote-baseline.yml` | Push to `main`, dispatch from `main` | `promote-guard` and `promote`: `ubuntu-24.04` |
16491649
| `release-ruleset-probe.yml` | Dispatch from `main` | `authz` and `probe`: `ubuntu-24.04` |
16501650

@@ -1668,15 +1668,15 @@ Trusted post workflows and PR proof legs build cold; no trusted job restores an
16681668
`HOSTED_JOB_RUNNERS` pins the hosted x86 and arm64 jobs and the native profile runner matrix; `GKP_JOB_RUNNERS` separately pins the protected release jobs and sizes. Every pinned job runs directly on its hosted image. PR checks are
16691669
`build`, `build-debug`, `corpus-parity-subset`, `legs` and `result` in `gke-pr.yml`. With the arm leg, the full set is `build`, `build-debug`, `build-arm`, `corpus-parity-subset`, `legs` and `result` in `gke-pr.yml`.
16701670

1671-
Release uses protected GKE runner group 15:
1672-
`release` and `notify-hub` use `gkp-od-4cpu-8gb-amd64`;
1671+
Release uses protected GKE runner group 15 for its x64 artifact only:
1672+
`release` and `notify-hub` run on hosted `ubuntu-24.04` and keep their `release` environment;
16731673
Linux x64 `prebuilt` uses `gkp-od-16cpu-32gb-amd64-128gb`; baseline `promote` uses hosted `ubuntu-24.04`.
16741674
The release ruleset probe runs `authz` and `probe` on hosted `ubuntu-24.04`.
16751675
Release's ARM artifact cell retains hosted `ubuntu-22.04-arm`;
16761676
1.8.0 has no macOS test or artifact cells, and
16771677
promotion runs on a push to `main` plus manual dispatch from `main`.
16781678

1679-
Release and notify-hub use the on-demand 4 CPU/8 GB set to prevent preemption after tag push.
1679+
Release and notify-hub run on hosted `ubuntu-24.04`, which no scale-set preemption can interrupt after tag push.
16801680
Release computes versions, stamps JSON catalogs and publishes with git/gh; notify-hub
16811681
mints an App token and dispatches an API request. The protected 2 CPU/4 GB set
16821682
runs promotion's API/tree checks, Node campaign tooling and Cloud SQL proxy; the

‎docs/reference/ci-runner-routing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ parity verdict; an outside contributor cannot post them. GitHub Actions is advis
55
The PR build (`ci.yml`) runs only the `fast`-labelled CTest rows (`ctest -L '^fast$'`); the post-merge legs (`ci-post.yml`) run
66
every row, and a documentation-only change runs preflight alone among verification jobs.
77
The seven PR/post-merge verification and documentation workflows start with the canon `authz` job and every other job in them needs it; release uses its status gate and baseline promotion its `promote-guard`. Pull-request
8-
proof workflows and post jobs use exact hosted pins: `ubuntu-24.04` for x86 and `ubuntu-24.04-arm` for the arm64 leg. Promotion, the ruleset probe and docs deployment also run on hosted `ubuntu-24.04`, and keep their `release`/`release-bot` environments. Only `release.yml` still names protected scale sets: `release` and `notify-hub` use `gkp-od-4cpu-8gb-amd64` in protected runner group 15. Hosted PR light jobs and `docs.yml`'s `build` retain `ubuntu-24.04`; post guards and aggregates use the same pin.
8+
proof workflows and post jobs use exact hosted pins: `ubuntu-24.04` for x86 and `ubuntu-24.04-arm` for the arm64 leg. Promotion, the ruleset probe and docs deployment also run on hosted `ubuntu-24.04`, and keep their `release`/`release-bot` environments. Only `release.yml` still names a protected scale set, in protected runner group 15: its x86_64 `prebuilt` cell. `release` and `notify-hub` run on hosted `ubuntu-24.04` and keep their `release` environment. Hosted PR light jobs and `docs.yml`'s `build` retain `ubuntu-24.04`; post guards and aggregates use the same pin.
99
Hosted fork PRs receive no secrets, so they need no VM-only fork-refusal hook. Retired main/post VM labels remain refused. The retired `gke-*` scale sets and paid larger-runner labels are refused too. Group 14's mirror `.github/runner-group-allowlist.txt` is empty because no workflow uses a group-14 label; group 15's required ref (`release.yml`) is in `.github/runner-group-15-allowlist.txt`, with live membership confirmed by a maintainer.
1010
Release's x86_64 prebuilt cell uses `gkp-od-16cpu-32gb-amd64-128gb`, installs Eigen and asserts exactly 3.4.0 before CMake; the arm64 cell retains hosted `ubuntu-22.04-arm`. App-token jobs keep their exact `release`/`release-bot` environments. The `build-gate` job in `ci-post.yml` and `native-live.yml`'s `native-live-gate` always run and refuse a non-success authz result first.
1111
See `docs/ci.md` for the class table and docs-only exception. The commands below run the full sets.

‎scripts/check_prebuilt_glibc.py‎

Lines changed: 110 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,18 @@
66
GNU ar/readelf must already be installed in the OS default executable search
77
path. Never extract archives, load libraries, execute payloads, or write files.
88
9+
--baseline-root DIR adds the released-baseline comparison. DIR is an already
10+
unpacked released archive (the workflow checks its digest and unpacks it; this
11+
tool still extracts nothing). The same strict inspection reads every library
12+
under DIR, and the highest numeric GLIBC_ tag over all staged libraries may not
13+
exceed the highest over all baseline libraries: equal or lower passes, higher
14+
fails. A tree with no numeric tag ranks below any tag. The baseline is read, not
15+
judged: the fixed ceiling and the __isoc23_* rule apply to the staging tree
16+
only, since a released archive may predate them. A missing, empty or partly
17+
uninspectable baseline, or one overlapping the staging root, fails closed. No
18+
expected value is configured anywhere: the verdict is a function of the two
19+
inspected trees.
20+
921
All symlinks (including directory links), protected/key/config/.env* paths,
1022
unreadable entries, thin archives, non-ELF members, and wrong ELF types fail.
1123
Archives must contain REL objects with inspectable symbol tables; shared objects
@@ -65,8 +77,13 @@ class InspectionError(Exception):
6577
pass
6678

6779

80+
# Fields added to every record while the released baseline is being read.
81+
EVENT_CONTEXT: dict[str, str] = {}
82+
83+
6884
def emit(event: str, **fields) -> None:
69-
print(json.dumps({"event": event, **fields}, ensure_ascii=True), flush=True)
85+
print(json.dumps({"event": event, **EVENT_CONTEXT, **fields}, ensure_ascii=True),
86+
flush=True)
7087

7188

7289
def issue(issues: list[dict], path: Path, cause: str, **fields) -> None:
@@ -282,6 +299,42 @@ def maximum_version(names: list[str]) -> str | None:
282299
return name[len("GLIBC_"):]
283300

284301

302+
def tree_maximum(rows: list[dict]) -> tuple[int, ...] | None:
303+
"""Highest numeric GLIBC_ tag over every inspected library of one tree."""
304+
highest = maximum_version(sorted({name for row in rows for name in row["versions"]}))
305+
return None if highest is None else tuple(int(part) for part in highest.split("."))
306+
307+
308+
def version_text(version: tuple[int, ...] | None) -> str | None:
309+
return None if version is None else ".".join(str(part) for part in version)
310+
311+
312+
def glibc_label(version: tuple[int, ...] | None) -> str:
313+
return "none" if version is None else "GLIBC_" + version_text(version)
314+
315+
316+
def exceeds_baseline(candidate: tuple[int, ...] | None,
317+
baseline: tuple[int, ...] | None) -> bool:
318+
"""True when the staged tree's highest numeric tag is above the baseline's.
319+
320+
Components compare with zero padding, so 2.36 equals 2.36.0. No numeric tag
321+
at all ranks below every tag: a staged tag above a tagless baseline exceeds
322+
it, and a tagless staged tree never does.
323+
"""
324+
if candidate is None:
325+
return False
326+
if baseline is None:
327+
return True
328+
width = max(len(candidate), len(baseline))
329+
return (candidate + (0,) * (width - len(candidate))
330+
> baseline + (0,) * (width - len(baseline)))
331+
332+
333+
def roots_overlap(left: Path, right: Path) -> bool:
334+
first, second = (Path(os.path.abspath(path)) for path in (left, right))
335+
return first == second or first in second.parents or second in first.parents
336+
337+
285338
def version_sections(lines: list[str], dynamic_count: int | None) -> set[str]:
286339
headers = [(index, VERSION_HEADER.fullmatch(line))
287340
for index, line in enumerate(lines) if line.startswith("Version ")]
@@ -493,7 +546,7 @@ def policy_issues(path: Path, member: dict, issues: list[dict]) -> None:
493546

494547

495548
def inspect_target(root_fd: int, target: dict, directories: dict,
496-
tools: dict[str, str], issues: list[dict]) -> dict:
549+
tools: dict[str, str], issues: list[dict], policy: bool = True) -> dict:
497550
path = target["path"]
498551
row = {
499552
"path": str(path), "kind": target["kind"], "identity": target["identity"],
@@ -580,7 +633,8 @@ def inspect_target(root_fd: int, target: dict, directories: dict,
580633
try:
581634
member.update(parse_elf(lines, "REL" if expected_members is not None else "DYN"))
582635
member["inspected"] = True
583-
policy_issues(path, member, issues)
636+
if policy:
637+
policy_issues(path, member, issues)
584638
except (InspectionError, ValueError) as error:
585639
issue(issues, path, str(error), member=name, ordinal=ordinal)
586640
complete = False
@@ -613,16 +667,12 @@ def inspect_target(root_fd: int, target: dict, directories: dict,
613667
return row
614668

615669

616-
def main(argv: list[str] | None = None) -> int:
617-
parser = argparse.ArgumentParser(description=__doc__)
618-
parser.add_argument("staging_root", type=Path, help="explicit installed prebuilt staging root")
619-
args = parser.parse_args(argv)
620-
issues = []
670+
def inspect_tree(root: Path, issues: list[dict], policy: bool) -> tuple[list[dict], list[dict]]:
671+
"""Inventory and inspect one explicit root; the fixed policy is for staging only."""
621672
targets = []
622673
results = []
623674
root_fd = None
624-
root = args.staging_root
625-
emit("root", path=str(root), fixed_glibc_ceiling="2.36")
675+
emit("root", path=str(root), **({"fixed_glibc_ceiling": "2.36"} if policy else {}))
626676
try:
627677
if sys.platform != "linux":
628678
raise InspectionError("Linux descriptor-based ELF inspection is required")
@@ -638,7 +688,7 @@ def main(argv: list[str] | None = None) -> int:
638688
issue(issues, root, "no shipped .so/.so.*/.a libraries in staging inventory")
639689
tools = installed_tools(root, issues) if targets else {}
640690
for target in targets:
641-
results.append(inspect_target(root_fd, target, directories, tools, issues))
691+
results.append(inspect_target(root_fd, target, directories, tools, issues, policy))
642692
# Re-open the path and re-inventory metadata to catch late additions,
643693
# removals, renames, or a replacement root; payloads are never re-read.
644694
final_fd = open_root(root)
@@ -657,12 +707,59 @@ def main(argv: list[str] | None = None) -> int:
657707
finally:
658708
if root_fd is not None:
659709
os.close(root_fd)
710+
return targets, results
711+
712+
713+
def main(argv: list[str] | None = None) -> int:
714+
parser = argparse.ArgumentParser(description=__doc__)
715+
parser.add_argument("staging_root", type=Path, help="explicit installed prebuilt staging root")
716+
parser.add_argument("--baseline-root", default=None, metavar="DIR",
717+
help="explicit unpacked released archive; the staging tree's highest "
718+
"numeric GLIBC_ tag may not exceed the highest found there")
719+
args = parser.parse_args(argv)
720+
if args.baseline_root is not None and not args.baseline_root:
721+
parser.error("--baseline-root must name a directory")
722+
issues = []
723+
targets, results = inspect_tree(args.staging_root, issues, True)
660724
inspected = sum(row["inspected"] for row in results)
661-
success = bool(targets) and inspected == len(targets) and not issues
725+
complete = bool(targets) and inspected == len(targets)
726+
baseline_fields = {}
727+
if args.baseline_root is not None:
728+
baseline_root = Path(args.baseline_root)
729+
baseline_targets = []
730+
baseline_results = []
731+
if roots_overlap(args.staging_root, baseline_root):
732+
issue(issues, baseline_root.absolute(),
733+
"baseline root must be separate from the staging root")
734+
else:
735+
EVENT_CONTEXT["role"] = "baseline"
736+
try:
737+
baseline_targets, baseline_results = inspect_tree(baseline_root, issues, False)
738+
finally:
739+
EVENT_CONTEXT.clear()
740+
baseline_inspected = sum(row["inspected"] for row in baseline_results)
741+
baseline_fields = {"baseline_discovered": len(baseline_targets),
742+
"baseline_inspected": baseline_inspected}
743+
complete = (complete and bool(baseline_targets)
744+
and baseline_inspected == len(baseline_targets))
745+
if complete:
746+
staged_highest = tree_maximum(results)
747+
released_highest = tree_maximum(baseline_results)
748+
above = exceeds_baseline(staged_highest, released_highest)
749+
emit("baseline_comparison", staged_max_glibc=version_text(staged_highest),
750+
baseline_max_glibc=version_text(released_highest), exceeds=above)
751+
if above:
752+
issue(issues, args.staging_root.absolute(),
753+
f"staged highest {glibc_label(staged_highest)} exceeds released baseline "
754+
f"highest {glibc_label(released_highest)}",
755+
staged_max_glibc=version_text(staged_highest),
756+
baseline_max_glibc=version_text(released_highest),
757+
baseline_root=str(baseline_root.absolute()))
758+
success = complete and not issues
662759
emit("summary", verdict="PASS" if success else "FAIL", discovered=len(targets),
663760
inspected=inspected, archive_members=sum(
664761
sum(member["inspected"] for member in row["members"]) for row in results),
665-
issues=len(issues))
762+
issues=len(issues), **baseline_fields)
666763
return 0 if success else 1
667764

668765

0 commit comments

Comments
 (0)