Skip to content

Commit 894bf4f

Browse files
luisleo526claude
andcommitted
Merge main into the exit-binding change
Plain merge of main into this branch, with no conflicts. Main now has the hosted release and notify-hub jobs and the glibc comparison with the v1.10.0 archive (#768), which edit the release workflow, the CI documents and the preflight and glibc scripts; this branch changes none of those files. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2 parents c8308d4 + b09f79d commit 894bf4f

9 files changed

Lines changed: 443 additions & 84 deletions

File tree

‎.github/actionlint.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
# The unchanged release workflow retains its protected runner labels.
1+
# Release keeps one protected runner label, on its x86_64 prebuilt cell.
22
self-hosted-runner:
33
labels:
4-
- gkp-od-4cpu-8gb-amd64
54
- gkp-od-16cpu-32gb-amd64-128gb

‎.github/workflows/release.yml‎

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ env:
7777

7878
jobs:
7979
release:
80-
runs-on: gkp-od-4cpu-8gb-amd64
80+
runs-on: ubuntu-24.04
8181
environment: release
8282

8383
outputs:
@@ -485,7 +485,32 @@ jobs:
485485
python3 scripts/release_version.py check-install \
486486
--prefix "${STAGE_DIR}" --expect "${NEXT}"
487487
488-
# Both dry-run and publication inspect every installed .a/.so/.so.*.
488+
# The glibc comparison baseline is this triple's archive from the v1.10.0
489+
# release: the real released bytes, pinned by sha256 and unpacked here.
490+
# The checker below only reads the unpacked libraries.
491+
- name: Fetch the v1.10.0 baseline archive for the glibc comparison
492+
env:
493+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
494+
TRIPLE: ${{ matrix.triple }}
495+
run: |
496+
set -euo pipefail
497+
case "${TRIPLE}" in
498+
linux-x86_64) digest=cb412fbae9e8ba515de41dc31026c77d6c5e309693247bee26bb93c67d3c3543 ;;
499+
linux-aarch64) digest=982b16da2d5f5ee7c274615895484fce36bd740052a14d59b66576dca496d7bb ;;
500+
*) echo "::error::no v1.10.0 baseline archive for ${TRIPLE}" >&2; exit 1 ;;
501+
esac
502+
archive="pineforge-v1.10.0-${TRIPLE}.tar.gz"
503+
root="${RUNNER_TEMP}/glibc-baseline-${TRIPLE}"
504+
mkdir "${root}" "${root}/download" "${root}/tree"
505+
gh release download v1.10.0 --repo "${GITHUB_REPOSITORY}" \
506+
--pattern "${archive}" --dir "${root}/download"
507+
echo "${digest} ${root}/download/${archive}" | sha256sum -c -
508+
tar -xzf "${root}/download/${archive}" -C "${root}/tree" \
509+
--no-same-owner --no-same-permissions
510+
echo "BASELINE_DIR=${root}/tree" >> "${GITHUB_ENV}"
511+
512+
# Both dry-run and publication inspect every installed .a/.so/.so.* and
513+
# compare their highest glibc tag with the baseline fetched above.
489514
# Keep raw checker output and its actual exit status, including setup errors.
490515
- name: Check every staged library's glibc floor
491516
env:
@@ -494,7 +519,7 @@ jobs:
494519
set -euo pipefail
495520
log="${RUNNER_TEMP}/prebuilt-glibc-${TRIPLE}"
496521
mkdir -p "${log}"
497-
if python3 scripts/check_prebuilt_glibc.py "${STAGE_DIR}" > "${log}/stdout.jsonl" 2> "${log}/stderr.txt"; then
522+
if python3 scripts/check_prebuilt_glibc.py "${STAGE_DIR}" --baseline-root "${BASELINE_DIR}" > "${log}/stdout.jsonl" 2> "${log}/stderr.txt"; then
498523
checker_rc=0
499524
else checker_rc=$?; fi
500525
printf '%s\n' "${checker_rc}" > "${log}/exitcode.txt"
@@ -565,7 +590,7 @@ jobs:
565590
notify-hub:
566591
needs: [release, prebuilt]
567592
if: ${{ !inputs.dry_run }}
568-
runs-on: gkp-od-4cpu-8gb-amd64
593+
runs-on: ubuntu-24.04
569594
environment: release
570595
steps:
571596
- name: Mint org App token (hub dispatch)

‎docs/ci.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ Python refuse a system-wide install (PEP 668), and `.venv/` is ignored. The
113113
and fails, never skips, without it.
114114

115115
Actionlint only accepts declared custom self-hosted runner labels; preflight
116-
checks the declarations in `.github/actionlint.yaml`, which now declare only the two protected `gkp-od-*` labels of `release.yml`, including its x86_64 release matrix cell. Hosted workflows declare nothing. Before a workflow moves onto a `gke-*` label, add its ref to `.github/runner-group-allowlist.txt` (group 14, currently empty because no workflow uses one); protected `gkp-*` refs belong in `.github/runner-group-15-allowlist.txt` (group 15, currently only `release.yml`). Both mirrors are binding; a maintainer must confirm the corresponding live org workflow/ref entries before enabling the routes.
116+
checks the declarations in `.github/actionlint.yaml`, which now declare only the protected `gkp-od-*` label of `release.yml`'s x86_64 release matrix cell. Hosted workflows declare nothing. Before a workflow moves onto a `gke-*` label, add its ref to `.github/runner-group-allowlist.txt` (group 14, currently empty because no workflow uses one); protected `gkp-*` refs belong in `.github/runner-group-15-allowlist.txt` (group 15, currently only `release.yml`). Both mirrors are binding; a maintainer must confirm the corresponding live org workflow/ref entries before enabling the routes.
117117

118118
ORG-ACTIONPINS is enforced by preflight's `ci-workflow-contract`: every
119119
third-party `uses:` in `.github/workflows/*.yml` (also `.yaml`) and nested
@@ -1644,7 +1644,7 @@ The native matrix is exactly `1..4` with `--ctest-shard I/4`; live sanitizers ar
16441644
| `corpus-parity.yml` | Schedule, dispatch, post CI callers | `authz`, full parity and subset: `ubuntu-24.04` |
16451645
| `native-live.yml` | Dispatch and post CI callers on `main` | `authz`, both aggregates and the native shards: `ubuntu-24.04`; `matrix.runner`: live-sanitizers and live-tsan both `ubuntu-24.04` |
16461646
| `docs-post.yml` | Push to `main`, dispatch | `authz` and credentialed `build`: `ubuntu-24.04` |
1647-
| `release.yml` | Dispatch from `main` | `release` and `notify-hub`: `gkp-od-4cpu-8gb-amd64`; x86_64 `prebuilt`: `gkp-od-16cpu-32gb-amd64-128gb`; arm64 `prebuilt`: hosted `ubuntu-22.04-arm` |
1647+
| `release.yml` | Dispatch from `main` | `release` and `notify-hub`: hosted `ubuntu-24.04`; x86_64 `prebuilt`: `gkp-od-16cpu-32gb-amd64-128gb`; arm64 `prebuilt`: hosted `ubuntu-22.04-arm` |
16481648
| `promote-baseline.yml` | Push to `main`, dispatch from `main` | `promote-guard` and `promote`: `ubuntu-24.04` |
16491649
| `release-ruleset-probe.yml` | Dispatch from `main` | `authz` and `probe`: `ubuntu-24.04` |
16501650

@@ -1668,15 +1668,15 @@ Trusted post workflows and PR proof legs build cold; no trusted job restores an
16681668
`HOSTED_JOB_RUNNERS` pins the hosted x86 and arm64 jobs and the native profile runner matrix; `GKP_JOB_RUNNERS` separately pins the protected release jobs and sizes. Every pinned job runs directly on its hosted image. PR checks are
16691669
`build`, `build-debug`, `corpus-parity-subset`, `legs` and `result` in `gke-pr.yml`. With the arm leg, the full set is `build`, `build-debug`, `build-arm`, `corpus-parity-subset`, `legs` and `result` in `gke-pr.yml`.
16701670

1671-
Release uses protected GKE runner group 15:
1672-
`release` and `notify-hub` use `gkp-od-4cpu-8gb-amd64`;
1671+
Release uses protected GKE runner group 15 for its x64 artifact only:
1672+
`release` and `notify-hub` run on hosted `ubuntu-24.04` and keep their `release` environment;
16731673
Linux x64 `prebuilt` uses `gkp-od-16cpu-32gb-amd64-128gb`; baseline `promote` uses hosted `ubuntu-24.04`.
16741674
The release ruleset probe runs `authz` and `probe` on hosted `ubuntu-24.04`.
16751675
Release's ARM artifact cell retains hosted `ubuntu-22.04-arm`;
16761676
1.8.0 has no macOS test or artifact cells, and
16771677
promotion runs on a push to `main` plus manual dispatch from `main`.
16781678

1679-
Release and notify-hub use the on-demand 4 CPU/8 GB set to prevent preemption after tag push.
1679+
Release and notify-hub run on hosted `ubuntu-24.04`, which no scale-set preemption can interrupt after tag push.
16801680
Release computes versions, stamps JSON catalogs and publishes with git/gh; notify-hub
16811681
mints an App token and dispatches an API request. The protected 2 CPU/4 GB set
16821682
runs promotion's API/tree checks, Node campaign tooling and Cloud SQL proxy; the

‎docs/reference/ci-runner-routing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ parity verdict; an outside contributor cannot post them. GitHub Actions is advis
55
The PR build (`ci.yml`) runs only the `fast`-labelled CTest rows (`ctest -L '^fast$'`); the post-merge legs (`ci-post.yml`) run
66
every row, and a documentation-only change runs preflight alone among verification jobs.
77
The seven PR/post-merge verification and documentation workflows start with the canon `authz` job and every other job in them needs it; release uses its status gate and baseline promotion its `promote-guard`. Pull-request
8-
proof workflows and post jobs use exact hosted pins: `ubuntu-24.04` for x86 and `ubuntu-24.04-arm` for the arm64 leg. Promotion, the ruleset probe and docs deployment also run on hosted `ubuntu-24.04`, and keep their `release`/`release-bot` environments. Only `release.yml` still names protected scale sets: `release` and `notify-hub` use `gkp-od-4cpu-8gb-amd64` in protected runner group 15. Hosted PR light jobs and `docs.yml`'s `build` retain `ubuntu-24.04`; post guards and aggregates use the same pin.
8+
proof workflows and post jobs use exact hosted pins: `ubuntu-24.04` for x86 and `ubuntu-24.04-arm` for the arm64 leg. Promotion, the ruleset probe and docs deployment also run on hosted `ubuntu-24.04`, and keep their `release`/`release-bot` environments. Only `release.yml` still names a protected scale set, in protected runner group 15: its x86_64 `prebuilt` cell. `release` and `notify-hub` run on hosted `ubuntu-24.04` and keep their `release` environment. Hosted PR light jobs and `docs.yml`'s `build` retain `ubuntu-24.04`; post guards and aggregates use the same pin.
99
Hosted fork PRs receive no secrets, so they need no VM-only fork-refusal hook. Retired main/post VM labels remain refused. The retired `gke-*` scale sets and paid larger-runner labels are refused too. Group 14's mirror `.github/runner-group-allowlist.txt` is empty because no workflow uses a group-14 label; group 15's required ref (`release.yml`) is in `.github/runner-group-15-allowlist.txt`, with live membership confirmed by a maintainer.
1010
Release's x86_64 prebuilt cell uses `gkp-od-16cpu-32gb-amd64-128gb`, installs Eigen and asserts exactly 3.4.0 before CMake; the arm64 cell retains hosted `ubuntu-22.04-arm`. App-token jobs keep their exact `release`/`release-bot` environments. The `build-gate` job in `ci-post.yml` and `native-live.yml`'s `native-live-gate` always run and refuse a non-success authz result first.
1111
See `docs/ci.md` for the class table and docs-only exception. The commands below run the full sets.

‎scripts/check_prebuilt_glibc.py‎

Lines changed: 110 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,18 @@
66
GNU ar/readelf must already be installed in the OS default executable search
77
path. Never extract archives, load libraries, execute payloads, or write files.
88
9+
--baseline-root DIR adds the released-baseline comparison. DIR is an already
10+
unpacked released archive (the workflow checks its digest and unpacks it; this
11+
tool still extracts nothing). The same strict inspection reads every library
12+
under DIR, and the highest numeric GLIBC_ tag over all staged libraries may not
13+
exceed the highest over all baseline libraries: equal or lower passes, higher
14+
fails. A tree with no numeric tag ranks below any tag. The baseline is read, not
15+
judged: the fixed ceiling and the __isoc23_* rule apply to the staging tree
16+
only, since a released archive may predate them. A missing, empty or partly
17+
uninspectable baseline, or one overlapping the staging root, fails closed. No
18+
expected value is configured anywhere: the verdict is a function of the two
19+
inspected trees.
20+
921
All symlinks (including directory links), protected/key/config/.env* paths,
1022
unreadable entries, thin archives, non-ELF members, and wrong ELF types fail.
1123
Archives must contain REL objects with inspectable symbol tables; shared objects
@@ -65,8 +77,13 @@ class InspectionError(Exception):
6577
pass
6678

6779

80+
# Fields added to every record while the released baseline is being read.
81+
EVENT_CONTEXT: dict[str, str] = {}
82+
83+
6884
def emit(event: str, **fields) -> None:
69-
print(json.dumps({"event": event, **fields}, ensure_ascii=True), flush=True)
85+
print(json.dumps({"event": event, **EVENT_CONTEXT, **fields}, ensure_ascii=True),
86+
flush=True)
7087

7188

7289
def issue(issues: list[dict], path: Path, cause: str, **fields) -> None:
@@ -282,6 +299,42 @@ def maximum_version(names: list[str]) -> str | None:
282299
return name[len("GLIBC_"):]
283300

284301

302+
def tree_maximum(rows: list[dict]) -> tuple[int, ...] | None:
303+
"""Highest numeric GLIBC_ tag over every inspected library of one tree."""
304+
highest = maximum_version(sorted({name for row in rows for name in row["versions"]}))
305+
return None if highest is None else tuple(int(part) for part in highest.split("."))
306+
307+
308+
def version_text(version: tuple[int, ...] | None) -> str | None:
309+
return None if version is None else ".".join(str(part) for part in version)
310+
311+
312+
def glibc_label(version: tuple[int, ...] | None) -> str:
313+
return "none" if version is None else "GLIBC_" + version_text(version)
314+
315+
316+
def exceeds_baseline(candidate: tuple[int, ...] | None,
317+
baseline: tuple[int, ...] | None) -> bool:
318+
"""True when the staged tree's highest numeric tag is above the baseline's.
319+
320+
Components compare with zero padding, so 2.36 equals 2.36.0. No numeric tag
321+
at all ranks below every tag: a staged tag above a tagless baseline exceeds
322+
it, and a tagless staged tree never does.
323+
"""
324+
if candidate is None:
325+
return False
326+
if baseline is None:
327+
return True
328+
width = max(len(candidate), len(baseline))
329+
return (candidate + (0,) * (width - len(candidate))
330+
> baseline + (0,) * (width - len(baseline)))
331+
332+
333+
def roots_overlap(left: Path, right: Path) -> bool:
334+
first, second = (Path(os.path.abspath(path)) for path in (left, right))
335+
return first == second or first in second.parents or second in first.parents
336+
337+
285338
def version_sections(lines: list[str], dynamic_count: int | None) -> set[str]:
286339
headers = [(index, VERSION_HEADER.fullmatch(line))
287340
for index, line in enumerate(lines) if line.startswith("Version ")]
@@ -493,7 +546,7 @@ def policy_issues(path: Path, member: dict, issues: list[dict]) -> None:
493546

494547

495548
def inspect_target(root_fd: int, target: dict, directories: dict,
496-
tools: dict[str, str], issues: list[dict]) -> dict:
549+
tools: dict[str, str], issues: list[dict], policy: bool = True) -> dict:
497550
path = target["path"]
498551
row = {
499552
"path": str(path), "kind": target["kind"], "identity": target["identity"],
@@ -580,7 +633,8 @@ def inspect_target(root_fd: int, target: dict, directories: dict,
580633
try:
581634
member.update(parse_elf(lines, "REL" if expected_members is not None else "DYN"))
582635
member["inspected"] = True
583-
policy_issues(path, member, issues)
636+
if policy:
637+
policy_issues(path, member, issues)
584638
except (InspectionError, ValueError) as error:
585639
issue(issues, path, str(error), member=name, ordinal=ordinal)
586640
complete = False
@@ -613,16 +667,12 @@ def inspect_target(root_fd: int, target: dict, directories: dict,
613667
return row
614668

615669

616-
def main(argv: list[str] | None = None) -> int:
617-
parser = argparse.ArgumentParser(description=__doc__)
618-
parser.add_argument("staging_root", type=Path, help="explicit installed prebuilt staging root")
619-
args = parser.parse_args(argv)
620-
issues = []
670+
def inspect_tree(root: Path, issues: list[dict], policy: bool) -> tuple[list[dict], list[dict]]:
671+
"""Inventory and inspect one explicit root; the fixed policy is for staging only."""
621672
targets = []
622673
results = []
623674
root_fd = None
624-
root = args.staging_root
625-
emit("root", path=str(root), fixed_glibc_ceiling="2.36")
675+
emit("root", path=str(root), **({"fixed_glibc_ceiling": "2.36"} if policy else {}))
626676
try:
627677
if sys.platform != "linux":
628678
raise InspectionError("Linux descriptor-based ELF inspection is required")
@@ -638,7 +688,7 @@ def main(argv: list[str] | None = None) -> int:
638688
issue(issues, root, "no shipped .so/.so.*/.a libraries in staging inventory")
639689
tools = installed_tools(root, issues) if targets else {}
640690
for target in targets:
641-
results.append(inspect_target(root_fd, target, directories, tools, issues))
691+
results.append(inspect_target(root_fd, target, directories, tools, issues, policy))
642692
# Re-open the path and re-inventory metadata to catch late additions,
643693
# removals, renames, or a replacement root; payloads are never re-read.
644694
final_fd = open_root(root)
@@ -657,12 +707,59 @@ def main(argv: list[str] | None = None) -> int:
657707
finally:
658708
if root_fd is not None:
659709
os.close(root_fd)
710+
return targets, results
711+
712+
713+
def main(argv: list[str] | None = None) -> int:
714+
parser = argparse.ArgumentParser(description=__doc__)
715+
parser.add_argument("staging_root", type=Path, help="explicit installed prebuilt staging root")
716+
parser.add_argument("--baseline-root", default=None, metavar="DIR",
717+
help="explicit unpacked released archive; the staging tree's highest "
718+
"numeric GLIBC_ tag may not exceed the highest found there")
719+
args = parser.parse_args(argv)
720+
if args.baseline_root is not None and not args.baseline_root:
721+
parser.error("--baseline-root must name a directory")
722+
issues = []
723+
targets, results = inspect_tree(args.staging_root, issues, True)
660724
inspected = sum(row["inspected"] for row in results)
661-
success = bool(targets) and inspected == len(targets) and not issues
725+
complete = bool(targets) and inspected == len(targets)
726+
baseline_fields = {}
727+
if args.baseline_root is not None:
728+
baseline_root = Path(args.baseline_root)
729+
baseline_targets = []
730+
baseline_results = []
731+
if roots_overlap(args.staging_root, baseline_root):
732+
issue(issues, baseline_root.absolute(),
733+
"baseline root must be separate from the staging root")
734+
else:
735+
EVENT_CONTEXT["role"] = "baseline"
736+
try:
737+
baseline_targets, baseline_results = inspect_tree(baseline_root, issues, False)
738+
finally:
739+
EVENT_CONTEXT.clear()
740+
baseline_inspected = sum(row["inspected"] for row in baseline_results)
741+
baseline_fields = {"baseline_discovered": len(baseline_targets),
742+
"baseline_inspected": baseline_inspected}
743+
complete = (complete and bool(baseline_targets)
744+
and baseline_inspected == len(baseline_targets))
745+
if complete:
746+
staged_highest = tree_maximum(results)
747+
released_highest = tree_maximum(baseline_results)
748+
above = exceeds_baseline(staged_highest, released_highest)
749+
emit("baseline_comparison", staged_max_glibc=version_text(staged_highest),
750+
baseline_max_glibc=version_text(released_highest), exceeds=above)
751+
if above:
752+
issue(issues, args.staging_root.absolute(),
753+
f"staged highest {glibc_label(staged_highest)} exceeds released baseline "
754+
f"highest {glibc_label(released_highest)}",
755+
staged_max_glibc=version_text(staged_highest),
756+
baseline_max_glibc=version_text(released_highest),
757+
baseline_root=str(baseline_root.absolute()))
758+
success = complete and not issues
662759
emit("summary", verdict="PASS" if success else "FAIL", discovered=len(targets),
663760
inspected=inspected, archive_members=sum(
664761
sum(member["inspected"] for member in row["members"]) for row in results),
665-
issues=len(issues))
762+
issues=len(issues), **baseline_fields)
666763
return 0 if success else 1
667764

668765

0 commit comments

Comments
 (0)