Skip to content

Build and test compatible release archives - #736

Merged
luisleo526 merged 1 commit into
mainfrom
ci/prebuilt-bookworm-public-r2
Nov 4, 2026
Merged

luisleo526 merged 1 commit into
mainfrom
ci/prebuilt-bookworm-public-r2

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

Build and check Linux release archives with a digest-pinned Bookworm toolchain. Before upload, verify and unpack the packed archive, then link and run the native kernel host against its full shipped library.

Apply strict secret-expression checks only to prebuilt; retain main's credential behavior for every other job. Keep prebuilt token-only with its existing matrix and runner pins, and pin the delegated archive-consumer source. Correct the environment inventory and isolated workflow fixtures.

Validation on this head: whole preflight504tests and whole release-gate289tests passed across eight shards each. Updated-head required CI, final review and the first real archive-consumer dry run remain pending.

🤖 Generated with Claude Code

Build x86_64 archives in the pinned Bookworm image and check their real
glibc requirements. Consume the packed bytes before upload. Keep prebuilt
on its existing token-only credential surface, scope strict credential
scanning to prebuilt, and pin the archive consumer with negative controls.

Intentionally replace the documentation introduced by c70e828 and the
intermediate wording introduced by f637684. These are the named deletions
and reverts in docs/ci.md, with their resulting policy:

- c70e828: "Linux x64 `prebuilt` uses `gkp-16cpu-32gb-amd64-128gb`."
  The release path now uses the corresponding on-demand protected label.
- c70e828: "The protected 2 CPU/4 GB set fits each orchestration job:"
  Release and notify-hub use the on-demand 4 CPU/8 GB set; promotion and
  the ruleset probe retain their stated protected routing.
- c70e828: "Docs uses the 4 CPU/8 GB set for Doxygen and Graphviz over the
  full public header set before uploading the site to Pages."
  The replacement paragraph preserves that allocation and purpose.
- c70e828: "The x86_64 prebuilt set reserves 16 CPU/32 GB and 128 GB disk
  for compiling, staging and packing the static library."
  The replacement keeps those resources and names the preloaded image.
- c70e828: "Both Linux artifact cells install `libeigen3-dev` with
  `sudo apt-get update` and `sudo apt-get install -y libeigen3-dev`;"
  ARM installs the Ubuntu package; x86_64 uses the pinned Bookworm package.
- c70e828: "Eigen is not baked into the protected image."
  The x86_64 Docker build uses the immutable image's pinned Eigen package,
  and both builds verify the consumed headers before CMake.
- c70e828: "Before CMake, x86_64 must read `Eigen/src/Core/util/Macros.h`
  and match exactly" now applies to both builds with explicit Eigen3_DIR.
- f637684: "ARM installs exactly `libeigen3-dev=3.4.0-4`; x86_64 checks
  that package in its image." The replacement distinguishes the Ubuntu
  ARM package from Bookworm's exact 3.4.0-4 package.
- f637684: "Hosted exceptions cover PR light jobs; release uses protected
  GKE and post light jobs use bare scalar GKE names."
  The preserved summary and detailed release inventory state the resulting
  hosted exceptions, protected release routing, and bare post light jobs.
- f637684: the release table's "`gkp-2cpu-4gb-amd64`; x86_64 `prebuilt`:"
  wording is replaced by the explicit on-demand release label inventory.

This commit preserves the complete tested archive-publication tree while
making every intentional documentation deletion or revert explicit.
Comment on lines +4511 to +4512
self.assertTrue({'test_version_policy_mutation_breaks_real_rejection_assertion',
'test_isoc23_policy_mutation_breaks_real_rejection_assertion'} <= names)
import sys
import tarfile
import tempfile
import unittest
import subprocess
import sys
import tempfile
import unittest

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; formal verification proved 0 of 5 changed function(s) (1 sampled, not proven, 4 not verified).

Formal verification. PR-changed functions: 1/5 verified (0 proven, 1 may-equivalent, 0 distinguished) · 4 not verified (4 vacuous).

Not verified on this run: app\_token\_environment\_findings (vacuous: never exercised), check\_commands (vacuous: never exercised), main (vacuous: never exercised), runner\_findings (vacuous: never exercised).


Graphify review — findings

Adds a Bookworm-container build path for the x86_64 release prebuilt: build_prebuilt_bookworm.sh admits the preloaded toolchain image against its pinned digest, builds inside it, then runs in --consume mode on the packed tarball (sidecar checksum, unpack, native host run) before upload. The generic Configure/Build/Install steps now run only for linux-aarch64, with Eigen3_DIR pinned and FetchContent disabled, and ARM gets its own Eigen 3.4.0 header assertion, while the existing x86_64 Eigen assertion still reads host headers rather than the container's. Adds a glibc-floor check over every staged .a/.so in dry-run and publish, keeping raw checker output and its real exit code (setup errors included), and sets persist-credentials: false on the prebuilt checkout. Moves release and notify-hub to on-demand runners to avoid preemption after tag push, and extends preflight and its tests to pin the new steps, labels, shard matrices and the glibc checker.

Worth a look

  • Docker image config ID is compared to the pinned repository digest — scripts/build_prebuilt_bookworm.sh:101 · Escalate · high
    • NOT verified (no proof, no reproducing execution) — needs human review
  • Configure/Build/Install now skipped for every non-aarch64 triple, including non-Linux slices — .github/workflows/release.yml:450 · Escalate · high
    • NOT verified (no proof, no reproducing execution) — needs human review
  • x86 Eigen check validates host headers after the artifact was built elsewhere — .github/workflows/release.yml:411 · Escalate · medium
    • NOT verified (no proof, no reproducing execution) — needs human review

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1540 functions depend on the 1519 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: ci_workflow_findings() — 81 callers, 24 callees
  • worse: check_commands() — 22 callers, 5 callees
  • worse: _job_values() — 29 callers, 3 callees
  • worse: _job_block() — 17 callers, 3 callees
  • worse: runner_findings() — 2 callers, 25 callees
  • new: release_prebuilt_workflow_findings() — 6 callers, 6 callees
  • worse: _holds_credentials() — 3 callers, 7 callees
  • worse: main() — 1 callers, 13 callees
  • …and 4 more — each is listed as a finding

Verification — 1540 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 1540 function(s) in the blast radius were not formally verified this run

Docs that may be stale (advisory)

…and 10 more.

Formal verification

No difference found (not proven): No behavior difference found in \_holds\_credentials (not a proof).

The verifier ran both versions of \_holds\_credentials on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify app\_token\_environment\_findings.

The verifier did not have enough to check app\_token\_environment\_findings, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 2 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

Could not verify: Could not verify check\_commands.

The verifier did not have enough to check check\_commands, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 90 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify main.

The verifier did not have enough to check main, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify runner\_findings.

The verifier did not have enough to check runner\_findings, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 2 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

· 6 grounded finding(s) anchored inline below; 6 more finding(s) on lines outside this diff (see the check run).

@@ -0,0 +1,421 @@
#!/usr/bin/env bash

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — build_prebuilt_bookworm.sh script

fans out to 9 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

raise


def discover(root: Path, root_fd: int, issues: list[dict]) -> tuple[list[dict], dict]:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — discover()

fans out to 8 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

**context, reference=reference)


def inspect_target(root_fd: int, target: dict, directories: dict,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — inspect_target()

fans out to 12 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return row


def main(argv: list[str] | None = None) -> int:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — main()

fans out to 8 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Comment thread scripts/ci_preflight.py
return False


def _holds_credentials(workflow: str, body: str, *, effective_permissions: bool = False,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — _holds_credentials()

fans out to 7 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Comment thread scripts/ci_preflight.py
return []


def release_prebuilt_workflow_findings(workflow: str) -> list[str]:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression — release_prebuilt_workflow_findings()

fans out to 6 callees (efferent coupling); 6 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

@luisleo526

Copy link
Copy Markdown
Collaborator Author

Review of bd1fa2f (one commit on main ace3079; it replaces #733, whose last head was 74d0476). GO WITH ONE CONDITION: preflight, corpus-parity-subset and build-arm green on this head. The tree is identical to the one I reviewed on #733, and the commit message names every sentence the documentation-revert checker flagged. I ran nothing: the tree and message statements come from the git objects, the CI statements from the job logs.

Condition

  1. Required jobs green on this head. preflight and corpus-parity-subset were both cancelled by runner shutdown signals, 36 seconds apart (jobs 126872532377 at 21:29:51Z and 126872532494 at 21:30:27Z: "The runner has received a shutdown signal"). Neither reports a check or test failure. In preflight, every stage that finished before the signal passed, including both documentation-revert stages (below), but the eight unit shards were killed at 415.9 s ("killed by signal 15"), so this head has no complete preflight run yet. The same tree passed all 504 tests across eight shards on the Build and test compatible release archives #733 head (run 41911017092); that is the same bytes, not a run on this head. Both jobs need a re-run. build-arm was still running. admit, both authz, both build, build-debug, both CodeQL analyses and Graphify pass.

What I checked

  • Tree identity. bd1fa2f has one parent, main's head ace3079. Its tree id, c8cf929520cd0c4809f90c6585bc067ede619d78, equals that of 74d0476, and docs/ci.md is byte-identical between the two. So every file I read on Build and test compatible release archives #733 (the prebuilt-only credential scan, the archive-consumer pin, the tests, the workflow) is unchanged, and my findings there stand as they were.
  • Namings, as the checker sees them. On this tree against this parent, the six findings (run 41911017092, doc-reverts-net) were all deletions of sentences introduced by c70e828, which is an ancestor of main. The message carries c70e828 and quotes five of the six (the gkp-16cpu-32gb-amd64-128gb line, the 2 CPU/4 GB set, the x86_64 prebuilt set, the two Eigen sentences). On this head both stages pass: doc-reverts "1 commits since ace3079: 0 unnamed sentence deletions or reverts ... OK" and doc-reverts-net "ace3079 -> bd1fa2f: 0 unnamed sentence deletions or reverts ... OK". The message's account of the resulting policy matches the head's docs/ci.md for the points I compared (the 4 CPU/8 GB set for Docs, the preloaded image, ARM's Ubuntu package against Bookworm's 3.4.0-4, the Eigen header check before CMake for both builds).
  • Public text. The message contains no internal names.

Smaller points (P3)

  • The message's list is not the checker's list. The sixth finding, the group 15 mirror sentence (docs/ci.md:1696-1698, which now ends with the operational-confirmation items where it used to end with the release dry run), is covered by the c70e828 hash only; the message calls its list "the named deletions and reverts" but does not include it. Three further bullets name f637684, a commit of Build and test compatible release archives #733's own branch that is not in main's history, so two of them (the "Hosted exceptions ..." and "ARM installs exactly ..." sentences) describe text main never had. Neither matters to the checker, and I do not ask for an amend: a new sha would restart every check for a wording point.
  • Carried over unchanged from my comments on Build and test compatible release archives #733. The ARM slice still has no archive consumer, and the glibc gate sees only explicit version tags and __isoc23_* names in .a members; docs/ci.md:1972 still says "configure, link and run diagnostics" for a consumer that has no configure stage, and the file has no sentence for the glibc gate; the operational inputs and the first real archive-consumer dry run are still pending, as the body says (the helper prints the image list and the inspect JSON to the job log, so confirm the image config holds nothing you would not publish); and, pre-existing, main's default credential scan does not recognise a whole toJSON(secrets) context in the other jobs.

@luisleo526
luisleo526 merged commit bd1fa2f into main Nov 4, 2026
19 of 23 checks passed

This branch was previously deployed

1 inactive deployment
release-bot — bd1fa2f0 Deployed Nov 4, 2026 by luisleo526 via promote #430
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant