Repository navigation
release gate: require a green ci-post proof run for the released commit - #526
Merged
Merged
Conversation
scripts/test_ci_release_gate.py only. Adds an opt-in green ci-post fixture to FakeAPI (the runs read and the jobs read of the proof commit; unknown endpoints still raise), a strict FakeHistory (first parent plus changed paths, no classifier of its own) and 40 rows in CiPostGateTests: plain, docs-only-ancestor and phase-two / release-bump passes with the evidence named in the green line; refusals for no run, running, failed, cancelled, a newer red run next to an older green, a run for another sha or branch, each of the four build jobs skipped / failed / missing, a jobs list shorter than its total, rate limit, 5xx, transport and parse errors, a truncated or 1000-capped runs page, a docs walk blocked by a non-documentation path or the old side of a rename, an empty diff, 21 docs-only commits, a root commit and unreadable history. The gate does not read ci-post yet, so the new rows are expected RED on this commit (they pass the `history` keyword gate() does not accept, and assert `ci-post: <code>` fragments); every existing row is unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ere skipped (R-A rows) and ignores an older unfinished run A ci-post run where preflight fails and the four build jobs are skipped is the most common real red. An implementation that treats "all builds skipped" as skipped-only before looking at the run conclusion would walk past that red run to an older green one. Rows pin the check order: failed build job (newer-run-failed), then run-level failure/cancelled/timed_out (not-success), then full success, else continue; and that an older queued/in_progress run below the proving run is never read. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…un (walk only across docs-only commits) The release gate accepted a sha whose ci-post run was skipped or red, because it only read the PR status contexts. It now also requires a ci-post run for the proof sha (head_branch main, any event) whose four build jobs and run conclusion are success. The proof sha is the release sha, or the nearest first-parent ancestor reached across docs-only commits only (at most 20). Runs are examined newest first (created_at, then id), at most 10, one jobs read per completed run. A failed build job refuses with newer-run-failed; a failed/cancelled/timed_out run conclusion refuses with not-success whatever its build legs show (skipped legs after a failed preflight must not let the walk slide to an older green run); a full success is the proof; any other run is skipped-only and the walk continues to the next older run. Refusals carry the codes no-run, run-in-progress, newer-run-failed, not-success, no-full-run, unreadable, walk-blocked and walk-too-long. The required job names are one tuple pinned against the ci-post.yml matrices by a workflow-pin test. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…e gate git_history is the only production reader of first-parent history for the ci-post walk, and every ci-post row injects a fake. These rows build a throwaway repository per test and call the real function: root commit, normal commit, rename (both sides), newline and non-ASCII paths, merge (first parent only), unknown sha, empty commit, and no repository. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ts its walk skips (F-1 rows, red first) A docs-only release sha whose own ci-post run is red (for example preflight failed on doc-anchors) must not be released on the strength of a built ancestor's green run. Rows pin: every skipped docs-only commit (tip first) has a newest run that exists and is completed with conclusion success, judged by the run-level conclusion only with no jobs read; docs-commit-red and docs-commit-no-run refusals; newest run by (created_at, id) in both response orders; unreadable / paginated / 1000-limit runs reads; the 20-commit cap decided before any docs read; phase 2 and release-bump paths; and that a success docs run with skipped builds is fine while a red built ancestor is still refused. The docs-walk helpers register a successful docs run per skipped sha and the exact read lists now include those reads. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…docs-only commit its walk skips The walk across docs-only commits ignored the runs of the commits it skipped, so a docs-only release sha whose own ci-post run was red (for example preflight failed on doc-anchors) could be released on a built ancestor's green run. _ci_post_proof now returns the skipped docs-only shas (tip first). Before the proof commit's runs are read, each skipped sha's newest ci-post run (same runs helper: main, exact sha, any event, newest by created_at then id) must exist and be completed with conclusion success, judged by the run-level conclusion only with no jobs read: ci-post: docs-commit-no-run sha <sha> or ci-post: docs-commit-red <run id> sha <sha>. The 20-commit cap is still decided during the walk, before any docs-run read. A docs run that succeeded with skipped build jobs is fine. Also narrows the walk's except to (OSError, ValueError), what the production history seam raises. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…the release gate (compare by commit, not run time) The gate keeps only runs whose head_sha is the commit being examined (the proof sha, or one skipped docs-only sha). Rows pin that a failed or in-progress run of the parent commit, created and completed later than the cut commit's green run and even listed in the same response, is neither blocking nor read, while flipping only its head_sha to the examined commit flips the verdict (newer-run-failed, not-success, run-in-progress for the proof; docs-commit-red for a skipped docs-only commit). The docs-chain rows also pin the exact runs and jobs reads. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Main moved by #517 and #518 (run_json, its tests, one docs page); none of the release-gate files changed. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… of the proof sha (red first) CORE-TOP ruled (10:51): on the proof sha a newer completed run blocks unless its run-level conclusion is success; neutral, action_required, stale, startup_failure, skipped and null each refuse with ci-post: not-success naming the conclusion, for every examined run, while a failed build job keeps the more specific newer-run-failed. Only a success run whose build legs are not a full success (for example all four skipped) still falls through. Two existing rows follow the ruling and keep their meaning: the loop of test_skipped_only_newer_run_allows_older_full_green narrows to run-level success (the run-level skipped case now belongs to the refusal rows), and the middle run of test_only_skipped_runs_and_a_missing_matrix_name_give_no_full_run is a success run with skipped legs instead of a run-level skipped run. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…f the proof sha A newer completed ci-post run of the proof sha whose run-level conclusion is neutral, action_required, stale, startup_failure, skipped or null was walked past and an older green run released. The run-level conclusion must now be success for every examined completed run, or the gate refuses with ci-post: not-success naming the conclusion; a failed build job still takes priority (newer-run-failed), and only a success run whose build legs are not a full success falls through to an older run. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…s final rule), replacing the first fail-closed reading A newer completed ci-post run of the proof sha: a failed/cancelled/timed_out build job refuses newer-run-failed (first, over any run conclusion); a run-level skipped run falls through only when all four build legs are skipped (otherwise it refuses not-success as contradictory data); every other non-success run conclusion (neutral, action_required, stale, startup_failure, null) refuses not-success; a run-level success run falls through only when all four legs are present and each success or skipped, a leg neutral, null, unknown or missing refuses not-success naming the leg. The proof run itself needs run-level success and all four legs success. Existing rows follow the table and keep their meaning: the run-level-skipped all-skipped shapes fall through again, test_other_completed_run_conclusions_refuse expects not-success, and the uncertain-or-missing-cell row becomes a refusal row. New rows name each table cell, including 132 failed-build-priority cases. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
… CORE-TOP's table Replaces the first fail-closed reading (any non-success run conclusion). A newer completed ci-post run that is not a full green is passed over for an older proof only when it is a run-level success whose four build legs are all present and each success or skipped, or a run-level skipped run whose four legs are all skipped; it is then counted as a newer skipped-only run. Every other conclusion (neutral, action_required, stale, startup_failure, null, or skipped with a non-skipped leg) refuses ci-post: not-success naming the conclusion and the contradicting leg, and a run-level success with a neutral, null, unknown or missing leg refuses not-success naming the leg while an older run exists to be protected. A failed, cancelled or timed-out build job keeps priority (newer-run-failed). The proof run is unchanged: run-level success and all four legs success, otherwise no-full-run. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…it for a missing build leg A newer run-level success run with a missing required build leg refuses not-success while an older run exists to be protected, and ends in no-full-run when it is the only examined run. Two rows pin both sides with the same defective run for each of the four required build names. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Main moved by #519 (fd-flag tests only); none of the release-gate files changed. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Comment only; the code is unchanged. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Oct 26, 2026
Adds one bullet to the CI and release process list from the merged change #526: the release gate also requires a green, non-skipped post-merge run of the released commit. Also corrects the scope of the mkstemp sentence in the #521 bullet to /tmp and /var/tmp. Documentation only. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Oct 26, 2026
Main moved by scripts and tests only (#525, #521, #526); CHANGELOG.md is unchanged on main since the base of this branch, so the merge result is main's tree with this branch's CHANGELOG.md. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526
added a commit
that referenced
this pull request
Oct 26, 2026
…ase (#526) A newer run with a failed, cancelled or timed-out build job refuses the release gate; the previous wording said the run as such. Documentation only. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The release gate (
scripts/ci_release_gate.py) now also requires a successful, non-skippedci-postrun for the commit being released. Until now it read only thepineforge/verifyandpineforge/paritystatuses: the engine dry run 40260918204 printedrelease gate green: PR #518 head 9325b640...; verify=success; parity=successfor a release of main 2561cfb whose own ci-post legs were never read. This closes that gap.Rule
Proof commit: the release sha, or, when the release sha is docs-only, the nearest first-parent ancestor that is not docs-only (docs-only by the merge gate's own classifier
ci_docs_only.is_documentation; at most 20 docs-only commits, thenwalk-too-long; unreadable history ->walk-blocked).Every skipped docs-only commit (the release tip included) must have a newest ci-post run that exists and is completed with conclusion
success(run-level conclusion only, no jobs read): elsedocs-commit-no-run/docs-commit-red.For the proof commit, runs of that exact sha on
main(any event) are examined newest first (created_at, then id), at most 10, one jobs read per examined completed run. Newer-run table:ci-post: newer-run-failed(first, over any run conclusion)skipped, all four build legs skippedskipped, any other leg shapenot-success(contradictory data), naming the legsnot-success, whatever the legs showsuccess, all four legs present, each success or skippedsuccess, a leg neutral / null / unknown / missingnot-successnaming the leg (while an older run exists to protect)run-in-progressThe proof run itself needs run-level
successAND all four build legssuccess(build (ubuntu-24.04|macos-26, Debug|Release), pinned to theci-post.ymlmatrices by a workflow-pin test); a skipped leg there isno-full-run. Judgement call: a run-levelsuccessrun with a bad/missing leg that is the LAST examined run (nothing older to protect, or the tenth examined run) ends inno-full-runinstead ofnot-success: both are refusals, only the typed reason differs (pinned from both sides bytest_single_run_with_a_missing_leg_gives_no_full_runandtest_missing_leg_run_with_an_older_proof_refuses_not_success).Read count
1 paginated runs read per skipped docs-only commit (at most 20, typically 0 or 1, no jobs reads), then 1 paginated runs read for the proof sha plus 1 jobs read per examined completed proof run (at most 10). A plain pass with a non-docs release sha: 1 + 1.
Scope notes
head_branch == main): they are not main evidence. Runs are matched byhead_shaof the examined commit (filter at_ci_post_runs); a red or running run of another commit never counts, whatever its run time (rows:test_later_parent_failure_only_blocks_after_release_sha_flipand the three siblings flip onlyhead_sha).release.ymlgate step has nodry_runguard, so adry_run=truedispatch executes it. The Actions API read scope of the release App token is unproven until that step runs: the engine dry run at the freeze commit is its first live proof.Existing test lines changed (against main)
The diff against main removes lines in
scripts/test_ci_release_gate.pyin exactly six existing methods; everything else in the file is added, no method is deleted:GateTests.assert_no_pr_fallback(takes the newci_postkeyword).GateTests.run_gate(opt-in green ci-post fixture and thehistory=seam).GateTests.test_green_reads_both_newest_statuses_and_names_the_reviewed_tree,GateTests.test_main_proof_with_full_contexts_green_passes_without_pr_reads,ReleaseBumpTests.assert_bump_green: the expected stdout gains; ci-post run <id> for <sha>.ReleaseRulesetTests.test_removing_each_new_condition_admits_its_unsafe_input: the full-module mutant getshistory=and the suffix.In
scripts/ci_release_gate.pythe removed lines are 13: the old module docstring sentence, thegate()signature (newhistoryparameter), its return statements (the evidence now ends with the ci-post evidence), the phase-one / release-bump wrapper lines folded into the new flow, and themain(...)signature and call line (gainshistory). The public CLI (gate --repo --sha --ref) is unchanged;gate()andmain()gain an optional injectedhistoryseam that defaults togit_history.Evidence
Every test run below was on the project's cloud runner; none ran on a developer machine.
test_missing_leg_run_with_an_older_proof_refuses_not_success) and always refusing (dropping the older-run condition) fails 25 rows (all four cases oftest_single_run_with_a_missing_leg_gives_no_full_run); never refusing in the run-levelskippedbranch fails 25 rows; never refusing in the branch for failed / cancelled / timed-out / other conclusions fails 47 rows. The success branch (both directions), the skipped branch and the other-conclusions branch are each mutation-checked.🤖 Generated with Claude Code