Repository navigation
Verdict integrity round 2: bind rows to sealed waves, strict single-lane authorization, immutable waves, sealed refutations, retained packets - #124
Merged
seathatflowsinourveins merged 7 commits intoSep 23, 2026
Conversation
…o their sealed wave Independent review of catalog #122 (head 2f19193), findings 1, 2, 3, 5 and 6, plus 7 and 9. Each has a regression test that fails on 97a56e2. 1. landscape.py recomputes a new-wave row's agreement from both sealed returns' winner_keys, resolved through the retained packet, and requires a recorded row's winners to equal the chosen lane's set (same_winner/codex_absent: claude; disagree: the adjudication's winner_lane); an unrecorded new-wave row carries no winners. 2. --allow-single-lane needs a dated docs/decisions/ record carrying the line "single-lane-authorization: <catalog>/<layer_id>"; the record's sha256 is stored on the row (lanes.single_lane_decision_sha256) and re-checked in CI. 3. A new wave's --write refuses once its run-manifest.json exists unless --append-rows names only absent rows, and never overwrites a sealed file with other bytes. Rows store lanes.run_manifest_sha256 and lanes.adjudication_sha256; landscape.py verifies both and rejects any file in a new wave's sealed folder that no row or the run manifest references. 5. Re-vendor layer-verdict-lane.js from agent-lab 6aab001 (sha256 2eb9c10d...), registered in lane-provenance.json; the return schema gains the refutation summary, claude_lane.py copies it, codex_lane.py drops it from the strict copy, and record_verdicts.py and landscape.py reject a Claude return whose final was refuted or unknown or lacks a lens vote. 6. --write retains the wave's packets and SHA256SUMS under <sealed_base>/packets/ (manifest retained_packets) and refuses withheld keys; CI re-checks hashes and withheld keys with the policy lane_packets.py now imports from landscape.py. 7. claude_lane.py and codex_lane.py write <lane>/failures.json; the run manifest records a "failed" outcome with its reason. 9. The kept "adopted" marker is documented; the candidate-only note is withheld. Findings 4 and 8 are left for a later PR. GRANDFATHERED_RUN_IDS and every 20260922 artifact are unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…d adjudications, stray wave folders Independent review of fcee72b: - MEDIUM: withheld_packet_keys checked only the positions withhold_popularity strips, so nested release/popularity keys, a top-level newcomers list, a non-null review_status and decisions[].selection passed CI and --write. scripts/landscape.py now walks the whole packet (is_withheld_packet_key, shared with lane_packets.withhold_popularity, which strips at any depth), rejects the disposition labels on candidate/component copies, and requires the always-listed policy labels in withheld[] (proof of --withhold-labels). Exemptions: requirement-gated archived/license and the packet's checked_at. Real 2026-09-23 packets leaked upstream.latest_flag (tag release/2025-11-28); it is now withheld. Default 2026-09-22 packets are byte-identical. - LOW: the run manifest entry lists a sealed adjudication's sha256; landscape.py requires it to equal the row's lanes.adjudication_sha256 and re-hashes the file. - LOW: a layer-verdicts-<id> folder with a non-alphanumeric id is rejected, not skipped. - INFO (lane policy edge): kept deliberately and recorded in docs/decisions/2026-09-23-verdict-integrity.md with its overturn condition. Regression tests (fail on fcee72b): test_lane_packets RecursiveWithheldKeyTests; test_landscape test_retained_packets_are_checked_at_every_depth, test_a_sealed_adjudication_is_bound_by_the_run_manifest, test_a_layer_verdicts_folder_no_row_can_name_is_rejected; test_record_verdicts ReviewOf122Tests test_a_packet_carrying_a_disposition_label_or_a_nested_withheld_key_is_refused, test_the_run_manifest_lists_each_sealed_adjudication. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
enabled auto-merge (squash)
September 23, 2026 15:54
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 238c754ef2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… at e6953e5 Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- claude_lane: delete a layer's previous return when its layer now fails or its packet is lost, before failures.json is written. - codex_lane: delete the old output for a layer whose attempts all fail. - record_verdicts: --append-rows fails before writing when a named row has no packet or lane return in the work dir. - landscape: a retained SHA256SUMS must list every retained packet exactly once with its hash; malformed or duplicate lines are rejected. - landscape: a sealed winner key must name an adopted candidate. Each fix has a regression test that fails on the previous code. The new codex_lane.py hash is appended to lane-provenance.json. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
record_verdicts now applies landscape.py's retained-SHA256SUMS rule before any write: exactly the retained packets, each once, with its actual sha256. A --write can no longer seal a wave CI would reject, since a sealed wave is never re-recorded. The test PacketWriter keeps one line per packet, as lane_packets.py does, and the three SHA256SUMS tests now expect the refusal and still check landscape.py against a wave tampered after sealing. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
deleted the
claude/verdict-integrity-2-20260923
branch
September 23, 2026 18:00
This was referenced Sep 23, 2026
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 23, 2026
…r every changed verdict row (#135) * Add the required verdict-review-gate check for changed layer-verdict rows With one maintainer and zero required approvals, a PR changing a verdict row could merge unreviewed. scripts/verdict_review_gate.py compares ledger rows at the merge base with the head, keyed by (catalog, layer_id, run_id), and requires for every changed non-grandfathered row: a registered wave document and run manifest listing the row; registered, hash-matching sealed returns from two distinct families; the agreement recomputed from the returns (review finding 1); winners resolved through the sealed packet, failing closed without it (finding 6); a both-family, both-order, unrefuted adjudication for disagree rows; an explicit docs/decisions single-lane-authorization line (finding 2); receipt-derived platform_status-only changes; and frozen earlier wave entries (finding 4). It then runs landscape.py and build_verdicts.py --check. validate.yml gains the verdict-review-gate job (every pull_request, no path filter; contents: read; harden-runner first; no persisted credentials; base sha via env). The target main ruleset adds the context; the coordinator applies it after merge. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Bind a changed row's winner fields and platform values to the sealed evidence A changed non-grandfathered row now also needs each winner's evidence_class and why_selected to be the chosen lane's and its pin to be the sealed packet's; a pending row may carry no winners; every changed platform_status value must be the one scripts/platform_status.py derives. Changes under the sealed verdict artifacts or the landscape/sota-convergence catalogs run the repository validators even when no row changed, so a deleted sealed lane file fails. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Close the verdict-review-gate review findings: decoy ledger, base-owned rules, published fields - Read rows from build_verdicts.LEDGER_FILES and fail when the head's landscape manifest names other ledgers (a decoy could be validated in place of the published ledger). - Run the base commit's gate against the PR checkout in CI, and fail a verdict change that also changes the gate's trust base in the same comparison. - Tie a changed winner's repository and recipe_ref to the sealed packet candidate, and the published alternatives, verdict_overturn_when and overturn_protocol to what record_verdicts derives from the sealed returns; open_gaps changes are re-checked. - Fail closed (exit 2) when a base or head ledger, manifest or wave registry exists but cannot be read or parsed. - Add a negative-control test for each closed bypass and record the review in the decision. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Close the second verdict-review-gate review: row continuity, semantic compare, #124 bindings F1: a base row cannot disappear, roll back to an older or grandfathered wave, or move to any wave but the newest registered one (then it needs the full new-wave evidence); one row per layer. F2: the PR-editable job definition is recorded as an accepted residual with its rejected alternatives, mitigations and overturn; a test asserts the push-to-main run executes the gate. F3: rows, wave documents and the registry compare on parsed values, so a pure reformat passes and is not a verdict change for the trust-base rule; documented that a changed row value in a non-grandfathered wave needs a new recorded wave. F4: TRUST_PATHS adds scripts/validate.py, lane-provenance.json and the host-receipt schema; a test derives the transitive imports and rule inputs from the modules. F5: on pull_request the base is the merge commit's first parent, cross-checked against the payload base.sha; push-to-main uses github.event.before and fails closed on all-zero. F6: packets resolve through the run manifest (packet_sha256, retained_packets, packets_sha256sums) and carry no withheld key at any depth; run_manifest_sha256, adjudication_sha256 and single_lane_decision_sha256 are required and bound, with #124's names. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Record the second-review measurements and tolerate #124's adjudication message The measured entry lists the new controls, the four real-checkout F1 mutations (each exit 1), the real-ledger reformat (exit 0 with both validators) and the #124 overlay run. Two adjudication tests accept the message #124's run_manifest_row_issue reports first; the no-new-wave move message no longer prints None. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Derive a changed row's verdict_status from its sealed evidence; fail closed on git errors and duplicate keys Third review of the verdict-review-gate: a recorded newest-wave row could be relabelled pending_lanes or no_selection with its winners cleared and still pass. The gate now derives the status record_verdicts.py writes for the sealed evidence (agreement, adjudication, single-lane decision, remaining alternatives) and rejects no_selection on a new-wave row. A failing git diff/ls-files raises a read error (exit 2); every parse rejects duplicate JSON keys; #124's constant values are pinned so the alignment test asserts before #124 merges. The free-form newest-wave fields are documented. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Bind winner refs and pin to the sealed evidence; require base-present single-lane authorizations Fourth review of the verdict-review-gate: - G1: every winner field except platform_status must equal record_verdicts.build_winners output (evidence_refs and pin included, no extra key), and platform_status derives from receipts bound to the sealed pin and refs; a pin the packet lacks binds only when the base's row candidates carry it. The canonical repository index must derive the same alternatives at the base. - G2: a single-lane decision record must exist byte-identical at the base. - G3: the trust-path test derives head-side reads with an audit hook; adoption/manifest.json platform_profiles is bound as a rule input; base tree reads use ls-tree and fail closed. - G4: the workflow asserts HEAD is the merge commit whose second parent is the PR head. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Record the fourth verdict-review-gate review, its fixes and real-checkout mutations Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Refresh the evidence manifest hashes of the gate, its docs, record and workflow Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Hold raised platform_status to base-trusted evidence; freeze registered SOTA manifests Fifth review of the verdict-review-gate. A changed platform_status value may rank no higher than platform_status() derives from only the cited evidence files and host receipts already at the base with the same bytes and registered there with that sha256, so a sealed citation made to resolve, or a receipt added, in the same PR cannot raise a status. The SOTA manifest every base wave registers, the newest included, keeps its pointer and value. merge_base fails closed on git errors other than no common history. The decision record adds the self-attested sealed-returns residual and rewords the overturn line and workflow comment as a consistency check. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Record the fifth verdict-review-gate review measurements and real-checkout mutations Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Re-derive an unchanged platform_status whose pin or evidence changed; fall back to the base row's candidate pin Sixth review: a candidates-only pin change with a same-PR receipt could keep a declared status the sealed pin does not support. The no-packet-pin fallback now comes from the base row, and an unchanged value is skipped only when pin, evidence_refs and evidence_class are unchanged. Three negative controls, each mutation-checked; two low findings recorded as residuals with proposed fixes. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Close the review of #135: retarget re-run, superseded-wave freeze, strict bootstrap and type-strict values - H1: the pull_request trigger adds the edited type (a base-branch change) and the verdict-review-gate job fails closed on a pull_request whose base branch (PR_BASE_REF from github.base_ref, through env) is not main, so a retargeted PR cannot keep a stale green run. - M1: when the head registers a wave newer than the base's newest, that newest wave's registry entry (sha256 included) and document are frozen byte for byte. - L4: the head's own gate runs only when base...HEAD adds scripts/verdict_review_gate.py (A); any other base without the gate fails closed. - L5: frozen values compare as canonical json.dumps(sort_keys=True), so 1, 1.0 and true differ. - Hardening: changed paths are listed with git diff -z / git ls-files -z and split on NUL. - L3: the decision record states that the pinning tests are head-defined too, and corrects the "visible workflow edit" claim with the retarget path and its fix. Each new negative control is mutation-checked (reverting its fix fails its test). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Close round 2 of the #135 review: one new current wave per change, base on origin/main A newly registered wave must be the only new run id, the head's newest and newer than every base wave, so build_verdicts.py --check always regenerates it; two new waves or a backdated wave fail. The PR gate step also requires the merge commit's first parent to be a commit on origin/main, so a merge commit built on a stacked branch fails closed. Fix the record's H1 cross-reference and document both checks. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Type-strict derivation checks and no unregistering the newest wave (#135 round 2 lows) Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Record the live retarget test of the #135 H1 fix and close the round-2 review lows A throwaway PR (#143) showed that the edited run after a retarget checks out the merge commit built on the old base; the round-2 origin/main ancestry check failed it closed, and close and reopen rebuilt it on main, where the gate passed. The error message now names that recovery. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: seathatflowsinourveins <[email protected]> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 23, 2026
… policy (#144) * Make the gate apply the landscape validator's withheld-key policy instead of a drifted copy The gate's own withheld_packet_keys predated #124 and lacked the top-level, copy-disposition and withheld[] label checks; importing scripts/landscape.withheld_packet_keys from the base makes a key the tooling owner adds (gap_receipts, blind-lanes round 2) apply here with no gate change. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Record the independent review of the withheld-policy import and correct the record's wording Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: seathatflowsinourveins <[email protected]> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the #122 review findings (agent-lab-e9,
review122/findings.md) before the 20260923 re-record of all 32 rows. No ledger row changes. The regression tests fail on 97a56e2: 26 failures and 4 errors.landscape.pyrecomputes the agreement from both sealed returns'winner_keys, resolved through the retained packet. The winners must equal the chosen lane's (same_winner or codex_absent: claude; disagree: the adjudication'swinner_lane). A row that isn't recorded carries no winners. Relabelling agreement or swapping in the losing lane's winners fails.docs/decisions/and contain the exact linesingle-lane-authorization: <catalog>/<layer_id>. Its sha256 is stored aslanes.single_lane_decision_sha256and re-checked.--writerefuses oncerun-manifest.jsonexists, unless--append-rowsnames only absent rows.lanes.run_manifest_sha256and, for disagree rows,lanes.adjudication_sha256. The run manifest also lists adjudications.unrefutedand both lens votes on the sealed round arefalse. This is checked at record time and in CI.e6953e5) is re-vendored byte for byte (sha2562eb9c10d…7fbb).claude_lane.pycopies the refutation summary and lists layers without a final infailures.json.packets/and listed in the run manifest.*_at,latest,prerelease,pin_behind_upstream,newcomer.stripped_packet_sha256must name a retained packet.failedlane outcome with a reason (finding 7);adoptedmarker is documented and candidate-onlynoteis stripped (finding 9);Review
Checks (merged with current main)
python3 -m unittest: 2,960 tests OK, 339 skipped.validate.py,evidence_manifest.py --check,landscape.py,build_verdicts.py --check,component_matrix.py --checkandtrading_gates.py --check: all pass.🤖 Generated with Claude Code