Repository navigation
Split the OSV SARIF upload into a tool-free job; fix the blind comparison's confirmed defects and file its evidence - #121
Conversation
…ind-comparison evidence Correct the stale harden-runner ubuntu-job count (18 of 22 -> 20 of 24, measured at HEAD with the exact counting method), fix dependabot.yml's "github-actions only" header and every doc claim that Dependabot owns only GitHub Actions references now that a deliberately vulnerable pip fixture entry exists, and fix the decision record: the stale "only the osv-scanner job holds security-events: write" claim (zizmor-sarif-upload also holds it), the "separate upload job scans twice" rationale that conflicts with the chosen design, and add the missing Evidence/Alternatives/Decision/Overturn elements to the zizmor-online, Scorecard SARIF and dependency-review sections. Add mutation-checked assertions covering the OSV/zizmor SARIF upload failure-path semantics to tests/test_workflow_hardening.py. Record the "Blind comparison (2026-09-23)" section in the decision record and copy its comparison artifacts into evidence/artifacts/blind-comparison-20260923/ (with the judgments file's scratch/home paths redacted and a README listing every file's sha256), registered in manifests/evidence.json. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Fix the review's confirmed high/medium findings: - security-scan.yml section 3 never had only one job holding security-events: write (zizmor-online held it from the founding commit a9e5180); say so, with the verification command that shows the count. - Restate the OSV separate-upload-job rejection honestly: it would not scan twice, and the real trade-off is the write scope sitting on the job that runs a curl-installed binary; mark it keep-but-compare. - Reword the dependabot.yml header and the two docs/github-automation.md passages: ignore: urllib3 suppresses security-update PRs for the fixture pin; open-pull-requests-limit: 0 only stops version updates; alerts still appear via the dependency graph regardless. - Restore the sealed 2026-09-22 lane:claude Dependabot why_not_default text verbatim in catalogs/landscape/foundation.json, and move the pip-fixture correction into a labelled coordinator annotation in the layer's limitations list instead of overwriting lane output; regenerate layer-verdicts-20260922.json and the handbook with build_verdicts.py --write. - Section 5: cite the observed hosted upload (the first push run on 4970ba0, dispatch run 35824151483) and scorecard-action's documented example workflow at its pinned SHA, not just the source diff. - Section 4: name zizmor's actual online-only audit classes instead of claiming offline analysis covers no secrets class (its secrets audits are static). Section 6: replace the #97/#98 "nothing to flag" inference with what those PRs actually show (gate latency, not threshold behavior). Fix the low findings: - tests/test_workflow_hardening.py: the failure-path test read a fixed line offset after each step/job marker, so a reordered key or a folded if: block scalar could pass vacuously. Add step_block()/ block_if() helpers that search the whole block and resolve folded scalars, and assert zizmor-online's analyzer step carries no continue-on-error (a crash there could otherwise still upload a partial SARIF). Mutation-verified: reordered uses:/if: with the cancelled() guard dropped, added continue-on-error to the analyzer step, and folded the upload job's if: with a hidden always() -- each mutation failed the test before being reverted. Re-registered all seven changed files in manifests/evidence.json with scripts/host_receipts.register_file. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…eview findings The osv-scanner job held security-events: write while running the installed scanner on every event; it is now contents: read and a needs/!cancelled() osv-sarif-upload job (no shell step or installed tool) uploads, matching the zizmor split. Record sections 3 and 5 are rewritten to match (Scorecard upload now cited from the code-scanning analyses list). The lane-sourced catalog text is restored verbatim and its staleness recorded instead; the redacted judgments copy now replaces only path tokens; the folded-if bypass in the artifact-step test is closed. Mutation-checked: a folded always(), a lost !cancelled() and a write scope back on the scan job each fail the suite. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…roof step checks, corrected citations Artifact names now include run_attempt so a re-run cannot hit a same-name conflict. step_block() spans the whole list item and block_if() collects plain-scalar continuations, so a leading '- if:' or an always() on a continuation line is caught (mutation-checked). Docs: harden-runner 21 of 25 ubuntu jobs after the split; Scorecard layout cited to the example the README links (ossf/scorecard scorecard-analysis.yml @ d13ba3f) and the upload to paginated analysis ids; the unmerged-branch commit dropped from the grep list; the redaction and upload-job wording corrected. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d4c9c5f7bc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| `<scratch>/`, keeping each file name (2 occurrences; every other byte is unchanged). `SEALED-KEY.json`, arm checkouts, `*.diff` files and battery JSON | ||
| files are deliberately not copied here. |
There was a problem hiding this comment.
Retain the omitted comparison outputs and inputs
The newly added evidence explicitly omits the battery JSON outputs, so an auditor cannot verify the recorded per-check details or command results from actual returned output; the packet contains only normalized booleans and exit codes. This is not recoverable by rerunning the committed harness either, because battery_catalog.py references the absent ../gh-settings/after input, causing B13 to become non-applicable rather than reproducing the reported 13/13 result. Retain sanitized battery outputs and immutable input/revision references with the closure evidence.
AGENTS.md reference: AGENTS.md:L10-L15
Useful? React with 👍 / 👎.
…rivate horizon repository (#1194) * tools(landing): add horizon_land.sh, the local-first lander for the private horizon repository horizon_land.sh lands pull requests of seathatflowsinourveins/horizon, which has no hosted CI, no scripts/validate.py, no evidence manifest and no Codex connector. Its main ruleset requires the commit statuses lander/local-ci, lander/verdict-claude and lander/verdict-gpt (strict), so the lander is the only merge path: it checks the local evidence, posts the three statuses at the exact head and squash-merges pinned with --match-head-commit. Checks, in order: the arguments and inputs (HORIZON_CLAUDE_VERDICT, HORIZON_GPT_VERDICT, HORIZON_MUSE_VERDICT as a TRIAL, LOCAL_CI_RECEIPTS), with the other landers' inputs refused before any call; its own horizon.lock; the pull request's head must equal the read head (no head-following); origin/main must be an ancestor of the head and GitHub must report neither BEHIND nor DIRTY (the branch is never updated here); the verdicts (nas_land.sh's acceptance plus a last Gate/Verdict line naming the full 40-hex head, the right Family line, two different files); the receipts (CI: exit <code> at <head>: <command>, the last line per command counts, sh tests/hooks_test.sh required, every command exit 0); main's CODEOWNERS owned by the merging account alone, and GitHub's codeowners/errors at that main reporting none; then a draft is marked ready, after the local refusals. Correction #121 is carried as nas_land.sh carries it (the CC's ruling of 2026-10-15): a verdict of any family whose text states a landing gate (nas's four phrases, any letter case) needs GATES_DONE, a receipt that names the full 40-hex head (the shared block's lg_receipt_ok given the full head, so the 8-digit prefix alone is refused), or exit 2 before any change on GitHub; it is read again at the final gate (exit 7 when its bytes changed) and recorded as Lander-gates-done. Statuses: each success carries "<file name> sha256 <digest>" (local-ci: one digest over the receipt files). An EXIT trap, with HUP, INT and TERM trapped, posts failure for all three unless the merge succeeded. The merge state is polled until CLEAN. The squash body ends in a trailer paragraph (Lander-head, Lander-local-ci with one Lander-local-ci-receipt line per file, Lander-verdict-claude, Lander-verdict-gpt, Lander-verdict-muse when set, Lander-gates-done when a verdict states a landing gate, Lander-codeowners). final_gate re-reads the head, the verdicts, GATES_DONE and the receipts and their digests, the combined status (exact descriptions, contexts compared case-insensitively), CODEOWNERS, origin/main and the merge state, with nothing but the merge after it. Exit codes: 2 inputs, verdicts or a stated landing gate without GATES_DONE, 3 head moved, 5 receipts or statuses, 6 merge state not settled, 7 ready, body or merge failed, or a verdict, receipt or GATES_DONE changed after its post, 8 BLOCKED or CODEOWNERS, 11, 12 and 14 as in the other landers, 13 not up to date or main unreadable or moved, 15 (new) a status POST failed, 129, 130 and 143 a signal. The shared land-gate-fns block is nas_land.sh's byte for byte (now a three-way test); nas_land.sh and uet_land.sh are unchanged. Tests: HorizonEndToEndTests with its own scratch repository, the fake gh's status POST and codeowners/errors branches, and horizon joining the existing script-text checks. README, its digest row and the manifest rows updated. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Fold the #1194 reviews: names-only statuses and trailers, base binding, PROPOSAL labels The GPT read (CHANGES_REQUESTED at 34add43) and the CC's read (PASS with items), each finding and its fix: - GPT P1, private digests published: the statuses and the trailers named each private file's sha256. Now the three descriptions are file names (local-ci: the receipt names, one space apart) and every Lander-* trailer names files only; CODEOWNERS keeps its sha256, its preimage being main's own file. The digests stay local: compared at the final gate and kept in a landing receipt, $LANDING_COORD/horizon-landings/<pr>-<head12>.json, written once after the merge with mode 0400 (exclusive create; its place checked before ready and the posts, exit 7; a failed write after the merge, new exit 16). - GPT P2, base not bound: the head and baseRefName are read together; the base must be main before ready and the posts and again at the final gate (exit 13, the name withheld). Cites github/docs about-code-owners.md:44 and cli v2.102.0 pkg/cmd/pr/merge/http.go:47-50, :77-80. - GPT P3, terminator: the full head must be a whole word (no letter, digit or _ after it), in a verdict and in GATES_DONE; head+A is refused. - GPT P3, post-merge report: the merge commit is read with its exit checked and printed in its typed shape only; a failed read is exit 12 (the receipt then records no merge commit). - GPT P3, body guard: #1194's base, main at the first reads and its first head registered in PUBLIC_COMMITS with their URLs (each read on GitHub), and the first head's horizon_land.sh digest in PUBLIC_DIGESTS. - CC P2, "the only merge path": labelled PROPOSAL, pointing to #1195's decision record, with the residual stated: GitHub checks only the contexts' states and anyone with write access can set one (available-rules-for-rulesets.md:176, :178), so a web-UI merge after the posts is not refused. - CC P2, ruleset mechanics: strict cited at available-rules-for-rulesets.md:188-193 and the checks at :172-178; what those lines do not say is labelled PROPOSAL. - CC P3, the merge-state lag: "measured on us-equities-trading #58, not documented by GitHub". - CC P3, UNSTABLE: MergeStateStatus cited at schema.docs.graphql:26511-26554 (:26551-26553). - CC P3, a second signal during revocation: trap '' HUP INT TERM is hz_revoke's first statement. Tests, each failing at 34add43 (9 tests, 34 failures) and against single-rule mutants: no digest or 7-digit prefix of a private file in a description or the body, the receipt keeping them; the receipt's place and its write; a base other than main, at the start and as a late retarget flip; head+A, G, 0 or _; the post-merge read with a stdout sentinel; a TERM during the first revocation POST. The shared block stays nas_land.sh's byte for byte; nas_land.sh and uet_land.sh are unchanged. README, its digest row and the manifest rows refreshed. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Fold the #1194 delta reads: required local workflow wording, signal-safe merge finalisation, fail-closed CODEOWNERS and receipt lists The command center's delta read at b4489b1 (PASS with micro items), the GPT delta read (ACK with four P3s) and the local review (three P2s), in one commit: - Wording matches #1195's Decision 4: the lander is "the required local workflow" (GitHub enforces three contexts and freshness; the web-UI residual), and the self-audit is "the PROPOSAL self-audit", never "planned" (CC P2-1). - The merge is finalised whole: HUP, INT and TERM are ignored from just before the merge call until the landing receipt is written and the merge reported, so a signal neither revokes a merged head's statuses nor loses the receipt (CC P3-3, local review P2-2); when exits 12 and 16 both apply, 12 is the exit (documented and tested). - CODEOWNERS fails closed: a git ls-tree or read that fails is exit 8, and the next location is tried only after a successful lookup with no entry (local review P2-1). - LOCAL_CI_RECEIPTS with a leading, trailing or doubled ':' is exit 2 before any call; read -a dropped a trailing empty field (local review P2-3). - uet#58's lag is worded as measured there, not documented by GitHub (CC P3-1); the noclobber and trap '' texts cite the installed bash 5.3 bash(1) (CC P3-2). - Comments no longer describe a posted digest (GPT P3-4). - The README digest row and the published-digest list follow the new script bytes; b4489b1's script digest is registered with its URL. Fail-first: the four new tests fail 7 times against b4489b1. Follow-ups, not taken here: GPT P3-1 (the shared author check, in the block that stays byte-identical to nas), P3-2 (a late local symlink swap after the receipt check) and P3-3 (a documentation assertion for the recovery limits). Co-Authored-By: Claude Opus 5.5 <[email protected]> * Reconcile a failed merge call before revoking; cite bash for the receipt split and pipefail The command center stopped the auto-land at 4fad273 on the local review's P2, and the GPT micro read there was an ACK with five P3s; one commit: - A failed merge call is reconciled before anything is revoked (local review P2): GitHub can commit the merge and the reply be lost, so the state is read three times, two seconds apart. MERGED at the read head continues as a landing (HZ_MERGED, the receipt, the report); OPEN or CLOSED there on all three reads is exit 7 and the EXIT trap revokes; anything else is the new exit 17: the outcome is unknown, the statuses are kept (HZ_KEEP), nothing is revoked, and the pull request is checked by hand. README exit codes and step list updated; the old merge-failure test now serves an OPEN pull request. Fail-first: the new test fails against 4fad273. - GPT P3-4: the empty-field comment cites bash(1) EXPANSION, Word Splitting ("field terminators"; adjacent delimiters "produce a null field"), and the pipefail comment quotes SHELL GRAMMAR, Pipelines (the rightmost non-zero status), both from the installed bash 5.3 (5.3-2ubuntu1). - GPT P3-5 is the PR body's wording (aligned there); P3-1, P3-2 and P3-3 stay Follow-ups. - The README digest row and the published-digest list follow the new bytes; 4fad273 and its script digest are registered with their URLs. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
This is the preregistered graft after the blind comparison of the two catalog automation implementations. Closure (tools/compare/closure.mjs) retained the merged implementation, and the publication gate passed: 3 of 3 blind refuters did not refute, with 0 leaks.
Security change
osv-scannernow runs with onlycontents: read.osv-sarif-uploadjob (needs: osv-scanner,!cancelled()) holdssecurity-events: writeand runs no shell step or installed tool. This matches the zizmor split, so the job running the installed scanner no longer holds a token that can dismiss the CodeQL alerts thecode_scanningrule gates on.run_attempt, so a re-run is safe.Tests
!cancelled()) and the zizmor artifact step (no forced run).if:parsers now also catch a leading- if:, folded block scalars and plain-scalar continuation lines.Docs and evidence
ossf/scorecardexample atd13ba3fthat the README links) and the observed-upload analysis ids, the dependabot header, and the upload wording.evidence/artifacts/blind-comparison-20260923/: preregistration with amendment 1, protocol, packet, closure, battery, harness, and a path-redacted judgments file with the original's sha256.Checks
osv-sarif-uploadon GitHub for the first time.🤖 Generated with Claude Code