feat(evidence): fork-based signing workflow + publishing docs#1451
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Enterprise Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughAdds a Bash helper, a Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/evidence-sign-unsigned.sh:
- Around line 44-45: The signer lookup in evidence-sign-unsigned.sh currently
masks `yq` or YAML parsing failures by falling back to "null", which can
misclassify signed pointers as unsigned. Update the logic around the `yq eval`
call that populates `signer` so failures from `yq` or invalid input cause the
script to fail closed instead of echoing "null", and keep the unsigned check in
the existing `signer` handling block.
In @.github/workflows/evidence-publish.yaml:
- Around line 98-115: The evidence publishing workflow currently loses
already-signed pointer patches if a later signing step fails, so the partial
work is never committed. Update the sign-and-commit flow in
evidence-publish.yaml around the “Sign unsigned evidence pointers” and “Commit
signed pointers” steps so that successful pointer patches are preserved and
committed even when a later signing operation errors; ensure the sign script or
its wrapper records partial success and the commit step still stages and pushes
any signed changes from recipes/evidence/.
In `@docs/contributor/evidence-publishing.md`:
- Around line 65-68: The example in the evidence publishing guide uses the wrong
commit-signing flag for external contributors. Update the command shown in the
contributor docs near the evidence-publishing example so it reflects the repo’s
real policy: use git commit -s for external contributors’ DCO sign-off, and only
mention git commit -S for NVIDIA org members if needed. Make the example wording
in that section consistent with the symbols and commands shown there, especially
git commit and the evidence/<slug>.yaml workflow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 7826b7cc-a867-496c-913e-76ac9d6a80be
📒 Files selected for processing (4)
.github/scripts/evidence-sign-unsigned.sh.github/workflows/evidence-publish.yamldocs/contributor/evidence-publishing.mddocs/index.yml
Coverage Report ✅
Coverage BadgeNo Go source files changed in this PR. |
njhensley
left a comment
There was a problem hiding this comment.
📋 Multi-persona review — PR #1451
Method: 3 independent persona passes (Security/CI supply-chain · Bash/Reliability · Docs/DX), each finding independently confirmed, refuted, or re-tiered by a senior meta-reviewer against the resolved code (CLI source on main, the repo's required_signatures ruleset, and GitHub Actions if: semantics). Line links pinned to head 00228d4.
Tier legend: 🔴 Blocker · 🟠 Major · 🟡 Minor · 🔵 Nitpick · ✅ Confirmed non-issue
Overall assessment — Approve with comments (resolve the two 🟠 first)
The two-phase split — push the network-light leg locally with --no-sign, do the Fulcio-bound leg in CI where Sigstore is reachable — is a sound design, and the workflow_dispatch-on-the-fork framing genuinely defuses the worst class of CI risk (no pull_request_target, no untrusted-ref exposure). No hard blocker. Two Major issues are merge-relevant: a workflow step whose if: silently defeats the script's own fail-closed exit, and an unsigned bot commit-back that contradicts both the doc's "fully signed pointer" wording and the repo's required_signatures ruleset. The remaining items are robustness, a real naming-confusion trap, and polish. Notably, the scariest candidate — a signed-in-registry-but-un-patched-pointer race — was refuted on inspection of SignExisting.
✅ Confirmed non-issues (checked and cleared)
- No signed-in-registry-but-un-patched-pointer race. The highest-proposed-tier candidate was refuted:
SignExistingcallsValidateSignablePointer(rejects a non-nil signer withErrCodeConflict), and on success setsAttestations[0].Signerand writes the file. A successful sign therefore always mutates the pointer, sogit diff --quietcannot find "nothing to commit" after a real--no-signpointer is signed; an idempotent re-sign returns non-zero and counts asfailed, notsigned. #aicr-evidence-sign,#aicr-evidence-publish,#aicr-validate, and../design/007-recipe-evidence.mdall resolve on currentmain(lychee-safe). CLI flag claims are accurate:--no-signon bothvalidateandevidence publish; positional bundle dir;evidence verifyreports unsigned as a non-failing "pending signature".concurrencywithcancel-in-progress: falseis correct — a mid-sign cancel could orphan a signed bundle from its un-patched pointer.- Fork-only
workflow_dispatchtrust model is sound: nopull_request_target, no untrusted-ref injection;actions/checkoutandactions/setup-goare SHA-pinned;timeout-minutes: 20set.
Summary
| Tier | Count | Items |
|---|---|---|
| 🔴 Blocker | 0 | — |
| 🟠 Major | 2 | commit-step if: defeats fail-closed exit · unsigned bot commit vs required_signatures + doc wording |
| 🟡 Minor | 6 | yq fail-open · persisted token across build · yq unpinned/local-incompat · bare git push · publish-vs-sign naming · id-token "default" misstatement |
| 🔵 Nitpick | 5 | nav casing · inert [skip ci] · skipped not in output · concurrent-push warning · go mod verify |
Recommendation: Approve with comments. No hard blocker. Fix the two 🟠 (one-line success() && guard; commit-back via peter-evans/create-pull-request, which also clears the persisted-token and bare-push items) before merge — the second could escalate to 🔴 if required_signatures blocks the PR merge. The 🟡/🔵 items are safe to address in the same pass or defer in writing.
Generated via a multi-persona + adversarial-meta-reviewer pass; findings independently re-derived from resolved code before surviving.
00228d4 to
e17b4b3
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/evidence-publish.yaml:
- Around line 122-129: The workflow-generated commit in the evidence publishing
step is missing a DCO sign-off trailer, which can cause the PR-head commit to
fail DCO checks for external contributors. Update the commit creation in the
evidence publish workflow by using the existing git commit step with a sign-off
option so the generated commit includes a Signed-off-by trailer, and keep the
change localized around the git commit command in the evidence publishing job.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: fa653afd-ed62-4014-add4-22d66ea24910
📒 Files selected for processing (4)
.github/scripts/evidence-sign-unsigned.sh.github/workflows/evidence-publish.yamldocs/contributor/evidence-publishing.mddocs/index.yml
e17b4b3 to
a2f91da
Compare
njhensley
left a comment
There was a problem hiding this comment.
✅ Approve — with two non-blocking notes
Re-reviewed at head e17b4b3. The revision resolves 11 of 13 findings from the multi-persona pass, including both 🟠 Majors:
- F1 (commit-step
if:) — nowalways() && …with documented intent: partial-failure still commits the pointers that did sign so the registry never holds a signed bundle with an unsigned committed pointer, while the job still fails to surface the error. Deliberate, well-reasoned. - F2 (unsigned bot commit) — documented that the commit relies on the eventual squash-merge for the repo's commit-signing policy;
[skip ci]dropped; doc softened from "fully signed pointer" to "a pointer." - F4 yq fail-open → fixed (fail-closed on parse error). F5
persist-credentials: false+ inline push token. F6 mikefarah-yq dependency documented. F7 non-fast-forward documented as recoverable. F9id-token"default" wording corrected. F11[skip ci]removed. F12skippedadded to$GITHUB_OUTPUT. F13 concurrent-push warning added. F14go mod verifyadded.
Two stragglers remain, both non-blocking (inline) — neither holds up the approve. Nice work on the turnaround.
Tier legend: 🟡 Minor · 🔵 Nitpick
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/evidence-publish.yaml:
- Around line 94-101: The build step for the aicr binary assumes the output
directory already exists, so `go build -o ./bin/aicr` can fail in this workflow.
Update the workflow around the `Build aicr` step to create `./bin` first, either
by adding a directory creation command before `go build` or by reusing an
existing setup step, so the `go build` command in the `Build aicr` job succeeds
reliably.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 5d4fa0ec-43de-492f-a009-e3ec11ad6837
📒 Files selected for processing (4)
.github/scripts/evidence-sign-unsigned.sh.github/workflows/evidence-publish.yamldocs/contributor/evidence-publishing.mddocs/index.yml
Add the Fulcio-bound signing leg of two-phase evidence publish (#1433) and document the connectivity workaround it exists for (#1436). Workflow (.github/workflows/evidence-publish.yaml): - workflow_dispatch, no inputs — the committed pointer is the work item. - permissions: id-token: write (ambient keyless OIDC) + contents: write (commit the patched pointer back). - Runs in a contributor's fork: builds aicr, discovers every committed pointer with an empty signer block, signs the bundle each references with the runner's OIDC identity (`aicr evidence sign`), and commits the patched pointers back. Clean no-op when nothing is unsigned. - Option B (auto-trigger on push) is documented in the header as a follow-up with its loop-guard requirements; Option A (manual) lands first. Discovery script (.github/scripts/evidence-sign-unsigned.sh): - "unsigned" = first attestation has no signer (the --no-sign state); already-signed pointers are skipped, so re-runs are idempotent. - Fails closed with an actionable message when a bundle can't be pulled (a private fork package returns HTTP 403 on the pre-sign pull). Docs (docs/contributor/evidence-publishing.md, nav entry): - Explains the Fulcio/Sigstore IP-level block on corporate VPNs, the recommended split-leg path (local --no-sign push, then sign in CI), the public-GHCR-package requirement, and the local-only fallback. Cross-links ADR-007 and the aicr evidence CLI reference. Smoke-tested: discovery skips both committed (signed) pointers (no-op); a temp unsigned pointer is detected and routed to `aicr evidence sign`, which fails closed with the public-package hint when the bundle can't be pulled. shellcheck + actionlint + yamllint clean.
a2f91da to
96130c1
Compare
njhensley
left a comment
There was a problem hiding this comment.
✅ Approve — re-reviewed at head 96130c1.
All 13 findings from the multi-persona pass are now resolved, including the two stragglers from the prior round:
- F8 (publish-vs-sign naming) — doc now explains the filename: "The filename says publish because it anticipates a future auto-publish leg (see the workflow header); today it only signs." Exactly the note that closes the loop.
- F10 (nav vs H1 casing) — H1 aligned to Title Case to match the nav.
Bonus hardening since the last revision: the bot commit-back now does git commit -s (DCO sign-off matching the github-actions[bot] author), with the rationale documented inline; step 2 of the doc mirrors it. Verified against CONTRIBUTING.md — external contributors -s, NVIDIA org members -S — the doc's claim is accurate.
CI is green across Lint, Test, CLI E2E, Security Scan, Fern Check (anchors/lychee), and actionlint; tests / E2E is still in progress (merge stays gated on it). Clean approve — nice, thorough iteration. 🟢
Summary
Adds the Fulcio-bound signing leg of two-phase evidence publish: a
workflow_dispatchworkflow a contributor runs in their fork to sign committed-but-unsigned evidence pointers, plus a contributor doc explaining the Sigstore connectivity problem this works around.Motivation / Context
Final piece of the evidence-gate pilot epic. Keyless signing reaches
fulcio.sigstore.dev/rekor.sigstore.dev, which corporate VPNs and some home networks block at the IP level. The fix is to keep the network-light push leg local (--no-sign, merged in #1445) and move only the signing leg to CI, where Sigstore is reachable. This consumes theaicr evidence signCLI merged in #1446.Fixes: #1433, #1436
Related: #1431, #1434
Type of Change
Component(s) Affected
docs/,examples/)Implementation Notes
.github/workflows/evidence-publish.yaml—workflow_dispatch, no inputs (the committed pointer is the work item).id-token: write(ambient keyless OIDC) +contents: write(commit the patched pointer back). Builds aicr, runs the discovery script, commits signed pointers back. Clean no-op when nothing is unsigned. Option B (auto-trigger on push) is documented in the header as a follow-up with its loop-guard requirements; Option A (manual) lands first per the issue..github/scripts/evidence-sign-unsigned.sh— "unsigned" = first attestation has nosignerblock (the--no-signstate); already-signed pointers are skipped (idempotent re-runs). Fails closed with an actionable message when a bundle can't be pulled (a private fork package → HTTP 403 on the pre-sign pull).docs/contributor/evidence-publishing.md(+ nav entry) — the Fulcio block, the split-leg path, the public-package requirement, and the local-only fallback.pull_request_target/untrusted-ref exposure (manual dispatch on the fork's own code).Testing
Local smoke tests (built aicr, ran the discovery script):
signed=0, clean no-opaicr evidence sign→ fails closed with the public-package hint when the bundle can't be pulledshellcheck -S warning+actionlint+yamllintall clean.Risk Assessment
Rollout notes: Contributors enable Actions in their fork and make their
aicr-evidencepackage public (documented). Upstream behavior is unchanged.Checklist
shellcheck,actionlint,yamllint)git commit -S)