Skip to content

aicr verify-evidence command for reviewers and CI #753

Description

@mchmarny

Parent: #750
Depends on: evidence bundle format child issue (filed alongside)

Summary

New top-level CLI command aicr verify-evidence <bundle> that maintainers and CI use to verify a recipe-evidence bundle produced by aicr validate --emit-evidence.

Why a separate command

Different audience from aicr validate:

  • aicr validate is run by recipe authors / cluster operators against live infrastructure.
  • aicr verify-evidence is run by reviewers and CI against an offline artifact, with no cluster required.

Keeping the verification surface isolated also makes the verification logic auditable and minimal — the maintainer's trust assumption is "this command does exactly the cryptographic + schema checks I need."

Verification steps

  1. Signature. Verify the in-toto attestation via cosign verify-blob-attestation with the expected OIDC issuer/identity. Record the signer identity.
  2. Schema. Validate the predicate against the published JSON Schema (recipe-evidence-v1). Reject unknown predicate types or unsupported schema versions.
  3. Inventory. Verify every file referenced in the manifest exists in the bundle and its content hash matches.
  4. Recipe ↔ snapshot binding. Run the fingerprint matcher (from the snapshot issue) — confirm the snapshot's fingerprint satisfies every dimension of the recipe's criteria. Reject if it doesn't, with a clear diff.
  5. Inline constraints. Replay the recipe's inline constraints against the snapshot (today's aicr validate --no-cluster path). These are deterministic and should still pass on the reviewer's machine.
  6. Phase results. Read the CTRF outputs and surface per-check status. Don't re-run validators; that defeats the purpose.
  7. BOM cross-reference. Confirm the bundled CycloneDX BOM matches the recipe's expected component set.
  8. Output. Emit a structured JSON report and a Markdown summary suitable for embedding in a PR review comment.

Markdown summary shape

## Evidence verification — <recipe-name>

**Signer:** [email protected] (via GitHub OIDC, github-actions identity)
**AICR:** v0.13.0  •  **Schema:** recipe-evidence-v1.0.0  •  **Bundle SHA:** sha256:abc…

### Cluster fingerprint
✅ service=eks, accelerator=h100, os=ubuntu, k8s=1.33.4 — matches recipe criteria

### Phase results
✅ deployment   12/12 checks passed
✅ performance   3/3 checks passed (NCCL all-reduce: 392 GB/s, threshold 380)
✅ conformance   9/9 checks passed

### Inline constraints
✅ All 6 inline constraints satisfied against snapshot

### BOM
22 components, 71 unique images, all digests resolved

CLI shape

aicr verify-evidence ./evidence-bundle.tar.gz
aicr verify-evidence ./evidence-bundle.tar.gz --output-format json
aicr verify-evidence ./evidence-bundle.tar.gz --output-markdown summary.md
aicr verify-evidence ./evidence-bundle.tar.gz \
  --expected-issuer https://token.actions.githubusercontent.com \
  --expected-identity-regex 'https://github.com/.*/\.github/workflows/.*'

Exit codes:

  • 0 — bundle valid, all checks passed
  • 1 — bundle valid, but evidence shows failed checks (informational success path)
  • 2 — bundle invalid (signature, schema, fingerprint mismatch, hash mismatch)

Success criteria

  • aicr verify-evidence runs offline (no cluster, no network beyond Rekor lookup).
  • A maintainer can run it locally and trust the output, or wire the same binary into CI for the same answer.
  • Markdown summary is small enough to paste into a PR comment but dense enough to support a merge decision.
  • Tampering with any file in the bundle fails verification.
  • Resigning the bundle with a different identity surfaces the new signer in the report.

Out of scope

  • Re-running validators against a cluster. Verification is an offline cryptographic + schema operation.
  • Long-running watch / polling modes.

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions