Skip to content

Recipe contribution workflow with CI evidence gate #751

Description

@mchmarny

Parent: #750
Depends on: aicr verify-evidence command (filed alongside)

Summary

Document the end-to-end community recipe contribution workflow and add a CI gate that runs aicr verify-evidence on every PR touching recipes/overlays/ (or other recipe-defining paths). The gate posts a Markdown verification summary as a structured review comment so a maintainer can approve a recipe for hardware they don't have.

Workflow (contributor view)

  1. Author a new overlay or modify an existing one.
  2. Apply it to a real cluster matching the recipe's criteria.
  3. Run aicr snapshot on that cluster.
  4. Run aicr validate --recipe <r> --snapshot <s> --emit-evidence ./evidence/.
  5. Attach the resulting evidence-bundle.tar.gz to the PR (uploaded artifact, release asset, or large-file attachment — exact mechanism TBD per repo policy).
  6. Open the PR. CI verifies the bundle and posts a summary. Maintainers review the bundle alongside the diff.

Workflow (maintainer / CI view)

  1. CI detects a recipe-touching PR.
  2. CI fetches the bundle (path TBD — likely evidence/<recipe>.tar.gz checked into the PR, or a separate signed-artifact link).
  3. CI runs aicr verify-evidence <bundle> with expected issuer/identity constraints set to public OIDC providers AICR trusts (initially GitHub Actions OIDC).
  4. CI posts the Markdown summary as a review comment. A failing bundle blocks merge via a required check.

Deliverables

  • .github/PULL_REQUEST_TEMPLATE/recipe.md — recipe-specific PR template requiring evidence bundle attachment, contributor declaration of test environment, and link to relevant criteria documentation.
  • .github/workflows/recipe-evidence.yaml — workflow that runs on pull_request events touching recipes/overlays/** (and other recipe paths). Runs aicr verify-evidence, posts a sticky review comment, sets a required check status.
  • CONTRIBUTING.md update — new section: "Contributing a recipe for hardware AICR maintainers can't access" — walks through the contributor workflow with a worked example and an FAQ (what if I don't have an OIDC identity, what if my test failed but I think the failure is environmental, etc.).
  • docs/integrator/recipe-development.md update — link from the existing recipe authoring docs to the contribution workflow page.

Acceptance criteria for the gate

  • A PR adding a recipe without an evidence bundle fails the required check.
  • A PR with an invalid bundle (bad signature, schema mismatch, fingerprint mismatch) fails the required check.
  • A PR with a valid bundle whose evidence shows failing validator checks does not auto-fail the required check — it posts the summary and lets the maintainer decide. Failed-evidence PRs are sometimes intentional (e.g., known-issue documentation, work in progress).
  • Required-check status, comment text, and exit codes are deterministic so contributors can self-debug.

Open questions

  • Bundle delivery mechanism. Embedding tar.gz in the PR diff is awkward. Options: (a) GitHub Releases asset on the contributor's fork, (b) cosign-signed artifact in a registry referenced from the PR body, (c) checked-in under recipes/evidence/<recipe>/. We should pick one and document it. Starting with (c) is the simplest path for v1.
  • Scope of "recipe-touching." Initially recipes/overlays/**. Should mixin or component changes also require evidence? Probably yes once the workflow is mature; out of scope for v1.
  • Multiple bundles per PR. If a PR adds a recipe variant covering several cloud / accelerator combinations, must each be evidence-backed? Yes — one bundle per criteria combination — but the gate UX should handle the n-way case cleanly.

Success criteria

  • A worked example PR exists in the docs (or as a real PR-as-fixture) that walks through contributor → CI → maintainer.
  • A maintainer with no access to a particular hardware combination can confidently approve a PR for it on the strength of the bundle and the CI summary.
  • The CI gate is robust: clear failure messages, no false negatives on bundle parsing, friendly to first-time contributors.

Out of scope

  • Rewarding contributors with specific badges/recognition (different policy work).
  • Hosting bundles outside the PR lifecycle for long-term retrieval (cosign + Rekor handle the durable record).

Metadata

Metadata

Assignees

Fields

No fields configured for Enhancement.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions