Skip to content

Evidence gate: curated 'protected recipes' set (implicit by committed pointer) #1432

Description

@njhensley

Part of #1431.

Summary

Distinguish recipes the evidence gate actively protects from the ~61 leaf recipes that have no evidence yet. A recipe is protected implicitly — iff it has a committed pointer in recipes/evidence/. Today only gb200-eks-ubuntu-training and h100-gke-cos-training carry pointers; every other recipe renders as "missing," which makes the gate comment on broad-impact PRs (e.g. #1418) look alarming despite being warning-only.

Motivation / Context

  • Evidence is hardware-gated: a bundle is only valid on a cluster matching the recipe's full criteria (accelerator × service × OS × intent × platform). Most clusters aren't readily available, and *-kind-* recipes can't produce real evidence. "Add pointers to all 63 recipes" is impractical.
  • The signal we want is: for the recipes we've chosen to protect, is their committed evidence stale or invalid? Everything else is best-effort.
  • Using "has a committed pointer" as the definition means zero new files to maintain — the set grows naturally as evidence is added, and converges with the maturing testgrid work (the recipes we validate on real hardware).

Proposed approach

  • In .github/scripts/recipe-evidence-check.sh, treat "protected" as "has a pointer file in recipes/evidence/."
  • Split the report into Protected (full status: pointer / verify / digest-match, drift highlighted) and Other affected (collapsed <details> summary count, or omitted). Keep warning-only semantics.

Acceptance criteria

  • Gate comment clearly separates protected recipes (have pointers) from "no evidence yet" recipes.
  • On a broad-impact PR, the all-recipes list is collapsed/summarized, not the headline.
  • The implicit "has a pointer = protected" definition is documented in ADR-007 / 09-ci-and-maintainer.md.

References

  • .github/scripts/recipe-evidence-check.sh (affected-detection + report rendering)
  • .github/workflows/recipe-evidence-comment.yaml
  • recipes/evidence/{gb200-eks-ubuntu-training,h100-gke-cos-training}.yaml
  • ADR-007 trust model: docs/design/007-recipe-evidence.md

Metadata

Metadata

Assignees

Fields

No fields configured for Enhancement.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions