Re-validated gb200-eks-ubuntu-training on a GB200 EKS cluster (all phases
passed) after the aws-efa v0.5.29 / eks-pin bump (NVIDIA#1418) changed the
rendered recipe digest to 5177d2ed. Updates the evidence pointer to the
freshly signed bundle so the recipe-evidence drift gate no longer reports
it stale.

Signed bundle: ghcr.io/yuanchen8911/aicr-evidence:gb200-eks-ubuntu-training-a3fe1a7e7bcb
Rekor log index: 1930094136

Re-validated both recipes after the aws-efa v0.5.29 / eks-pin bump (NVIDIA#1418)
changed their rendered digests, and refreshed the evidence pointers to the
freshly signed bundles so the recipe-evidence drift gate no longer reports
them stale.

- gb200-eks-ubuntu-training: GB200 EKS, all phases passed; bundle
  ghcr.io/yuanchen8911/aicr-evidence:gb200-eks-ubuntu-training-a3fe1a7e7bcb
  (rekorLogIndex 1930094136)
- h100-gke-cos-training: H100 GKE/COS, all phases passed (NCCL 338 GB/s);
  bundle ghcr.io/yuanchen8911/aicr-evidence:h100-gke-cos-training-ee1657fedd59
  (rekorLogIndex 1931193974)

Re-validated both recipes after the aws-efa v0.5.29 / eks-pin bump (NVIDIA#1418)
changed their rendered digests, and refreshed the evidence pointers to the
freshly signed bundles so the recipe-evidence drift gate no longer reports
them stale.

- gb200-eks-ubuntu-training: GB200 EKS, all phases passed; bundle
  ghcr.io/yuanchen8911/aicr-evidence:gb200-eks-ubuntu-training-a3fe1a7e7bcb
  (rekorLogIndex 1930094136)
- h100-gke-cos-training: H100 GKE/COS, all phases passed (NCCL 338 GB/s);
  bundle ghcr.io/yuanchen8911/aicr-evidence:h100-gke-cos-training-ee1657fedd59
  (rekorLogIndex 1931193974)

The container-images BOM renders each chart at its registry
defaultVersion, but at resolution that default is only a fallback: a
componentRef version/tag pin in base.yaml, an overlay, or a mixin
overrides it. So the BOM reflects what recipes install only when no pin
diverges. A pin that drifts from the default (most dangerously on a
component whose sole consumer diverges) makes the BOM advertise a
version no recipe installs -- the #1418 aws-efa class of bug.

Add TestOverlayVersionPinsMatchRegistry: every base/overlay/mixin
componentRefs version/tag must equal the component's registry default
unless declared in versionPinExemptions with a justification. Runs under
make test -> make qualify. Fails on undeclared drift, stale exemptions,
and vacuous discovery.

Each exemption binds a single divergence via expectedPin/expectedDefault:
if either the recipe pin or the registry default later moves, the guard
fails so the divergence is re-reviewed rather than a new, unvetted
version silently inheriting the old blessing.

Fix the one live divergence it caught: base.yaml pinned nvsentinel
v1.3.0 while the registry default (and BOM) advertised v1.9.0 -- every
recipe shipped v1.3.0. Align base to v1.9.0 so reality matches the BOM.
Record the intentional aks kube-prometheus-stack 83.7.0 divergence (#700)
as an exemption.

Document the single-source version model in docs/contributor/recipe.md.

Fixes: #1424
Signed-off-by: Mark Chmarny <[email protected]>

The container-images BOM renders each chart at its registry
defaultVersion, but at resolution that default is only a fallback: a
componentRef version/tag pin in base.yaml, an overlay, or a mixin
overrides it. So the BOM reflects what recipes install only when no pin
diverges. A pin that drifts from the default (most dangerously on a
component whose sole consumer diverges) makes the BOM advertise a
version no recipe installs -- the #1418 aws-efa class of bug.

Enforce both halves of #1424's acceptance:

- TestOverlayVersionPinsMatchRegistry (pkg/recipe): every
  base/overlay/mixin componentRefs version/tag must equal the
  component's registry default unless declared in versionPinExemptions
  with a justification. Each exemption binds one divergence via
  expectedPin/expectedDefault, so a later move of either the pin or the
  default re-triggers review. Fails on undeclared drift, stale
  exemptions, and vacuous discovery.
- TestCommittedBOMVersionsMatchRegistry (tools/bom): the committed
  docs/user/container-images.md version column must match the registry
  pins. This catches a coordinated bump (registry + pins moved together)
  that forgot make bom-docs -- which the recipe guard alone would pass.
  Version-only diff, no Helm rendering, runs under make test.

Both run under make test -> make qualify.

Fix the one live divergence caught: base.yaml pinned nvsentinel v1.3.0
while the registry default (and BOM) advertised v1.9.0 -- every recipe
shipped v1.3.0. Align base to v1.9.0 so reality matches the BOM. Record
the intentional aks kube-prometheus-stack 83.7.0 divergence (#700) as an
exemption.

Document the single-source version model and the two gates in
docs/contributor/recipe.md.

Fixes: #1424
Signed-off-by: Mark Chmarny <[email protected]>

The container-images BOM renders each chart at its registry
defaultVersion, but at resolution that default is only a fallback: a
componentRef version/tag pin in base.yaml, an overlay, or a mixin
overrides it. So the BOM reflects what recipes install only when no pin
diverges. A pin that drifts from the default (most dangerously on a
component whose sole consumer diverges) makes the BOM advertise a
version no recipe installs -- the #1418 aws-efa class of bug.

Enforce both halves of #1424's acceptance:

- TestOverlayVersionPinsMatchRegistry (pkg/recipe): every
  base/overlay/mixin componentRefs version/tag must equal the
  component's registry default unless declared in versionPinExemptions
  with a justification. Each exemption binds one divergence via
  expectedPin/expectedDefault, so a later move of either the pin or the
  default re-triggers review. It also resolves every overlay and fails
  if an exemption's registry default is installed by no recipe (the
  sole-consumer policy). Fails on undeclared drift, stale/unsafe
  exemptions, and vacuous discovery.
- TestCommittedBOMVersionsMatchRegistry (tools/bom): the committed
  docs/user/container-images.md version column must match the registry
  pins. This catches a coordinated bump (registry + pins moved together)
  that forgot make bom-docs -- which the recipe guard alone would pass.
  Version-only diff, no Helm rendering, runs under make test.

Both run under make test -> make qualify.

Fix the one live divergence caught: base.yaml pinned nvsentinel v1.3.0
while the registry default (and BOM) advertised v1.9.0 -- every recipe
shipped v1.3.0. Align base to v1.9.0 so reality matches the BOM. Record
the intentional aks kube-prometheus-stack 83.7.0 divergence (#700) as an
exemption.

Realign stale pins in examples/recipes/{eks-training,aks-training,
eks-gb200-ubuntu-training-with-validation}.yaml to resolved output
(gpu-operator v26.3.2, nvsentinel v1.9.0, AKS kube-prometheus-stack
83.7.0), preserving their curated form.

Document the single-source version model and the two gates in
docs/contributor/recipe.md.

Fixes: #1424
Signed-off-by: Mark Chmarny <[email protected]>

The container-images BOM renders each chart at its registry
defaultVersion, but at resolution that default is only a fallback: a
componentRef version/tag pin in base.yaml, an overlay, or a mixin
overrides it. So the BOM reflects what recipes install only when no pin
diverges. A pin that drifts from the default (most dangerously on a
component whose sole consumer diverges) makes the BOM advertise a
version no recipe installs -- the #1418 aws-efa class of bug.

Enforce both halves of #1424's acceptance:

- TestOverlayVersionPinsMatchRegistry (pkg/recipe): every
  base/overlay/mixin componentRefs version/tag must equal the
  component's registry default unless declared in versionPinExemptions
  with a justification. Each exemption binds one divergence via
  expectedPin/expectedDefault, so a later move of either the pin or the
  default re-triggers review. It also resolves every overlay and fails
  if an exemption's registry default is installed by no recipe (the
  sole-consumer policy). Fails on undeclared drift, stale/unsafe
  exemptions, and vacuous discovery.
- TestCommittedBOMVersionsMatchRegistry (tools/bom): the committed
  docs/user/container-images.md version column must match the registry
  pins. This catches a coordinated bump (registry + pins moved together)
  that forgot make bom-docs -- which the recipe guard alone would pass.
  Version-only diff, no Helm rendering, runs under make test.

Both run under make test -> make qualify.

Fix the one live divergence caught: base.yaml pinned nvsentinel v1.3.0
while the registry default (and BOM) advertised v1.9.0 -- every recipe
shipped v1.3.0. Align base to v1.9.0 so reality matches the BOM. Record
the intentional aks kube-prometheus-stack 83.7.0 divergence (#700) as an
exemption.

Realign stale pins in examples/recipes/{eks-training,aks-training,
eks-gb200-ubuntu-training-with-validation}.yaml to resolved output
(gpu-operator v26.3.2, nvsentinel v1.9.0, AKS kube-prometheus-stack
83.7.0), preserving their curated form.

Document the single-source version model and the two gates in
docs/contributor/recipe.md.

Fixes: #1424
Signed-off-by: Mark Chmarny <[email protected]>