Summary

Two *-any-training.yaml criteria-wildcard overlays exist today:

• recipes/overlays/gb200-any-training.yaml — contributes nccl-all-reduce-bw >= 350 to every GB200 + training query.
• recipes/overlays/b200-any-training.yaml — same pattern for B200.

No equivalent exists for h100, rtx-pro-6000, a100, or l40. The H100 leaves carry per-leaf nccl-all-reduce-bw thresholds tuned to each service's network fabric (EFA, TCPXO, RDMA over Converged Ethernet, etc.) — a single cross-service wildcard threshold is misleading at best because the same number is rarely correct for two fabrics.

This issue establishes the principle and removes the wildcards.

Principle

NCCL bandwidth thresholds live per-leaf, not in *-any-training.yaml wildcards. Each concrete training leaf declares its own validation.performance block with a threshold measured on (or anchored to) that leaf's specific service + accelerator + network fabric.

Cross-cutting accelerator-wide concerns (deployment-phase floor: 4 standard checks + Deployment.gpu-operator.version pin) continue to live in <accelerator>-any.yaml wildcards — that pattern (established by #1001) is fabric-independent and applies cleanly across every service for a given accelerator.

Current state

Scope

Deletions:

• recipes/overlays/gb200-any-training.yaml

(b200-any-training.yaml is deleted by #1004 as a natural consequence — that PR ships b200-gke-cos-training.yaml with its own NCCL threshold, leaving the wildcard unreferenced.)

Backfill required by the GB200 deletion:

• recipes/overlays/gb200-oke-training.yaml — add a validation.performance block. Two options:
  • (a) Measure NCCL all-reduce on the OCI testbed and ship a real nccl-all-reduce-bw-net / nccl-all-reduce-bw-nvls constraint, mirroring gb200-eks-training's split. This is part of feat(validation): add performance-phase constraints to OKE overlays once OCI testbed lands #1007's OCI testbed work.
  • (b) If feat(validation): add performance-phase constraints to OKE overlays once OCI testbed lands #1007 hasn't landed yet, add gb200-oke-* to pkg/recipe/validation_phase_floor_test.go::knownGaps referencing feat(validation): add performance-phase constraints to OKE overlays once OCI testbed lands #1007, then ship the real perf block when feat(validation): add performance-phase constraints to OKE overlays once OCI testbed lands #1007 closes.

Doc updates:

• docs/design/005-overlay-refactoring.md
• docs/integrator/recipe-development.md
• docs/contributor/data.md

Each currently mentions *-any-training.yaml as an example of criteria-wildcard overlays. Update to either (i) reference <accelerator>-any.yaml (the accelerator-scoped deployment floor) instead, or (ii) leave a brief historical note that the intent-scoped wildcard shape was retired and explain why.

Floor-test: confirm TestOverlayValidationPhaseFloor still passes after the deletion (it should — performance-phase coverage for gb200-oke-* is then carried by the leaf's own block or knownGaps).

Coordination with #1004

#1004 (GKE B200 service-bound overlays) is the proximate trigger and independently deletes b200-any-training.yaml. This issue handles the GB200 side. They are independent and can land in either order.

Out of scope

• Adding <accelerator>-any.yaml deployment-floor wildcards for a100 / l40 — there are no overlays at all for those accelerators today (tracked in feat(recipes): add concrete A100 service-bound overlays #1002 / feat(recipes): add concrete L40 / L40S service-bound overlays #1003). Their floor wildcards land when the first concrete leaf does.
• Changing the <accelerator>-any.yaml accelerator-wildcard pattern itself — that pattern is fabric-independent (carries Deployment.gpu-operator.version + the 4 standard deployment checks), correct, and stays.

Acceptance

1. recipes/overlays/gb200-any-training.yaml removed.
2. gb200-oke-training.yaml either ships its own validation.performance block, or gb200-oke-* are added to knownGaps with a #1007 reference.
3. go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloor passes.
4. aicr recipe --service oke --accelerator gb200 --intent training --format yaml resolves successfully and surfaces a meaningful (or explicitly-skipped) performance phase.
5. Doc references updated.
6. make qualify clean.

Related

• feat(recipe): deliver deployment-phase floor at per-accelerator wildcards #1001 — per-accelerator deployment-floor wildcard pattern (<accelerator>-any.yaml); this issue is the symmetric cleanup for the intent-scoped wildcard pattern.
• feat(recipes): add concrete GKE B200 service-bound overlays #1004 — GKE B200 service-bound overlays (deletes b200-any-training.yaml as a natural consequence).
• feat(validation): add performance-phase constraints to OKE overlays once OCI testbed lands #1007 — OCI testbed (GB200 / OKE performance constraints; supplies the real numbers for the gb200-oke-training backfill).
• feat(validation): add performance-phase constraints to LKE overlays once Linode testbed lands #1008 — Linode testbed (informs the floor-test knownGaps shape).
• Close deployment-phase validation coverage gaps for accelerator-bound GPU recipes #969 — validation-phase coverage audit (original trigger for the wildcard refactor work).