Summary

The B200 accelerator has only the wildcard stub b200-any-training.yaml (added by #436) and no concrete service-bound overlays. A user running aicr recipe --service gke --accelerator b200 --intent <any> resolves only the wildcard NCCL threshold, with no GKE-COS GPU operator config, no GPUDirect-TCPXO components, no platform variant.

Scope rescoped to GKE first, anchored on the GCP testbed nvcf-dgxc-k8s-gcp-azne1-prd7. EKS / AKS / OKE B200 coverage is tracked as separate follow-up issues so each can land paced by its own testbed availability.

Motivation / Context

Surfaced as an explicit "out of scope (track separately)" item in #969 — the validation-phase coverage audit listed b200-any-training as a stub missing D + C, but the per-leaf fix pattern doesn't apply to wildcard overlays. The fix is concrete service-bound overlays at non-wildcard level.

Picking GKE first because the testbed (a4-series B200 on GCP, cluster nvcf-dgxc-k8s-gcp-azne1-prd7) is the cluster currently available to validate the resolved recipe and bundle end-to-end. EKS p6-b200 is GA but no internal AICR testbed is wired up yet.

Suggested scope (GKE only)

Mirror the H100 / GB200 leaf pattern adapted to GKE-COS. GKE is COS-only by AICR convention — there is no -ubuntu- variant for any GKE leaf today (see recipes/overlays/h100-gke-cos-*.yaml), so B200 GKE follows the same shape.

This PR:

• b200-any.yaml — per-accelerator wildcard carrying the deployment-phase floor (4 standard checks + Deployment.gpu-operator.version >= v25.10.0). Mirrors gb200-any.yaml from feat(recipe): deliver deployment-phase floor at per-accelerator wildcards #1001 so every accelerator-bound B200 query inherits a real version assertion.
• b200-gke-cos-training.yaml (base: gke-cos-training)
• b200-gke-cos-inference.yaml (base: gke-cos-inference)
• b200-gke-cos-training-kubeflow.yaml (mixin: platform-kubeflow)
• b200-gke-cos-inference-dynamo.yaml (dynamo + grove, DRA, K8s ≥ 1.34)
• b200-gke-cos-training-slurm.yaml — include iff Blackwell-on-Slurm is a real workload pattern, otherwise drop and reuse the H100 pattern verbatim when a B200/Slurm requester appears.

B200-vs-GB200 deltas to honor on GKE:

• Host CPU is x86 (datacenter Blackwell on standard x86 host). Use the real tuning-gke.yaml for nodewright-customizations (same as h100-gke-cos-*), not the GB200 no-op tuning.yaml.
• No NVreg_GrdmaPciTopoCheckOverride=1 kernel-module-params override — that flag exists for GB200's Grace PCI topology with EFA. GKE uses GPUDirect-TCPXO over Ethernet (no EFA, no analogous workaround). Verify on testbed before omitting permanently.
• Single-fabric NCCL (no MNNVL / NVL72 IMEX domain) — keep the existing nccl-all-reduce-bw >= 350 from b200-any-training.yaml rather than splitting into net + nvls like GB200.
• GPU operator version floor >= v25.10.0 (Blackwell support stabilized in 25.10, matches GB200).
• K8s version floor >= 1.32.4 for standard leaves, >= 1.34 for the dynamo leaf (DRA GA).
• GPUDirect-TCPXO via the existing gke-nccl-tcpxo componentRef (nccl-tcpxo-installer.yaml + nri-device-injector.yaml) — required for multi-NIC inter-node NCCL on B200, identical to the H100/GKE pattern.

Cluster anchor

Reference cluster: nvcf-dgxc-k8s-gcp-azne1-prd7 (GCP, asia-northeast1, NVCF prod). Machine type, COS image version, K8s version, GPU operator pin, and multi-NIC networking config will be lifted from this cluster's cluster-spec.yaml and used to set the constraint floors and componentRef parameters.

Out of scope (file separately)

• EKS B200 (p6-b200.48xlarge) — separate issue once AICR EKS B200 testbed exists.
• AKS B200 (Standard_ND96isr_B200_v6) — separate issue, blocked on Azure testbed.
• OKE B200 (BM.GPU.B200.8) — separate issue, blocked on OCI testbed.
• GB200 (Grace-Blackwell superchip) — distinct gb200 enum value, distinct overlays already exist.
• MIG profiles on B200 — defer until MIG on Blackwell is GA in gpu-operator.

Per-PR checklist (acceptance)

1. go test ./pkg/recipe/... -run TestOverlayValidationPhaseFloor — pass with no knownGaps entry for any new leaf.
2. aicr recipe --service gke --accelerator b200 --intent training --format yaml hydrates the gpu-operator block, version pin, NCCL check, and TCPXO componentRef.
3. make bom-docs if any chart pin changes (none expected in PR 1).
4. make qualify clean.
5. Decide whether b200-any-training.yaml survives alongside the new b200-any.yaml — GB200 kept both (intent-scoped perf floor + accelerator-wide deployment floor). Recommend the same.

Related

• Add B200 accelerator type support #436 — Add B200 accelerator type support (CLOSED, added the wildcard stub)
• Close deployment-phase validation coverage gaps for accelerator-bound GPU recipes #969 — validation-phase coverage audit
• feat(recipe): deliver deployment-phase floor at per-accelerator wildcards #1001 — per-accelerator wildcard deployment floor (gb200-any.yaml, h100-any.yaml, rtx-pro-6000-any.yaml)
• H100 / GKE overlay set: recipes/overlays/h100-gke-cos-*.yaml (closest reference pattern)
• GB200 overlay set: recipes/overlays/gb200-*.yaml (closest accelerator reference; note GB200-specific ARM / EFA deltas above)